Adjustable geometry controller for three-dimensional surgical planning based on multi-plane reconstruction
By introducing multi-plane reconstruction technology in 3D surgical planning, using adjustable geometric controllers and reference surface projection methods, the unintuitive and resource-consuming problems of traditional geometric controllers in rotation and translation operations are solved, achieving more accurate and efficient The three-dimensional space operation improves the accuracy and user experience of surgical planning.
Patent Information
- Application Number
- CN202410344000.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-25
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-03-25
AI Technical Summary
The geometric controllers in traditional 3D surgical planning have problems such as unintuitive, resource-consuming and poor user experience in rotation and translation operations, especially when precise positioning and operation in three-dimensional space, iterative adjustments are required, which affects the accuracy of surgical planning. and efficiency.
Adopting an adjustable geometric controller based on multi-plane reconstruction, operates through three mutually perpendicular planes, uses plane normal vector direction drag and rotation axis adjustment to achieve translation and rotation, and optimizes mouse operations with reference surface projection method to reduce computing resources Consumption and improve operational intuitiveness.
It realizes more accurate and intuitive object operation in three-dimensional space, reduces the iterative adjustment process, improves the accuracy and efficiency of surgical planning, and reduces the consumption of computing resources.
Smart Images

Figure CN118942646A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of three-dimensional surgery planning, and in particular relates to an adjustable geometric controller for three-dimensional surgery planning based on multi-planar reconstruction. Background Art
[0002] In orthopedics, doctors usually need to use imaging to observe the patient's bone structure in order to accurately diagnose and formulate treatment plans. Although traditional two-dimensional images can provide certain information, they may have some limitations when observing complex bone structures, such as occlusion, overlap, and blur.
[0003] With the continuous advancement of medical imaging technology, doctors can obtain more accurate and detailed patient bone data and use computer software to accurately reconstruct the bones in three dimensions. Orthopedic three-dimensional volume reconstruction technology generates individualized bone models and volume calculation results based on the patient's specific conditions, which can effectively reduce human errors and help doctors develop more accurate and detailed personalized surgical planning plans.
[0004] 3D surgical planning can provide more accurate anatomical information, including bone structure, joint morphology, soft tissue, etc. Through 3D reconstruction and simulation operations, doctors can better understand the patient's specific conditions, better select surgical methods and surgical instruments, and thus make more accurate surgical plans. 3D surgical planning can help doctors determine the best surgical path and operation method in advance, better assess surgical risks and postoperative effects, reduce surgical risks, improve surgical results, and reduce surgical time and trauma.
[0005] Rotation and translation are basic operations in 3D surgical planning, and 3D geometric controllers are basic tools for 3D object manipulation. Currently, there are many urgent problems that need to be solved in 3D surgical planning software, such as how to accurately position the spinal correction, how to accurately position the mouse interactive operation in 3D space, and how to more accurately measure the Cobb angle and intervertebral distance.
[0006] The standard geometric controller in 3D surgical planning consists of three mutually perpendicular axes to perform operations such as translation and rotation of objects. The controller is generally bound to the local coordinate system of the object being operated. The object geometric controller in traditional 3D surgical planning has the following problems:
[0007] (1) Rotation can only be performed by selecting the X, Y, and Z axes of the local coordinate system of the object. This is equivalent to decomposing the rotation operation in three-dimensional space into the Euler angles of the X, Y, and Z axes to achieve the final rotation state of the object. During the operation, it is necessary to iterate and fine-tune each rotation axis one by one.
[0008] (2) Translation operations can only be performed on the X, Y, and Z axes of the local coordinate system of the selected object, which is equivalent to decomposing the translation operation of the object in three-dimensional space into translation operations in the X, Y, and Z axis directions. During the operation, it is necessary to iterate and fine-tune each direction one by one to move to the final appropriate position.
[0009] (3) Regardless of rotation or translation, the direction and size of the displacement of the object in the 3D surgical simulation, the direction of the rotation axis, the position and the size of the rotation angle, all need to be determined by comparing with the relative positions of other bones and organs in 3D space. Since the two-dimensional computer display can only observe the relative position between the selected object and the reference object from the specified camera direction, in order to confirm the position and direction of the selected object after movement, it is necessary to modify and confirm from different angles by changing the camera direction, and this process often requires repeated iterations.
[0010] In 3D surgical planning software, how to correctly and reasonably map mouse operations to the position and direction changes of the operated object in 3D is crucial to the overall UI design and user experience, and further affects the accuracy and practicality of the overall surgical planning operation. Traditional 3D object geometry controllers cannot solve the above problems well, so it is often necessary to use three view windows in the coronal, sagittal, and axial directions at the same time, provide observations in different directions through the linkage of three views, and rotate and translate the object in three directions respectively. The linkage of three views consumes three times the computer resources and is not intuitive in operation. Users need to observe the other two views while operating one view to obtain position reference information, and often require repeated iterations and fine-tuning in three directions to determine the final position and direction of the controlled object.
[0011] Therefore, further improvements are made to the above problems. Summary of the invention
[0012] The main purpose of the present invention is to provide an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, which is committed to optimizing operations such as translation and rotation of 3D objects in three-dimensional space, expanding the application of 3D modeling technology in medical imaging and surgical planning, and solving the problem of precise positioning and operation through mouse interaction in three-dimensional space.
[0013] To achieve the above objectives, the present invention provides an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, including a three-dimensional MPR geometric controller. The three-dimensional MPR geometric controller includes three mutually perpendicular planes, each plane includes four edges and four endpoints of the circumscribed DICOM image, and the plane is displaced in the direction of the plane normal vector by dragging the edge of the plane (using a mouse), and the area size of the plane is changed by dragging the endpoint (using a mouse). After selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged, so as to adjust the normal vector direction of the plane and the orthogonal constraint plane of the plane in a linked manner, wherein:
[0014] In the initial state, three orthogonal planes parallel to the coronal plane, sagittal plane, and axial plane are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes. At the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis of the 3D object operation.
[0015] In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects.
[0016] During measurement, point selection measurement and plane measurement are realized through the three-dimensional MPR geometry controller.
[0017] As a further preferred technical solution of the above technical solution, for operation interaction:
[0018] First, the position and direction of the controller's translation axis and rotation axis are adjusted by translating and rotating the MPR plane, and the translation direction and rotation axis direction of the controlled object are confirmed and calibrated by the 2D cross-section of the DICOM image using the MPR plane;
[0019] When you need to translate an object, select the target object, then hold down a preset key (preferably the Shift key) and drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object;
[0020] When the object needs to be rotated, first click to select the rotation axis, hold down the preset key (preferably the Shift key) and use the mouse to drag the edge of the plane where the rotation axis is located and perpendicular to the rotation axis to adjust the rotation angle. The closer the distance to the center point is, the greater the rotation amplitude, and the farther the distance from the center point is, the smaller the rotation amplitude (which is convenient for doctors to perform precise control), thereby realizing the rotation function of the object.
[0021] As a further preferred technical solution of the above technical solution, a reference plane projection method is used to implement the specific operation of the mouse on the MPR plane, wherein:
[0022] The 3D MPR geometry controller is composed of the DICOM bounding box B(x min ,x max ,y min ,y max ,z min ,z max ), the controller center point O and the normal vectors (n1, n2, n3) of the three orthogonal planes, each MPR section is determined by the normal vectors n, viewX, viewUP, and the plane vertices (p0, p1, p2) are calculated according to the tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point and the bounding box B in the direction of the normal vector;
[0023] In the initial state, the normal vectors of the three planes are set to the (x, y, z) coordinate axes of the bounding box by default. When the MPR plane is transformed, the new tangent points of the plane and the bounding box are calculated based on the transformed center point O and the plane vectors n, viewX, viewUP, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated.
[0024] As a further preferred technical solution of the above technical solution, under the translation operation, the mouse selects the border of the MPR plane, and the normal vector of the plane passing through the center point is used as the translation axis. The selection of the MPR plane border is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction (dop) intersects with the border of the MPR plane. At the same time, the spatial relationship between the bounding box and the MPR plane border needs to be considered, and the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box are calculated. When the border of the MPR plane intersects with the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distance of all the six planes of the bounding box, the plane is considered to be selected, and its intersection point is set as the initial coordinate point pw0 of the mouse operation;
[0025] In the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the rotation axis vertices t0 and t1 and the intersection distance L. At the same time, calculate the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box. When the intersection distance L is less than the intersection distance of all the six planes of the bounding box, the rotation axis is considered to be selected;
[0026] Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after moving is pw1. Regardless of translation or rotation operation, you first need to determine a suitable reference plane, and project points pw0 and pw1 onto the reference plane according to the camera projection direction. In the translation operation, the translation vector is obtained based on the relationship between the projection point and the translation axis. In the rotation operation, the rotation angle is obtained through the positional relationship between the projection point and the rotation axis.
[0027] As a further preferred technical solution of the above technical solution, in the translation operation, when the mouse moves, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop, and the obtained projection points pw′0 and pw′1 are used to determine the size of the translation vector by the inner product of the displacement vector pw′0-pw′1 and the translation axis.
[0028] In the translation operation, the center point O of the MPR geometry controller is moved according to the translation vector, and the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated;
[0029] During the rotation operation, the projection plane is selected as a plane with the rotation axis as the normal vector, and the center point of the plane is the intersection point O of the perpendicular line from the mouse world coordinate system coordinate point pw0 to the rotation axis. t Sure;
[0030] During the rotation operation, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop to obtain the projection points pw′0 and pw′1. The rotation angle is given by the vector u=pw1-O t and v = pw0-O t Through the inverse cosine function:
[0031]
[0032] Solve to obtain;
[0033] Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the transformation of any spatial point by the rotation operation can be performed by the following steps:
[0034] Translation operation:
[0035] Assuming the origin is (x0, y0, z0), the formula for translating a point (x, y, z) to a new coordinate (x′, y′, z′) is:
[0036] x′=x-x0
[0037] y′=y-y0
[0038] Z′=z-z0;
[0039] Rotation operation:
[0040] The rotation matrix for rotating around the normal vector (a, b, c) by an angle θ can be expressed as:
[0041]
[0042] Reverse translation operation:
[0043] The formula for translating the coordinates (x′, y′, z′) to the original position (x″, y″, z″) is:
[0044] x″=x′+x0
[0045] y″=y′+y0
[0046] z″=z′+z0;
[0047] During the rotation operation, the normal vectors of each plane of the MPR geometry controller are transformed according to the rotation operation, and then the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.
[0048] As a further preferred technical solution of the above technical solution, for the point selection measurement method of the controller, the point is selected by adjusting the center point position of the three-dimensional MPR geometric controller, and the center point of the intersection of the three planes is directly marked as the endpoint and adjusted.
[0049] As a further preferred technical solution of the above technical solution, for the plane measurement method, the three-dimensional MPR geometry controller inserts an adjustment plane in any direction and marks it.
[0050] To achieve the above objectives, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction is implemented.
[0051] To achieve the above objectives, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 An adjustable geometry controller for three-dimensional surgical planning based on multi-planar reconstruction is shown.
[0053] Figure 2 The actual operation interface of the geometry controller is shown.
[0054] Figure 3Shows the geometry controller switching from planar mode to mesh mode.
[0055] Figure 4 The geometry controller is shown in an enlarged state, and precise control can also be achieved through operation of the adjustment axis.
[0056] Figure 5 A scene showing an object being moved via a geometry controller.
[0057] Figure 6 A scene showing the rotation of an object via a geometry controller.
[0058] Figure 7 The standard MPR three-view diagram is shown.
[0059] Figure 8 Shown is the structure of the MPR plane in the MPR geometry controller.
[0060] Fig. 9 The translation operation of the MPR geometry controller is shown. DETAILED DESCRIPTION
[0062] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.
[0063] In the preferred embodiments of the present invention, those skilled in the art should note that the objects and the like involved in the present invention may be regarded as prior art.
[0064] Preferred embodiments.
[0065] MPR (Multi-Planar Reconstruction) technology is a medical image processing technology that can convert three-dimensional medical image data into two-dimensional images displayed on different planes. This technology is commonly used in imaging modalities such as CT and MRI. By selecting different reconstruction algorithms, images can be presented on multiple planes such as the coronal plane, sagittal plane, and axial plane, thereby improving doctors' observation and diagnosis of the patient's internal tissue structure.
[0066] The MPR plane is generated by slicing and reconstructing three-dimensional medical imaging data. Different slicing planes can be selected according to the doctor's needs to provide more comprehensive and detailed anatomical information.
[0067] The MPR plane can provide plane reference and positioning in a specific direction, helping doctors to more accurately evaluate the three-dimensional structure of the spine, assist doctors in surgical navigation, implant selection and surgical path planning to minimize surgical risks and improve surgical accuracy and safety.
[0068] This scheme introduces the MPR geometry controller into three-dimensional surgical planning to control three-dimensional objects.
[0069] like Figure 1-9 As shown, the present invention discloses an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, including a three-dimensional MPR geometric controller, wherein the three-dimensional MPR geometric controller includes three mutually perpendicular planes, each plane includes four edges and four endpoints of the circumscribed DICOM image, and the plane is displaced in the direction of the plane normal vector by dragging the edge of the plane (using a mouse), and the area size of the plane is changed by dragging the endpoint (using a mouse), and after selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged, so as to adjust the normal vector direction of the plane and the orthogonal constraint plane of the plane in linkage (the plane can adjust the opacity, and can also be switched to a grid mode, which can reduce occlusion while still providing a reference, so that the doctor can observe the three-dimensional section in real time and directly adjust the target), wherein:
[0070] In the initial state, three orthogonal planes parallel to the coronal plane, sagittal plane, and axial plane are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes (to facilitate the doctor to have the ability to make quick and accurate adjustments under magnification), and at the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis of the 3D object operation;
[0071] In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects.
[0072] During measurement, point selection measurement and plane measurement are realized through the three-dimensional MPR geometry controller.
[0073] Specifically, for operation interactions:
[0074] First, the position and direction of the controller's translation axis and rotation axis are adjusted by translating and rotating the MPR plane, and the translation direction and rotation axis direction of the controlled object are confirmed and calibrated by the 2D cross-section of the DICOM image using the MPR plane;
[0075] When you need to translate an object, select the target object, then hold down a preset key (preferably the Shift key) and drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object;
[0076] When the object needs to be rotated, first click to select the rotation axis, hold down the preset key (preferably the Shift key) and use the mouse to drag the edge of the plane where the rotation axis is located and perpendicular to the rotation axis to adjust the rotation angle. The closer the distance to the center point is, the greater the rotation amplitude, and the farther the distance from the center point is, the smaller the rotation amplitude (which is convenient for doctors to perform precise control), thereby realizing the rotation function of the object.
[0077] In the specific implementation, the operation interaction of the 3D MPR geometry controller needs to correctly and reasonably map the mouse operation to the position and direction change of the operated object in 3D to achieve an accurate user operation experience. The mouse operation of the MPR geometry controller includes the selection operation of the MPR plane and the rotation axis and the drag and shift operation of the mouse after selection. In the traditional MPR algorithm, it is necessary to display the 2D views of the three MPR planes at the same time, and adjust the position and direction of the other two orthogonally linked MPR planes by controlling the coordinate axis control points in the 2D view, such as Figure 7 As shown. Manipulating the MPR plane through a two-dimensional view will bring about problems such as unintuitive operation and poor user interaction experience. The present invention adopts a reference surface projection method to realize the specific operation of the mouse on the MPR plane.
[0078] The 3D MPR geometry controller is composed of the DICOM bounding box B(x min ,x max ,y min ,y max ,z min ,z max ), the controller center point O and the normal vectors (n1, n2, n3) of the three orthogonal planes are determined, such as Figure 8 As shown. Each MPR section is determined by the normal vector n, viewX, viewUP, and the plane vertices (p0, p1, p2) are calculated based on the tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point and the bounding box B in the direction of the normal vector.
[0079] In the initial state, the normal vectors of the three planes are set to the (x, y, z) coordinate axes of the bounding box by default. When the MPR plane is transformed, the new tangent points of the plane and the bounding box are calculated based on the transformed center point O and the plane vectors n, viewX, viewUP, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated;
[0080] During the translation operation, the mouse passes through the border of the selected MPR plane, and the normal vector of the plane through the center point is used as the translation axis. The selection of the MPR plane border is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the border of the MPR plane. At the same time, the spatial relationship between the bounding box and the MPR plane border needs to be considered, and the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box are calculated. When the border of the MPR plane intersects with the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distance of all the six planes of the bounding box, the plane is considered to be selected, and its intersection point is set as the initial coordinate point pw0 of the mouse operation.
[0081] During the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the rotation axis vertices t0 and t1 and the intersection distance L. At the same time, calculate the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box. When the intersection distance L is less than the intersection distance of all the six planes of the bounding box, the rotation axis is considered to be selected.
[0082] Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after moving is pw1. Regardless of translation or rotation operation, you first need to determine a suitable reference plane, and project points pw0 and pw1 onto the reference plane according to the camera projection direction. In the translation operation, the translation vector is obtained based on the relationship between the projection point and the translation axis. In the rotation operation, the rotation angle is obtained through the positional relationship between the projection point and the rotation axis.
[0083] The reference projection plane in the translation operation is selected as one of the six faces of the DICOM bounding box. The specific method is as follows:
[0084] 1. First, get the direction of the camera dop and the initial world coordinate point where the mouse is pressed as pw0.
[0085] 2. Then, traverse the six faces and select the planes that meet the conditions as the target planes for transformation through a series of criteria.
[0086] 3. The screening criteria include:
[0087] 3.1 If the dot product of the face normal and the camera projection direction is less than the given threshold EPSILON, the face is perpendicular to or deviates from the camera direction and is removed.
[0088] 3.2 If the dot product of the face normal and the slice normal is close to 1, the face is removed to ensure that the selected face and the slice are not in the same plane.
[0089] 3.3 If the above screening conditions are met, the face index number and the dot product value of the face normal and the camera projection direction are added to the candidate list.
[0090] 4. If the length of the candidate list is greater than or equal to 2, sort them in descending order according to the inner product of the plane normal vector and the camera projection vector. Select the first face in the candidate list that is also the face closest to the initial world coordinate point as the reference plane.
[0091] In translation operation, such as Fig. 9 As shown, when the mouse moves, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop, and the obtained projection points pw′0 and pw′1 are determined by the inner product of the displacement vector pw′0-pw′1 and the translation axis.
[0092] In the translation operation, the center point O of the MPR geometry controller is moved according to the translation vector, and the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.
[0093] During the rotation operation, the projection plane is selected as a plane with the rotation axis as the normal vector, and the center point of the plane is the intersection point O of the perpendicular line from the mouse world coordinate system coordinate point pw0 to the rotation axis. t Sure.
[0094] During the rotation operation, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop to obtain the projection points pw′0 and pw′1. The rotation angle is given by the vector u=pw1-O t and v = pw0-O t Through the inverse cosine function:
[0095]
[0096] Solved and obtained.
[0097] Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the transformation of any spatial point by the rotation operation can be performed by the following steps:
[0098] 1. Translation operation:
[0099] Assuming the origin is (x0, y0, z0), the formula for translating a point (x, y, z) to a new coordinate (x′, y′, z′) is:
[0100] x′=x-x0
[0101] y′=y-y0
[0102] z′=z-z0
[0103] 2. Rotation operation:
[0104] The rotation matrix for rotating around the normal vector (a, b, c) by an angle θ can be expressed as:
[0105]
[0106] 3. Reverse translation operation
[0107] The formula for translating the coordinates (x′, y′, z′) to the original position (x″, y″, z″) is:
[0108] x″=x′+x0
[0109] y″=y′+y0
[0110] z″=z′+z0
[0111] During the rotation operation, the normal vectors of each plane of the MPR geometry controller are transformed according to the rotation operation, and then the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.
[0112] More specifically, for the point selection measurement method of the controller, the point is selected by adjusting the center point position of the three-dimensional MPR geometric controller, and the center point of the intersection of the three planes is directly marked as the endpoint and adjusted.
[0113] Furthermore, for the plane measurement method, the three-dimensional MPR geometry controller inserts an adjustment plane in any direction and marks it. A parallel and close adjustment plane is inserted at the first position required by the object (for example, the lower end plate of the vertebra), marked as the lower end plate, and a parallel and close adjustment plane is inserted at the second position required by the object (the upper end plate of the next vertebra), marked as the upper end plate. The average distance between the two adjustment planes is the distance between the two positions (that is, the intervertebral distance between the two vertebrae). The measurement of angles such as the Cobb angle can also be completed using a plane. After determining the target vertebra and setting the corresponding adjustment plane, the projection angle of the two planes on the coronal plane of the human body is the Cobb angle.
[0114] Preferably, all measurement data in the present solution can be tracked in real time. The geometric controller of the present solution allows the inserted measurement points and measurement planes to be linked and bound to the target object. If the measured object undergoes displacement, rotation, etc., the measurement points and measurement planes bound to it also undergo linkage displacement and rotation in the same manner, and the corresponding measurement values will also change in real time.
[0115] This scheme can be used not only for surgical planning of STL models, but also for volume reconstruction models based on volume rendering.
[0116] For the attached pictures:
[0117] Figure 1The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction is shown. The controller uses the plane mode by default, which consists of three mutually perpendicular planes, control axes, and control points.
[0118] Figure 2 The actual operation interface of the geometric controller is shown, in which the STL, volume rendering model and controller can be adjusted to complete the relevant operations of surgical planning.
[0119] Figure 3 The grid mode of the geometry controller is shown. The geometry controller mainly has three modes: grid, plane and transparent, and each plane can adjust the opacity. Users can choose according to their needs.
[0120] Figure 4 The geometric controller in the magnified state is shown. The line segment generated by the intersection of every two planes in the geometric controller is the adjustment axis. In the magnified state, the target object can also be precisely controlled by adjusting the axis. In this solution, before moving or rotating the object, it is necessary to adjust the geometric controller first to determine the translation direction or rotation axis of the object, and then perform subsequent operations after determination.
[0121] Figure 5 The following is a scene showing how to move an object using a geometry controller.
[0122] First, confirm the target moving direction of the object. Second, move or rotate the planes of the geometry controller so that one of the planes is parallel to the target direction. Then, select the target object to be moved, hold down the "shift" key, and drag the plane or its adjustment axis in the geometry controller that is perpendicular to the target direction to move the target object in the target direction. When the desired position is reached, release the mouse to complete the object movement operation.
[0123] Figure 6 The following figure shows a scene of rotating an object through a geometry controller, which specifically includes the following steps:
[0124] First, confirm the target rotation direction and rotation axis of the object. Secondly, move or rotate the plane of the geometry controller so that one of the adjustment axes is consistent with the expected rotation axis position and angle. At this time, the adjustment axis is the target rotation axis. Then, select the target object to be rotated, click the target rotation axis, hold down the "shift" key, and drag the plane or adjustment axis of the geometry controller through the target rotation axis to rotate the target object around the target rotation axis. After reaching the expected position, release the mouse to complete the rotation operation of the object.
[0125] Figure 7The MPR plane manipulation scheme of the standard MPR three-view is shown, and the position and direction of the other two MPR planes are adjusted by the control points of the coordinate axes in the 2D view. MPR plane manipulation through two-dimensional views will bring problems such as unintuitive operation and poor user interaction experience in 3D surgical planning.
[0126] Figure 8 The structure of the MPR plane in the MPR geometry controller is shown, including boundary points, coordinate vectors, and coordinate axis vertices. According to the transformed center point O and the plane vectors n, viewX, viewUP, the new tangent points between the plane and the bounding box are calculated, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated.
[0127] Fig. 9 The MPR geometry controller uses a three-dimensional translation operation algorithm of the mouse with a dynamically selected projection plane. First, the mouse selects the translation axis, dynamically calculates and selects the reference projection plane of the mouse, calculates the displacement vector on the reference plane through the projection points on the reference plane before and after the mouse moves, and obtains the translation vector through the dot product with the translation axis vector, thereby updating the selected MPR plane to the new position.
[0128] For the present invention:
[0129] 1. In this solution, the translation direction or rotation axis of the object is first determined in advance by adjusting the MPR geometry controller. Since the translation and rotation operations of the object are based on the determined movement axis or rotation axis, the mouse interaction is converted into single-dimensional operations such as translation along the movement axis or rotation along the rotation axis, and there is no need to decompose the translation and rotation operations into the directional components of the rectangular coordinates. Therefore, the 3D operation based on the MPR geometry controller is concise, clear and efficient, and can be done in one step, avoiding the iterative adjustment process of the traditional geometry controller in various directions.
[0130] 2. In this solution, the rotation axis or translation direction of the object operation is obtained through the intersection line between the MPR planes. Since each MPR plane is a cross-section of the overall 3D space, the MPR plane provides an effective spatial position reference plane in three-dimensional space, which can be used intuitively to determine the position relationship between 3D space objects in cross-sections in different directions, such as distance, alignment, angle, etc. MPR contains planes in three orthogonal directions in 3D space, so this solution only needs a single view window to accurately and effectively provide position references in different directions. Similar solutions require three view windows to be opened at the same time to provide viewing angles in different directions. At the same time, due to the lack of plane reference, the positioning of objects can only be adjusted and operated iteratively by observing in different directions.
[0131] 3. The objects in medical 3D surgical planning are STL models based on DICOM medical images. This solution uses MPR sections generated from DICOM raw data, which can perform 3D positioning based on reference planes with important clinical medical significance, and calibrate and confirm the movement direction and rotation point of objects in 3D planning, realizing 2D and 3D operation reference and association in a multi-data, multi-modal manner.
[0132] In summary, this solution uses the MPR plane to provide 2D image references and 3D spatial position references that are of medical significance in 3D surgical planning, making up for the lack of references in traditional geometric controllers, which can only rely on observation for spatial calibration. At the same time, the reference provided by the adjusted MPR plane is used to pre-plan the translation direction or rotation angle and direction of the object before operation, avoiding the traditional geometric controller, which can only be operated from the rectangular coordinate direction and repeatedly perform the iterative process of 3D spatial calibration due to the inability to adjust the position and direction of its own coordinate axis. Therefore, the MPR adjustable geometric controller proposed in this solution can provide more accurate and efficient operation of the controlled object in 3D surgical planning, which is an important development and innovation in 3D geometric controller technology.
[0133] The present invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction is implemented.
[0134] The present invention also discloses a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction is implemented.
[0135] It is worth mentioning that the technical features such as objects involved in the patent application of this invention should be regarded as prior art. The specific structure, working principle and possible control method and spatial layout method of these technical features can be selected by conventional methods in the field and should not be regarded as the inventive point of the patent of this invention. The patent of this invention will not be further elaborated.
[0136] For those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. An adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction, characterized in that: The invention comprises a three-dimensional MPR geometry controller, wherein the three-dimensional MPR geometry controller comprises three mutually perpendicular planes, each plane comprises four edges and four endpoints of the circumscribed DICOM image, and the plane is displaced in the direction of the plane normal vector by dragging the edge of the plane, and the area size of the plane is changed by dragging the endpoints. After selecting the rotation axis, the edge of the plane where the rotation axis is located and perpendicular to the rotation axis is dragged, so as to adjust the normal vector direction of the plane and the orthogonal constraint plane of the plane in linkage, wherein: In the initial state, three orthogonal planes parallel to the coronal plane, sagittal plane, and axial plane are used as the initial adjustment planes. The intersection of the three planes is the center control point. The center control point is provided with a direction vector with an arrow parallel to the three planes. At the center control point, a straight line parallel to the normal vectors of each plane is drawn through the center point, which is the center adjustment axis. The center adjustment axis of the 3D MPR geometry controller is adjusted by the translation and rotation of the plane, and can be used as the rotation axis and translation direction axis of the 3D object operation. In terms of operational interaction, the 3D MPR geometric controller is used to implement operations including translation and rotation of objects. During measurement, point selection measurement and plane measurement are realized through the three-dimensional MPR geometry controller.
2. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 1, characterized in that: For operational interactions: First, the position and direction of the controller's translation axis and rotation axis are adjusted by translating and rotating the MPR plane, and the translation direction and rotation axis direction of the controlled object are confirmed and calibrated by the 2D cross-section of the DICOM image using the MPR plane; When you need to translate an object, select the target object, then hold down the preset button and drag the translation axis or the corresponding plane border with the mouse to achieve the translation function of the object; When you need to rotate an object, first click to select the rotation axis. Press and hold the preset button while dragging the edge of the plane where the rotation axis is located and perpendicular to the rotation axis with the mouse to adjust the rotation angle. The closer the distance to the center point is, the greater the rotation amplitude is, and the farther the distance from the center point is, the smaller the rotation amplitude is, thereby realizing the rotation function of the object.
3. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 2, characterized in that: The reference plane projection method is used to realize the specific operation of the mouse on the MPR plane, where: The 3D MPR geometry controller is composed of the DICOM bounding box B(x min ,x max ,y min ,y max ,z min ,z max ), the controller center point O and the normal vectors (n1, n2, n3) of the three orthogonal planes, each MPR section is determined by the normal vectors n, viewX, viewUP, and the plane vertices (p0, p1, p2) are calculated according to the tangent points of the plane and the bounding box. The rotation axis and translation axis of the plane are determined by the two intersection points (t0, t1) of the center point and the bounding box B in the direction of the normal vector; In the initial state, the normal vectors of the three planes are set to the (x, y, z) coordinate axes of the bounding box by default. When the MPR plane is transformed, the new tangent points of the plane and the bounding box are calculated based on the transformed center point O and the plane vectors n, viewX, viewUP, and the plane vertices (p0, p1, p2) and intersection points (t0, t1) are updated.
4. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 3, characterized in that: In the translation operation, the mouse selects the border of the MPR plane, and the normal vector of the plane through the center point is used as the translation axis. The selection of the MPR plane border is determined by detecting whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the border of the MPR plane. At the same time, the spatial relationship between the bounding box and the MPR plane border needs to be considered, and the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box are calculated. When the border of the MPR plane intersects with the projection of the mouse world coordinate point along the camera direction and the intersection distance is less than the intersection distance of all the six planes of the bounding box, the plane is considered to be selected, and its intersection point is set as the initial coordinate point pw0 of the mouse operation; In the rotation operation, directly calculate whether the projection of the world coordinate point clicked by the mouse along the camera direction intersects with the rotation axis vertices t0 and t1 and the intersection distance L. At the same time, calculate the intersection and distance of the mouse world coordinate point along the camera direction in the six planes of the DICOM bounding box. When the intersection distance L is less than the intersection distance of all the six planes of the bounding box, the rotation axis is considered to be selected; Assume that the initial world coordinate point when the mouse is pressed is pw0, and the world coordinate point after moving is pw1. Regardless of translation or rotation operation, you first need to determine a suitable reference plane, and project points pw0 and pw1 onto the reference plane according to the camera projection direction. In the translation operation, the translation vector is obtained based on the relationship between the projection point and the translation axis. In the rotation operation, the rotation angle is obtained through the positional relationship between the projection point and the rotation axis.
5. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 4, characterized in that: In the translation operation, when the mouse moves, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop. The obtained projection points pw′0 and pw′1 are used to determine the size of the translation vector by the inner product of the displacement vector pw′0-pw′1 and the translation axis. In the translation operation, the center point O of the MPR geometry controller is moved according to the translation vector, and the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated; During the rotation operation, the projection plane is selected as a plane with the rotation axis as the normal vector, and the center point of the plane is the intersection point O of the perpendicular line from the mouse world coordinate system coordinate point pw0 to the rotation axis. t Sure; During the rotation operation, the initial world coordinate point pw0 and the current world coordinate point pw1 are projected onto the reference plane according to the camera projection direction dop to obtain the projection points pw′0 and pw′1. The rotation angle is given by the vector u=pw1-O t and v = pw0-O t Through the inverse cosine function: Solve to obtain; Given the rotation axis normal vector n = (a, b, c) and the rotation angle θ, the transformation of any spatial point by the rotation operation can be performed by the following steps: Translation operation: Assuming the origin is (x0, y0, z0), the formula for translating a point (x, y, z) to a new coordinate (x′, y′, z′) is: x′=x-x0 y′=y-y0 Z′=z-z0; Rotation operation: The rotation matrix for rotating around the normal vector (a, b, c) by an angle θ can be expressed as: Reverse translation operation: The formula for translating the coordinates (x′, y′, z′) to the original position (x″, y″, z″) is: x″=x′+x0 y″=y′+y0 z″=z′+z0; During the rotation operation, the normal vectors of each plane of the MPR geometry controller are transformed according to the rotation operation, and then the plane vertices (p0, p1, p2) and the rotation axis vertices (t0, t1) of each MPR plane are updated.
6. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 5, characterized in that: For the point selection measurement method of the controller, the point is selected through the center point position of the three-dimensional MPR geometric controller, and the center point of the intersection of the three planes is directly marked as the endpoint and adjusted.
7. The adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction according to claim 6, characterized in that: For plane measurement, the 3D MPR geometry controller inserts and marks adjustment planes in any direction.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, an adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction as described in any one of claims 1 to 7 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the adjustable geometric controller for three-dimensional surgical planning based on multi-planar reconstruction as claimed in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Blood vessel computer aided iconography evaluating system
CN101923607A
Direct volume rendering method based on transfer function with two-dimensional image being interactive interface
CN103745496A
Three-dimensional registering method for single image
CN106327532A
Two-dimensional medical image and three-dimensional model positioning linkage method and system based on VTK
CN115049710A
MPR three-dimensional reconstruction method of medical image
CN116402875A
