VTK-based linkage method and system for 2D medical imaging and 3D model positioning

Abstract

The present invention belongs to the technical field of medical image data processing, and provides a VTK-based two-dimensional medical image and three-dimensional model positioning linkage method and system, which obtains two-dimensional medical image data, obtains a section image of any position and direction based on the two-dimensional medical image data; based on the section image, through MPR positioning line interaction, rotates to obtain any section; displays the section image in a three-dimensional model with matching position, rotates and moves interactively in two-dimensional space to cause section image changes, and in the three-dimensional model window, uses the MPR image intersection as a pick-up point of interest, moves and transforms the point to switch different sections to perform linkage between two-dimensional medical images and three-dimensional models. By interactive operation, the two-dimensional and three-dimensional models are placed in the same coordinate space for observation, which can more intuitively reflect the spatial relationship between the two.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image data processing, and in particular relates to a VTK-based two-dimensional medical image and three-dimensional model positioning linkage method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of modern medical imaging technology, two-dimensional images and three-dimensional model spatial positioning have key guiding significance for guiding surgical approaches.

[0004] Currently, two-dimensional images and three-dimensional models are displayed in different windows, and mainly rely on people to establish spatial connections independently. This method is prone to deviations, and it is impossible to accurately place two-dimensional images and three-dimensional models in a unified coordinate space for observation through interactive operations. It is not convenient to display the spatial relationship between the two, and it is impossible to achieve accurate positioning. Summary of the invention

[0005] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a VTK-based two-dimensional medical image and three-dimensional model positioning linkage method and system, which can more intuitively reflect the spatial relationship between the two-dimensional and three-dimensional models by placing them in the same coordinate space for observation, thereby achieving accurate positioning.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a VTK-based two-dimensional medical image and three-dimensional model positioning linkage method, comprising the following steps:

[0008] Acquire two-dimensional medical imaging data, and acquire cross-sectional images at any position and direction based on the two-dimensional medical imaging data;

[0009] Based on the section image, any section can be obtained by rotating through the interaction of MPR positioning lines;

[0010] The section image is displayed in a three-dimensional model with matching position. Rotation and movement in two-dimensional space can trigger changes in the section image. In the three-dimensional model window, the intersection of the MPR image is used as the picking point of interest. The point is moved and transformed to switch between different sections to achieve linkage between two-dimensional medical imaging and three-dimensional models.

[0011] The second aspect of the present invention provides a VTK-based two-dimensional medical image and three-dimensional model positioning linkage system, comprising:

[0012] A section image acquisition module is used to acquire two-dimensional medical image data, and acquire section images at any position and direction based on the two-dimensional medical image data;

[0013] Positioning interaction module, used to rotate and obtain any section based on the section image through MPR positioning line interaction;

[0014] The linkage module is used to display the section image in the three-dimensional model with matching position, and to rotate and move interactively in the two-dimensional space to cause the section image to change. In the three-dimensional model window, the intersection of the MPR image is used as the picking point of interest, and the point is moved and transformed to switch different sections to achieve linkage between the two-dimensional medical image and the three-dimensional model.

[0015] A third aspect of the present invention provides a computer-readable storage medium.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the VTK-based two-dimensional medical image and three-dimensional model positioning linkage method as described above.

[0017] A fourth aspect of the present invention provides a computer device.

[0018] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the VTK-based two-dimensional medical image and three-dimensional model positioning linkage method as described above are implemented.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The present invention causes changes in cross-sectional images through interactive rotation and movement in two-dimensional space. In the three-dimensional model window, the intersection of the MPR image is used as a picking point of interest, and the point is moved and transformed to switch different cross-sectional surfaces to perform linkage between the two-dimensional medical image and the three-dimensional model. The two-dimensional image and the three-dimensional model are accurately placed in a unified coordinate space for observation through interactive operations, which can more intuitively reflect the spatial relationship between the two, thereby achieving precise positioning.

[0021] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0023] Figure 1 It is a flow chart of a method for linking two-dimensional medical image and three-dimensional model positioning based on VTK according to an embodiment of the present invention;

[0024] Figure 2 This is the process of using the VTK class to perform pipeline operations in an embodiment of the present invention;

[0025] Figure 3 It is the interactive judgment of movement and rotation according to the embodiment of the present invention;

[0026] Figure 4 This is the rotation matrix implementation process of the embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0029] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0030] Explanation of terms

[0031] VTK: Visualization Toolkit (VTK) is an open source, cross-platform, parallel processing graphics application library for 3D computer graphics, image processing, and visualization.

[0032] Among them, VTKImageReslice, Vtktransform, VTKImageStack and MPR are all available class names.

[0033] The vtkImageReslice class can cut out a planar image from a point in the volume data along different directions; vtkImageReslice is the Swiss Army knife of image geometry filters: it can rotate, flip, scale, resample, deform, and fill image data with reasonable efficiency.

[0034] vtktransform is used for spatial geometric transformations, including translation, rotation, and scaling.

[0035] VtkImageStack is used to manage a stack of images. Multiple layers of imported images can be added to this manager. Each image is assigned a layer number through its property object. It is this layer number that determines the compositing order: images with higher layers are drawn on top of images with lower layers. The image stack has a SetActiveLayer method to control which layer is used for interaction and selection. It is mainly used here to store vtkImageActor corresponding to different slice images.

[0036] vtkImageData is a class used by vtk to identify the topology and geometric relationship of image data. It is usually used in conjunction with pipeline connection SetInputData().

[0037] vtkImageMapToColors is responsible for image color mapping, and vtkLookUpTable is responsible for generating the color lookup table.

[0038] VTK's rendering pipeline consists of two parts: the visualization pipeline and the rendering engine. The visualization pipeline is responsible for reading and generating data, and the rendering engine renders the obtained data into objects in the window.

[0039] The rendering engine includes vtkActor, vtkRender, vtkRenderWindow

[0040] vtkImageActor inherits from vtkActor and is used to display images.

[0041] vtkRenderWindow is responsible for managing the window creation and rendering process. vtkRenderWindow contains vtkRender collection, rendering parameters, etc.

[0042] vtkRender is responsible for managing the rendering process of the scene. The objects that make up the scene, including vtkActor, camera (VTKCamera), and lighting (vtkLight), are integrated into a vtkRender object. There can be multiple vtkRender objects in a vtkRenderwindow.

[0043] Embodiment 1

[0044] like Figure 1 As shown, this embodiment provides a VTK-based two-dimensional medical image and three-dimensional model positioning linkage method, including the following steps:

[0045] Step 1: Acquire two-dimensional medical image data, and acquire a section at any position and direction based on the two-dimensional medical image data;

[0046] Step 2: Based on the section image, rotate and obtain any section through the interaction of MPR positioning lines;

[0047] Step 3: Display the cross-sectional image in the three-dimensional model with matching position;

[0048] Step 4: Rotate and move interactively in two-dimensional space to cause changes in the cross-sectional image. In the three-dimensional model window, use the intersection of the MPR image as the point of interest, move and transform the point to switch between different cross-sectional planes to link the two-dimensional medical image and the three-dimensional model.

[0049] like Figure 2 As shown, as one or more embodiments, in step 1, the step of acquiring a section at any position and direction based on two-dimensional medical image data includes:

[0050] Based on two-dimensional medical imaging data, VTKImageReslice is used to rotate, flip, scale, and resample the image. The specific method of setting VTKImageReslice is: by setting the input setInputData(), the output volume SetOutputDimensionality(2), and the interpolation mode SetInterpolationModeToNearestNeighbor().

[0051] The key to updating and obtaining slices at any position and direction is to set ResliceAxes through SetResliceAxes(), which is a vtkMatrix4x4 matrix, where the first, second, and third columns are the vector coordinates of the new coordinate system in the world coordinate system, and the fourth column represents the coordinate value of the new coordinate origin in the world coordinate system.

[0052] These three normal vectors must satisfy the right-hand rule, that is, X cross Y is Z, Y cross Z is X, and Z cross X is Y.

[0053] As one or more embodiments, in step 2, based on the section image, an arbitrary section is obtained by rotating through the MPR positioning line interaction, specifically including:

[0054] like Figure 3 As shown, step 201: use the center area of ​​the crosshair positioning line as the picking movement area, and any line segment range of the crosshair as the picking rotation area, use the vtkpointpicker mouse to pick up the point of interest, compare the spatial distance from the current point picked to the crosshair positioning line with the threshold, and judge whether it is picked up in the moving / rotating effective area. The specific judgment process is as follows:

[0055] In the rendering window, line 1 and line 2 intersect to form a crosshair. If the distance between the pick position and line 1 is less than the threshold, and the distance between the pick position and line 2 is less than the threshold, the pick position can be moved. If the distance between the pick position and one of the lines is less than the threshold, and the distance between the pick position and the other line is greater than the threshold, it is determined that it can be rotated. If the distance between the pick position and both lines is greater than the threshold, it cannot be moved or rotated.

[0056] like Figure 4 The figure shows the process of rotation matrix, step 202: if the object is picked up in the effective movement / rotation area, a rotation operation is performed to calculate the rotation matrix of the corresponding section.

[0057] The specific steps of performing the rotation operation and calculating the rotation matrix of the corresponding section include:

[0058] (1) Calculation of rotation angle θ: The rotation angle θ is calculated based on the space vector formed by the rotation center point centerPos and the interaction points lastCursorPos and currentCursorPos.

[0059] (2) Rotation matrix calculation: First, move the rotation axis to the origin, use the rotation angle θ to calculate the intermediate rotation matrix R(θ) of the rotation around any space vector in three-dimensional space, and move the rotation axis to its original position;

[0060] The matrix operation expression of the entire process of obtaining the section rotation matrix is: T(C).R(θ).T(-C).

[0061] The right-hand rule is used to calculate the direction of the rotation axis vector according to the vector cross product, and the exact rotation axis is obtained through the cross positioning line focusPos.

[0062] The entire process of obtaining the slice rotation matrix uses the VTK class to implement three operations including:

[0063] Use vtktransform() to translate the rotation axis to the origin of the coordinate axis;

[0064] Use vtktranformRotateWXYZ() to get the rotation matrix;

[0065] Use vtktransform to translate the rotation axis to its original position;

[0066] It is known that the rotation transformation can be calculated by matrix multiplication. Multiply the rotation matrix with the original coordinate axis matrix using vtkMatrix4x4::Multiply4x4() to obtain the new coordinate axis matrix.

[0067] Step 203: Move the cross point (i.e., the focus of the crosshairs) and switch to different sections parallel to the coordinate axis. Here, the new coordinate axis origin of ResliceAxes is calculated through geometric transformation to obtain a new coordinate matrix, which is actually achieved by changing the value of the fourth column of the ResliceAxes matrix.

[0068] As one or more embodiments, in step 3, displaying the section image in the three-dimensional model with matching position specifically includes:

[0069] Set up the slice image actor through the visualization pipeline;

[0070] Use VTKImageStack class objects m_pImageStack2D[3] and m_pImageStack3D[3], m_pImageStack2D[3] stores the slice image actor in the MPR window, where m_pImageStack3D is used to store the slice image actor in the 3D model window, m_Renderer2D[crntViewLabel]->AddActor(m_ImageStack2D[crntViewLabel];

[0071] The renderer VTKRender adds the slice image actor to the rendering pipeline;

[0072] m_pImageStack3D sets the same transformation matrix matrix through SetUserMatrix.

[0073] Based on this, any slices are displayed in the 3D model and the positions are matched.

[0074] As one or more embodiments, in step 4, the rotation and movement interaction in the two-dimensional space triggers changes in the slice matrix and the slice image. The SetUserMatrix() function is used to move and rotate the object m_pImageStack3D, affecting the switching of slices at different positions in the three-dimensional view, thereby realizing the linkage between the two-dimensional image and the three-dimensional model.

[0075] In the 3D model window, the intersection of the MPR image is used as the picking point of interest. By moving and transforming the point, different sections can be switched to achieve 2D and 3D linkage.

[0076] Embodiment 2

[0077] This embodiment provides a VTK-based two-dimensional medical image and three-dimensional model positioning linkage system, including:

[0078] A section image acquisition module is used to acquire two-dimensional medical image data, and acquire section images at any position and direction based on the two-dimensional medical image data;

[0079] Positioning interaction module, used to rotate and obtain any section based on the section image through MPR positioning line interaction;

[0080] The linkage module is used to display the section image in the three-dimensional model with matching position, and to rotate and move interactively in the two-dimensional space to cause the section image to change. In the three-dimensional model window, the intersection of the MPR image is used as the picking point of interest, and the point is moved and transformed to switch different sections to achieve linkage between the two-dimensional medical image and the three-dimensional model.

[0081] Embodiment 3

[0082] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the VTK-based two-dimensional medical image and three-dimensional model positioning linkage method as described above are implemented.

[0083] Embodiment 4

[0084] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the VTK-based two-dimensional medical image and three-dimensional model positioning linkage method as described above are implemented.

[0085] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0086] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0087] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0088] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0089] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A VTK-based two-dimensional medical image and three-dimensional model positioning linkage method, characterized in that: The steps include: Acquire two-dimensional medical imaging data, and acquire cross-sectional images at any position and direction based on the two-dimensional medical imaging data; Based on the section image, any section can be obtained by rotating through the interaction of MPR positioning lines; The cross-sectional image is displayed in the 3D model with matching position. The cross-sectional image changes are triggered by rotating and moving in the 2D space. In the 3D model window, the intersection of the MPR image is used as the point of interest. The point is moved and transformed to switch between different cross-sectional planes to link the 2D medical image and the 3D model. The method of rotating and acquiring any section based on the section image through the interaction of the MPR positioning line specifically includes: The center area of ​​the crosshairs is used as the picking movement area, and any line segment of the crosshairs is used as the picking rotation area. The mouse is used to pick the point of interest, and the spatial distance from the current point to the crosshairs is compared with the threshold. If the point is picked within the effective movement / rotation area, the rotation operation is performed and the rotation matrix of the corresponding section is calculated; the focus of the crosshairs is moved to switch to different sections parallel to the coordinate axis; The comparison of the spatial distance between the current point picked up and the cross positioning line and the threshold specifically includes: In the rendering window, the first line and the second line intersect to form a crosshair. If the distance between the pick position and the first line is less than the threshold, and the distance between the pick position and the second line is less than the threshold, the pick position can be moved; if the distance between the pick position and one of the lines is less than the threshold, and the distance between the pick position and the other line is greater than the threshold, it is determined that it can be rotated; if the distance between the pick position and the two lines is greater than the threshold, it cannot be moved or rotated; The step of displaying the section image in the three-dimensional model with matching position includes: Setting up slice images through the visualization pipeline; Use the VTKImageStack class object to store the slice image in the MPR window; The renderer adds the slice image to the rendering pipeline.

2. The VTK-based two-dimensional medical image and three-dimensional model positioning linkage method according to claim 1, characterized in that: The method of obtaining a section image at any position and direction based on two-dimensional medical imaging data is specifically: using VTKImageReslice to rotate, flip, scale and resample the image. The specific method of setting VTKImageReslice is: by setting the input, output volume and interpolation method.

3. The VTK-based two-dimensional medical image and three-dimensional model positioning linkage method according to claim 1, characterized in that: The picking up of the moving / rotating effective area, performing the rotation operation, and calculating the rotation matrix of the corresponding section include: The rotation angle is calculated based on the space vector formed by the rotation center point and the interaction point; Based on the rotation angle, use the right-hand rule to calculate the direction of the rotation axis vector by cross multiplication of the vectors, and get the exact rotation axis through the crosshair focus; Based on the exact rotation axis, use vtktranformRotateWXYZ to get the rotation matrix.

4. The VTK-based two-dimensional medical image and three-dimensional model positioning linkage method according to claim 1, characterized in that: The key to obtaining slices at any position and direction is to set ResliceAxes, which is a vtkMatrix4x4 matrix, in which the first, second, and third columns are the vector coordinates of the new coordinate system in the world coordinate system, and the fourth column represents the coordinate value of the new coordinate origin in the world coordinate system.

5. The VTK-based 2D medical image and 3D model positioning linkage system is characterized by: include: A section image acquisition module is used to acquire two-dimensional medical image data, and acquire section images at any position and direction based on the two-dimensional medical image data; Positioning interaction module, used to rotate and obtain any section based on the section image through MPR positioning line interaction; The linkage module is used to display the section image in the 3D model with matching position, and to rotate and move interactively in the 2D space to cause the section image to change. In the 3D model window, the intersection of the MPR image is used as the picking point of interest, and the point is moved and transformed to switch different sections to link the 2D medical image and the 3D model; The method of rotating and acquiring any section based on the section image through the interaction of the MPR positioning line specifically includes: The center area of ​​the crosshairs is used as the picking movement area, and any line segment of the crosshairs is used as the picking rotation area. The mouse is used to pick the point of interest, and the spatial distance from the current point to the crosshairs is compared with the threshold. If the point is picked within the effective movement / rotation area, the rotation operation is performed and the rotation matrix of the corresponding section is calculated; the focus of the crosshairs is moved to switch to different sections parallel to the coordinate axis; The comparison of the spatial distance between the current point picked up and the cross positioning line and the threshold specifically includes: In the rendering window, the first line and the second line intersect to form a crosshair. If the distance between the pick position and the first line is less than the threshold, and the distance between the pick position and the second line is less than the threshold, the pick position can be moved; if the distance between the pick position and one of the lines is less than the threshold, and the distance between the pick position and the other line is greater than the threshold, it is determined that it can be rotated; if the distance between the pick position and the two lines is greater than the threshold, it cannot be moved or rotated; The step of displaying the section image in the three-dimensional model with matching position includes: Setting up slice images through the visualization pipeline; Use the VTKImageStack class object to store the slice image in the MPR window; The renderer adds the slice image to the rendering pipeline.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the VTK-based two-dimensional medical image and three-dimensional model positioning linkage method as described in any one of claims 1-4 are implemented.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the VTK-based two-dimensional medical image and three-dimensional model positioning linkage method as described in any one of claims 1-4 are implemented.

Images