Visualization method and visualization system of medical image

Abstract

The medical image visualization method comprises the following steps: tracking coordinates of a left pupil and a right pupil of a viewer in a predefined space coordinate system in real time; the three-dimensional array is rendered in real time according to the coordinates of the left pupil and the coordinates of the right pupil, a rendered image corresponding to the left eye and a rendered image corresponding to the right eye are obtained, and the three-dimensional array analyzes medical image data of the anatomical structure and comprises density information or gray information of the anatomical structure at different spatial positions; and presenting the rendered image corresponding to the left eye and the rendered image corresponding to the right eye, so that the rendered image corresponding to the left eye enters the left eye of the viewer, and the rendered image corresponding to the right eye enters the right eye of the viewer. The visualization method enables a viewer to visually see a stereoscopic rendered image of an anatomical structure. The invention further provides a medical image visualization system.

Description

Technical Field

[0001] This invention relates to the field of medical image visualization, and more particularly to a method and system for visualizing medical images. Background Technology

[0002] With the continuous advancement of medical technology, medical imaging has become an important supporting tool in clinical diagnosis and treatment. Traditional medical imaging techniques such as CT and MRI provide two-dimensional image data. However, in traditional doctor-patient communication, verbal descriptions of this two-dimensional image data are abstract and difficult to understand, struggle to convey spatial hierarchy, fail to intuitively demonstrate the patient's condition, and cannot intuitively reconstruct individual anatomical details.

[0003] These limitations make it difficult for patients to accurately construct a three-dimensional understanding of their own lesions and surgical plans, thus failing to provide intuitive support for treatment decisions. Summary of the Invention

[0004] The purpose of this invention is to provide a method for visualizing medical images, which can intuitively present three-dimensional rendered images of anatomical structures.

[0005] Another object of the present invention is to provide a medical image visualization system that can intuitively present three-dimensional rendered images of anatomical structures.

[0006] The medical image visualization method provided by this invention includes: real-time tracking of the coordinates of the viewer's left and right pupils in a predefined spatial coordinate system; real-time rendering of a three-dimensional array based on the coordinates of the left and right pupils respectively, to obtain a rendered image corresponding to the left eye and a rendered image corresponding to the right eye, wherein the three-dimensional array is parsed from medical image data of anatomical structures and includes density information or grayscale information of anatomical structures at different spatial locations; and presenting the rendered image corresponding to the left eye and the rendered image corresponding to the right eye, so that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye.

[0007] By obtaining the coordinates of the left and right pupils, a three-dimensional array is rendered in real time based on the coordinates of the left and right pupils, and the rendered images of the corresponding left and right eyes are presented. The rendered images of the corresponding left and right eyes are then entered into the viewer's left and right eyes respectively, allowing the viewer to intuitively see a three-dimensional rendered image of the anatomical structure.

[0008] In another illustrative implementation of the medical image visualization method, rendering includes: applying a lighting model to a three-dimensional array to perform ray stepping and sampling, density interpolation and classification, light scattering and absorption, sampling and integration along the lighting direction, tone mapping and exposure adjustment, and local contrast enhancement and noise reduction to obtain a rendered image. This medical image visualization method can produce high-quality rendered images with a sense of depth.

[0009] In another illustrative embodiment of the medical image visualization method, the steps are as follows: Presenting a rendered image corresponding to the left eye and a rendered image corresponding to the right eye, such that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye. Specifically, this involves adjusting the refraction angle of light in real time based on the coordinates of the viewer's left and right pupils, so that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye. This medical image visualization method can adjust the refraction angle of light in real time according to the coordinates of the viewer's left and right pupils, preventing ghosting.

[0010] In another illustrative embodiment of the medical image visualization method, the method further includes adjusting at least one of the geometric state, display state, and structural state of the rendered image based on a received gesture signal. This medical image visualization method enables interaction between the rendered image and the viewer.

[0011] In another illustrative embodiment of the medical image visualization method, the medical image data is CT image data and / or MRI image data.

[0012] This invention also provides a medical image visualization system, including a first display module and a calculation module. The first display module is configured to track the coordinates of the viewer's left and right pupils in a predefined spatial coordinate system in real time. The calculation module is configured to render a three-dimensional array in real time based on the coordinates of the left and right pupils, respectively, to obtain a rendered image for the left eye and a rendered image for the right eye. The three-dimensional array is parsed from medical image data of anatomical structures and includes density or grayscale information of the anatomical structures at different spatial locations. The first display module is further configured to present the rendered image for the left eye and the rendered image for the right eye, so that the rendered image for the left eye enters the viewer's left eye and the rendered image for the right eye enters the viewer's right eye.

[0013] This medical image visualization system can intuitively present three-dimensional rendered images of anatomical structures.

[0014] In another illustrative embodiment of the medical image visualization system, the first display module is further configured to adjust the refraction angle of light in real time according to the coordinates of the viewer's left and right pupils, so that the rendered image corresponding to the left eye enters the viewer's left eye, and the rendered image corresponding to the right eye enters the viewer's right eye. This medical image visualization system can adjust the refraction angle of light in real time according to the coordinates of the viewer's left and right pupils to prevent ghosting.

[0015] In another illustrative embodiment of the medical image visualization system, the first display module is a spatial reality display screen. This facilitates the presentation of high-quality, stereoscopic rendered images, while simultaneously tracking the coordinates of the viewer's left and right pupils and adjusting the optical refraction angles to prevent ghosting.

[0016] In another illustrative embodiment of the medical image visualization system, an input module is also included. The input module is a motion controller and is configured to generate gesture signals based on the viewer's movements. The calculation module is further configured to adjust at least one of the geometric state, display state, and structural state of the rendered image based on the received gesture signals. This facilitates interaction between the viewer and the rendered image.

[0017] In another illustrative embodiment of the medical image visualization system, the medical image data is CT image data and / or MRI image data. Attached Figure Description

[0018] The following figures are for illustrative purposes only and do not limit the scope of the invention.

[0019] Figure 1 A flowchart illustrating one implementation of a method for visualizing medical images.

[0020] Figure 2 For explanation Figure 1 The visualization method shown presents a rendered image of the medical image.

[0021] Figure 3 A schematic structural block diagram of one illustrative implementation of a medical image visualization system.

[0022] Label Explanation

[0023] 10 First Display Module

[0024] 20 Calculation Module

[0025] 30 Second display module

[0026] 40 Input Module Detailed Implementation

[0027] To provide a clearer understanding of the technical features, objectives, and effects of the invention, specific embodiments of the invention are now described with reference to the accompanying drawings, in which the same reference numerals denote the same parts.

[0028] In this document, “illustrative” means “serving as an example, illustration or description”, and any illustration or implementation described herein as “illustrative” should not be construed as a more preferred or advantageous technical solution.

[0029] In this document, terms such as "first," "second," etc., do not indicate degree of importance or order, but are used only to distinguish them for the purpose of description. For the sake of brevity, each figure only schematically shows the parts relevant to the invention and does not represent their actual structure as a product. Nouns and pronouns referring to people in this patent application are not limited to specific genders.

[0030] Figure 1 A flowchart illustrating one implementation of a method for visualizing medical images. See also... Figure 1 In an illustrative embodiment, the method for visualizing medical images includes steps S10 to S30.

[0031] S10: Real-time tracking of the viewer's left and right pupils in a predefined spatial coordinate system. The viewer is, for example, located in front of the display screen, and the spatial coordinate system is, for example, defined as a three-dimensional coordinate system with the geometric center of the display screen as its origin. The display screen is, for example, a spatial reality display screen, which includes, for example, several infrared sensors to track the viewer's left and right pupils in the predefined spatial coordinate system in real time. However, this is not limited to this; devices implementing virtual reality (VR) and augmented reality (AR) technologies can also track the viewer's left and right pupils in a predefined spatial coordinate system in real time. Of course, the spatial coordinate system should be predefined according to the device used.

[0032] S20: The 3D array is rendered in real time based on the coordinates of the left and right pupils, respectively, to obtain the rendered images of the left and right eyes. The 3D array is parsed from medical image data of anatomical structures and includes density or grayscale information of the anatomical structures at different spatial locations.

[0033] Medical image data includes, for example, standard DICOM format CT and / or MRI images from PACS systems or local file systems. The parsing methods are standard in the field of medical image processing and will not be elaborated upon here. Density information reflects the absorption of X-rays by the tissue. Grayscale information is the digital representation of density information. These three-dimensional arrays form the basis of the digital three-dimensional anatomical model.

[0034] Real-time rendering processes can employ Monte Carlo ray tracing models, for example. However, they are not limited to this; real-time rendering processes can also utilize existing rendering techniques such as surface rendering and volumetric rendering, with volumetric rendering including ray casting.

[0035] S30: Presents a rendered image corresponding to the left eye and a rendered image corresponding to the right eye, so that the rendered image corresponding to the left eye enters the viewer's left eye, and the rendered image corresponding to the right eye enters the viewer's right eye. The presented rendered images are as follows: Figure 2 As shown.

[0036] A viewer, for example, is positioned in front of a display screen configured to present a rendered image corresponding to the left eye and a rendered image corresponding to the right eye. The rendered image corresponding to the left eye enters the viewer's left eye, and the rendered image corresponding to the right eye enters the viewer's right eye. After the left and right eyes receive independent rendered images with parallax, the visual cortex of the brain automatically calculates the difference between the two rendered images (e.g., the left eye sees a different image than the right eye). Figure 2 The left side of the heart is shown, and the right eye sees something like this. Figure 2 (showing the right side of the heart), thus allowing viewers to see a stereoscopic rendered image in naked-eye 3D.

[0037] Similarly, based on the principle of parallax, viewers wearing devices that implement Virtual Reality (VR) and Augmented Reality (AR) technologies can also see rendered images with a three-dimensional effect.

[0038] By obtaining the coordinates of the left and right pupils, a three-dimensional array is rendered in real time, and the rendered images of the corresponding left and right eyes are presented. This allows the rendered images of the left and right eyes to enter the viewer's left and right eyes respectively, enabling the viewer to intuitively see a three-dimensional rendered image of the anatomical structure.

[0039] Specifically, in the illustrative embodiment, step S20 includes, for example, applying a lighting model to the three-dimensional array to perform ray stepping and sampling, density interpolation and classification, light scattering and absorption, sampling and integration along the lighting direction, tone mapping and exposure adjustment, and local contrast enhancement and noise reduction to obtain a rendered image. The lighting model is, for example, a Monte Carlo ray tracing model, which is based on the Monte Carlo ray tracing algorithm.

[0040] Monte Carlo ray tracing is a probabilistic and statistical rendering algorithm widely used in computer graphics, especially in handling complex lighting, indirect lighting, and global illumination scenarios. It estimates the path of light through random sampling, thereby simulating real-world lighting effects.

[0041] The rendering equation for the Monte Carlo ray tracing model is shown in the following equation:

[0042] in: Radiance, representing the radiance at a given location. ,direction The light intensity on; Indicates a position in space. It is the location where scattering occurs. It is the observation or launch position; It is an integral variable The upper limit represents the boundary position of the medium in that dimension; Indicates the direction of light propagation. It is the direction of the incident light at the scattering point. It is the direction of emission after scattering; Optical Depth, representing the distance from... arrive The cumulative light attenuation along the path is expressed by the formula: ,in, It is the extinction coefficient, which is the sum of the absorption coefficient and the scattering coefficient; Scattering probability, representing the location The scattering ability of the medium; The phase function describes the direction of light. Scattered to Probability distribution of direction; The incident radiance (which can be the "input" light from a light source, ambient light, or other scattering processes); For full-space solid angle, Its solid angle covers all possible directions.

[0043] The rendering equations described above can be numerically approximated using Monte Carlo integration. This method of numerical approximation using Monte Carlo integration is existing technology and will not be elaborated upon here. It is worth noting that the rendering equations for the Monte Carlo ray tracing algorithm can also be other forms of existing Monte Carlo rendering equations.

[0044] Ray stepping involves a ray starting from a point and "progressing" along its direction, calculating the distance between the current position and the scene at each step, until it approaches an intersection point or exceeds the maximum number of steps. Ray stepping can efficiently handle complex geometry, but the step size needs to be carefully considered based on the rendering object; too large a step size may miss sampling points, while too small a step size is time-consuming. Sampling aims to approximate the integrals in the rendering equations using random samples. After each intersection point, the reflection, refraction, and diffuse reflection directions are selected probabilistically. Ray stepping and sampling constitute the core flow of the Monte Carlo ray tracing algorithm: ray stepping finds the intersection point, sampling determines the ray direction, tracing the sampled rays, and accumulating contributions.

[0045] Density interpolation converts discrete medium density samples into a continuous density field, solving the density continuity problem in medium ray tracing and ensuring the accuracy of ray propagation calculations. Classification categorizes scene elements (mediums, objects, and rays) into different classes and applies targeted processing strategies (such as material parameters) to improve the efficiency and realism of Monte Carlo ray tracing algorithms.

[0046] Light scattering and absorption are the core physical processes that simulate the energy interaction of light as it propagates through a medium, and they directly affect the physical accuracy of the rendering results.

[0047] Sampling along the lighting direction refers to emitting sampled rays directly from the shading point to the light source region. Integration is the Monte Carlo solution of the rendering equation. Sampling and integration along the lighting direction are used to solve the sampling efficiency problem of small light sources, avoid noise, accelerate convergence, support complex lighting effects, and ensure energy conservation.

[0048] Tone mapping and exposure adjustment convert the physically accurate radiance values ​​generated by the algorithm into visualization results adapted to ordinary display devices, while preserving the realism and detail of the scene.

[0049] The goal of local contrast enhancement is to improve the contrast of local details (such as dark textures and object edges) without changing the overall brightness distribution of the image, making the image clearer and more in line with human visual perception. Local contrast enhancement is usually combined with adaptive adjustments during image post-processing or rendering, such as adaptive histogram equalization. Noise reduction refers to removing random noise, such as graininess, from Monte Carlo rendering results while preserving image details such as edges and textures. Noise in Monte Carlo rendering originates from finite sampling; sampling errors cause pixel value fluctuations, forming noise. Noise reduction methods can be divided into traditional methods and deep learning-based methods, such as spatial filtering and Gaussian filtering.

[0050] The Monte Carlo ray tracing model takes a 3D array as input and outputs rendered images corresponding to the left and right eyes, respectively. Each pixel in the rendered image records the brightness and color information accumulated by light rays in 3D space. Compared to surface rendering techniques and traditional volumetric rendering techniques such as ray casting, the Monte Carlo ray tracing model can generate higher quality and more realistic rendered images.

[0051] Specifically, step S30 presents the rendered image, for example, using a glasses-free 3D method. In an illustrative embodiment, step S30 specifically includes, for example: The viewer is positioned in front of a display screen, such as a spatial reality display screen, which is configured to receive an image in which the rendered image corresponding to the left eye and the rendered image corresponding to the right eye are interwoven (hereinafter referred to as a subpixel interwoven image, where even-numbered columns of subpixels in the subpixel interwoven image are, for example, rendered image data corresponding to the left eye, and odd-numbered columns of subpixels are, for example, rendered image data corresponding to the right eye). After receiving the subpixel interleaved image, the even-numbered columns of subpixels on the display screen present, for example, the rendered image corresponding to the left eye, and the odd-numbered columns of subpixels present, for example, the rendered image corresponding to the right eye. The display screen includes, for example, several semi-cylindrical microlenses covering the surface of the subpixels on the display screen. Each microlens includes, for example, liquid crystal molecules and has a liquid crystal control area. When no voltage is applied, the multiple liquid crystal molecules are arranged in parallel, and the curvature of the microlens is the initial curvature. The display screen calculates the curvature compensation value required for each microlens based on the coordinates of the left and right pupils acquired in real time and converts it into an analog voltage signal. Under the action of the applied voltage, each liquid crystal molecule tilts in a direction perpendicular to the arrangement direction, causing a change in the local refractive index, thereby changing the curvature of the microlens and adjusting the refraction angle of the light, so that the rendered image corresponding to the left eye enters the left eye of the moved viewer, and the rendered image corresponding to the right eye enters the right eye of the moved viewer.

[0052] This medical image visualization method can adjust the angle of light refraction in real time according to the viewer's pupil position to prevent ghosting.

[0053] In the illustrative embodiments, such as Figure 1 As shown, the medical image visualization method further includes step S40: adjusting at least one of the geometric state, display state, and structural state of the rendered image based on the received gesture signal.

[0054] Viewers control at least one of the geometric, display, and structural states of the rendered image through defined gestures such as waving, grabbing, and tapping. Geometric states include size, orientation, and position, which correspondingly cause the rendered image to zoom in, zoom out, rotate, or move. Display states include making an organ visible or hidden. Structural states include displaying the image as a whole or displaying individual organs separately.

[0055] It should be noted that after parsing medical image data to obtain a 3D array, the 3D array can be processed to achieve organ segmentation. Methods for processing the 3D array include, for example, combining existing techniques such as threshold segmentation and gradient edge detection, which will not be elaborated upon here. Based on the organ-segmented 3D array, the resulting rendered image has the function of making a particular organ visible or hidden, as well as displaying each organ separately. Viewers can use predefined gestures to segment the organs, allowing them to focus on the organs of interest.

[0056] like Figure 3 As shown, the present invention also provides a medical image visualization system, which includes a first display module 10 and a computing module 20. The first display module 10 can be the display screen described above, and its operation process is as described in steps S10 and S30 above, which will not be repeated here.

[0057] The first display module 10 is, for example, a spatial reality display screen, which is configured to track the coordinates of the viewer's left and right pupils in a predefined spatial coordinate system in real time.

[0058] The calculation module 20 is configured to receive the coordinates of the left pupil and the right pupil tracked by the first display module 10, and to render the three-dimensional array in real time based on the coordinates of the left and right pupils respectively, to obtain the rendered images of the corresponding left and right eyes. The three-dimensional array is parsed from medical image data of anatomical structures and includes density or grayscale information of the anatomical structures at different spatial locations. The calculation module 20 is also configured to transmit the rendered images of the corresponding left and right eyes to the first display module 10 in a subpixel interleaved image manner.

[0059] The first display module 10 is also configured to present a rendered image corresponding to the left eye and a rendered image corresponding to the right eye, so that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye.

[0060] This medical image visualization system can intuitively present three-dimensional rendered images of anatomical structures.

[0061] Specifically, in the illustrative embodiment, the first display module 10 is further configured to adjust the refraction angle of light in real time according to the coordinates of the viewer's left pupil and right pupil, so that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye.

[0062] In the illustrative embodiment, the calculation module 20 is further configured to apply a lighting model to perform ray stepping and sampling, density interpolation and classification, light scattering and absorption, sampling and integration along the lighting direction, tone mapping and exposure adjustment, and local contrast enhancement and noise reduction on the three-dimensional array to obtain a rendered image. The rendering process of the calculation module 20 is as described in step S20 above, and will not be repeated here. This medical image visualization system can obtain high-fidelity rendered images with a three-dimensional feel.

[0063] The medical image visualization system also includes a second display module 30, such as a standard computer screen. The second display module 30 is signal-connected to the computing module 20 and is configured to receive and display medical image data and render images. This facilitates the implementation of medical image visualization methods. Figure 3 As shown, both the computing module 20 and the second display module 30 are integrated into the computer (e.g., Figure 3 The module shown is within the dashed box.

[0064] The medical image visualization system also includes an input module 40. Input module 40 is a motion controller configured to generate gesture signals based on the viewer's movements. The gesture signals have been described previously and will not be repeated here. Input module 40 may be, for example, a Leap Motion motion controller. The Leap Motion motion controller is a motion controller designed for personal computers, characterized by its small size, low price, and high gesture recognition accuracy. Using the Leap Motion motion controller's motion-sensing interaction technology to replace a mouse can effectively help viewers perform intuitive interactive operations on the rendered images, providing a superior user experience.

[0065] The calculation module 20 is also configured to adjust at least one of the geometric state, display state, and structural state of the rendered image based on the received gesture signal. The geometric state, display state, and structural state have been described previously and will not be repeated here. The adjusted rendered image can be displayed on the first display module 10, thereby facilitating interaction between the viewer and the rendered image.

[0066] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0067] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention and are not intended to limit the scope of protection of the present invention. All equivalent implementation schemes or modifications made without departing from the spirit of the present invention, such as combinations, divisions or repetitions of features, should be included within the scope of protection of the present invention.

Claims

1. A method for visualizing medical images, characterized in that, include: Real-time tracking of the viewer's left and right pupils in a predefined spatial coordinate system; The three-dimensional array is rendered in real time based on the coordinates of the left pupil and the right pupil, respectively, to obtain the rendered image of the left eye and the rendered image of the right eye. The three-dimensional array is parsed from medical image data of anatomical structures and includes density information or grayscale information of the anatomical structures at different spatial locations. as well as The rendered image corresponding to the left eye and the rendered image corresponding to the right eye are presented, so that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye.

2. The method for visualizing medical images as described in claim 1, characterized in that, The rendering process includes: applying a lighting model to the three-dimensional array to perform ray stepping and sampling, density interpolation and classification, light scattering and absorption, sampling and integration along the lighting direction, tone mapping and exposure adjustment, and local contrast enhancement and noise reduction, to obtain the rendered image.

3. The method for visualizing medical images as described in claim 1, characterized in that, Steps: Present the rendered image corresponding to the left eye and the rendered image corresponding to the right eye, so that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye. Specifically, adjust the refraction angle of light in real time according to the coordinates of the viewer's left pupil and right pupil, so that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye.

4. The method for visualizing medical images as described in claim 1, characterized in that, Also includes: Based on the received gesture signal, at least one of the geometric state, display state, and structural state of the rendered image is adjusted.

5. The method for visualizing medical images as described in claim 1, characterized in that, The medical imaging data refers to CT imaging data and / or MRI imaging data.

6. A medical image visualization system, characterized in that, include: The first display module (10) is configured to track the coordinates of the viewer’s left and right pupils in a predefined spatial coordinate system in real time. as well as The calculation module (20) is configured to render a three-dimensional array in real time according to the coordinates of the left pupil and the right pupil, respectively, to obtain a rendered image of the corresponding left eye and a rendered image of the corresponding right eye. The three-dimensional array is parsed from medical image data of anatomical structures and includes density information or grayscale information of the anatomical structures at different spatial locations. The first display module (10) is also configured to present the rendered image of the corresponding left eye and the rendered image of the corresponding right eye, so that the rendered image of the corresponding left eye enters the viewer's left eye and the rendered image of the corresponding right eye enters the viewer's right eye.

7. The medical image visualization system as described in claim 6, characterized in that, The first display module (10) is also configured to adjust the refraction angle of light in real time according to the coordinates of the viewer's left pupil and right pupil, so that the rendered image corresponding to the left eye enters the viewer's left eye and the rendered image corresponding to the right eye enters the viewer's right eye.

8. The medical image visualization system as described in claim 7, characterized in that, The first display module (10) is a spatial reality display screen.

9. The medical image visualization system as described in claim 6, characterized in that, It also includes an input module (40), which is a motion controller and is configured to generate gesture signals based on the viewer's actions. The calculation module (20) is also configured to adjust at least one of the geometric state, display state, and structural state of the rendered image based on the received gesture signals.

10. The medical image visualization system as described in claim 6, characterized in that, The medical imaging data refers to CT imaging data and / or MRI imaging data.

Images