A method for rapid 3D visualization and annotation of medical images based on virtual reality

Abstract

This virtual reality-based system for rapid 3D visualization and annotation of medical images uses a 2D medical image set as input, converts the 2D images into 3D volume data, and analyzes the scalar values ​​and gradients of the volume data. The volume data is then volume-rendered and rendered with a lighting model, and the resulting visualization is displayed on the screen of the virtual reality device. During operation, the system uses geodesic distance to reduce occlusion of irrelevant volume data around the viewpoint in real time. Furthermore, through the interactive capabilities of the virtual reality device and system, users can freely explore the virtual scene without spatial constraints and annotate the medical data as needed. This system allows users to observe and annotate volume data in a virtual scene in an intuitive and natural interactive manner. It also supports users stepping inside the volume data to observe complete internal structural information without occlusion or interference from irrelevant voxels.

Description

Technical Field

[0001] The present invention belongs to the fields of virtual reality and medical image visualization, and particularly relates to a method for rapid three-dimensional visualization and annotation of medical images based on virtual reality. Background Art

[0002] Medical image visualization is a crucial tool for conveying anatomical information. Traditional two-dimensional medical images lack three-dimensional information, requiring viewers to visualize the three-dimensional structures represented by the two-dimensional images. Clinical applications involving medical imaging, such as surgical path planning and medical image segmentation, require layer-by-layer manipulation of two-dimensional images, leading to the risk of task failure or patient harm. To address this, methods have emerged for achieving three-dimensional visualization of two-dimensional medical image sets through volume rendering. This method converts stacked two-dimensional medical image sets into three-dimensional scalar volume data input, and uses volume rendering to directly present the anatomical structures in three dimensions on the screen.

[0003] Virtual reality technology provides users with an immersive 3D virtual world, allowing them to focus more closely on the task at hand. Head-mounted VR devices track the user's head movements in real time, updating the viewpoint within the VR scene. This allows users to observe and explore 3D visualizations naturally and quickly. Furthermore, compared to traditional 2D interactive operations on a screen, VR technology offers a 3D interactive mode, making user interaction with volume data more natural and intuitive.

[0004] In medical applications, users often need to observe structures located within an object. This can cause occlusions in the field of view caused by unrelated voxels between the target area and the user's viewpoint. This occlusion problem is exacerbated in volume rendering scenarios in virtual reality. When a user wearing a head-mounted display walks into the volume data, occlusions can surround the viewpoint, causing the user to lose their sense of orientation within the volume data, thus hindering clinical tasks. Current solutions include the following:

[0005] Application date: 2021 / 4 / 30, application number: CN202110482041.9, patent name: A minimally invasive surgical simulation method and system based on virtual reality, the minimally invasive surgical simulation system based on virtual reality includes: a data preprocessing module, a data import module, a central control module, a three-dimensional image reconstruction module, a three-dimensional stereo image generation module, a stereo image processing module, a surgical simulation module, a warning reminder module, an effect evaluation module, a data storage module, and an update display module. The present invention utilizes the virtual environment provided by the VR device to allow doctors to observe the three-dimensional model of the surgical site in an all-round manner in the virtual space, so that doctors can have a full and sufficient understanding of the patient's condition; the simulated surgical operation platform performed by the VR device in the virtual space facilitates doctors to simulate before surgery to achieve more specific preoperative planning; the three-dimensional medical imaging model presented in virtual reality is more realistic, realizing the interaction between doctors and three-dimensional imaging data.

[0006] However, the virtual reality system described in this paper does not allow users to freely observe and explore the interior of the volume data within the virtual scene. This is because the user's viewpoint typically maintains a certain distance from the target area to ensure a good view of the target area. When the viewpoint enters the volume data, irrelevant volume data and noise data surrounding the viewpoint will block the viewpoint, making it difficult for the user to clearly see the target area. The system described in this paper uses an occlusion reduction method based on geodesic distance to ensure a clear view of the target area. Surface enhancement methods are also used to ensure that the volume data structure information is intact while reducing occlusion. Summary of the Invention

[0007] In order to solve the problems existing in the prior art, the present invention proposes a method for rapid three-dimensional visualization and annotation of medical images based on virtual reality. After the medical image set is input into the system, the system can convert the two-dimensional image set into three-dimensional scalar volume data. And the data is visualized in three dimensions according to the set transfer function and volume rendering based on the ray casting method. With the help of virtual reality technology, users can freely explore the target volume data in the virtual scene. When it is necessary to observe the internal structure, the user can enter the volume data and directly observe the target area. The adaptive occlusion reduction method will reduce the occlusion around the viewpoint and adaptively retain the target area structure. In addition, to assist in the implementation of medical tasks, the system has a series of interactive functions including text annotations and image labeling.

[0008] The present invention provides a method for rapid three-dimensional visualization and annotation of medical images based on virtual reality, which is characterized in that the method comprises the following steps:

[0009] S1, a medical image data conversion step, after obtaining a two-dimensional image data set, extracting scalar values ​​of image pixels, and converting the pixels in the image data set into three-dimensional scalar volume data according to the image coordinates of the pixels and the order of the two-dimensional images;

[0010] The image data conversion process is as follows:

[0011] First, according to the medical image data format, relevant information of the medical image is extracted, including: pixel data, pixel value data type, pixel size, image size, and scaling information; according to the medical image information, the pixel data is converted into a three-dimensional array storing pixel scalar values; then, according to the medical image size information, the three-dimensional array is rendered into a single-channel three-dimensional texture that conforms to the image ratio, wherein the single-channel pixel value of the texture is a linearly scaled image scalar;

[0012] S2, a volume data analysis step, inputting the three-dimensional scalar volume data obtained in step S1, calculating the three-dimensional scalar value gradient data of the three-dimensional scalar volume data, the scalar value range in the volume data, the scalar value distribution, and the scalar value gradient distribution; and then obtaining the scalar value distribution peak value, the scalar value gradient distribution peak value, the scalar value distribution mutation position, and the scalar value gradient distribution mutation position;

[0013] The three-dimensional scalar volume data gradient calculation is replaced by the numerical difference method. The scalar value of this point is f(x i ,y i , z i ), whose gradient The calculation is in the form of scalar value difference of adjacent points, and its gradient The calculation is as follows:

[0014]

[0015] S3, a data volume rendering step, inputting the volume data from step S1 and the scalar value gradient data and volume data analysis results from step S2; performing three-dimensional visualization rendering on the volume data using a ray casting volume rendering method according to a set transfer function; and performing additional Blinn-Phong rendering on portions of the volume data where the gradient is higher than a threshold value according to the scalar value gradient data;

[0016] Step S3 includes the following sub-steps:

[0017] S301, a transfer function setting sub-step, in which a scalar value-opacity transfer function and a scalar value-color transfer function for volume rendering are set, wherein the two transfer functions respectively define the opacity value and color value corresponding to the scalar value of a voxel in the volume data;

[0018] S302, a transfer function rendering sub-step, in which the opacity transfer function and the color transfer function are rendered on two two-dimensional textures respectively; the opacity transfer function is a single-channel texture, and the color transfer function is an RGB three-channel texture;

[0019] S303, a ray casting volume rendering sub-step, in which the 3D texture of the volume data obtained in step S1 is sampled using a ray casting method, and the color and opacity values ​​corresponding to the sampled voxels are determined based on the transfer function texture obtained in sub-step S302; the color and opacity values ​​are then integrated and blended to achieve 3D visualization;

[0020] S304, Blinn-Phong rendering sub-step: After sampling voxels and determining corresponding color values ​​in sub-step S303, this sub-step performs Blinn-Phong rendering on sampling points whose gradient values ​​are greater than a set threshold, and the obtained new color values ​​replace the original color values ​​of the sampling points;

[0021] S4, a real-time adaptive occlusion reduction step, wherein the geodesic distance from each voxel of the 3D scalar volume data obtained in step S1 to the viewpoint is calculated frame by frame in real time based on the camera position input by the head-mounted virtual reality device; an occlusion reduction range is defined based on the 3D geodesic distance field, and a 3D voxel opacity reduction field is calculated; occlusion reduction processing is performed on the volume data based on the opacity reduction field; surface opacity is further enhanced; and then the volume rendering result is updated;

[0022] Step S4 includes the following sub-steps:

[0023] S401, a geodesic distance field generation sub-step, in which the geodesic distance from each voxel in the volume data to the viewpoint is calculated in real time frame by frame, and a three-dimensional geodesic distance field is formed;

[0024] The geodesic distance between the voxel and the viewpoint in sub-step S401 is calculated as follows:

[0025]

[0026] in, is the desired voxel position, is the viewpoint position, Γ represents and A path between represent and The set of all paths between, l(Γ) is the length of the path, is the gradient of the infinitesimal element s along the path, and γ is the set gradient weight coefficient. The gradient weight coefficient γ is manually set and initialized automatically using the gradient weight and spatial distance weights. The geodesic distance field is numerically calculated using the two-scan method. The sampling resolution of the geodesic distance field is 1 / 8 of the volume data resolution, and the calculated geodesic distance field is upsampled in the GPU.

[0027] S402, an occlusion reduction coefficient field generation sub-step, in which an occlusion reduction range is defined based on the three-dimensional geodesic distance field, and the occlusion reduction coefficients of the voxels within the range are calculated; then, an occlusion reduction process is performed on the voxels within the range;

[0028] The occlusion reduction range in sub-step S402 is determined by the manually set threshold d T Determine that the voxels whose geodesic distance d is less than the threshold are included in the occlusion attenuation range; the occlusion attenuation coefficient The calculation is as follows:

[0029]

[0030] Occlusion reduction is achieved by reducing the opacity value of the voxels within the range, which is calculated as follows:

[0031]

[0032] in is the opacity value after processing;

[0033] S403, a surface enhancement sub-step, in which opacity values ​​of voxels in the volume data whose gradients are higher than a threshold are enhanced;

[0034] In sub-step S403, the opacity value of voxels with gradients above a threshold is increased. The threshold is the same as the gradient threshold in sub-step S304. The opacity enhancement is calculated as follows:

[0035]

[0036] in, is the opacity value after enhancement; a is the enhancement coefficient, which is less than 1 and can be set manually;

[0037] S5, a virtual reality interaction step, implements corresponding interactive functions according to the interactive instructions input by the virtual reality device; including: occlusion reduction adjustment function, viewpoint movement and camera adjustment function, and medical volume data annotation function; after the interactive function adjusts the volume data and rendering, the updated rendered image is displayed on the virtual reality device screen.

[0038] As a further improvement of the present invention, the medical imaging data supported in step S1 include DICOM format, RAW format, and NiFTI format.

[0039] As a further improvement of the present invention, the transfer functions in sub-step S301 are piecewise linear functions that map scalar values ​​to opacity values ​​and color values ​​respectively; the transfer function is set by editing the control points of the piecewise linear function and then performing linear interpolation on the function within the control points.

[0040] As a further improvement of the present invention, sub-step S303 adopts a ray casting sampling method to perform fixed-step sampling along a sampling line in a back-to-front order; the number of samples is fixed, and the sampling step is determined by the longest straight line distance in the volume data; the sampling line is emitted from the viewpoint to each pixel on the screen;

[0041] The color integration method used in the sub-step ray casting method integrates the opacity according to Lambert's law and performs the following iterative calculations on opacity and color respectively:

[0042]

[0043] in, For sampling to The opacity integral result is, for The opacity value of the point; For sampling to The color integration result is for The color value of the point.

[0044] As a further improvement of the present invention, the ray casting method in sub-step S303 adopts an acceleration method of empty sampling skipping and early termination of sampling; the empty sampling skipping acceleration method skips voxels with opacity values ​​less than 0.05; and the early termination of sampling stops sampling after the opacity integral value exceeds 1.

[0045] As a further improvement of the present invention, when the gradient threshold in sub-step S304 is not actively set, the gradient obtained in step S2 is located at the position where the gradient distribution mutation occurs and exceeds the 75% quantile, or the 75% quantile position is directly used.

[0046] As a further improvement of the present invention, the occlusion reduction function of step S5 includes a reduction range expansion function and a gradient weight adjustment function; the two functions respectively adjust the geodesic distance field threshold d in step S4 through the joystick of the virtual reality hand controller device. T It is achieved by adjusting the gradient weight coefficient γ; the joystick and parameter growth rate are linearly related.

[0047] As a further improvement of the present invention, the viewpoint movement and camera adjustment functions of step S5 include a viewpoint smooth movement function, a viewpoint zooming function, and a viewpoint teleporting function; the viewpoint smooth movement function is used by using a trigger button of a virtual reality hand controller device; pressing the trigger button and dragging the hand controller can manifest the effect of dragging the viewpoint to a desired direction in the virtual scene; the viewpoint zooming function is used by using a joystick of the virtual reality hand controller device; pushing the joystick controls the size ratio of the viewpoint relative to the volume data; the viewpoint teleporting function is used by using a button of the virtual reality hand controller device; pointing the cursor of the hand controller in the virtual scene to the target position, and pressing a button to reset the viewpoint position to the cursor position.

[0048] As a further improvement of the present invention, the medical volume data annotation function of step S5 includes a line drawing function and an area marking function; the two functions are respectively implemented by the buttons of the left and right hand controllers; when the user presses the button of the left hand controller, a line will be drawn at the position where the cursor passes in the virtual scene; when the user presses the button of the right hand controller, a certain range around the position where the cursor passes in the virtual scene will be marked as a highlighted area; the line thickness, line color, highlight area range, and highlight area color are manually set.

[0049] Compared with the prior art, the beneficial technical effects brought about by the present invention are as follows:

[0050] 1. This method for rapid 3D visualization and annotation of medical images based on virtual reality leverages the high immersion and 3D interactivity of virtual reality technology, combined with volume rendering methods, to rapidly achieve 3D visualization of 2D medical images. This allows users to explore and interact with volume data in a natural and intuitive way. Users can directly approach a target area of ​​interest and observe it from any perspective using this system.

[0051] 2. The virtual reality-based rapid 3D medical image visualization and annotation method of this invention leverages the geodesic distance field's sensitivity to both spatial distance and scalar changes. This method generates a soft segmentation field, effectively eliminating visual field obstructions while preserving the target region. This method also avoids methods such as region segmentation and deep learning denoising, which require extensive manual intervention and computational time. This ensures the system's ease of use and real-time operation (rendering speeds can reach 60 frames per second).

[0052] 3. The present invention's method for rapid 3D visualization and annotation of medical images based on virtual reality incorporates a series of interactive operations tailored to virtual reality systems to aid in the execution of medical visualization tasks. Viewpoint-related interactive operations directly manipulate virtual scenes, mitigating the large workable space requirements of traditional virtual reality technology. Volumetric data annotation-related interactive operations allow users to highlight regions directly on volume data and create symbols and text annotations without the need for third-party tools. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of a method for rapid three-dimensional visualization and annotation of medical images based on virtual reality in an embodiment of the present invention;

[0054] Figure 2 Draw example graphs of the results for the data volume;

[0055] Figure 3 This is an example of the rendering results of the real-time adaptive occlusion reduction step;

[0056] Figure 4 An example diagram of volume data annotation. DETAILED DESCRIPTION

[0057] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0058] This embodiment discloses a method for rapid three-dimensional visualization and annotation of medical images based on virtual reality. Figure 1 As shown, the method includes the following steps:

[0059] S1. Medical image data conversion step: After obtaining a two-dimensional image dataset, extract the scalar value of the image pixel, and convert the pixels in the image dataset into three-dimensional scalar volume data based on the image coordinates of the pixels and the order of the two-dimensional image.

[0060] The medical imaging data supported by step S1 include DICOM format, RAW format, and NiFTI format.

[0061] The image data conversion process extracts relevant information about the medical image, including pixel data, pixel value data type, pixel size, image size, and scaling information, based on the medical image data format. Based on the medical image information, the pixel data is converted into a three-dimensional array storing pixel scalar values. Based on the medical image size information, the three-dimensional array is then rendered into a single-channel three-dimensional texture that conforms to the image scale. The texture's single-channel pixel values ​​are linearly scaled image scalar values.

[0062] S2, a volume data analysis step, inputs the three-dimensional scalar volume data obtained in step S1, calculates the three-dimensional scalar value gradient data of the three-dimensional scalar volume data, the scalar value range in the volume data, the scalar value distribution, and the scalar value gradient distribution. Furthermore, the scalar value distribution peak, the scalar value gradient distribution peak, the scalar value distribution mutation location, and the scalar value gradient distribution mutation location are obtained.

[0063] The gradient calculation of voxels in three-dimensional scalar volume data is replaced by the numerical difference method of the scalar values ​​of the voxels around the voxel.

[0064] S3, data volume rendering step, this step performs preliminary 3D visualization of the volume data, and the results are as follows Figure 2 As shown in Figure 1, the volume data from step S1, the scalar-valued gradient data from step S2, and the volume data analysis results are input. Based on the specified transfer function, the volume data is rendered in 3D using a ray casting volume rendering method. Based on the scalar-valued gradient data, additional Blinn-Phong rendering is performed on portions of the volume data where the gradient exceeds a threshold.

[0065] Step S3 includes the following sub-steps:

[0066] S301, transfer function setting sub-step, this step sets the scalar value-opacity transfer function and scalar value-color transfer function used for volume rendering. The two transfer functions respectively specify the opacity value and color value corresponding to the scalar value of the voxel in the volume data;

[0067] S302, a transfer function rendering sub-step, in which the opacity transfer function and the color transfer function are rendered on two two-dimensional textures respectively; the opacity transfer function is a single-channel texture, and the color transfer function is an RGB three-channel texture;

[0068] S303, a ray casting volume rendering sub-step, in which the 3D texture of the volume data obtained in step S1 is sampled using a ray casting method, and the color and opacity values ​​corresponding to the sampled voxels are determined based on the transfer function texture obtained in sub-step S302; the color and opacity values ​​are then integrated and blended to achieve 3D visualization;

[0069] S304, Blinn-Phong rendering sub-step: After sampling voxels and determining corresponding color values ​​in sub-step S303, this sub-step performs Blinn-Phong rendering on sampling points whose gradient values ​​are greater than a set threshold, and the obtained new color values ​​replace the original color values ​​of the sampling points.

[0070] The transfer function of sub-step S301 is a piecewise linear function. The transfer function is set by editing the control points of the piecewise linear function and then performing linear interpolation on the function within the control points.

[0071] The raycast sampling method in substep S303 uses a fixed-step sampling method along the sampling line, starting from the back and ending at the front. The number of samples is fixed at 512, and the sampling step is determined by the longest straight line distance within the volume data. The sampling line is emitted from the viewpoint to each pixel on the screen.

[0072] When the gradient threshold is not actively set in sub-step S304, the gradient obtained in step S2 is located at the position where the gradient distribution mutation occurs and exceeds the 75% quantile, or the 75% quantile position is directly used.

[0073] S4, real-time adaptive occlusion reduction step, based on the camera position input by the head-mounted virtual reality device, calculates the geodesic distance from each voxel of the three-dimensional scalar volume data obtained in step S1 to the viewpoint in real time frame by frame. The occlusion reduction range is defined according to the three-dimensional geodesic distance field, and the three-dimensional voxel opacity reduction field is calculated. The volume data is occluded according to the opacity reduction field; in addition, the surface opacity is further enhanced. The volume rendering result is then updated. The example rendering result is as follows Figure 3 shown.

[0074] Furthermore, step S4 includes the following sub-steps:

[0075] S401, a geodesic distance field generation sub-step, in which the geodesic distance from each voxel in the volume data to the viewpoint is calculated in real time frame by frame, and a three-dimensional geodesic distance field is formed;

[0076] S402, an occlusion reduction coefficient field generation sub-step, in which an occlusion reduction range is defined based on the three-dimensional geodesic distance field, and the occlusion reduction coefficients of the voxels within the range are calculated; then, an occlusion reduction process is performed on the voxels within the range;

[0077] S403: Surface enhancement sub-step: This sub-step enhances the opacity value of voxels in the volume data whose gradient is higher than a threshold.

[0078] In substep S401, the gradient weight coefficient γ used in the geodesic distance calculation is automatically initialized using a combination of gradient weights and spatial distance weights. This can be manually adjusted during system operation. The geodesic distance field is calculated using a two-scan method. The sampling resolution of the geodesic distance field is set to 1 / 8 the volume data resolution, and the calculated geodesic distance field is upsampled on the GPU.

[0079] In sub-step S403, the opacity value of the voxels with gradients higher than a threshold is increased, where the threshold is the same as the gradient threshold in sub-step S304, wherein the enhancement coefficient is set to a=0.2.

[0080] S5. The VR interaction step implements corresponding interactive functions based on the interactive commands input by the VR device. These include: occlusion reduction adjustment, viewpoint movement and camera adjustment, and medical volume data annotation. After the interactive functions adjust the volume data and rendering, the updated rendered image is displayed on the VR device screen.

[0081] The occlusion reduction function in step S5 includes a reduction range expansion function and a gradient weight adjustment function. The two functions are respectively assigned to the x-axis and y-axis of the joystick of the left hand controller of the virtual reality device, and the geodesic distance field threshold d in step S4 is linearly changed by pushing the joystick. T And the gradient weight coefficient γ. When the joystick is pushed to full scale, the threshold d T The changing speed of is 5 / s, and the changing speed of the gradient weight coefficient γ is 0.05 / s.

[0082] The viewpoint movement and camera adjustment functions in step S5 include smooth viewpoint movement, zooming in and out, and teleporting. The smooth viewpoint movement function is activated using the trigger button on the VR hand controller. Pressing the trigger button and dragging the hand controller will cause the viewpoint to move in the desired direction in the virtual scene. The viewpoint zooming function is activated using the VR hand controller's joystick. Pushing the joystick controls the viewpoint's relative size to the volume data. The viewpoint teleporting function is activated using the VR hand controller's buttons. Point the hand controller's cursor at the desired location in the virtual scene and press a button to reset the viewpoint to the cursor's location.

[0083] The medical volume data annotation function of step S5 includes line drawing function and area marking function. The two functions are realized by the buttons of the left and right hand controllers respectively. After the user presses the button of the left hand controller, a line will be drawn at the position where the cursor passes in the virtual scene. After the user presses the button of the right hand controller, a certain range around the position where the cursor passes in the virtual scene will be marked as a highlighted area. The line thickness, line color, highlight area range, and highlight area color are manually set. Figure 4 shown.

[0084] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent variation based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A method for rapid 3D visualization and annotation of medical images based on virtual reality, characterized by: The method comprises the following steps: S1, a medical image data conversion step, after obtaining a two-dimensional image data set, extracting scalar values ​​of image pixels, and converting the pixels in the image data set into three-dimensional scalar volume data according to the image coordinates of the pixels and the order of the two-dimensional images; The image data conversion process is as follows: First, according to the medical image data format, relevant information of the medical image is extracted, including: pixel data, pixel value data type, pixel size, image size, and scaling information; according to the medical image information, the pixel data is converted into a three-dimensional array storing pixel scalar values; then, according to the medical image size information, the three-dimensional array is rendered into a single-channel three-dimensional texture that conforms to the image ratio, wherein the single-channel pixel value of the texture is a linearly scaled image scalar; S2, a volume data analysis step, inputting the three-dimensional scalar volume data obtained in step S1, calculating the three-dimensional scalar value gradient data of the three-dimensional scalar volume data, the scalar value range in the volume data, the scalar value distribution, and the scalar value gradient distribution; and then obtaining the scalar value distribution peak value, the scalar value gradient distribution peak value, the scalar value distribution mutation position, and the scalar value gradient distribution mutation position; The three-dimensional scalar volume data gradient calculation uses the numerical difference method. The scalar value of this point is f(x i ,y i , z i ), whose gradient The calculation is in the form of the difference of scalar values ​​of adjacent points, as follows: S3, a data volume rendering step, inputting the volume data from step S1 and the scalar value gradient data and volume data analysis results from step S2; performing three-dimensional visualization rendering on the volume data using a ray casting volume rendering method according to a set transfer function; and performing additional Blinn-Phong rendering on portions of the volume data where the gradient is higher than a threshold value according to the scalar value gradient data; Step S3 includes the following sub-steps: S301, a transfer function setting sub-step, in which a scalar value-opacity transfer function and a scalar value-color transfer function for volume rendering are set, wherein the two transfer functions respectively define the opacity value and color value corresponding to the scalar value of a voxel in the volume data; S302, a transfer function rendering sub-step, in which the opacity transfer function and the color transfer function are rendered on two two-dimensional textures respectively; the opacity transfer function is a single-channel texture, and the color transfer function is an RGB three-channel texture; S303, a ray casting volume rendering sub-step, in which the 3D texture of the volume data obtained in step S1 is sampled using a ray casting method, and the color and opacity values ​​corresponding to the sampled voxels are determined based on the transfer function texture obtained in sub-step S302; the color and opacity values ​​are then integrated and blended to achieve 3D visualization; S304, Blinn-Phong rendering sub-step: After sampling voxels and determining corresponding color values ​​in sub-step S303, this sub-step performs Blinn-Phong rendering on sampling points whose gradient values ​​are greater than a set threshold, and the obtained new color values ​​replace the original color values ​​of the sampling points; S4, a real-time adaptive occlusion reduction step, wherein the geodesic distance from each voxel of the 3D scalar volume data obtained in step S1 to the viewpoint is calculated frame by frame in real time based on the camera position input by the head-mounted virtual reality device; an occlusion reduction range is defined based on the 3D geodesic distance field, and a 3D voxel opacity reduction field is calculated; occlusion reduction processing is performed on the volume data based on the opacity reduction field; surface opacity is further enhanced; and then the volume rendering result is updated; Step S4 includes the following sub-steps: S401, a geodesic distance field generation sub-step, in which the geodesic distance from each voxel in the volume data to the viewpoint is calculated in real time frame by frame, and a three-dimensional geodesic distance field is formed; The geodesic distance between the voxel and the viewpoint in sub-step S401 is calculated as follows: in, is the desired voxel position, is the viewpoint position, Γ represents and A path between represent and The set of all paths between, l(Γ) is the length of the path, is the gradient of the infinitesimal element s along the path, and γ is the set gradient weight coefficient. The gradient weight coefficient γ is manually set and initialized automatically using the gradient weight and spatial distance weights. The geodesic distance field is numerically calculated using the two-scan method. The sampling resolution of the geodesic distance field is 1 / 8 of the volume data resolution, and the calculated geodesic distance field is upsampled in the GPU. S402, an occlusion reduction coefficient field generation sub-step, in which an occlusion reduction range is defined based on the three-dimensional geodesic distance field, and the occlusion reduction coefficients of the voxels within the range are calculated; then, an occlusion reduction process is performed on the voxels within the range; The occlusion reduction range in sub-step S402 is determined by the manually set threshold d T Determine that the voxels with geodesic distance d less than the threshold are included in the occlusion attenuation range; occlusion attenuation coefficient The calculation is as follows: Occlusion reduction is achieved by reducing the opacity value of the voxels within the range, which is calculated as follows: in is the opacity value after processing; S403, a surface enhancement sub-step, in which opacity values ​​of voxels in the volume data whose gradients are higher than a threshold are enhanced; In sub-step S403, the opacity value of voxels with gradients above a threshold is increased. The threshold is the same as the gradient threshold in sub-step S304. The opacity enhancement is calculated as follows: in, is the opacity value after enhancement; a is the enhancement coefficient, which is less than 1 and can be set manually; S5, a virtual reality interaction step, implements corresponding interactive functions according to the interactive instructions input by the virtual reality device; including: occlusion reduction adjustment function, viewpoint movement and camera adjustment function, and medical volume data annotation function; after the interactive function adjusts the volume data and rendering, the updated rendered image is displayed on the virtual reality device screen.

2. The method for rapid three-dimensional visualization and annotation of medical images based on virtual reality according to claim 1, characterized in that: The medical imaging data supported in step S1 include DICOM format, RAW format, and NiFTI format.

3. The method for rapid three-dimensional visualization and annotation of medical images based on virtual reality according to claim 1, characterized in that: The transfer functions in sub-step S301 are piecewise linear functions that map scalar values ​​to opacity values ​​and color values ​​respectively; the transfer functions are set by editing the control points of the piecewise linear functions and then performing linear interpolation on the functions within the control points.

4. The method for rapid three-dimensional visualization and annotation of medical images based on virtual reality according to claim 1, characterized in that: In sub-step S303, a ray casting sampling method is used to perform fixed-step sampling along a sampling line in a back-to-front order. The number of samples is fixed, and the sampling step is determined by the longest straight line distance in the volume data. The sampling line is emitted from the viewpoint to each pixel on the screen. The color integration method used in the sub-step ray casting method integrates the opacity according to Lambert's law and performs the following iterative calculations on opacity and color respectively: in, For sampling to The opacity integral result is, for The opacity value of the point; For sampling to The color integration result is for The color value of the point.

5. The method for rapid three-dimensional visualization and annotation of medical images based on virtual reality according to claim 4, characterized in that: In sub-step S303, the ray casting method adopts an acceleration method of empty sampling skipping and early termination of sampling; the empty sampling skipping acceleration method skips voxels with opacity values ​​less than 0.05; and the early termination of sampling stops sampling after the opacity integral value exceeds 1.

6. The method for rapid 3D visualization and annotation of medical images based on virtual reality according to claim 1, characterized in that: When the gradient threshold is not actively set in sub-step S304, the gradient obtained in step S2 is located at the position where the gradient distribution mutation occurs and exceeds the 75% quantile, or the 75% quantile position is directly used.

7. The method for rapid 3D visualization and annotation of medical images based on virtual reality according to claim 1, characterized in that: The occlusion reduction function of step S5 includes a reduction range expansion function and a gradient weight adjustment function; the two functions respectively adjust the geodesic distance field threshold d in step S4 through the joystick of the virtual reality hand controller device. T It is achieved by adjusting the gradient weight coefficient γ; the joystick and parameter growth rate are linearly related.

8. The method for rapid three-dimensional visualization and annotation of medical images based on virtual reality according to claim 1, characterized in that: The viewpoint movement and camera adjustment functions of step S5 include a viewpoint smooth movement function, a viewpoint zooming function, and a viewpoint teleporting function; the viewpoint smooth movement function is used by using the trigger button of the virtual reality hand controller device; pressing the trigger button and dragging the hand controller can show the effect of dragging the viewpoint to the desired direction in the virtual scene; the viewpoint zooming function is used by the joystick of the virtual reality hand controller device; pushing the joystick controls the size ratio of the viewpoint relative to the volume data; the viewpoint teleporting function is used by the button of the virtual reality hand controller device; pointing the cursor of the hand controller in the virtual scene to the target position, and pressing the button to reset the viewpoint position to the cursor position.

9. The method for rapid 3D visualization and annotation of medical images based on virtual reality according to claim 1, characterized in that: The medical volume data annotation function of step S5 includes line drawing and area marking functions; the two functions are implemented through the buttons of the left and right hand controllers respectively; after the user presses the button of the left hand controller, a line will be drawn at the position where the cursor passes in the virtual scene; after the user presses the button of the right hand controller, a certain range around the position where the cursor passes in the virtual scene will be marked as a highlighted area; the line thickness, line color, highlight area range, and highlight area color are manually set.

Images