3D visualization methods, head-mounted devices, and storage media for medical imaging data

Abstract

This application relates to the field of medical imaging technology and provides a three-dimensional visualization method, head-mounted device, and storage medium for medical image data. The method includes: parsing the two-dimensional grayscale value matrix of the corresponding cross-section from the binary pixel data of each two-dimensional image file in the medical image file sequence, and constructing it as three-dimensional volume data of the target body tissue; determining the incident and exit points of multiple projection rays emitted from the target observation viewpoint to the volume bounding box of the three-dimensional volume data on the volume bounding box; performing step-by-step sampling along the direction of the projection rays starting from the incident point; determining the target color and target opacity of the target voxel based on the grayscale value of the target voxel corresponding to the projection sampling position, until sampling reaches the exit point; and determining the color of the corresponding pixel on the projection surface based on the target color and target opacity of the target voxel corresponding to all projection sampling positions to obtain a three-dimensional visualization rendering image, thereby improving the efficiency and safety of surgical operations.

Description

Technical Field

[0001] This application belongs to the field of medical imaging technology, and in particular relates to a three-dimensional visualization method for medical image data, a head-mounted device, and a storage medium. Background Technology

[0002] With the continuous development of medical imaging technology, surgical navigation systems based on medical imaging data have become an indispensable auxiliary tool in surgical procedures. These systems analyze and render image data of the target body tissue acquired by medical imaging equipment, and generate a three-dimensional visualization image of the target body tissue on the display, thereby providing doctors with operational references for intraoperative positioning, path planning, and risk area identification.

[0003] However, traditional surgical navigation systems are mostly deployed on computer platforms, generating 3D visualizations that are displayed on a computer screen. This forces surgeons to frequently switch their gaze between the surgical area and the computer screen during surgery to confirm the spatial relationship between surgical instruments and target tissues in real time. This can easily distract the surgeon, reducing surgical efficiency and accuracy, and increasing surgical risks. Therefore, it is necessary to design a surgical navigation system that can run directly on a head-mounted device to assist surgeons in acquiring and viewing medical imaging data in real time during surgery. Traditional surgical navigation systems, however, rely heavily on computer-based medical image data parsing methods, which are not compatible with head-mounted devices. Summary of the Invention

[0004] In view of this, embodiments of this application provide a three-dimensional visualization method for medical image data, a head-mounted device, and a storage medium to solve the technical problem that traditional surgical navigation systems are mostly deployed on computer platforms and cannot be adapted to head-mounted devices, resulting in low surgical efficiency and accuracy and high surgical risks.

[0005] In a first aspect, embodiments of this application provide a three-dimensional visualization method for medical image data, applied to a head-mounted device; the method includes:

[0006] Obtain a sequence of medical image files to be processed; the sequence of medical image files includes multiple two-dimensional image files corresponding to multiple scanned cross-sections of the target body tissue, and each two-dimensional image file includes image metadata and binary pixel data of the corresponding cross-section;

[0007] For each of the two-dimensional image files, based on the pixel encoding information in the image metadata, the two-dimensional grayscale matrix of the corresponding cross-section is parsed from the binary pixel data;

[0008] Based on the display parameters and spatial location information in each of the image metadata, all the two-dimensional grayscale value matrices are constructed into three-dimensional volume data corresponding to the target body tissue;

[0009] Multiple projection rays are emitted from the target observation perspective toward the volume bounding box of the three-dimensional volume data; each projection ray corresponds to a different pixel on the projection surface.

[0010] For each of the projected rays, the incident point and exit point of the projected ray on the volume bounding box are determined. Starting from the incident point, step-by-step sampling is performed along the direction of the projected ray. At each projection sampling position, the target color and target opacity of the target voxel are determined based on the gray value of the target voxel corresponding to the projection sampling position in the three-dimensional volume data. This process continues until the exit point is sampled. Then, the color of the corresponding pixel of the projected ray on the projection surface is determined based on the target color and target opacity of the target voxels corresponding to all the projection sampling positions on the projected ray.

[0011] A three-dimensional visualization rendering image is obtained based on the colors of all the pixels on the projection surface.

[0012] In one alternative implementation of the first aspect, it also includes:

[0013] When a trigger operation is received for the target virtual button or a target virtual gesture is received, the grayscale values ​​of the three-dimensional volume data are sampled in the cross section, sagittal plane and coronal plane respectively to obtain the two-dimensional grayscale value matrices corresponding to the cross section, the sagittal plane and the coronal plane respectively.

[0014] The two-dimensional grayscale value matrices corresponding to the cross section, the sagittal plane, and the coronal plane are respectively converted into corresponding two-dimensional cross-sectional images.

[0015] In one optional implementation of the first aspect, determining the target color and target opacity of the target voxel based on the grayscale value of the target voxel corresponding to the projection sampling position in the three-dimensional volume data includes:

[0016] Obtain the grayscale value of the target voxel corresponding to the projection sampling position from the three-dimensional volume data;

[0017] Based on the grayscale value of the target voxel, the initial color and initial opacity of the target voxel are determined by a preset transfer function;

[0018] If the gray value of the target voxel is greater than a preset gray value threshold, then the cumulative sampling color and cumulative sampling opacity of the target voxel under preset lighting conditions are determined, and the cumulative sampling color and the cumulative sampling opacity are respectively determined as the target color and target opacity of the target voxel.

[0019] If the grayscale value of the target voxel is less than or equal to the preset grayscale threshold, then the initial color and the initial opacity are determined as the target color and target opacity of the target voxel, respectively.

[0020] In one alternative implementation of the first aspect, determining the cumulative sampled color and cumulative sampled opacity of the target voxel under preset illumination conditions includes:

[0021] The incident point of the illumination light emitted from the preset light source toward the target voxel on the volume bounding box is determined, and step sampling is performed along the direction of the illumination light starting from the incident point of the illumination light. When the target voxel is sampled, the cumulative sampled color and the cumulative sampled opacity of the target voxel in the direction of the illumination light are determined.

[0022] In one optional implementation of the first aspect, step sampling is performed along the direction of the illumination light. After sampling the target voxel, the cumulative sampled color and cumulative sampled opacity of the target voxel in the direction of the illumination light are determined, including:

[0023] At each illumination sampling position on the illumination light, the target color and target opacity of the target voxel corresponding to the illumination sampling position in the three-dimensional volume data are determined by the preset transfer function based on the gray value of the target voxel at the illumination sampling position.

[0024] After sampling the target voxel corresponding to the projection sampling position, the cumulative sampling color and cumulative sampling opacity of the target voxel corresponding to the projection sampling position in the direction of the illumination light are determined according to the target color and target opacity of all target voxels in the direction of the illumination light using the following formula:

[0025] ;

[0026] ;

[0027] in, C For the cumulative sampled color, n The total number of samples taken along the direction of the irradiated light. C i The first ray of the irradiation iThe target color of the target voxel. A j The first ray of the irradiation j The target opacity of the target voxel. Used to indicate the first of the irradiated rays i The combined transmittance of all target voxels preceding the target voxel. A The cumulative sampling opacity.

[0028] In one optional implementation of the first aspect, parsing a two-dimensional grayscale matrix corresponding to the cross-section from the binary pixel data based on pixel encoding information in the image metadata includes:

[0029] Based on the pixel encoding information in the image metadata, the integer grayscale value matrix corresponding to the cross-section is parsed from the binary pixel data;

[0030] The integer grayscale value matrix is ​​normalized to obtain the floating-point grayscale value matrix of the corresponding cross-section, and the floating-point grayscale value matrix is ​​determined as the two-dimensional grayscale value matrix.

[0031] Secondly, embodiments of this application provide a head-mounted device, comprising:

[0032] The first acquisition unit is used to acquire a sequence of medical image files to be processed; the sequence of medical image files includes multiple two-dimensional image files corresponding to multiple scanned cross-sections of the target body tissue, and each two-dimensional image file includes image metadata and binary pixel data of the corresponding cross-section.

[0033] The pixel parsing unit is used to parse the corresponding two-dimensional grayscale matrix from the binary pixel data for each two-dimensional image file based on the pixel encoding information in the image metadata.

[0034] The three-dimensional volume construction unit is used to construct the three-dimensional volume data corresponding to the target body tissue from all the two-dimensional grayscale value matrices based on the display parameters and spatial location information in each of the image metadata.

[0035] The first determining unit is used to emit multiple projection rays to the volume bounding box of the three-dimensional volume data based on the target viewing perspective; each projection ray corresponds to a different pixel on the projection surface.

[0036] The second determining unit is configured to, for each of the projected rays, determine the incident point and exit point of the projected ray on the volume bounding box, and, starting from the incident point, perform step-by-step sampling along the direction of the projected ray. At each projection sampling position, based on the grayscale value of the target voxel corresponding to the projection sampling position in the three-dimensional volume data, determine the target color and target opacity of the target voxel, until sampling reaches the exit point. Then, based on the target color and target opacity of the target voxels corresponding to all the projection sampling positions on the projected ray, determine the color of the corresponding pixel of the projected ray on the projection surface.

[0037] A rendering and display unit is used to obtain a three-dimensional visualization rendering image based on the colors of all the pixels on the projection surface.

[0038] In one alternative implementation of the first aspect, it also includes:

[0039] The three-view sampling unit is used to sample the grayscale values ​​of the three-dimensional volume data in the cross section, sagittal plane and coronal plane respectively when a trigger operation for the target virtual button or a target virtual gesture is received, so as to obtain the two-dimensional grayscale value matrix corresponding to the cross section, the sagittal plane and the coronal plane respectively.

[0040] The three-view conversion unit is used to convert the two-dimensional grayscale value matrices corresponding to the cross section, the sagittal plane and the coronal plane into corresponding two-dimensional cross-sectional images.

[0041] Thirdly, embodiments of this application provide another head-mounted device, including a memory and a computer program stored in the memory and executable on a processor, wherein the processor executes the computer program to implement the method as described in any of the optional implementations of the first aspect above.

[0042] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the optional implementations of the first aspect above.

[0043] Fifthly, embodiments of this application provide a computer program product that, when run on a head-mounted device, causes the head-mounted device to implement the method described in any optional implementation of the first aspect.

[0044] Implementing the three-dimensional visualization method for medical image data, the head-mounted device, the computer-readable storage medium, and the computer program product provided in the embodiments of this application has the following beneficial effects:

[0045] The 3D visualization method for medical image data provided in this application enables the parsing, reconstruction, and rendering of medical image file sequences directly on a head-mounted device without relying on a computer platform. This allows doctors to directly view real-time 3D visualization images of target body tissues in the surgical area during surgery, avoiding the problem of frequently switching between the surgical area and the computer screen, thus improving surgical efficiency and safety. By employing volumetric rendering using volumetric projection in the head-mounted device, 3D visualization rendering images can be obtained without cumbersome image segmentation, resulting in better cross-platform compatibility and adaptability to various embedded graphics engines, making it particularly suitable for resource-constrained head-mounted devices. Furthermore, this method can be completed directly on the head-mounted device, reducing system latency and the risk of data loss, and ensuring the real-time nature and integrity of medical image data. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A schematic flowchart illustrating a three-dimensional visualization method for medical image data provided in an embodiment of this application;

[0048] Figure 2 This is a schematic diagram of a three-dimensional volume data construction process provided in an embodiment of this application;

[0049] Figure 3 A schematic diagram illustrating the implementation process of S102 in a three-dimensional visualization method for medical image data provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of a volume rendering process provided in an embodiment of this application;

[0051] Figure 5 A schematic flowchart illustrating a three-dimensional visualization method for medical image data, provided as another embodiment of this application;

[0052] Figure 6 This is a schematic diagram of the structure of a head-mounted device provided in an embodiment of this application;

[0053] Figure 7 This is a schematic diagram of the structure of a head-mounted device provided in another embodiment of this application. Detailed Implementation

[0054] The following embodiments are only used to illustrate the technical solutions of this application more clearly, and are therefore only examples and should not be used to limit the scope of protection of this application.

[0055] In the description of the embodiments of this application, the technical terms "comprising," "including," "having," and any variations thereof all mean "including but not limited to," unless otherwise specifically emphasized. In the description of the embodiments of this application, unless otherwise stated, the technical term "multiple" refers to two or more, and the technical terms "at least one" or "one or more" refer to one, two, or more than two. The technical terms "first," "second," etc., are only used to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary / secondary relationship of the indicated technical features. The technical term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0056] Based on the description in the background section, traditional surgical navigation systems, in addition to easily distracting doctors, also require doctors to manually operate the computer equipment when they need to view volume rendering images of target body tissues or update their three-view drawings during surgery. This is not only cumbersome, but also easily disrupts the sterile state of the surgical environment, thereby increasing the risk of surgical infection.

[0057] In view of this, this application first provides a method for three-dimensional visualization of medical image data applied to head-mounted devices. This method is adaptable to different system platforms, such as Windows, Linux, macOS, iOS, and Android. Head-mounted devices may include, for example, mixed reality (MR) devices, virtual reality (VR) devices, augmented reality (AR) devices, or smart glasses. This application does not limit the type of head-mounted device.

[0058] Figure 1 This is a schematic flowchart illustrating a three-dimensional visualization method for medical image data provided in an embodiment of this application. Figure 1 As shown, the method may include S101 to S106, as detailed below:

[0059] S101, Obtain the medical image file sequence to be processed; the medical image file sequence includes multiple two-dimensional image files corresponding to multiple scanned cross sections of the target body tissue, and each two-dimensional image file includes image metadata and binary pixel data of the corresponding cross section.

[0060] Among them, a medical image file sequence can be a collection of two-dimensional image files corresponding to the scanned cross-sections after a medical imaging device performs a tomographic scan on a target body tissue.

[0061] The medical image file sequence to be processed can refer to a sequence of medical image files of target body tissues that need to be visualized in three dimensions and / or displayed in three views on a head-mounted device. For example, such as... Figure 2 As shown, if each scanned section of the target body tissue corresponds to a two-dimensional image file 21, then the set 200 of the multiple two-dimensional image files 21 is the medical image file sequence of the target body tissue.

[0062] Medical imaging equipment may include, but is not limited to, computed tomography (CT) equipment or magnetic resonance imaging (MRI) equipment.

[0063] The target body tissue can be any body tissue of a human or other animal, such as the chest or abdomen.

[0064] For example, the medical image file sequence can be a file sequence in digital imaging and communications in medicine (DICOM) format, or it can be a file sequence in other formats. This application embodiment does not limit the format of the medical image file sequence.

[0065] It should be understood that because 3D medical image data is difficult to store and inconvenient to view, medical imaging equipment typically outputs 3D medical image data in the form of 2D tomographic image sequences. In practical applications, to enable digital processing systems to recognize and process medical image data and ensure data fidelity, medical imaging equipment stores the grayscale values ​​of each tomographic plane in binary form and outputs the corresponding binary pixel data when performing tomographic scanning of the target body tissue. Simultaneously, to ensure that this binary pixel data can be correctly parsed and used for 3D visualization rendering, the medical imaging equipment also outputs image metadata describing the information of each scanned tomographic plane. The image metadata and binary pixel data of each scanned tomographic plane together constitute the corresponding 2D image file. Different 2D image files will have at least some different data in their image metadata, while their binary pixel data will be completely different.

[0066] For example, image metadata may include patient information, pixel encoding information, spatial location information, and display parameters. This application does not limit the content of the image metadata.

[0067] For example, patient information may include patient name, patient identification (ID), patient age, patient gender, and allergy history. The patient ID may be, for example, the patient's medical record number.

[0068] For example, pixel encoding information may include pixel format, pixel bit depth, and pixel arrangement.

[0069] The pixel format describes the storage type of the grayscale value of each pixel in the corresponding cross-section. Value types can include, for example, unsigned integer, signed integer, or floating-point.

[0070] Pixel bit depth describes the number of binary bits used to represent the grayscale value of each pixel in a corresponding cross-section. This number of binary bits can include the allocated bit depth and the storage location. The allocated bit depth is defined as the number of bits allocated to the grayscale value of each pixel, such as 8 bits, 12 bits, or 16 bits. The storage bit depth defines the number of effective bits used to represent the grayscale value within the allocated bit depth.

[0071] The pixel arrangement can be used to describe the organizational order of each pixel in a corresponding cross-section. The organizational order can include, for example, a row-by-row arrangement or an interleaved arrangement.

[0072] For example, spatial location information may include fault plane location, fault plane direction, pixel spacing, fault plane thickness, and fault plane spacing, etc.

[0073] The fault plane location can be used to describe the position of the fault plane in the virtual three-dimensional space, which can be represented by the three-dimensional coordinates of the first pixel at the top left corner of the fault plane in the virtual three-dimensional space.

[0074] The orientation of a fracture plane can be used to describe the orientation of the fracture plane relative to a reference coordinate system (such as the world coordinate system or the patient coordinate system). It can be represented by two sets of direction cosine vectors, which can be used to describe the direction of the row and column directions of the fracture plane in virtual three-dimensional space.

[0075] Pixel pitch can be used to describe the physical distance between adjacent pixels in the plane of the slice, and it can be used to define the resolution of a two-dimensional tomographic image.

[0076] The thickness of a fracture plane can be used to describe the physical thickness of the fracture plane in the scanning direction.

[0077] The fault plane spacing can be used to describe the distance between two adjacent fault planes in virtual three-dimensional space.

[0078] For example, display parameters may include window width, window level, grayscale slope, and grayscale intercept.

[0079] The window width defines the width of the grayscale value range, which controls the contrast of the final displayed image. Generally, a narrower window width produces a higher contrast.

[0080] Window level can be used to determine the center position of a grayscale value range, which can control the overall brightness of the final displayed image.

[0081] The grayscale slope can be used to describe the proportional relationship between each binary grayscale value in binary pixel data and the intensity of the true physical signal. The intensity of the true physical signal represents the strength of the raw physical signal detected by medical imaging equipment when performing a tomographic scan of a target body tissue. It reflects the physical properties of the target body tissue, such as density, absorption coefficient, and tissue composition.

[0082] The grayscale intercept can be used as a reference offset to describe the strength of a real physical signal. When used in conjunction with the grayscale slope, it can convert binary grayscale values ​​into grayscale values ​​with physical meaning.

[0083] Optionally, the head-mounted device may store multiple medical image file sequences in its internal memory. Based on this, the head-mounted device can retrieve the medical image file sequences to be processed from its internal memory.

[0084] Optionally, the head-mounted device can establish a communication connection with a computer device. Based on this, the head-mounted device can acquire sequences of medical image files to be processed from the computer device. For example, the computer device may include, but is not limited to, devices such as desktop computers, laptops, tablets, or mobile phones.

[0085] Optionally, the head-mounted device can acquire the sequence of medical image files to be processed using a single thread. Alternatively, the head-mounted device can also acquire the sequence of medical image files to be processed using multiple threads. This application embodiment does not limit the method by which the head-mounted device acquires the sequence of medical image files to be processed.

[0086] Optionally, the head-mounted device can adaptively update the medical image file sequence based on the user's update operation. For example, if the user wants to change any one or more two-dimensional image files in the medical image file sequence, the head-mounted device can update the selected two-dimensional image files in the medical image file sequence with new two-dimensional image files.

[0087] S102: For each two-dimensional image file, based on the pixel encoding information in the image metadata, parse the two-dimensional grayscale matrix of the corresponding cross-section from the binary pixel data.

[0088] The two-dimensional grayscale matrix represents the grayscale distribution of each pixel in the corresponding cross-section. It typically consists of several rows and columns, with each element corresponding to the grayscale value of a pixel, representing the signal intensity or density information of the target body tissue at that pixel. The signal intensity of the target body tissue can be used to describe its response signal amplitude under specific imaging energy.

[0089] Optionally, S102 may include, for example, Figure 3 S1021 to S1022 are described in detail below:

[0090] S1021, Based on the pixel encoding information in the image metadata, parse the integer gray value matrix of the corresponding cross-section from the binary pixel data.

[0091] Optionally, the head-mounted device can read data blocks one by one, starting from the beginning bit of the binary pixel data and using the allocated number of bits in the pixel encoding information as the unit. For each data block read, the head-mounted device can parse the data block into corresponding integer grayscale values ​​according to the pixel format and storage bits in the pixel encoding information. After each integer grayscale value is parsed, the head-mounted device can fill it into the corresponding position in a pre-created two-dimensional matrix according to the pixel arrangement in the pixel encoding information, until the entire two-dimensional matrix is ​​completely filled, thus obtaining the integer grayscale value matrix of the corresponding cross-section. For example, the integer grayscale value matrices corresponding to each cross-section can be respectively as follows: Figure 2 As shown in 22.

[0092] S1022, normalize the integer gray value matrix to obtain the floating-point gray value matrix of the corresponding cross-section, and determine the floating-point gray value matrix as a two-dimensional gray value matrix.

[0093] For example, for each integer grayscale value matrix, the head-mounted device can determine the maximum and minimum grayscale values ​​in the matrix and calculate a first difference between them. For each integer grayscale value in the matrix, the device can calculate a second difference between that value and the minimum value. The ratio of the second difference to the first difference is determined as the floating-point grayscale value corresponding to that integer grayscale value. All floating-point grayscale values ​​constitute a floating-point grayscale value matrix. For example, each two-dimensional grayscale value matrix can be configured as follows: Figure 2 As shown in 23.

[0094] S103, based on the display parameters and spatial location information in the metadata of each image, constructs all two-dimensional grayscale value matrices into three-dimensional volume data corresponding to the target body tissue.

[0095] Three-dimensional volumetric data is used to describe the grayscale distribution of a target body tissue in three-dimensional space, and is a digital representation of the physical structure and density characteristics of the target body tissue. For example, three-dimensional volumetric data consists of a number of regularly arranged voxels, each voxel corresponding to a tiny volume unit of the target body tissue, and the grayscale value of each voxel is used to characterize the signal intensity or density information at its corresponding volume unit.

[0096] Optionally, the head-mounted device can pre-initialize a three-dimensional array as a container for holding three-dimensional volume data. The dimensions (width, height, and depth) of this container can be pre-calculated based on the spatial location information in all two-dimensional image files. Based on this, the head-mounted device can traverse each two-dimensional grayscale matrix. For each pixel in each two-dimensional grayscale matrix, based on the spatial location parameters corresponding to that grayscale matrix, mathematical operations such as affine transformations are used to calculate the voxel position corresponding to that pixel in the aforementioned container, and the normalized grayscale value of that pixel is then filled into the corresponding voxel position. It should be understood that when all pixels in all two-dimensional grayscale matrices have been mapped and filled, the three-dimensional volume data is obtained. For example, the three-dimensional volume data of a target body tissue can be obtained as follows: Figure 2 As shown in 24.

[0097] Optionally, after the head-mounted device generates the 3D volumetric data corresponding to the target body tissue, it can store the 3D volumetric data in a resource folder for later use. For example, when a user wants to view the 3D volumetric data corresponding to the target body tissue again later, they can directly query the 3D volumetric data in the resource folder without waiting for the head-mounted device to regenerate the 3D volumetric data. This not only saves the head-mounted device's computing resources and power consumption but also shortens the waiting time for the user to view the 3D volumetric data.

[0098] S104, based on the target observation perspective, emits multiple projection rays to the volume bounding box of the three-dimensional volume data; each projection ray corresponds to a different pixel on the projection surface.

[0099] The target viewing angle represents the user's observation position and line of sight in the virtual 3D space. This position and direction of sight correspond to the spatial position and orientation of the virtual camera in the volumetric rendering system of the head-mounted device, respectively. The volumetric bounding box of the 3D volume data defines the geometric boundaries of the 3D volume data; it can be an axis-aligned bounding box. The projection plane refers to the imaging plane of the virtual camera, i.e., the 2D plane onto which the volumetric bounding box of the 3D volume data is mapped after projection.

[0100] For example, such as Figure 4 As shown, if the user's observation position in the virtual three-dimensional space is point A, and the line of sight is directed toward the volume bounding box 24 of the three-dimensional volume data, then each projected ray can be 41.

[0101] S105: For each projected ray, determine the incident point and exit point of the projected ray on the volume bounding box, and start from the incident point to perform step-by-step sampling along the direction of the projected ray. At each projected sampling position, determine the target color and target opacity of the target voxel based on the gray value of the target voxel corresponding to the projected sampling position in the 3D volume data. Continue sampling until the exit point is reached. Then, determine the color of the corresponding pixel on the projection surface based on the target color and target opacity of the target voxel corresponding to all projected sampling positions on the projected ray.

[0102] It should be noted that, given the target's viewing angle, the spatial position of the volume bounding box of the 3D volume data, and the spatial position of the projection plane, the spatial positions of the incident and exit points of each projected ray relative to the volume bounding box can be determined. Specific methods for determining the spatial positions of the incident and exit points of each projected ray can be found in relevant technologies and will not be detailed here.

[0103] Optionally, the spatial location of the incident point can be determined by its coordinates in the world coordinate system. x in , y in , z in The spatial location of the launch point can be determined by its coordinates in the world coordinate system. x out , y out , z out Each projected sampling position can be represented by its coordinates in the world coordinate system. x ts , y ts , z ts )express.

[0104] Optionally, for each projected ray, given that the spatial location of its incident point is known, the head-mounted device can determine the coordinates of each projected sampling position in the world coordinate system using the following formula (1):

[0105] P k ( i )= P in + k ( i )×Δ s × d ;Formula (1)

[0106] in,P k ( i ) is the first i The coordinates of each sampling location in the world coordinate system P in Let be the coordinates of the point of incidence of the projected ray on the bounding box of the volume in the world coordinate system. k ( i ) is the first i The number of sampling steps corresponding to each sampling position, Δ s The sampling step size, d is the unit vector of the projected ray.

[0107] The sampling step count for each projection sampling position can refer to the cumulative number of steps taken along the direction of the projected ray from the incident point to that projection sampling position. For example, the sampling step count for the first projection sampling position after the incident point can be 1, and the sampling step count for the second projection sampling position can be 2.

[0108] The gray value of the target voxel corresponding to the projection sampling position in the 3D volume data can refer to the gray value of the target voxel obtained by sampling the 3D volume data at the projection sampling position.

[0109] Optionally, in step S105, the target color and target opacity of the target voxel are determined based on the grayscale value of the target voxel corresponding to the projection sampling position in the three-dimensional volume data. Specifically, this may include steps 1.1 to 1.4, as detailed below:

[0110] Step 1.1: Obtain the grayscale value of the target voxel corresponding to the projection sampling position from the 3D volume data.

[0111] It should be understood that the voxel indices of the 3D volume data are discrete integer coordinates, while the coordinates of the projected sampling position are continuous floating-point numbers. Therefore, the head-mounted device can first transform the coordinates of the projected sampling position from the world coordinate system to the local coordinate system of the 3D volume data. The world coordinate system is the unified coordinate system of the entire 3D virtual scene. The local coordinate system is the coordinate system of the 3D volume data itself, which typically uses a corner point of the 3D volume data as its origin (0,0,0). Based on this, the coordinates of its opposite corner point in the local coordinate system are (…). width -1, height -1, depth -1). width , height and depth These represent the number of voxels in the three dimensions of the three-dimensional volume data.

[0112] Since the coordinates of the projected sampling position are still floating-point numbers after conversion, they cannot directly correspond to integer voxel indices in the 3D voxel data. Therefore, head-mounted devices can use interpolation to determine the grayscale value of the target voxel corresponding to the projected sampling position. For example, head-mounted devices can use trilinear interpolation to calculate the pixel grayscale value corresponding to the projected sampling position by weighting the grayscale values ​​of all adjacent voxels of the target voxel where the projected sampling position is located and the relative position ratio of the projected sampling position in the target voxel.

[0113] Step 1.2: Determine the initial color and initial opacity of the target voxel using a preset transfer function based on the grayscale value of the target voxel.

[0114] Preset transfer functions can be used to define the correspondence between the grayscale values ​​of voxels and visualization parameters. For example, visualization parameters may include color and opacity. Based on this, preset transfer functions may include color transfer functions and opacity transfer functions. The color transfer function can be used to map the grayscale values ​​of voxels to colors, thereby facilitating the visual differentiation of tissue density or material properties in the rendering results. The opacity transfer function can be used to map the grayscale values ​​of voxels to transparency or opacity to control the visual visibility of different tissues.

[0115] Preset transfer functions can be stored in the internal memory of the head-mounted device and dynamically adjusted according to different image types (such as CT or MRI images) or display requirements to achieve flexible visual presentation effects. For example, higher grayscale values ​​can be configured as high-brightness colors (such as white) and opacity to highlight high-density tissues such as bones; medium grayscale values ​​can be configured as medium-brightness colors and semi-transparent to present medium-density structures such as soft tissues; and lower grayscale values ​​can be configured as low-brightness colors and completely transparent to filter out low-density areas such as air or background.

[0116] Based on this, the head-mounted device can retrieve a preset transfer function from its memory, and use the color transfer function and opacity transfer function in the preset transfer function to process the gray value of the target voxel corresponding to the projection sampling position, respectively, to obtain the initial color and initial opacity of the target pixel.

[0117] Step 1.3: If the gray value of the target voxel is greater than the preset gray value threshold, then determine the cumulative sampling color and cumulative sampling opacity of the target voxel under the preset illumination conditions, and determine the cumulative sampling color and cumulative sampling opacity as the target color and target opacity of the target voxel, respectively.

[0118] The preset grayscale threshold can be determined based on the type of body tissue to be observed. For example, if you want to observe high-density tissues such as bones, you can set a larger preset grayscale threshold to reduce interference from other tissues around the high-density tissue; if you want to observe low-density tissues such as soft tissues, you can set a smaller preset grayscale threshold to ensure that the low-density tissues can be identified and participate in rendering.

[0119] It should be understood that if the grayscale value of the target voxel corresponding to the projection sampling position is greater than a preset grayscale threshold, it indicates the presence of a tissue entity (such as bone) to be colored at that projection sampling position, which will contribute to the final pixel color. Therefore, to ensure the final 3D visualization rendering image presents realistic lighting effects and three-dimensional depth, it is necessary to determine the cumulative sampled color and cumulative sampled opacity of the target pixel corresponding to the projection sampling position under preset lighting conditions. In this case, the head-mounted device can emit illumination light from a preset light source towards the target pixel corresponding to the projection sampling position to simulate the attenuation of light due to occlusion by other voxels in the propagation path, thereby improving the accuracy of the calculation of the cumulative sampled color and cumulative sampled opacity of the target pixel.

[0120] In practical applications, the preset light source's illumination direction can maintain a reasonable angular relationship with the user's line of sight. This ensures that the main tissue areas receive sufficient light while creating appropriate shadows, enhancing the three-dimensionality of the final 3D visualization rendering image. Optionally, the preset light source's illumination direction can be adjusted in real time according to the user's line of sight to avoid light and shadow reversals caused by changes in the user's line of sight, improving the stability and realism of the brightness distribution in the 3D visualization rendering image.

[0121] Specifically, the head-mounted device can determine the incident point of the illumination light emitted from a preset light source toward the target voxel on the volume bounding box of the three-dimensional volume data, and start from the incident point of the illumination light to perform step-by-step sampling along the direction of the illumination light. When the target voxel corresponding to the projection sampling position is sampled, the cumulative sampling color and cumulative sampling opacity of the target voxel in the direction of the illumination light are determined.

[0122] Optionally, step-by-step sampling is performed along the direction of the illumination light. After sampling to the target voxel corresponding to the projection sampling position, the cumulative sampling color and cumulative sampling opacity of the target voxel in the direction of the illumination light are determined. Specifically, this may include the following steps 1.31 to 1.32:

[0123] Step 1.31: At each illumination sampling position on the illumination light, based on the gray value of the target voxel corresponding to the illumination sampling position in the three-dimensional volume data, determine the target color and target opacity of the target voxel corresponding to the illumination sampling position through a preset transfer function.

[0124] For each illumination sampling position on the illuminating light, the head-mounted device can use the color transfer function and opacity transfer function in the preset transfer function to process the gray value of the target voxel corresponding to the illumination sampling position, and obtain the target color and target opacity of the target pixel.

[0125] Step 1.32: After sampling the target voxel corresponding to the projection sampling position, based on the target color and target opacity of all target voxels in the direction of the illumination light, the cumulative sampling color and cumulative sampling opacity of the target voxel corresponding to the projection sampling position in the direction of the illumination light are determined by the following formulas (1) and (2):

[0126] ;Formula (1)

[0127] ;Formula (2)

[0128] in, C This represents the cumulative sampled color of the target pixel at the projection sampling position along the direction of the illuminating light. n This represents the total number of samples taken along the direction of the illuminating light. C i For the first light source i The target color of each target voxel. A j For the first light source j Target opacity of a target voxel Used to indicate the first of the illuminating rays i The combined transmittance of all target voxels preceding the target voxel. A The cumulative sampling opacity of the target pixel corresponding to the projection sampling position in the direction of the illumination light.

[0129] Step 1.4: If the grayscale value of the target voxel is less than or equal to the preset grayscale threshold, then the initial color and initial opacity are determined as the target color and target opacity of the target voxel, respectively.

[0130] It should be understood that if the grayscale value of the target voxel corresponding to the projection sampling position is less than or equal to a preset grayscale threshold, it indicates that the tissue density at that projection sampling position is low and does not significantly contribute to the final pixel color. Therefore, in order to improve rendering efficiency and simplify processing steps, the head-mounted device can directly determine the initial color and initial transparency of the target voxel corresponding to the projection sampling position as the target color and target opacity of the target voxel, respectively.

[0131] This application embodiment maps the grayscale values ​​of each voxel to corresponding colors and opacities, enabling three-dimensional visualization of target body tissues on a head-mounted device, and visually reflecting the structural differences between different body tissues such as bones, muscles, and fat in the three-dimensional visualization image.

[0132] S106, obtains a 3D visualization rendering image based on the colors of all pixels on the projection surface.

[0133] It should be understood that an image composed of all the colored pixels on the projection surface is a 3D visualization rendering image. Because the color of each pixel in a 3D renderable image takes into account the voxel grayscale value, color, and opacity of the 3D volume data, it can visually present a three-dimensional effect with a sense of depth.

[0134] As can be seen from the above, the 3D visualization method for medical image data provided in this embodiment can directly realize the parsing, reconstruction, and rendering of medical image file sequences on a head-mounted device without relying on a computer platform. This allows doctors to directly view the 3D visualization image of the target body tissue in real time in the surgical area during surgery, thereby avoiding the problem of frequently switching the line of sight between the surgical area and the computer device display screen, improving the efficiency and safety of surgical operations. By using volumetric rendering of the line of sight using volumetric projection in the head-mounted device, 3D visualization rendering images can be obtained without cumbersome image segmentation, thus having better cross-platform compatibility and being compatible with various embedded graphics engines, especially suitable for resource-constrained head-mounted devices. In addition, this method can be completed directly on the head-mounted device, thus reducing system latency and the risk of data loss, and ensuring the real-time performance and integrity of medical image data.

[0135] Figure 5 This is a schematic flowchart illustrating a three-dimensional visualization method for medical image data, provided as another embodiment of this application. Figure 5 As shown, with Figures 1 to 4 Compared to the corresponding embodiments, the three-dimensional visualization method for medical image data in this embodiment may further include S501 to S502, as detailed below:

[0136] S501, when a trigger operation is received for the target virtual button or a target virtual gesture is received, grayscale values ​​are sampled on the cross section, sagittal plane and coronal plane of the three-dimensional volume data respectively to obtain the two-dimensional grayscale value matrices corresponding to the cross section, sagittal plane and coronal plane respectively.

[0137] Both the target virtual buttons and the target virtual gestures are used to control the head-mounted device to display three views (front view / rear view, left view / right view, and top view / bottom view) of the target body tissue's three-dimensional volume data.

[0138] Based on this, when the head-mounted device receives a trigger operation on a virtual button or a virtual gesture from a target, it can sample the grayscale values ​​of the three-dimensional volume data in the cross-section, sagittal plane, and coronal plane respectively to obtain the two-dimensional grayscale value matrices corresponding to the cross-section, sagittal plane, and coronal plane respectively.

[0139] For example, the head-mounted device can extract the set of gray values ​​of the target voxels corresponding to each sampling plane from the three-dimensional volume data based on the preset sampling plane position parameters, thereby obtaining the two-dimensional gray value matrices corresponding to the cross section, sagittal plane and coronal plane respectively.

[0140] S502 converts the two-dimensional grayscale value matrices corresponding to the cross section, sagittal plane, and coronal plane into corresponding two-dimensional cross-sectional images.

[0141] The head-mounted device can normalize the two-dimensional grayscale value matrices corresponding to the transverse, sagittal, and coronal planes respectively, to obtain two-dimensional cross-sectional images corresponding to the transverse, sagittal, and coronal planes respectively.

[0142] The three-dimensional visualization method for medical imaging data provided in this embodiment allows users to control a head-mounted device to display three-view images of three-dimensional volume data via virtual gestures or virtual buttons, thereby enabling rapid switching and flexible viewing of transverse, sagittal, and coronal section images. Through this interactive display and control method, doctors can directly operate the head-mounted device to browse images within the intraoperative field of view without relying on external computers or physical input devices, significantly improving the convenience and real-time nature of the operation. Furthermore, by generating and displaying three-view images on the head-mounted device, this method can display the internal structure of the target body tissue from different spatial directions without altering the three-dimensional volume data itself. This facilitates doctors' accurate judgment of tissue boundaries, lesion locations, and spatial relationships, improving the accuracy and intuitiveness of intraoperative navigation.

[0143] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0144] Based on the three-dimensional visualization method for medical image data provided in the above embodiments, this application further provides an embodiment of a head-mounted device that implements the above method embodiments. Please refer to... Figure 6 This is a schematic diagram of a head-mounted device provided in an embodiment of this application. For ease of explanation, only the parts relevant to this embodiment are shown. Figure 6As shown, the head-mounted device 60 may include: a first acquisition unit 601, a pixel resolution unit 602, a 3D volume construction unit 603, a first determination unit 604, a second determination unit 605, and a rendering and display unit 606. Wherein:

[0145] The first acquisition unit 601 is used to acquire a sequence of medical image files to be processed; the sequence of medical image files includes multiple two-dimensional image files corresponding to multiple scanned cross sections of the target body tissue, and each two-dimensional image file includes image metadata and binary pixel data of the corresponding cross section;

[0146] The pixel parsing unit 602 is used to parse the two-dimensional gray value matrix of the corresponding cross-section from the binary pixel data for each of the two-dimensional image files, based on the pixel encoding information in the image metadata.

[0147] The three-dimensional volume construction unit 603 is used to construct three-dimensional volume data corresponding to the target body tissue based on the display parameters and spatial location information in each of the image metadata;

[0148] The first determining unit 604 is used to emit multiple projection rays to the volume bounding box of the three-dimensional volume data based on the target viewing perspective; each projection ray corresponds to a different pixel on the projection surface.

[0149] The second determining unit 605 is used to determine the incident point and exit point of each projected ray on the volume bounding box, and to perform step-by-step sampling along the direction of the projected ray starting from the incident point. At each projection sampling position, the target color and target opacity of the target voxel are determined according to the gray value of the target voxel corresponding to the projection sampling position in the three-dimensional volume data, until the exit point is sampled. Then, the color of the corresponding pixel of the projected ray on the projection surface is determined according to the target color and target opacity of the target voxels corresponding to all projection sampling positions on the projected ray.

[0150] The rendering display unit 606 is used to obtain a three-dimensional visualization rendering image based on the colors of all the pixels on the projection surface.

[0151] Optionally, the head-mounted device also includes a three-view sampling unit and a three-view transformation unit. Wherein:

[0152] The three-view sampling unit is used to sample the grayscale values ​​of the three-dimensional volume data in the cross section, sagittal plane, and coronal plane respectively when it receives an operation on the target virtual button or a target virtual gesture, so as to obtain the two-dimensional grayscale value matrix corresponding to the cross section, the sagittal plane, and the coronal plane respectively.

[0153] The three-view conversion unit is used to convert the two-dimensional grayscale value matrices corresponding to the cross section, the sagittal plane, and the coronal plane into corresponding two-dimensional cross-sectional images, respectively.

[0154] Optionally, the second determining unit 605 may specifically include a grayscale value acquisition unit, an initial parameter determining unit, and a target parameter determining unit. Wherein:

[0155] The grayscale value acquisition unit is used to acquire the grayscale value of the target voxel corresponding to the projection sampling position from the three-dimensional volume data.

[0156] The initial parameter determination unit is used to determine the initial color and initial opacity of the target voxel based on the grayscale value of the target voxel through a preset transfer function.

[0157] The target parameter determination unit is used to determine the cumulative sampling color and cumulative sampling opacity of the target voxel under a preset illumination condition if the gray value of the target voxel is greater than a preset gray value threshold, and to determine the cumulative sampling color and the cumulative sampling opacity as the target color and target opacity of the target voxel, respectively.

[0158] The target parameter determination unit is further configured to determine the initial color and the initial opacity as the target color and target opacity of the target voxel if the gray value of the target voxel is less than or equal to the preset gray value threshold.

[0159] Optionally, the target parameter determination unit is specifically used for:

[0160] The incident point of the illumination light emitted from the preset light source toward the target voxel on the volume bounding box is determined, and step sampling is performed along the direction of the illumination light starting from the incident point of the illumination light. When the target voxel is sampled, the cumulative sampled color and the cumulative sampled opacity of the target voxel in the direction of the illumination light are determined.

[0161] Optionally, the target parameter determination unit is also specifically used for:

[0162] At each illumination sampling position on the illumination light, the target color and target opacity of the target voxel corresponding to the illumination sampling position in the three-dimensional volume data are determined by the preset transfer function based on the gray value of the target voxel at the illumination sampling position.

[0163] After sampling the target voxel corresponding to the projection sampling position, the cumulative sampling color and cumulative sampling opacity of the target voxel corresponding to the projection sampling position in the direction of the illumination light are determined according to the target color and target opacity of all target voxels in the direction of the illumination light using the following formula:

[0164] ;

[0165] ;

[0166] in, C For the cumulative sampled color, n The total number of samples taken along the direction of the irradiated light. C i The first ray of the irradiation i The target color of the target voxel. A j The first ray of the irradiation j The target opacity of the target voxel. Used to indicate the first of the irradiated rays i The combined transmittance of all target voxels preceding the target voxel. A The cumulative sampling opacity.

[0167] Optionally, the pixel resolution unit 602 is specifically used for:

[0168] Based on the pixel encoding information in the image metadata, the integer grayscale value matrix corresponding to the cross-section is parsed from the binary pixel data;

[0169] The integer grayscale value matrix is ​​normalized to obtain the floating-point grayscale value matrix of the corresponding cross-section, and the floating-point grayscale value matrix is ​​determined as the two-dimensional grayscale value matrix.

[0170] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units is used as an example. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the head-mounted device can be divided into different functional units to complete all or part of the functions described above. The functional units in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of each unit in the above-described head-mounted device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0171] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a head-mounted device provided for another embodiment of this application. (See attached diagram.) Figure 7 As shown, the head-mounted device 7 provided in this embodiment may include: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as a program corresponding to a three-dimensional visualization method for medical image data. When the processor 70 executes the computer program 72, it implements the steps in the above-described embodiment of the three-dimensional visualization method for medical image data, for example... Figure 1 S101~S106 are shown. Or, when the processor 70 executes the computer program 72, it implements the functions of each unit in the above-described head-mounted device embodiment.

[0172] For example, computer program 72 can be divided into one or more modules / units, one or more of which are stored in memory 71 and executed by processor 70 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 72 in head-mounted device 7. For example, computer program 72 can be divided into a first acquisition unit, a pixel resolution unit, a 3D volume construction unit, a first determination unit, a second determination unit, and a rendering and display unit. For the specific functions of each unit, please refer to [link to relevant documentation]. Figure 6 The relevant descriptions in the corresponding embodiments are not repeated here.

[0173] Those skilled in the art will understand that Figure 7 This is merely an example of head-mounted device 7 and does not constitute a limitation on head-mounted device 7. It may include more or fewer components than illustrated, or combine certain components, or use different components.

[0174] The processor 70 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0175] The memory 71 can be an internal storage unit of the head-mounted device 7, such as a hard disk or RAM of the head-mounted device 7. The memory 71 can also be an external storage device of the head-mounted device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card equipped on the head-mounted device 7. Furthermore, the memory 71 can include both internal and external storage units of the head-mounted device 7. The memory 71 is used to store computer programs and other programs and data required by the head-mounted device. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0176] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various steps of the three-dimensional visualization method for medical image data described in the above method embodiments.

[0177] This application provides a computer program product that, when run on a head-mounted device, causes the head-mounted device to perform the steps described in the various method embodiments above.

[0178] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

[0179] It should be noted that, unless otherwise specified, all technical terms used in the embodiments of this application have the same meaning as commonly understood by those skilled in the art to which this application belongs. The technical terms used in the embodiments of this application are only used to explain specific embodiments of this application and are not intended to limit this application.

[0180] The term "embodiment" as used in the description of embodiments in this application means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0181] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0182] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for three-dimensional visualization of medical image data, characterized in that, The method is applied to a head-mounted device, and the method comprises: Obtaining a sequence of medical image files to be processed; the sequence of medical image files comprises a plurality of two-dimensional image files respectively corresponding to a plurality of scanned tomographic sections of a target body tissue, and each two-dimensional image file comprises image metadata and binary pixel data of a corresponding tomographic section; For each two-dimensional image file, a two-dimensional grayscale value matrix of the corresponding tomographic section is parsed from the binary pixel data according to pixel encoding information in the image metadata; Based on display parameters and spatial position information in each image metadata, all the two-dimensional grayscale value matrices are constructed into three-dimensional volume data corresponding to the target body tissue; Based on a target observation view angle, a plurality of projection rays are emitted to a volume bounding box of the three-dimensional volume data; each projection ray corresponds to a different pixel on a projection plane; For each projection ray, an incident point and an exit point of the projection ray on the volume bounding box are determined, and stepwise sampling is performed along the direction of the projection ray starting from the incident point; at each projection sampling position, a target color and a target opacity of a target voxel corresponding to the projection sampling position in the three-dimensional volume data are determined according to the grayscale value of the target voxel, until the exit point is reached, and the color of the pixel on the projection plane corresponding to the projection ray is determined according to the target colors and target opacities of all the target voxels corresponding to all the projection sampling positions on the projection ray; A three-dimensional visualization rendering image is obtained according to the colors of all the pixels on the projection plane; Determining the target color and the target opacity of the target voxel corresponding to the projection sampling position in the three-dimensional volume data comprises: Obtaining the grayscale value of the target voxel corresponding to the projection sampling position from the three-dimensional volume data; Determining an initial color and an initial opacity of the target voxel through a preset transfer function according to the grayscale value of the target voxel; If the grayscale value of the target voxel is greater than a preset grayscale threshold, an incident point of an illumination ray emitted from a preset light source to the target voxel on the volume bounding box is determined, and stepwise sampling is performed along the direction of the illumination ray starting from the incident point of the illumination ray; when the target voxel is reached, a cumulative sampling color and a cumulative sampling opacity of the target voxel in the direction of the illumination ray are determined, and the cumulative sampling color and the cumulative sampling opacity are determined as the target color and the target opacity of the target voxel, respectively; wherein the preset grayscale threshold is determined according to the type of the body tissue to be observed; If the grayscale value of the target voxel is less than or equal to the preset grayscale threshold, the initial color and the initial opacity are determined as the target color and the target opacity of the target voxel, respectively.

2. The method of claim 1, wherein, Further comprising: When a trigger operation for a target virtual button is received or a target virtual gesture is received, the three-dimensional volume data is sampled in gray scale values in a transverse plane, a sagittal plane and a coronal plane respectively to obtain a two-dimensional gray scale value matrix corresponding to the transverse plane, the sagittal plane and the coronal plane respectively; The two-dimensional gray scale value matrix corresponding to the transverse plane, the sagittal plane and the coronal plane is converted into a corresponding two-dimensional section image respectively.

3. The method of claim 1, wherein, The step-by-step sampling is performed along the direction of the illumination light ray, and when the target voxel is sampled, the accumulated sampling color and the accumulated sampling opacity of the target voxel in the direction of the illumination light ray are determined, including: At each illumination sampling position on the illumination light ray, the target color and the target opacity of the target voxel corresponding to the illumination sampling position are determined by the preset transfer function according to the gray scale value of the target voxel corresponding to the illumination sampling position in the three-dimensional volume data; When the target voxel corresponding to the projection sampling position is sampled, the accumulated sampling color and the accumulated sampling opacity of the target voxel corresponding to the projection sampling position in the direction of the illumination light ray are determined by the following formula according to the target color and the target opacity of all the target voxels in the direction of the illumination light ray: ； ； in, C For the cumulative sampled color, n The total number of samples taken along the direction of the irradiated light. C i The first irradiation light i The target color of the target voxel. A j The first irradiation light j The target opacity of the target voxel. Used to indicate the first of the irradiated rays i The combined transmittance of all target voxels preceding the target voxel. A The cumulative sampling opacity.

4. The method according to any one of claims 1 to 3, characterized in that, According to the pixel coding information in the image metadata, a two-dimensional gray scale value matrix corresponding to a section plane is parsed from the binary pixel data, including: According to the pixel coding information in the image metadata, an integer gray scale value matrix corresponding to a section plane is parsed from the binary pixel data; The integer gray scale value matrix is normalized to obtain a floating-point gray scale value matrix corresponding to a section plane, and the floating-point gray scale value matrix is determined as the two-dimensional gray scale value matrix.

5. A head-mounted device, comprising: Including: A first acquisition unit is configured to acquire a medical image file sequence to be processed; the medical image file sequence includes a plurality of two-dimensional image files corresponding to a plurality of scanned section planes of a target body tissue respectively, and each two-dimensional image file includes image metadata and binary pixel data of a corresponding section plane; A pixel parsing unit is configured to, for each two-dimensional image file, parse a two-dimensional gray scale value matrix of a corresponding section plane from the binary pixel data according to pixel coding information in the image metadata; A three-dimensional volume construction unit is configured to construct all the two-dimensional gray scale value matrices into three-dimensional volume data corresponding to the target body tissue based on display parameters and spatial position information in each image metadata; A first determination unit is configured to emit a plurality of projection light rays to a volume bounding box of the three-dimensional volume data based on a target observation view angle; each projection light ray corresponds to a different pixel on a projection plane respectively. The second determining unit is configured to determine, for each of the projection rays, an incident point and an exit point of the projection ray on the volume bounding box, and perform stepwise sampling along the direction of the projection ray from the incident point, and at each projection sampling position, determine a target color and a target opacity of a target voxel corresponding to the projection sampling position in the three-dimensional body data according to the grayscale value of the target voxel, until the exit point is reached, and then determine the color of a pixel on the projection plane corresponding to the projection ray according to the target color and the target opacity of all the target voxels corresponding to the projection sampling positions on the projection ray. The rendering display unit is configured to obtain a three-dimensional visualization rendering image according to the colors of all the pixels on the projection plane. The second determining unit includes a grayscale value obtaining unit, an initial parameter determining unit, and a target parameter determining unit. The grayscale value obtaining unit is configured to obtain the grayscale value of the target voxel corresponding to the projection sampling position from the three-dimensional body data. The initial parameter determining unit is configured to determine an initial color and an initial opacity of the target voxel by using a preset transfer function according to the grayscale value of the target voxel. The target parameter determining unit is configured to, if the grayscale value of the target voxel is greater than a preset grayscale threshold, determine an incident point of an illumination ray emitted from a preset light source to the target voxel on the volume bounding box, and perform stepwise sampling along the direction of the illumination ray from the incident point of the illumination ray, and when the target voxel is reached, determine a cumulative sampling color and a cumulative sampling opacity of the target voxel in the direction of the illumination ray, and determine the cumulative sampling color and the cumulative sampling opacity as the target color and the target opacity of the target voxel, respectively; wherein the preset grayscale threshold is determined according to the type of the body tissue to be observed. The target parameter determining unit is further configured to, if the grayscale value of the target voxel is less than or equal to the preset grayscale threshold, determine the initial color and the initial opacity as the target color and the target opacity of the target voxel, respectively.

6. The head-mounted device of claim 5, wherein, Further comprising: A three-view sampling unit is configured to, when a triggering operation for a target virtual key is received or a target virtual gesture is received, perform grayscale value sampling on the three-dimensional body data in a transverse plane, a sagittal plane, and a coronal plane, respectively, to obtain a two-dimensional grayscale value matrix corresponding to the transverse plane, the sagittal plane, and the coronal plane, respectively. A three-view conversion unit is configured to convert the two-dimensional grayscale value matrix corresponding to the transverse plane, the sagittal plane, and the coronal plane into a corresponding two-dimensional section image, respectively.

7. A head-mounted device, comprising: A computer program is stored in the memory and executable on the processor, and the processor executes the computer program to implement the method according to any one of claims 1-4.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executable on the processor to implement the method according to any one of claims 1-4.

Images