Vascular projection visualization method and apparatus, and storage medium and electronic device

Abstract

Provided in the present description are a vascular projection visualization method and apparatus, and a storage medium and an electronic device. The method comprises: acquiring a first image of a target region by computed tomography angiography and a second image of the target region by digital subtraction angiography; determining a vascular path and a plaque region in the first image, and constructing a mesh structure of the vascular path; on the basis of a positional relationship between the mesh structure and the plaque region in the first image, obtaining an occlusion relationship between the mesh structure and the plaque region; and on the basis of the occlusion relationship, rendering a mesh structure and the plaque region in the second image, so as to obtain a fused image. A mesh structure of a vascular path is constructed, and on the basis of an occlusion relationship between the mesh structure and a plaque region, the mesh structure and the plaque region are rendered in a second image to obtain a fused image, such that a positional relationship between the mesh structure and the plaque region in a mapping direction of the second image is intuitively reflected while the degree of vascular stenosis, the degree of vascular dilation, and the location of a plaque are displayed.

Description

A method, device, storage medium, and electronic device for visualizing blood vessel projection.

[0001] Cross-reference to related applications

[0002] The related applications of this application claim priority to Chinese Patent Application No. 202411498552.X, filed on October 24, 2024, entitled “A Method and Program Product for Visualizing Blood Vessel Projection”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This specification relates to the field of image processing technology, and in particular to a method, apparatus, storage medium, and electronic device for visualizing blood vessel projection. Background Technology

[0004] Computed tomography angiography (CTA) is a non-invasive imaging technique that provides three-dimensional images of blood vessels, including information such as the degree of stenosis, dilation, and plaque location. Digital subtraction angiography (DSA) is an invasive technique that uses X-rays obtained by injecting a contrast agent into the blood vessel. These X-ray images contain information about the vessel's course and the guidewire used to guide medical devices within the vessel.

[0005] In clinical practice, information such as the degree of vascular stenosis / dilation and plaque location from CTA is often mapped onto the two-dimensional plane of DSA by projecting a three-dimensional segmentation mask onto the CTA image. This allows physicians to understand the degree of vascular stenosis, dilation, and plaque location within the two-dimensional plane of the DSA. However, this method cannot reflect the positional relationship between the plaque and the blood vessel.

[0006] Therefore, this specification provides a method, apparatus, storage medium, and electronic device for visualizing blood vessel projection. Summary of the Invention

[0007] This specification provides a method, apparatus, storage medium, and electronic device for visualizing blood vessel projection, in order to partially solve the aforementioned problems existing in the prior art.

[0008] The following technical solution is adopted in this specification:

[0009] This specification provides a method for visualizing blood vessel projection, including:

[0010] A three-dimensional image of computed tomography angiography of the target area is acquired as the first image, and a two-dimensional image of digital subtraction angiography of the target area is acquired as the second image.

[0011] Determine the blood vessel paths and plaque regions in the first image, and construct a mesh structure for the blood vessel paths;

[0012] Based on the grid structure in the first image and the positional relationship of the patch regions, the occlusion relationship between the grid structure and the patch regions is obtained;

[0013] According to the occlusion relationship, the mesh structure and the patch region are rendered in the second image to obtain a fused image and display it.

[0014] Optionally, constructing the mesh structure of the vascular pathway specifically includes:

[0015] The cross-section of the blood vessel path is determined at preset intervals;

[0016] Based on the cross-section, determine the contour points of the blood vessel path;

[0017] Based on the contour points, a mesh structure for the blood vessel path is constructed.

[0018] Optionally, the cross-section of the vascular path is determined at preset intervals, specifically including:

[0019] Determine the centerline of the blood vessel path, and select center points on the centerline at preset intervals;

[0020] The cross-section of the blood vessel path is determined based on the center point.

[0021] Optionally, the cross-section of the vascular path passing through each center point is determined, specifically including:

[0022] Determine the tangent direction of each center point on the center line;

[0023] On a plane passing through each center point and perpendicular to the tangent direction of each center point, determine the cross-section of the blood vessel path passing through each center point.

[0024] Optionally, based on the positional relationship between the grid structure and the patch region in the first image, the occlusion relationship between the grid structure and the patch region is obtained, specifically including:

[0025] Determine the grid structure and the mapping direction of the patch region to the second image;

[0026] Based on the mapping direction and the positional relationship between the grid structure and the patch region in the first image, the occlusion relationship between the grid structure and the patch region is obtained, wherein the occlusion relationship includes the depth information of the grid structure and the depth information of the patch region;

[0027] Optionally, according to the occlusion relationship, the mesh structure and the patch region are rendered in the second image to obtain a fused image, specifically including:

[0028] Construct the patch structure of the patch region;

[0029] The patch structure and the grid lines of the mesh structure are used as elements to be rendered;

[0030] Based on the occlusion relationship, determine the unoccluded elements to be rendered from the elements to be rendered;

[0031] The unobstructed elements to be rendered are rendered into the second image to obtain the merged image.

[0032] Optionally, according to the occlusion relationship, the mesh structure and the patch region are rendered in the second image, specifically including:

[0033] Construct the patch structure of the patch region;

[0034] The patch structure and the mesh surface of the mesh structure are used as elements to be rendered;

[0035] Based on the occlusion relationship, determine the unoccluded elements to be rendered from the elements to be rendered;

[0036] The unoccluded elements to be rendered are rendered into the second image according to the depth information to obtain the fused image.

[0037] This specification provides a blood vessel projection visualization device, wherein:

[0038] The acquisition module is used to acquire a three-dimensional image of computed tomography angiography of the target area as a first image, and to acquire a two-dimensional image of digital subtraction angiography of the target area as a second image.

[0039] A mesh structure module is used to determine the blood vessel paths and plaque regions in the first image, and to construct a mesh structure for the blood vessel paths;

[0040] The occlusion relationship module is used to obtain the occlusion relationship between the grid structure and the patch region based on the positional relationship between the grid structure and the patch region in the first image;

[0041] The fusion module is used to render the mesh structure and the patch region in the second image according to the occlusion relationship, so as to obtain a fused image and display it.

[0042] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method for visualizing blood vessel projection.

[0043] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned method for visualizing blood vessel projection.

[0044] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0045] In a vascular projection visualization method provided in this specification, a three-dimensional image of computed tomography angiography and a two-dimensional image of digital subtraction angiography of the target area are acquired and used as the first image and the second image, respectively. The vascular path and plaque region in the first image are determined, and a mesh structure of the vascular path is constructed. Based on the positional relationship between the mesh structure and the plaque region in the first image, the occlusion relationship between the mesh structure and the plaque region is obtained. Based on the occlusion relationship, the mesh structure and the plaque region can be rendered in the second image to obtain a fused image.

[0046] As can be seen from the above method, by constructing a grid structure of the vascular path and rendering the grid structure and the plaque region in the second image based on the grid structure and the occlusion relationship of the plaque region, the fused image not only shows the degree of vascular stenosis, the degree of dilation and the location of the plaque, but also intuitively reflects the positional relationship between the grid structure and the plaque region in the three-dimensional space under the mapping direction of the second image. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and their descriptions and are used to explain this specification, but do not constitute an undue limitation of this specification.

[0048] In the attached diagram:

[0049] Figure 1 is a flowchart illustrating a method for visualizing blood vessel projection provided in this specification;

[0050] Figure 2 is a schematic diagram of a partial cross-section of a blood vessel provided in this specification;

[0051] Figure 3 is a schematic diagram of the connection of each contour point of adjacent center points provided in this specification;

[0052] Figure 4 is a schematic diagram of the grid lines and patch structure in the rendered mesh structure provided in this specification;

[0053] Figure 5 is a schematic diagram of the mesh surface and patch structure in the depth information-based mesh rendering mesh structure provided in this specification;

[0054] Figure 6 is a schematic diagram of a blood vessel projection visualization device provided in this specification;

[0055] Figure 7 is a schematic diagram of the electronic device corresponding to one of the blood vessel projection visualization methods provided in this specification. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0057] In the medical field, the precision and safety of surgery are paramount, especially in the treatment of vascular diseases. Clinically, physicians often need to combine information from computed tomography angiography (CTA) and digital subtraction angiography (DSA) to make more accurate decisions during surgery. For example, in endovascular repair of abdominal aortic aneurysms, CTA can provide the relationship between the aneurysm and the renal artery to determine the optimal stent placement; in cerebrovascular diseases, CTA can provide detailed anatomical information for intracranial aneurysm embolization and endovascular treatment of cerebral vascular stenosis; in coronary artery diseases, CTA can help assess the anatomy and calcification of the coronary arteries, providing important reference for stent implantation. Therefore, this specification does not limit the application of a single vascular projection visualization method to all scenarios of "CTA and DSA combined to obtain fused images," but for ease of description, the following explanation uses coronary artery disease as an example.

[0058] In coronary artery disease, CTA (Continuous Acupuncture and Traumatology) provides three-dimensional images of blood vessels, including the degree of stenosis, dilation, and plaque location, while DSA (Digital Subtraction Angiography) provides two-dimensional images of blood vessels, including the vessel course and guidewires used to guide medical devices. Therefore, based on the vascular projection visualization method provided in this specification, a grid structure of the vascular path is constructed from the three-dimensional CTA image. Based on this grid structure and the occlusion relationship of plaque regions, a fused image is obtained by rendering this grid structure and plaque regions in the two-dimensional DSA image. This image not only displays the degree of stenosis, dilation, and plaque location, but also intuitively reflects the positional relationships between blood vessels and between blood vessels and plaques attached to their surfaces in a specific mapping direction in three-dimensional space during surgery.

[0059] The process of implementing a blood vessel projection visualization method described in this specification involves the processing of image data. Therefore, in the embodiments of this specification, the blood vessel projection visualization process can be performed by a server. Of course, this specification does not limit the type of device that can perform the blood vessel projection visualization process; for example, personal computers and mobile terminals can be used. For ease of description, the following description uses a server as the execution entity. The server can be a single device or a combination of multiple devices, such as a distributed server; this specification does not impose any limitations on this.

[0060] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0061] Figure 1 is a flowchart illustrating a method for visualizing blood vessel projection provided in this specification, including the following steps:

[0062] S100: Acquire a three-dimensional image of computed tomography angiography of the target area as a first image, and acquire a two-dimensional image of digital subtraction angiography of the target area as a second image.

[0063] In one or more embodiments of this specification, in order to project and map the three-dimensional information in the three-dimensional image of computed tomography angiography (CTA) onto the two-dimensional plane of digital subtraction angiography (DSA) in a subsequent step, the server needs to acquire the three-dimensional image of CTA and the two-dimensional image of DSA in this step.

[0064] Specifically, the server can acquire a three-dimensional image of computed tomography angiography of the target area as the first image, and acquire a two-dimensional image of digital subtraction angiography of the target area as the second image.

[0065] S102: Determine the blood vessel paths and plaque regions in the first image, and construct a grid structure for the blood vessel paths.

[0066] In one or more embodiments of this specification, in order to determine the occlusion relationship between the blood vessel path and the plaque region in subsequent steps, in this step, the server needs to determine the blood vessel path and the plaque region in the first image and construct the mesh structure of the blood vessel path.

[0067] Specifically, the server can determine the blood vessel paths and plaque regions in the first image based on the first image, and construct a mesh structure for the blood vessel paths. This specification does not limit the specific method of determining the blood vessel paths and plaque regions in the first image; for example, a pre-trained semantic segmentation model can be used to perform semantic segmentation on the first image to obtain a blood vessel segmentation mask and a plaque segmentation mask. Alternatively, a deep learning network can be used to identify blood vessels and plaques in the first image.

[0068] It should be noted that this manual does not limit the specific method for constructing the mesh structure of the vascular path; it can be set according to actual needs. For example, the Moving Cubes algorithm in the Visualization Toolkit (VTK) can be used. The mesh structure of the vascular path can be constructed using the Moving Cubes algorithm, and then the windowed polygon data filter in VTK can be used to smooth and optimize the mesh structure.

[0069] S104: Based on the positional relationship between the grid structure and the patch region in the first image, obtain the occlusion relationship between the grid structure and the patch region.

[0070] In one or more embodiments of this specification, in order to obtain, in subsequent steps, both the degree of vascular stenosis, the degree of dilation, and the location of plaques, and to intuitively reflect the positional relationship between blood vessels and between blood vessels and plaques attached to their surfaces in a specific mapping direction in three-dimensional space, the server needs to obtain the occlusion relationship between the grid structure and the plaque region based on the positional relationship between the grid structure and the plaque region in the first image.

[0071] Specifically, the server can determine the occlusion relationship between the grid structure and the patch area based on the positional relationship between the grid structure and the patch area in the first image.

[0072] It should be noted that in clinical practice, to facilitate rapid examination of the course of blood vessels and plaque areas in a patient's coronary arteries, surgeons generally perform surgery from a default viewpoint. Therefore, the mapping direction corresponding to the occlusion relationship between the mesh structure and the plaque area can be this default viewpoint. This specification does not limit the specific method for determining the occlusion relationship between the mesh structure and the plaque area; it can be set according to actual needs. For example, the visibility of the mesh structure and the plaque area in the mapping direction of the second image can be determined using ray tracing, thereby determining the occlusion relationship between the mesh structure and the plaque area. In one or more embodiments of this specification, the server can use the projection direction of the second image in the first image as the mapping direction for mapping the mesh structure and the plaque area to the second image. Based on this mapping direction and the positional relationship of the mesh structure in the first image, the occlusion relationship between the mesh structure and the plaque area is obtained, wherein the occlusion relationship includes the depth information of the mesh structure and the depth information of the plaque area. In another embodiment, the server can use the projection direction of the second image in the first image as the mapping direction for mapping the grid structure and patch region to the second image. Based on this mapping direction and the positional relationship of the grid structure in the first image, the server determines the depth information of the grid structure and the depth information of the patch region. Based on the depth information of the grid structure and the patch region, the server determines the occlusion relationship between the grid structure and the patch region. For example, assuming that pixel a has greater depth information in the grid structure than in the patch region under the mapping direction of the grid structure and patch region to the second image, then based on the depth information of the grid structure and the patch region, the occlusion relationship between the grid structure and the patch region is determined to be "the grid structure is occluded by the patch region". A pixel is the smallest unit in the image. The occlusion relationship between the grid structure and the patch region can be the occlusion relationship between each pixel in the grid structure and each pixel in the patch region, or the occlusion relationship between each "preset size region" in the grid structure and each "preset size region" in the patch region. The preset size region can be a region composed of several pixels.

[0073] S106: According to the occlusion relationship, render the mesh structure and the patch region in the second image to obtain a fused image and display it.

[0074] In one or more embodiments of this specification, the mesh structure and patch regions can be rendered in the second image according to the occlusion relationship determined in step S104 to obtain a fused image and display it.

[0075] Specifically, the server can render the mesh structure and patch regions in the second image based on the occlusion relationship determined in step S104, obtain a fused image, and display it. This specification does not limit the specific rendering method; for example, a 3D rendering tool can be used to simulate the emission of point light sources, recording the pixel values ​​of the mesh structure and patch regions in the mapping direction of the second image in 3D space, thereby enabling the rendering of the mesh structure and patch regions in the second image based on the determined occlusion relationship.

[0076] As can be seen from the above method, by constructing a mesh structure of the vascular path and rendering the mesh structure and the plaque region in the second image based on the occlusion relationship between the mesh structure and the plaque region, the positional relationship between the blood vessels and the plaque region is considered when mapping information such as the degree of vascular stenosis, the degree of dilation and the location of the plaque to the second image. This allows the fused image to intuitively reflect the positional relationship between blood vessels and between blood vessels and the plaques attached to their surfaces in a specific mapping direction in three-dimensional space during surgery.

[0077] In addition, before determining the blood vessel path and plaque region based on the first image in step S102, image preprocessing is performed on the first image to standardize its image format and improve its image quality, so as to better identify the blood vessel path and plaque region in the first image subsequently. The image preprocessing process is as follows:

[0078] First, the server can resample the first image to the Radiological Anatomy Index (RAI). The server can preset a first voxel size for resampling the first image. The voxel size determines the spatial resolution of the image. Since different image scanning devices and scanning parameters may cause differences in image resolution, resampling to a uniform first voxel size standardizes the first image, ensuring greater consistency in subsequent image analysis and comparison.

[0079] Secondly, the server can pre-determine the second voxel size to segment the first image. The first image (3D image) contains a large amount of data. Processing the entire first image directly is time-consuming and memory-intensive. Segmenting the image makes it easier to manage computing resources, thereby improving processing efficiency.

[0080] Finally, the server can normalize the first image using preset window width and preset window level, normalizing the voxel values ​​to [0, 1]. That is, window width and window level are parameters used in image processing to adjust image contrast; image contrast normalization helps eliminate differences between different images. This specification does not limit the size of the preset window width and preset window level; they can be set according to actual needs. Since the window width determines the grayscale range of the image display, the window width should include the Hounsfield Unit (HU) value of the target structure. For example, blood vessels are typically between 150 HU and 300 HU, and calcified plaque areas are typically above 300 HU. Therefore, the window width can be 1000 HU to ensure that both blood vessels and calcified plaque areas are well displayed in the image. The window level is the HU value at the center of the window width, determining the image brightness. The window level should be set to the average HU value of the target structure to clearly display the target structure in the image. For example, the average HU value of blood vessels is approximately 250 HU; the window level can be set to 250 HU to clearly display blood vessels.

[0081] In addition, this specification provides a method for determining several cross-sections at preset intervals along the vascular path, then determining the contour points of the vascular path based on each determined cross-section, and constructing the mesh structure of the vascular path based on each determined contour point, as detailed below:

[0082] In one or more embodiments of this specification, the server first determines the centerline of the blood vessel path and selects a number of center points on the centerline at preset intervals, thereby determining the cross-section of the blood vessel path based on each center point. The cross-section is the area where the plane passing through the center point coincides with the blood vessel, and the size of the preset interval is not limited in this specification and can be set according to actual needs. In another embodiment, the determined cross-section is perpendicular to the tangent direction of its corresponding center point on the centerline. That is, for each center point, the server determines the tangent direction of that center point on the centerline and determines the cross-section of the blood vessel path passing through that center point on a plane that passes through the center point and is perpendicular to the tangent direction of that center point.

[0083] As shown in Figure 2, this specification provides the determination of each center point and the cross-section of the blood vessel path passing through each center point. The elliptical surface in the figure is the cross-section of the blood vessel path, and the interval between the center points corresponding to adjacent cross-sections is the preset interval.

[0084] Secondly, the server can determine the contour points of the blood vessel path on the boundaries of each determined cross-section. This specification does not limit the number of contour points determined, nor does it restrict whether the number of contour points on each cross-section is consistent. Finally, the server can use each determined contour point as a vertex constituting the outer surface of the blood vessel path, connecting the vertices to form the outer surface of the blood vessel path as a mesh structure. This specification does not limit the specific method of connecting the vertices, such as quadrilateral mesh generation. After connecting the contour points, the server can also smooth the connected mesh lines to make them smoother and more continuous. Of course, this specification does not limit the specific method of smoothing the connected mesh lines, such as basic spline (B-spline) interpolation.

[0085] In another embodiment, the number of contour points on each cross-section is consistent, meaning the server can select contour points on the boundaries of the cross-sections according to a preset number. Of course, this specification does not limit the specific method of selecting contour points based on a preset number, such as selecting at equal angles, or sequentially selecting contour points of equal arc length on the boundaries of each cross-section according to a preset number. That is, for each cross-section, the server can determine a first interval based on a preset number and the perimeter of the cross-section, and sequentially determine a preset number of contour points on the cross-section based on the first interval. When sequentially determining the preset number of contour points, the server can record the determination order of each contour point. Then, for each center point, according to the determined order, it sequentially connects the contour points corresponding to that center point. For each contour point in the determined order of the center point, from the contour points of adjacent center points, it determines contour points matching the determined order as matching contour points. The contour points in the determined order are then connected to each matching contour point to form triangular facets composed of contour points as vertices. The outer surface of the blood vessel path composed of these triangular facets serves as a mesh structure.

[0086] It should be noted that a triangular facet is a basic unit in 3D geometric modeling, consisting of a planar polygon composed of three vertices. This specification does not restrict the connection based on a predetermined order, determining matching contour points, and then connecting and determining the specific method of the triangular facet. Figure 3 shows a schematic diagram of the connection of contour points between adjacent center points provided in this specification. This figure includes a 3D diagram illustrating one connection method and three extended diagrams illustrating three different connection methods. In the 3D diagram, the determination order of the matching contour points for each contour point is the same predetermined order as the contour point and the next predetermined order. That is, if the determination order of the contour point is 1, then the determination order of the matching contour points for that contour point is 1 and 2. The three extended diagrams illustrating the connection methods from top to bottom are respectively: "The determination order of the matching contour points for each contour point is the same predetermined order as the contour point and the next predetermined order," "The determination order of the matching contour points for each contour point is the same predetermined order as the contour point, the previous predetermined order, and the next predetermined order," and "The determination order of the matching contour points for each contour point is the same predetermined order as the contour point and the previous predetermined order." Of course, the next determined order after the largest determined order is the smallest determined order, and the previous determined order after the smallest determined order is the largest determined order. That is, in this diagram, the next determined order after the determined order of 8 is 1, and the previous determined order after the determined order of 1 is 8.

[0087] In the above process, based on preset intervals and centerlines, the cross-section of the blood vessel path passing through each center point can be accurately determined. Further, based on the tangent direction of each center point on the centerline and a preset number, the cross-section of the blood vessel path passing through each center point is determined, generating a regular mesh structure. This regular mesh structure improves subsequent rendering effects, making the visual effect of the mesh structure closely match the actual blood vessel and more accurately reflecting the degree of vascular stenosis and dilation. Simultaneously, multiple connection methods can be freely selected to connect contour points to obtain triangular facets, offering high flexibility. Furthermore, triangular facets are the simplest polygons, and other polygons may exhibit problems such as planar distortion. The planarity of triangular facets ensures high-quality mesh structures built based on them.

[0088] In addition, this specification also provides details on constructing the patch structure of the patch region, and rendering the patch structure and the mesh lines of the mesh structure, as follows:

[0089] In one or more embodiments of this specification, the server first constructs a patch structure for the patch region determined in step S102. This specification does not limit the specific method of constructing the mesh structure of the patch region; it can be set according to actual needs. For example, the Marching Cubes algorithm in the Visualization Toolkit (VTK) is used to construct the patch structure of the patch region, and then the windowed polygon data filter in VTK can be used to smooth and optimize the patch structure.

[0090] Secondly, the server can use the patch structure and the grid lines of the mesh structure as elements to be rendered, and determine the unoccluded elements to be rendered from the elements to be rendered based on the occlusion relationship between the grid lines and the patch structure. Finally, the server can render the unoccluded elements to be rendered into the second image to obtain the merged image. This specification does not limit the rendering pixel values ​​of the grid lines and the patch structure; these can be set according to actual needs.

[0091] Following the previous example, a mesh structure for the vascular path is constructed. Based on the occlusion relationship between the mesh lines and plaque structures in the mesh structure, the unoccluded mesh lines and unoccluded plaque regions are rendered in the second image to obtain a fused image. Figure 4 shows a schematic diagram of the mesh lines and plaque structures in the rendering mesh structure provided in this specification.

[0092] In the above, based on the occlusion relationship, only the grid lines and plaque structure of the mesh structure are rendered in the second image, which intuitively reflects the positional relationship of the plaques attached to the posterior surface of the blood vessel in a specific mapping direction in three-dimensional space during the operation, thus avoiding the situation where the blood vessel occludes the plaques and causes the doctor to miss them.

[0093] In addition, this specification also provides details on constructing the patch structure of the patch region, and rendering the patch structure and the mesh surfaces of the mesh structure, as follows:

[0094] In one or more embodiments of this specification, the server may first construct a patch structure for the patch region determined in step S102. The specific details of constructing the patch structure will not be elaborated here.

[0095] Secondly, the server can use the patch structure and the mesh surface of the grid structure as elements to be rendered, and determine the unoccluded elements to be rendered from the elements to be rendered based on the occlusion relationship between the mesh surface and the patch structure. Finally, the server can render the unoccluded elements to be rendered into the second image according to the depth information to obtain the merged image. This specification does not limit the rendering pixel values ​​of the mesh surface; these can be set according to actual needs.

[0096] It should be noted that depth information affects the rendered pixel values ​​of the elements to be rendered in the second image. This specification does not limit the mapping relationship between depth information and rendered pixel values, nor the method of determining this mapping relationship. For example, using a 3D rendering tool, the rendering pixel values ​​of the mesh surface under different depth information and the rendering pixel values ​​of the patch structure under different depth information are recorded under a specific mapping direction in 3D space, and based on the recorded "mapping relationship between depth information and rendered pixel values", the mesh surface and the patch structure are rendered in the second image to obtain the fused image.

[0097] Following the previous example, a mesh structure for the vascular pathway is constructed. Based on the occlusion relationship between the mesh surfaces and plaque structures within the mesh structure, the unoccluded mesh surfaces and unoccluded plaque regions are rendered in the second image using depth information to obtain a fused image. Figure 5 shows a schematic diagram of the mesh surfaces and plaque structures rendered based on depth information in the mesh structure provided in this specification.

[0098] In the above, based on the occlusion relationship and depth information, the mesh surface of the mesh structure and the plaque structure are rendered in the second image, which can show the front-to-back relationship between blood vessels and plaque areas, so that doctors can clearly observe the course of blood vessels.

[0099] In addition, this manual also provides rendering methods for calcified and non-calcified plaque areas within the plaque region, as detailed below:

[0100] In one or more embodiments of this specification, the server can render the patch structure of the calcified region using a preset first pixel value. Simultaneously, it can render the patch structure of the non-calcified region using a preset second pixel value and a preset transparency.

[0101] It should be noted that calcified plaque areas increase the complexity and difficulty of revascularization surgery. Therefore, the server can render calcified and non-calcified plaque areas with different pixel values ​​in the second image, making it easier for doctors to distinguish between different types of plaque areas. Furthermore, calcified plaques are often surrounded by non-calcified plaques; therefore, considering the possibility of calcified plaque areas being occluded by non-calcified plaque areas, a preset transparency can be used to render the plaque structure of the non-calcified areas.

[0102] In the above, calcified and non-calcified patch regions are rendered separately using different pixel values ​​and different transparency, so that the patch regions can be clearly displayed in the fused image.

[0103] The above describes a method for visualizing blood vessel projection through one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for visualizing blood vessel projection, as shown in Figure 6.

[0104] The acquisition module 600 acquires a three-dimensional image of computed tomography angiography of the target area as a first image, and acquires a two-dimensional image of digital subtraction angiography of the target area as a second image.

[0105] The mesh structure module 601 determines the blood vessel path and plaque region in the first image and constructs the mesh structure of the blood vessel path.

[0106] The occlusion relationship module 602 obtains the occlusion relationship between the grid structure and the patch region based on the positional relationship between the grid structure and the patch region in the first image;

[0107] The fusion module 603 renders the mesh structure and the patch region in the second image according to the occlusion relationship to obtain and display the fused image.

[0108] Optionally, the mesh structure module 601 is specifically used to determine the cross-section of the blood vessel path at preset intervals; determine the contour points of the blood vessel path based on the cross-section; and construct the mesh structure of the blood vessel path based on the contour points.

[0109] Optionally, the grid structure module 601 is specifically used to determine the centerline of the blood vessel path and select center points on the centerline at preset intervals; and determine the cross-section of the blood vessel path based on the center points.

[0110] Optionally, the mesh structure module 601 is specifically used to determine the tangent direction of each center point on the center line; and to determine the cross-section of the blood vessel path passing through each center point on a plane that passes through each center point and is perpendicular to the tangent direction of each center point.

[0111] Optionally, the occlusion relationship module 602 is specifically used to determine the mapping direction of the mesh structure and the patch region to the second image; and to determine the occlusion relationship between the mesh structure and the patch region based on the positional relationship between the mesh structure and the patch region in the first image and the mapping direction, wherein the occlusion relationship includes the depth information of the mesh structure and the depth information of the patch region.

[0112] Optionally, the fusion module 603 is specifically used to construct the patch structure of the patch region; take the patch structure and the grid lines of the grid structure as elements to be rendered; determine the unoccluded elements to be rendered from the elements to be rendered according to the occlusion relationship; and render the unoccluded elements to be rendered into the second image to obtain the fused image.

[0113] Optionally, the fusion module 603 is specifically used to construct the patch structure of the patch region; take the patch structure and the mesh surface of the mesh structure as elements to be rendered; determine the unoccluded elements to be rendered from the elements to be rendered according to the occlusion relationship; and render the unoccluded elements to be rendered into the second image according to the depth information to obtain the fused image.

[0114] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute a blood vessel projection visualization method provided in Figure 1 above.

[0115] This specification also provides a schematic structural diagram of the electronic device shown in Figure 7. As shown in Figure 7, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the blood vessel projection visualization method of Figure 1. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0116] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog are the most commonly used. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0117] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0118] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0119] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0120] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0124] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0125] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0126] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0127] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0128] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0129] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0130] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0131] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this application.

Claims

1. A method for visualizing blood vessel projection, characterized in that, include: A three-dimensional image of computed tomography angiography of the target area is acquired as the first image, and a two-dimensional image of digital subtraction angiography of the target area is acquired as the second image. Determine the blood vessel paths and plaque regions in the first image, and construct a mesh structure for the blood vessel paths; Based on the grid structure in the first image and the positional relationship of the patch regions, the occlusion relationship between the grid structure and the patch regions is obtained; According to the occlusion relationship, the mesh structure and the patch region are rendered in the second image to obtain a fused image and display it.

2. The method as described in claim 1, characterized in that, Constructing the mesh structure of the vascular pathway specifically includes: The cross-section of the blood vessel path is determined at preset intervals; Based on the cross-section, determine the contour points of the blood vessel path; Based on the contour points, a mesh structure for the blood vessel path is constructed.

3. The method as described in claim 2, characterized in that, Determining the cross-section of the vascular path at preset intervals specifically includes: Determine the centerline of the blood vessel path, and select center points on the centerline at preset intervals; The cross-section of the blood vessel path is determined based on the center point.

4. The method as described in claim 3, characterized in that, Based on the center point, the cross-section of the blood vessel path is determined, specifically including: Determine the tangent direction of each center point on the center line; On a plane passing through each center point and perpendicular to the tangent direction of each center point, determine the cross-section of the blood vessel path passing through each center point.

5. The method as described in claim 1, characterized in that, Based on the grid structure and the positional relationship of the patch regions in the first image, the occlusion relationship between the grid structure and the patch regions is obtained, specifically including: Determine the grid structure and the mapping direction of the patch region to the second image; Based on the positional relationship between the mesh structure and the patch region in the first image, and the mapping direction, the occlusion relationship between the mesh structure and the patch region is determined, wherein the occlusion relationship includes the depth information of the mesh structure and the depth information of the patch region.

6. The method as described in claim 1, characterized in that, According to the occlusion relationship, the mesh structure and the patch region are rendered in the second image to obtain a fused image, specifically including: Construct the patch structure of the patch region; The patch structure and the grid lines of the mesh structure are used as elements to be rendered; Based on the occlusion relationship, determine the unoccluded elements to be rendered from the elements to be rendered; The unobstructed elements to be rendered are rendered into the second image to obtain the merged image.

7. The method as described in claim 5, characterized in that, According to the occlusion relationship, rendering the mesh structure and the patch region in the second image specifically includes: Construct the patch structure of the patch region; The patch structure and the mesh surface of the mesh structure are used as elements to be rendered; Based on the occlusion relationship, determine the unoccluded elements to be rendered from the elements to be rendered; The unoccluded elements to be rendered are rendered into the second image according to the depth information to obtain the fused image.

8. A blood vessel projection visualization device, characterized in that, in: The acquisition module is used to acquire a three-dimensional image of computed tomography angiography of the target area as a first image, and to acquire a two-dimensional image of digital subtraction angiography of the target area as a second image. A mesh structure module is used to determine the blood vessel paths and plaque regions in the first image, and to construct a mesh structure for the blood vessel paths; The occlusion relationship module is used to obtain the occlusion relationship between the grid structure and the patch region based on the positional relationship between the grid structure and the patch region in the first image; The fusion module is used to render the mesh structure and the patch region in the second image according to the occlusion relationship, so as to obtain a fused image and display it.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 7.

Images