Vascular projection visualization method and program product

Abstract

In the blood vessel projection visualization method and the program product provided by the invention, a first image of computed tomography angiography and a second image of digital subtraction angiography in a target area are acquired, a blood vessel path and a plaque area in the first image are determined, and a grid structure of the blood vessel path is constructed; and according to the position relationship between the grid structure and the plaque region in the first image, obtaining the shielding relationship between the grid structure and the plaque region, and based on the shielding relationship, rendering the grid structure and the plaque region in the second image to obtain a fused image. By constructing the grid structure of the blood vessel path and rendering the grid structure and the plaque region in the second image based on the shielding relationship between the grid structure and the plaque region to obtain the fused image, the blood vessel stenosis degree, the blood vessel dilation degree and the plaque position are displayed, and the blood vessel stenosis degree and the blood vessel dilation degree are displayed at the same time. And the position relationship between the grid structure and the plaque region in the mapping direction of the second image is intuitively reflected.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a blood vessel projection visualization method and a program product. Background Art

[0002] Computed Tomography Angiography (CTA) is a non-invasive imaging method that provides a three-dimensional image of the blood vessels, which contains information such as the degree of vascular stenosis, vascular dilation, and plaque location. Digital Subtraction Angiography (DSA) is an invasive examination method that obtains an X-ray image after injecting a contrast agent into the blood vessels. The X-ray image contains information such as the shape of the blood vessels and the guidewires that guide medical devices in the blood vessels.

[0003] In clinical practice, the 3D segmentation mask of CTA is often projected and overlaid on the 2D plane of DSA to map the information such as the degree of vascular stenosis / dilation and the location of plaques in CTA onto the 2D plane of DSA, so that doctors can understand the degree of vascular stenosis, dilation and location of plaques in the 2D plane of DSA. However, this method cannot reflect the positional relationship between plaques and blood vessels.

[0004] To this end, this specification provides a blood vessel projection visualization method and a program product. Summary of the invention

[0005] This specification provides a method and a program product to partially solve the above problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a blood vessel projection visualization method, comprising:

[0008] Acquiring a three-dimensional image of a computer tomography angiography of a target area as a first image, and acquiring a two-dimensional image of a digital subtraction angiography of the target area as a second image;

[0009] Determine a blood vessel path and a plaque area in the first image, and construct a grid structure of the blood vessel path;

[0010] Obtaining an occlusion relationship between the grid structure and the patch area according to a positional relationship between the grid structure and the patch area in the first image;

[0011] According to the occlusion relationship, the grid structure and the patch area are rendered in the second image to obtain a fused image and display it.

[0012] Optionally, constructing the grid structure of the blood vessel path specifically includes:

[0013] determining a cross section of the vascular path at predetermined intervals;

[0014] determining contour points of the blood vessel path according to the cross section;

[0015] A grid structure of the blood vessel path is constructed according to the contour points.

[0016] Optionally, determining the cross section of the blood vessel path at preset intervals specifically includes:

[0017] Determining a centerline of the vascular path and selecting center points on the centerline at preset intervals;

[0018] Based on the center point, a cross section of the blood vessel path is determined.

[0019] Optionally, determining the cross section of the blood vessel path passing through each center point specifically includes:

[0020] Determine the tangent direction of each center point on the center line;

[0021] On a plane passing through each center point and perpendicular to the tangent direction of each center point, a cross section of the blood vessel path passing through each center point is determined.

[0022] Optionally, obtaining an occlusion relationship between the grid structure and the patch area according to a positional relationship between the grid structure and the patch area in the first image specifically includes:

[0023] Determine a mapping direction in which the grid structure and the patch area are mapped to the second image;

[0024] Obtaining an occlusion relationship between the grid structure and the patch area according to the mapping direction and the positional relationship between the grid structure and the patch area in the first image, wherein the occlusion relationship includes depth information of the grid structure and depth information of the patch area;

[0025] Optionally, rendering the grid structure and the patch area in the second image according to the occlusion relationship to obtain a fused image specifically includes:

[0026] constructing a plaque structure of the plaque area;

[0027] Using the patch structure and the grid lines of the grid structure as elements to be rendered;

[0028] Determining, from the elements to be rendered, unobstructed elements to be rendered according to the obstruction relationship;

[0029] The unobstructed elements to be rendered are rendered into the second image to obtain a fused image.

[0030] Optionally, rendering the grid structure and the patch area in the second image according to the occlusion relationship specifically includes:

[0031] constructing a plaque structure of the plaque area;

[0032] Using the patch structure and the grid surface of the grid structure as elements to be rendered;

[0033] Determining, from the elements to be rendered, unobstructed elements to be rendered according to the obstruction relationship;

[0034] The unobstructed elements to be rendered are rendered into the second image according to the depth information to obtain a fused image.

[0035] This specification provides a blood vessel projection visualization device, wherein:

[0036] an acquisition module, configured to acquire a three-dimensional image of a computer tomography angiography of a target area as a first image, and to acquire a two-dimensional image of a digital subtraction angiography of the target area as a second image;

[0037] A grid structure module, used to determine the blood vessel path and the plaque area in the first image, and construct a grid structure of the blood vessel path;

[0038] An occlusion relationship module, configured to obtain an occlusion relationship between the grid structure and the patch area according to a positional relationship between the grid structure and the patch area in the first image;

[0039] A fusion module is used to render the grid structure and the patch area in the second image according to the occlusion relationship to obtain and display a fused image.

[0040] This specification provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned blood vessel projection visualization method is implemented.

[0041] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned blood vessel projection visualization method when executing the program.

[0042] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0043] In a blood vessel projection visualization method provided in the present specification, a three-dimensional image of computed tomography angiography and a two-dimensional image of digital subtraction angiography of a target area are obtained as a first image and a second image, respectively, the blood vessel path and the plaque area in the first image are determined, and a grid structure of the blood vessel path is constructed. According to the positional relationship between the grid structure and the plaque area in the first image, an occlusion relationship between the grid structure and the plaque area is obtained, so that based on the occlusion relationship, the grid structure and the plaque area can be rendered in the second image to obtain a fused image.

[0044] It can be seen from the above method that by constructing a grid structure of the vascular path and rendering the grid structure and the plaque area in the second image based on the occlusion relationship between the grid structure and the plaque area, a fused image is obtained. While showing the degree of stenosis, dilation and plaque location of the blood vessel, it intuitively reflects the positional relationship between the grid structure and the plaque area in the mapping direction of the second image in three-dimensional space. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide further understanding of this specification and constitute a part of this specification. The illustrative embodiments of this specification and their descriptions are used to explain this specification and do not constitute improper limitations on this specification.

[0046] In the attached picture:

[0047] Figure 1 A schematic diagram of a flow chart of a blood vessel projection visualization method provided in this specification;

[0048] Figure 2 A schematic diagram of a cross section of a portion of a blood vessel provided in this specification;

[0049] Figure 3 A schematic diagram of connecting contour points of adjacent center points provided in this specification;

[0050] Figure 4 A schematic diagram of a grid line and a patch structure in a rendering grid structure provided in this specification;

[0051] Figure 5 A schematic diagram of a mesh surface and a patch structure in a mesh structure rendered based on depth information provided in this specification;

[0052] Figure 6 A schematic diagram of a blood vessel projection visualization device provided in this specification;

[0053] Figure 7 The diagram is a structural diagram of an electronic device corresponding to a blood vessel projection visualization method provided in this specification. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] In the medical field, the accuracy and safety of surgery are crucial, especially when dealing with vascular-related diseases. In clinical practice, doctors often need to combine information from computed tomography angiography (CTA) and digital subtraction angiography (DSA) to make more accurate decisions during surgery. For example, in the endovascular repair of abdominal aortic aneurysm, CTA can provide the relationship between the aneurysm and the renal artery to determine the optimal stent position; in cerebrovascular disease, CTA can provide detailed anatomical information for intracranial aneurysm embolization and endovascular treatment of cerebral vascular stenosis; in coronary artery disease, CTA can help evaluate the anatomical structure and degree of calcification of the coronary artery, providing an important reference for stent implantation. Therefore, in this specification, a vascular projection visualization method is not limited to all application scenarios of "combining CTA and DSA to obtain a fused image", but for the sake of convenience, the following is an example of coronary artery disease.

[0056] In coronary artery disease, the three-dimensional image of CTA can provide information such as the degree of vascular stenosis, the degree of vascular dilation and the location of plaques, and the two-dimensional image of DSA can provide information such as the shape of the blood vessels and the guidewires that guide medical devices in the blood vessels. Therefore, based on a blood vessel projection visualization method provided in this specification, a grid structure of the blood vessel path is constructed according to the three-dimensional image of CTA, and based on the occlusion relationship between the grid structure and the plaque area, the grid structure and the plaque area are rendered in the two-dimensional image of DSA to obtain a fused image, which, while showing the degree of vascular stenosis, dilation and plaque location, intuitively reflects the positional relationship between blood vessels and between blood vessels and plaques attached to their surfaces in a specific mapping direction in three-dimensional space during surgery.

[0057] In this specification, the process of executing a blood vessel projection visualization method involves processing image data. Therefore, in the embodiments of this specification, the blood vessel projection visualization process can be executed by a server. Of course, this specification does not limit which device executes the blood vessel projection visualization process, for example, a personal computer and a mobile terminal perform blood vessel projection visualization. For the convenience of description, the following description is based on the server as the execution subject, wherein the server can be a single device, or composed of multiple devices, for example, a distributed server, and this specification does not limit this.

[0058] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0059] Figure 1 The flowchart of a blood vessel projection visualization method provided in this specification includes the following steps:

[0060] S100: Acquire a three-dimensional image of a target area by computer tomography angiography as a first image, and acquire a two-dimensional image of a digital subtraction angiography of the target area as a second image.

[0061] In one or more embodiments of the present specification, in order to realize the projection mapping of the three-dimensional information in the three-dimensional image of Computed Tomography Angiography (CTA) onto the two-dimensional plane of Digital Subtraction Angiography (DSA) in the subsequent steps, in this step, the server needs to obtain the three-dimensional image of CTA and the two-dimensional image of DSA.

[0062] Specifically, the server may obtain a three-dimensional image of a computer tomography angiography of the target area as a first image, and obtain a two-dimensional image of a digital subtraction angiography of the target area as a second image.

[0063] S102: Determine the blood vessel path and the plaque area in the first image, and construct a grid structure of the blood vessel path.

[0064] In one or more embodiments of the present specification, in order to determine the occlusion relationship between the vascular path and the plaque area in subsequent steps, in this step, the server needs to determine the vascular path and the plaque area in the first image and construct a grid structure of the vascular path.

[0065] Specifically, the server can determine the blood vessel path and plaque area in the first image based on the first image, and construct a grid structure of the blood vessel path. The present specification does not limit the specific method of determining the blood vessel path and plaque area in the first image based on the first image, such as using a pre-trained semantic segmentation model to perform semantic segmentation on the first image to obtain a blood vessel segmentation mask and a plaque segmentation mask. Alternatively, a deep learning network is used to identify blood vessels and plaques in the first image.

[0066] It should be noted that the specific method of constructing the mesh structure of the vascular path is not limited in this specification and can be set according to actual needs. For example, the Marching Cubes algorithm in the Visualization Toolkit (VTK). That is, the Marching Cubes algorithm is used to construct the mesh structure of the vascular path, and then the windowed polygon data filter in VTK can be used to smooth and optimize the mesh structure.

[0067] S104: Obtaining an occlusion relationship between the grid structure and the patch area according to a positional relationship between the grid structure and the patch area in the first image.

[0068] In one or more embodiments of the present specification, in order to obtain in subsequent steps both the degree of stenosis, dilation and location of the blood vessels and the location of the plaques attached to their surfaces in a specific mapping direction in three-dimensional space and to display the degree of stenosis, dilation and location of the blood vessels and to intuitively reflect the positional relationship between the blood vessels and the plaques attached to their surfaces during surgery, in this step, the server needs to obtain the occlusion relationship between the grid structure and the plaque area based on the positional relationship between the grid structure and the plaque area in the first image.

[0069] Specifically, the server may obtain the occlusion relationship between the grid structure and the patch area according to the positional relationship between the grid structure and the patch area in the first image.

[0070] It should be noted that in clinical practice, in order to facilitate rapid exploration of the course of blood vessels and plaque areas in the patient's coronary arteries, doctors generally perform surgery under the default viewing angle. Therefore, the mapping direction corresponding to the occlusion relationship between the grid structure and the plaque area can be the default viewing angle. The specific method for determining the occlusion relationship between the grid structure and the plaque area is not limited in this specification and can be set according to actual needs. For example, the visibility of the grid structure and the plaque area in the mapping direction of the second image is determined by a ray tracing method, thereby determining the occlusion relationship between the grid structure and the plaque area. In one or more embodiments of the present specification, the server may use the projection direction of the second image in the first image as the mapping direction of the grid structure and the plaque area to the second image, and obtain the occlusion relationship between the grid structure and the plaque area based on the mapping direction and the positional relationship of the grid structure in the first image, wherein the occlusion relationship includes the depth information of the grid structure and the depth information of the plaque area. In another embodiment, the server may use the projection direction of the second image in the first image as the mapping direction of the grid structure and the patch area to the second image, determine the depth information of the grid structure and the depth information of the patch area based on the mapping direction and the positional relationship of the grid structure in the first image, and determine the occlusion relationship between the grid structure and the patch area according to the depth information of the grid structure and the depth information of the patch area. For example, assuming that the depth information of pixel a in the grid structure is greater than the depth information of the patch area under the mapping direction of the grid structure and the patch area to the second image, then according to the depth information of the grid structure and the depth information of the patch area, the occlusion relationship between the grid structure and the patch area is determined as "the grid structure is occluded by the patch area". A pixel is the smallest unit in an image, that is, the occlusion relationship between the grid structure and the patch area may be an occlusion relationship between each pixel in the grid structure and each pixel in the patch area, or an occlusion relationship between each "preset size area" in the grid structure and each "preset size area" in the patch area, and the preset size area may be an area composed of several pixels.

[0071] S106: Rendering the grid structure and the patch area in the second image according to the occlusion relationship to obtain and display a fused image.

[0072] In one or more embodiments of the present specification, the grid structure and the patch area may be rendered in the second image according to the occlusion relationship determined in step S104 to obtain and display a fused image.

[0073] Specifically, the server can render the grid structure and the patch area in the second image according to the occlusion relationship determined in step S104, obtain a fused image and display it. The specific rendering method is not limited in this specification, for example, by using a three-dimensional rendering tool to simulate the light emission of a point light source, record the pixel values ​​of the grid structure and the patch area in the mapping direction of the second image in three-dimensional space, so that the grid structure and the patch area can be rendered in the second image based on the determined occlusion relationship.

[0074] It can be seen from the above method that by constructing a grid structure of the vascular path and rendering the grid structure and the plaque area in the second image based on the occlusion relationship between the grid structure and the plaque area, the positional relationship between the blood vessels and the plaque area is taken into account when mapping information such as the degree of stenosis, dilation and plaque position of the blood vessels to the second image, so that the fused image can intuitively reflect the positional relationship between the blood vessels during surgery and between the blood vessels and the plaques attached to their surfaces in a specific mapping direction in three-dimensional space.

[0075] In addition, before determining the vascular path and plaque area based on the first image in step S102, the first image is subjected to image preprocessing to standardize the image format of the first image and improve the image quality of the first image, so as to better identify the vascular path and plaque area in the first image later. The image preprocessing process is as follows:

[0076] First, the server may resample the first image to an anatomical reference coordinate system (RAI, RadiologicalAnatomy Index). The server may resample the first image by presetting a first voxel size. The voxel size determines the spatial resolution of the image. Since different image scanning devices and scanning parameters may result in differences in image resolution, resampling to a uniform first voxel size and standardizing the first image can ensure that subsequent image analysis and comparison are more consistent.

[0077] Secondly, the server can preset the second voxel size to slice the first image. The first image (three-dimensional image) contains a large amount of data. Directly processing the entire first image is time-consuming and memory-intensive. Slicing facilitates management of computing resources, thereby improving processing efficiency.

[0078] Finally, the server can normalize the first image by preset window width and preset window position, so that the value of the voxel is normalized to [0, 1], that is, the window width and window position are parameters used to adjust the image contrast in image processing, and the normalization of image contrast helps to eliminate the differences between different images. The size of the preset window width and preset window position is not limited in this specification, and can be set according to actual needs. Since the window width determines the grayscale range of the image display, the window width should include the Hounsfield Unit (HU) value of the target structure. For example, blood vessels are usually 150HU to 300HU, and calcified plaque areas are usually above 300HU. Therefore, the window width can be 1000HU to ensure that both blood vessels and calcified plaque areas are well displayed in the image. The window position is the HU value at the center of the window width, which determines the brightness of the image. The window position should be set to the average HU value of the target structure so that the target structure can be clearly displayed in the image. For example, the average HU value of the blood vessel is about 250HU, and the window position can be set to 250HU to clearly display the blood vessel.

[0079] In addition, this specification provides the contents of determining a plurality of cross sections at preset intervals on the vascular path, determining contour points of the vascular path according to the determined cross sections, and constructing a grid structure of the vascular path according to the determined contour points, as follows:

[0080] In one or more embodiments of the present specification, first, the server may determine the center line of the vascular path, and select a number of center points on the center line at preset intervals, so that the cross section of the vascular path can be determined based on each center point. The cross section is the area where the plane passing through the center point coincides with the blood vessel, and the size of the preset interval is not limited in the present specification and can be set according to actual needs. In another embodiment, the determined cross section is perpendicular to the tangent direction of the corresponding center point on the center line. That is, the server can determine the tangent direction of each center point on the center line, and determine the cross section of the vascular path passing through the center point on a plane passing through the center point and perpendicular to the tangent direction of the center point.

[0081] like Figure 2 As shown, the determination of each center point and the cross-section of the vascular path passing through each center point provided in this specification, the elliptical surface in the figure is the cross-section of the vascular path, and the interval between the center points corresponding to adjacent cross-sections is the preset interval.

[0082] Secondly, the server may determine the contour points of the vascular path on the boundaries of each determined cross section. This specification does not limit the number of contour points to be determined, and whether the number of contour points on each cross section is consistent. Finally, the server may use the determined contour points as vertices constituting the outer surface of the vascular path, connect the vertices, and use the outer surface of the vascular path constituted by the vertices as a grid structure. Among them, this specification does not limit the specific method of connecting the vertices, such as quadrilateral mesh generation. After connecting the contour points, the server may also make the grid lines smoother and more continuous by smoothing the connected grid lines. Of course, this specification does not limit the specific method of smoothing the connected grid lines, such as basic spline (B-spline) interpolation.

[0083] In another embodiment, the number of contour points on each cross section is the same, that is, the server can select contour points on the boundary of the cross section according to a preset number. Of course, the present specification does not limit the specific method of selecting contour points based on the preset number, such as selecting at equal angles, or selecting at equal arc lengths on the boundary of each cross section in sequence according to the preset number, that is, the server can determine a first interval for each cross section according to the preset number and the perimeter of the cross section, and determine a preset number of contour points on the cross section in sequence according to the first interval. When determining the preset number of contour points in sequence, the server can record the determination order of each contour point, and then for each center point, in accordance with the determination order, sequentially connect the contour points corresponding to the center point, and for each contour point of the center point in the determined order, determine the contour point that matches the determined order from the contour points of the adjacent center points according to the determined order as the matching contour point, and respectively connect the contour point of the determined order with each matching contour point to form a triangular facet composed of the contour points as vertices, and use the outer surface of the blood vessel path composed of the triangular facet as a mesh structure.

[0084] It should be noted that a triangular face is a basic unit in three-dimensional geometric modeling and is a planar polygon composed of three vertices. This specification does not limit the specific method of connecting based on determining the order, determining the matching contour points, and then connecting to determine the triangular face. Figure 3As shown, it is a schematic diagram of connecting each contour point of adjacent center points provided in this specification, which includes a stereogram illustrating a connection method and three extended diagrams illustrating the connection method. The determination order of the matching contour points of each contour point in the stereogram is the same determination order and the next determination order of the contour point, that is, if the determination order of the contour point is 1, then the determination order of the matching contour points of the contour point is 1 and 2. From top to bottom, the three extended diagrams illustrating the connection method are respectively "the determination order of the matching contour points of each contour point is the same determination order and the next determination order of the contour point", "the determination order of the matching contour points of each contour point is the same determination order, the previous determination order and the next determination order of the contour point", and "the determination order of the matching contour points of each contour point is the same determination order and the previous determination order of the contour point". Of course, the next determination order of the largest determination order is the smallest determination order, and the previous determination order of the smallest determination order is the largest determination order, that is, in the figure, the next determination order of the determination order of 8 is 1, and the previous determination order of the determination order of 1 is 8.

[0085] In the above, based on the preset interval and the center line, the cross section of the blood vessel path passing through each center point can be accurately determined. Further, according to the tangent direction of each center point on the center line and the preset number, the cross section of the blood vessel path passing through each center point can be determined, and a regular grid structure can be generated. The regular grid structure can improve the subsequent rendering effect, so that the visual effect of the grid structure fits the actual blood vessel and more accurately reflects the degree of vascular stenosis and vascular dilation. At the same time, a variety of connection methods can be freely selected to connect the contour points to obtain triangular facets, which is highly flexible. At the same time, triangular facets are the simplest polygons, and other polygons may have problems such as plane distortion, while the planarity of triangular facets makes the quality of the grid structure based on triangular facets high.

[0086] In addition, this specification also provides content for constructing a patch structure of a patch area and rendering the patch structure and the grid lines of the grid structure, as follows:

[0087] In one or more embodiments of the present specification, first, the server may construct a plaque structure of the plaque area based on the plaque area determined in step S102. The present specification does not limit the specific method of constructing the grid structure of the plaque area, which can be set according to actual needs. For example, the marching cubes algorithm in the visualization toolkit (VTK) uses the marching cubes algorithm to construct the plaque structure of the plaque area, and then the windowed polygon data filter in VTK can be used to smooth and optimize the plaque structure.

[0088] Secondly, the server may use the grid lines of the patch structure and the grid structure as elements to be rendered, and determine the unobstructed elements to be rendered from the elements to be rendered according to the occlusion relationship between the grid lines in the grid structure and the patch structure. Finally, the server may render the unobstructed elements to be rendered into the second image to obtain a fused image. The rendering pixel values ​​of the grid lines and the rendering pixel values ​​of the patch structure are not limited in this specification and can be set according to actual needs.

[0089] Following the above example, the grid structure of the blood vessel path is constructed, and based on the occlusion relationship between the grid lines and the plaque structure in the grid structure, the unoccluded grid lines and unoccluded plaque areas are rendered in the second image to obtain a fused image. Figure 4 The figure shows a schematic diagram of the grid lines and patch structures in the rendering grid structure provided in this specification.

[0090] In the above, based on the occlusion relationship, only the grid lines of the grid structure and the plaque structure are rendered in the second image, which intuitively reflects the positional relationship of the plaque attached to the posterior surface of the blood vessel during surgery in a specific mapping direction in three-dimensional space, avoiding the situation where the doctor misses the plaque due to the blood vessel occluding the plaque.

[0091] In addition, this specification also provides content for constructing a patch structure of a patch area and rendering the patch structure and the mesh surface of the mesh structure, as follows:

[0092] In one or more embodiments of the present specification, first, the server may construct a plaque structure of the plaque area based on the plaque area determined in step S102. The specific content of constructing the plaque structure is not repeated here.

[0093] Secondly, the server can use the patch structure and the grid surface of the grid structure as the elements to be rendered, and determine the unobstructed elements to be rendered from the elements to be rendered according to the occlusion relationship between the grid surface and the patch structure in the grid structure. Finally, the server can render the unobstructed elements to be rendered into the second image according to the depth information to obtain a fused image. The rendering pixel value of the grid surface is not limited in this specification and can be set according to actual needs.

[0094] It should be noted that the depth information affects the rendering pixel values ​​of the elements to be rendered in the second image. The mapping relationship between the depth information and the rendering pixel values, as well as the method for determining the mapping relationship, are not limited in this specification. For example, by using a three-dimensional rendering tool to simulate the way point light sources emit light, the rendering pixel values ​​of the mesh surface at different depth information and the rendering pixel values ​​of the patch structure at different depth information in a specific mapping direction in three-dimensional space are recorded, and based on the recorded "mapping relationship between depth information and rendering pixel values", the mesh surface and the patch structure are rendered in the second image to obtain a fused image.

[0095] Using the above example, the grid structure of the blood vessel path is constructed, and based on the occlusion relationship between the grid surface and the plaque structure in the grid structure, the unoccluded grid surface and the unoccluded plaque area are rendered in the second image based on the depth information to obtain a fused image. Figure 5 As shown, it is a schematic diagram of rendering the mesh surface and the patch structure in the mesh structure based on the depth information provided in this specification.

[0096] In the above, based on the occlusion relationship and depth information, the grid surface of the grid structure and the plaque structure are rendered in the second image, which can show the front-to-back relationship between the blood vessel and the plaque area, so that the doctor can clearly observe the shape of the blood vessel.

[0097] In addition, this specification also provides content for rendering the calcified plaque area and the non-calcified plaque area in the plaque area, as follows:

[0098] In one or more embodiments of the present specification, the server may render the plaque structure of the calcified area with a preset first pixel value, and at the same time, render the plaque structure of the non-calcified area with a preset second pixel value and a preset transparency.

[0099] It should be noted that in revascularization surgery, calcified plaque areas will increase the complexity and difficulty of the surgery. Therefore, the server can render the calcified plaque area and the non-calcified plaque area with different pixel values ​​in the second image, which can make it easier for doctors to distinguish different types of plaque areas. At the same time, calcified plaques are often wrapped by non-calcified plaques, that is, considering the situation where the calcified plaque area is blocked by the non-calcified plaque area, the plaque structure of the non-calcified area can be rendered with a preset transparency.

[0100] In the above, the calcified plaque area and the non-calcified plaque area are rendered respectively by different pixel values ​​and different transparency, so that the plaque area can be clearly displayed in the fused image.

[0101] The above is a blood vessel projection visualization method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding blood vessel projection visualization device, such as Figure 6 shown.

[0102] An acquisition module 600 acquires a three-dimensional image of a target area by computer tomography angiography as a first image, and acquires a two-dimensional image of a digital subtraction angiography of the target area as a second image;

[0103] A grid structure module 601 determines a blood vessel path and a plaque area in the first image, and constructs a grid structure of the blood vessel path;

[0104] An occlusion relationship module 602 obtains an occlusion relationship between the grid structure and the patch area according to a positional relationship between the grid structure and the patch area in the first image;

[0105] The fusion module 603 renders the grid structure and the patch area in the second image according to the occlusion relationship to obtain and display a fused image.

[0106] Optionally, the grid structure module 601 is specifically used to determine the cross-section of the vascular path at preset intervals; determine the contour points of the vascular path according to the cross-section; and construct the grid structure of the vascular path according to the contour points.

[0107] Optionally, the grid structure module 601 is specifically used to determine the center line of the blood vessel path, and select center points on the center line at preset intervals; and determine the cross section of the blood vessel path based on the center points.

[0108] Optionally, the grid structure module 601 is specifically used to determine the tangent direction of each center point on the center line; and determine the cross-section of the blood vessel path passing through each center point on a plane passing through each center point and perpendicular to the tangent direction of each center point.

[0109] Optionally, the occlusion relationship module 602 is specifically used to determine the mapping direction of the grid structure and the patch area to the second image; based on the positional relationship between the grid structure and the patch area in the first image, and the mapping direction, determine the occlusion relationship between the grid structure and the patch area, wherein the occlusion relationship includes the depth information of the grid structure and the depth information of the patch area.

[0110] Optionally, the fusion module 603 is specifically used to construct a patch structure of the patch area; use the patch structure and the grid lines of the grid structure as elements to be rendered; determine unobstructed elements to be rendered from the elements to be rendered according to the occlusion relationship; render the unobstructed elements to be rendered into the second image to obtain a fused image.

[0111] Optionally, the fusion module 603 is specifically used to construct a patch structure of the patch area; use the patch structure and the grid surface of the grid structure as elements to be rendered; determine unobstructed elements to be rendered from the elements to be rendered according to the occlusion relationship; render the unobstructed elements to be rendered into the second image according to the depth information to obtain a fused image.

[0112] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A blood vessel projection visualization method is provided.

[0113] This manual also provides Figure 7 The schematic structure diagram of the electronic device shown in FIG. Figure 7 As shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0114] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0115] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0116] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0117] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0118] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0119] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0120] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0122] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0123] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0124] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0125] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0126] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0128] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0129] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A blood vessel projection visualization method, characterized in that: include: Acquiring a three-dimensional image of a computer tomography angiography of a target area as a first image, and acquiring a two-dimensional image of a digital subtraction angiography of the target area as a second image; determining a blood vessel path and a plaque area in the first image, and constructing a grid structure of the blood vessel path; Obtaining an occlusion relationship between the grid structure and the patch area according to a positional relationship between the grid structure and the patch area in the first image; According to the occlusion relationship, the grid structure and the patch area are rendered in the second image to obtain a fused image and display it.

2. The method according to claim 1, characterized in that Constructing the grid structure of the blood vessel path specifically includes: determining a cross section of the vascular path at predetermined intervals; determining contour points of the blood vessel path according to the cross section; A grid structure of the blood vessel path is constructed according to the contour points.

3. The method according to claim 2, characterized in that Determining the cross-section of the blood vessel path at preset intervals specifically includes: Determining a centerline of the vascular path and selecting center points on the centerline at preset intervals; Based on the center point, a cross section of the blood vessel path is determined.

4. The method according to claim 3, characterized in that Determining the cross section of the blood vessel path according to the center point specifically includes: Determine the tangent direction of each center point on the center line; On a plane passing through each center point and perpendicular to the tangent direction of each center point, a cross section of the blood vessel path passing through each center point is determined.

5. The method according to claim 1, characterized in that: Obtaining an occlusion relationship between the grid structure and the patch area according to the positional relationship between the grid structure and the patch area in the first image specifically includes: Determine a mapping direction in which the grid structure and the patch area are mapped to the second image; According to the positional relationship between the grid structure and the patch area in the first image and the mapping direction, an occlusion relationship between the grid structure and the patch area is determined, wherein the occlusion relationship includes depth information of the grid structure and depth information of the patch area.

6. The method according to claim 1, characterized in that According to the occlusion relationship, rendering the grid structure and the patch area in the second image to obtain a fused image specifically includes: constructing a plaque structure of the plaque area; Using the patch structure and the grid lines of the grid structure as elements to be rendered; According to the occlusion relationship, determining unoccluded elements to be rendered from the elements to be rendered; The unobstructed elements to be rendered are rendered into the second image to obtain a fused image.

7. The method according to claim 5, characterized in that Rendering the grid structure and the patch area in the second image according to the occlusion relationship specifically includes: constructing a plaque structure of the plaque area; Using the patch structure and the grid surface of the grid structure as elements to be rendered; Determining, from the elements to be rendered, unobstructed elements to be rendered according to the obstruction relationship; The unobstructed elements to be rendered are rendered into the second image according to the depth information to obtain a fused image.

8. A blood vessel projection visualization device, characterized in that: in: an acquisition module, configured to acquire a three-dimensional image of a computer tomography angiography of a target area as a first image, and to acquire a two-dimensional image of a digital subtraction angiography of the target area as a second image; A grid structure module, used to determine the blood vessel path and the plaque area in the first image, and construct a grid structure of the blood vessel path; An occlusion relationship module, configured to obtain an occlusion relationship between the grid structure and the patch area according to a positional relationship between the grid structure and the patch area in the first image; A fusion module is used to render the grid structure and the patch area in the second image according to the occlusion relationship to obtain a fused image and display it.

9. A computer program product, characterized in that When the computer program product is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the method according to any one of claims 1 to 7 is implemented when the processor executes the program.

Images