A method and device for detecting coronary artery stenosis and plaque.

Abstract

This application provides a method and apparatus for detecting coronary artery stenosis and plaque, comprising: inputting a coronary angiography image into a pre-trained coronary artery inner and outer diameter segmentation model to obtain an inner and outer diameter segmentation result image; extracting the centerline of the coronary artery and obtaining a surface reconstruction image based on the centerline; determining the stenotic intervals on the coronary artery based on the inner and outer diameter segmentation result image; for each stenotic interval, inputting the corresponding interval surface reconstruction image from the surface reconstruction image into a pre-trained plaque classification model to determine the calcified plaque classification result and the non-calcified plaque classification result, and comprehensively determining the plaque type. Thus, by accurately segmenting the inner and outer diameters of the coronary artery using the coronary artery inner and outer diameter segmentation model, the stenotic interval can be quickly and accurately located; by decomposing the plaque classification task into two sub-tasks and combining the calcified plaque classification results and the non-calcified plaque classification results obtained from the two sub-tasks, the plaque type of the stenotic interval can be determined more accurately.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method and device for detecting coronary artery stenosis and plaque. Background Technology

[0002] In existing technologies, the task of identifying plaque and stenosis in blood vessels is usually performed through visual assessment, or automatically by extracting a straightened image of the entire centerline from the coronary artery segmentation results to obtain surface CPR, and then using neural networks to determine the presence of plaque and stenosis.

[0003] However, the former suffers from significant inter-observer variability and subjectivity; while the latter, based on CPR results at points along the centerline, requires substantial human and material resources in the data preparation phase. The highly complex labeling scheme, which involves labeling data points along the centerline, needs to be implemented for both narrowing rate and patchiness, making it an extremely labor-intensive process whose results are highly dependent on the accuracy of the labeling. Furthermore, this method, which uses deep learning to first segment and obtain the centerline and then combines it with centerline classification, lacks a theoretical explanation for the narrowing rate, and the dominance of background data negatively impacts the entire evaluation process. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and device for detecting coronary artery stenosis and plaque. By using a coronary artery inner and outer diameter segmentation model to accurately segment the inner and outer diameters of the coronary artery, the stenotic region can be located quickly and accurately, avoiding the complex process of preparing stenosis label data. By decomposing the plaque classification task into two sub-tasks and combining the calcified plaque classification results and non-calcified plaque classification results obtained from the two sub-tasks, the plaque type in the stenotic region can be determined more accurately.

[0005] This application provides a method for detecting coronary artery stenosis and plaque, the detection method comprising:

[0006] The coronary angiography image is input into a pre-trained coronary artery inner and outer diameter segmentation model to obtain the segmentation result map of the inner and outer diameters of the coronary artery in the coronary angiography image;

[0007] Extract the centerline of the coronary artery and obtain a surface reconstruction map based on the centerline;

[0008] Based on the inner and outer diameter segmentation results, the stenotic region on the coronary artery is determined;

[0009] For each narrow interval, the interval surface reconstruction map corresponding to the narrow interval in the surface reconstruction map is input into the pre-trained patch classification model to determine the classification result of calcified patches and non-calcified patches in the narrow interval;

[0010] Based on the classification results of calcified plaques and non-calcified plaques, the plaque type of the narrow interval is determined comprehensively; wherein, the plaque type includes calcified plaques, non-calcified plaques, mixed plaques, and no plaques.

[0011] Furthermore, when annotating the training data for training the coronary artery inner and outer diameter segmentation model, the coronary artery boundary in the training coronary angiography image is defined as the inner diameter of the coronary artery; the coronary artery boundary is extended outward by a predetermined number of pixels, and the extended coronary artery boundary is defined as the outer diameter of the coronary artery.

[0012] Furthermore, the step of comprehensively determining the plaque type of the narrow interval based on the calcified plaque classification results and the non-calcified plaque classification results includes:

[0013] If the calcified plaque classification result indicates that the narrow region has calcified plaques, and the non-calcified plaque classification result indicates that the narrow region has non-calcified plaques, then the plaque type of the narrow region is determined to be a mixed plaque.

[0014] If the calcified plaque classification result indicates that the narrow region has calcified plaques, and the non-calcified plaque classification result indicates that the narrow region does not have non-calcified plaques, then the plaque type of the narrow region is determined to be calcified plaques.

[0015] If the calcified plaque classification result indicates that the narrow region does not have calcified plaques, and the non-calcified plaque classification result indicates that the narrow region does not have non-calcified plaques, then the plaque type of the narrow region is determined to be plaque-free.

[0016] If the calcified plaque classification result indicates that the narrow region does not have calcified plaques, and the non-calcified plaque classification result indicates that the narrow region has non-calcified plaques, then the plaque type of the narrow region is determined to be non-calcified plaque.

[0017] Furthermore, determining the stenotic region on the coronary artery based on the inner and outer diameter segmentation results includes:

[0018] For each of the multiple center points included in the centerline, based on the slice result image corresponding to each center point in the inner and outer diameter segmentation result image, the equivalent inner diameter and equivalent outer diameter at each center point are determined;

[0019] The equivalent inner diameter variation line along the center line is obtained by fitting the equivalent inner diameter at each center point.

[0020] The equivalent outer diameter variation line along the centerline is obtained by fitting the equivalent outer diameter at each center point.

[0021] Based on the equivalent outer diameter variation line along the centerline and the equivalent inner diameter variation line along the centerline, the stenotic region on the coronary artery is determined.

[0022] Furthermore, determining the stenotic region on the coronary artery based on the inner and outer diameter segmentation results includes:

[0023] For each of the multiple center points included in the centerline, the equivalent stenotic vessel diameter at that center point is determined based on the slice result image corresponding to that center point in the inner and outer diameter segmentation result image.

[0024] For multiple sampling center points located before and after the center point on the center line, the surface reconstruction map is sampled with a sampling frame of a predetermined size, centered on the center point and each sampling center point, to obtain multiple surface reconstruction sampling maps.

[0025] The multiple surface reconstruction sampling maps are input into a pre-trained recurrent neural network to regress and predict the actual diameter at the center point;

[0026] The stenosis rate at the center point is determined based on the equivalent stenosis diameter and the actual diameter of the vessel at that center point.

[0027] Based on the stenosis rate at each center point, the stenosis interval on the coronary artery is determined.

[0028] Furthermore, for each narrow interval, the corresponding interval surface reconstruction map in the surface reconstruction map is input into a pre-trained patch classification model to determine the classification result of calcified patches and non-calcified patches for that narrow interval, including:

[0029] For each center point located within the narrow interval among the multiple center points included in the center line, the surface reconstruction map is sampled with a sampling frame of a predetermined size centered on the center point to obtain the interval surface reconstruction map;

[0030] The interval surface reconstruction map is input into the patch classification model to obtain the classification results of calcified patches and non-calcified patches; wherein, the first fully connected layer in the patch classification model is used to distinguish calcified patches to obtain the calcified patch classification result; the second fully connected layer in the patch classification model is used to distinguish non-calcified patches to obtain the non-calcified patch classification result.

[0031] This application embodiment also provides a device for detecting coronary artery stenosis and plaque, the detection device comprising:

[0032] The segmentation module is used to input the coronary angiography image into a pre-trained coronary artery inner and outer diameter segmentation model to obtain the segmentation result map of the inner and outer diameters of the coronary arteries in the coronary angiography image;

[0033] The first determining module is used to extract the centerline of the coronary artery and obtain a surface reconstruction map based on the centerline;

[0034] The second determining module is used to determine the stenotic region on the coronary artery based on the inner and outer diameter segmentation result map;

[0035] The classification module is used to input the interval surface reconstruction map corresponding to the narrow interval in the surface reconstruction map into the pre-trained patch classification model for each narrow interval, and determine the classification result of calcified patches and non-calcified patches for the narrow interval.

[0036] The comprehensive module is used to comprehensively determine the plaque type of the narrow interval based on the classification results of the calcified plaques and the classification results of the non-calcified plaques; wherein the plaque type includes calcified plaques, non-calcified plaques, mixed plaques, and no plaques.

[0037] Furthermore, the detection device also includes: a training module; during the process of training the coronary artery inner and outer diameter segmentation model, when the training module annotates the training data for training the coronary artery inner and outer diameter segmentation model, the coronary artery boundary in the training coronary angiography image is defined as the inner diameter of the coronary artery; the coronary artery boundary is extended outward by a predetermined number of pixels, and the extended coronary artery boundary is defined as the outer diameter of the coronary artery.

[0038] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the coronary artery stenosis and plaque detection method described above are performed.

[0039] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the coronary artery stenosis and plaque detection method described above.

[0040] This application provides a method and apparatus for detecting coronary artery stenosis and plaque. The detection method includes: inputting a coronary angiography image into a pre-trained coronary artery inner and outer diameter segmentation model to obtain a segmentation result map of the inner and outer diameters of the coronary artery in the coronary angiography image; extracting the centerline of the coronary artery and obtaining a surface reconstruction map based on the centerline; determining stenosis intervals on the coronary artery based on the inner and outer diameter segmentation result map; for each stenosis interval, inputting the corresponding interval surface reconstruction map from the surface reconstruction map into a pre-trained plaque classification model to determine the calcified plaque classification result and the non-calcified plaque classification result of the stenosis interval; and comprehensively determining the plaque type of the stenosis interval based on the calcified plaque classification result and the non-calcified plaque classification result; wherein the plaque type includes calcified plaque, non-calcified plaque, mixed plaque, and no plaque.

[0041] In this way, by using the coronary artery inner and outer diameter segmentation model to accurately segment the inner and outer diameters of the coronary arteries, the stenotic region can be located quickly and accurately, avoiding the complex process of preparing stenosis label data. By decomposing the plaque classification task into two sub-tasks and combining the calcified plaque classification results and non-calcified plaque classification results obtained from the two sub-tasks, the plaque type in the stenotic region can be determined more accurately.

[0042] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart of a method for detecting coronary artery stenosis and plaque provided in an embodiment of this application is shown;

[0045] Figure 2 This illustration shows a schematic diagram of the inner and outer diameters of a coronary artery provided in an embodiment of this application;

[0046] Figure 3 This illustration shows a slice of an inner and outer diameter segmentation result diagram provided in an embodiment of this application.

[0047] Figure 4 This illustration shows a schematic diagram of an equivalent outer diameter variation line and an equivalent inner diameter variation line along the center line provided in an embodiment of this application.

[0048] Figure 5 A schematic diagram of the structure of a coronary artery stenosis and plaque detection device provided in an embodiment of this application is shown;

[0049] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0051] Research has found that in existing technologies, the task of identifying plaque and stenosis in blood vessels is usually performed through visual assessment, or automatically by extracting a straightened image of the entire centerline from the coronary artery segmentation results to obtain surface CPR, and then using neural networks to determine the presence of plaque and stenosis.

[0052] However, the former suffers from significant inter-observer variability and subjectivity; while the latter, based on CPR results at points along the centerline, requires substantial human and material resources in the data preparation phase. The highly complex labeling scheme, which involves labeling data points along the centerline, needs to be implemented for both narrowing rate and patchiness, making it an extremely labor-intensive process whose results are highly dependent on the accuracy of the labeling. Furthermore, this method, which uses deep learning to first segment and obtain the centerline and then combines it with centerline classification, lacks a theoretical explanation for the narrowing rate, and the dominance of background data negatively impacts the entire evaluation process.

[0053] Based on this, embodiments of this application provide a method and device for detecting coronary artery stenosis and plaque. By using a coronary artery inner and outer diameter segmentation model to accurately segment the inner and outer diameters of the coronary artery, the stenotic region can be located quickly and accurately, avoiding the complex process of preparing stenosis label data. By decomposing the plaque classification task into two sub-tasks and combining the calcified plaque classification results and non-calcified plaque classification results obtained from the two sub-tasks, the plaque type in the stenotic region can be determined more accurately.

[0054] Please see Figure 1 , Figure 1This is a flowchart illustrating a method for detecting coronary artery stenosis and plaque, provided as an embodiment of this application. Figure 1 As shown in the embodiments of this application, the detection method includes:

[0055] S101. Input the coronary angiography image into the pre-trained coronary artery inner and outer diameter segmentation model to obtain the segmentation result map of the inner and outer diameters of the coronary artery in the coronary angiography image.

[0056] Coronary angiography is a commonly used and effective method for diagnosing coronary atherosclerotic heart disease (CAD). It is a relatively safe and reliable invasive diagnostic technique that is now widely used in clinical practice and is considered the "gold standard" for diagnosing CAD. It involves injecting a contrast agent into the coronary arteries through a catheter, and then obtaining 2D coronary angiographic images via X-ray.

[0057] It should be noted that the coronary artery consists of the intima and the adventitia, which have corresponding inner diameter (also called coronary cavity) and outer diameter; while plaque is a type of atherosclerosis that exists between the two. The vast majority of plaques will cause abnormalities in the inner and outer diameters of the blood vessel. These abnormalities are generally of two types: the outer diameter bulges outward or the inner diameter is concave inward. Therefore, accurately dividing the inner and outer diameter membranes becomes a prerequisite for determining the stenosis rate and plaque classification.

[0058] The coronary artery internal and external diameter segmentation model performs roughly the same segmentation of the vessel's inner and outer diameters, the only difference being the labels. Therefore, the coronary artery internal and external diameter segmentation model yields a binary segmentation result image. The outer diameter includes the plaque portion, while the inner diameter does not; that is, their label data are different. In practical implementation, the 3DUnet network can be used as the coronary artery internal and external diameter segmentation model to simultaneously segment the inner and outer diameters of the coronary arteries, obtaining the resulting internal and external diameter segmentation image.

[0059] Unlike IVUS images, it is difficult to find the difference between the inner and outer membranes in coronary angiography (CTA) images. This is because the difference between the inner, middle, and outer membranes in CTA images is less than 0.3 mm (equivalent to one pixel), so there is no visual difference, but the inner and outer membranes are actually present.

[0060] Therefore, during the training of the coronary artery inner and outer diameter segmentation model, when labeling the training data for the model, the coronary artery boundary in the training coronary angiography image is defined as the inner diameter of the coronary artery; the coronary artery boundary is extended outward by a predetermined number of pixels, and the extended boundary is defined as the outer diameter of the coronary artery. In one experiment, the predetermined pixel can be a single pixel.

[0061] After obtaining the labeled training data, the coronary artery inner and outer diameter segmentation model can be trained using any method in the existing technology, and this application does not impose any restrictions on this.

[0062] Please see Figure 2 and Figure 3 , Figure 2 A schematic diagram illustrating the inner and outer diameters of a coronary artery, provided for an embodiment of this application; Figure 3 This application provides a slice result diagram of an inner and outer diameter segmentation result diagram, wherein... Figure 3 It is a cross-sectional slice of a blood vessel, such as... Figure 2 and Figure 3 As shown in the diagram, the inner ring represents the inner diameter of the coronary artery, and the outer ring represents the outer diameter. When annotating training data, the inner diameter of the vessel is the area we see with the naked eye on a CTA, while the outer diameter is the area extended outward by one pixel based on the inner diameter.

[0063] When applying the coronary artery diameter segmentation model to segment coronary angiography images, the input to the 3DUnet model is the original CTA vessel image, and the label data is based on the original vessel image, such as... Figure 3 The outline of the blood vessels is shown in concentric circles.

[0064] S102. Extract the centerline of the coronary artery and obtain a surface reconstruction map based on the centerline.

[0065] In practice, the centerline of the coronary artery can be extracted using any method available in the prior art, and a curved planar reconstruction (CPR) image can be obtained based on the centerline of the coronary artery. Curved planar reconstruction (CPR) can straighten tubular images such as curved blood vessels and display them on a plane.

[0066] For example, the segmentation result of the blood vessel to be detected can be obtained from the segmentation result image of the inner and outer diameters. The center line is obtained by skeletalizing the segmented blood vessel, and the center line includes multiple path points. The CPR sequenced image is obtained based on the path points on the center line.

[0067] S103. Based on the inner and outer diameter segmentation results, determine the stenotic region on the coronary artery.

[0068] Here, the sliced ​​result diagram of the inner and outer diameter segmentation result diagram (such as...) Figure 3 The annular cross-section of the vessel shown reveals that in some sections, the inner and outer diameters of the coronary artery do not match well. Such mismatches may indicate plaque buildup leading to vascular bulging. The steps for identifying the narrowed sections in the coronary artery will be described in detail below.

[0069] In one possible implementation, step S103 may include:

[0070] S1031. For each of the multiple center points included in the center line, based on the slice result diagram corresponding to each center point in the inner and outer diameter segmentation result diagram, determine the equivalent inner diameter and equivalent outer diameter at each center point.

[0071] In this step, firstly, the slice result image corresponding to each center point in the inner and outer diameter segmentation result image is obtained; secondly, the area enclosed by the inner diameter and the area enclosed by the outer diameter of each center point are calculated; for example, such as Figure 3 As shown, the shapes enclosed by the inner and outer diameters can be divided into multiple sectors according to the square-shaped marking points and center points on the inner and outer diameters. The areas enclosed by the inner and outer diameters can be summed based on the areas of each sector. Then, the inner and outer diameter regions of the coronary artery can be equivalently represented as regular shapes, such as circles or ellipses, so that the equivalent inner diameter and equivalent outer diameter at each center point can be calculated based on the area.

[0072] S1032. Based on the equivalent inner diameter fitting at each center point, the equivalent inner diameter variation line along the center line is obtained.

[0073] S1033. Based on the equivalent outer diameter fitting at each center point, the equivalent outer diameter variation line along the center line is obtained.

[0074] S1034. Based on the equivalent outer diameter variation line along the centerline and the equivalent inner diameter variation line along the centerline, determine the stenotic interval on the coronary artery.

[0075] For S1032 to S1034, please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the equivalent outer diameter variation line and the equivalent inner diameter variation line along the center line, provided as embodiments of this application. Figure 4 As shown, Figure 4 The difference between the inner and outer diameters of the coronary arteries is large in a certain interval in the middle, and they cannot match well. Therefore, this interval can be defined as a narrow interval.

[0076] In another possible implementation, step S103 may include:

[0077] S1035. For each of the multiple center points included in the centerline, based on the slice result image corresponding to that center point in the inner and outer diameter segmentation result image, determine the equivalent stenotic blood vessel diameter at that center point.

[0078] In S1035, the equivalent inner diameter at the center point can be determined by referring to S1031, and the equivalent inner diameter can then be used as the equivalent vascular diameter at the stenosis.

[0079] S1036. For multiple sampling center points on the center line that are located before and after the center point, the surface reconstruction map is sampled with a sampling frame of a predetermined size, centered on the center point and each sampling center point, to obtain multiple surface reconstruction sampling maps.

[0080] S1037. Input the multiple surface reconstruction sampling maps into a pre-trained recurrent neural network to regress and predict the actual diameter at the center point.

[0081] For steps S1036 and S1037, the inner and outer diameter segmentation results can only help us accurately locate the narrowed blood vessel segment, but the original diameter of the blood vessel is unknown. Therefore, in this embodiment, a recurrent neural network can be used to regress the true diameter of the blood vessel.

[0082] The input to the recurrent neural network is a bounding box of a predetermined size, sampled from the center point on the centerline of the reconstructed blood vessel surface sequence. The output of the recurrent neural network is the actual diameter at the center point. Specifically, when predicting the actual diameter at a certain center point, the surface reconstruction image can be sampled with a bounding box of a predetermined size, using that center point as the center. Then, multiple center points on the centerline before and after the center point are used as sampling center points, and the surface reconstruction image is sampled with a bounding box of a predetermined size, using each of these sampling center points as the center, ultimately resulting in multiple surface reconstruction sampling images.

[0083] For example, you can select each center point, the two points before each center point, and the two points after each center point, and sample a 32*32*32 bounding box for each point on the surface reconstruction map. This is because the CPR sequence is not a square, and smaller boxes are more likely to tear vessel segments, resulting in no correlation between vessel segments. However, this correlation is very important. Suppose a box only contains half of the plaque, and the other half is located in the preceding box. If you simply predict based on this box, it is easy to make a prediction error because the starting point of this box contains the plaque, resulting in an underestimation of the regression diameter.

[0084] S1038. Based on the equivalent stenotic vessel diameter and the actual diameter at the center point, determine the stenosis rate at the center point.

[0085] In practical implementation, the stenosis rate can be determined based on the following formula: Stenosis rate = 1 - (Equivalent stenotic diameter / Vessel diameter). The equivalent inner diameter at the center point can be used as the equivalent stenotic diameter and substituted into the formula to calculate the stenosis rate.

[0086] S1039. Based on the stenosis rate at each center point, determine the stenosis interval on the coronary artery.

[0087] In this step, the narrow center point can be determined based on the narrowing rate at each center point; then, continuous narrow intervals can be divided according to the position, coordinates, or sequence number of the narrow center point.

[0088] Furthermore, after obtaining the stenosis rate at each center point, the stenosis grade can be determined; generally, the coronary artery stenosis grades can be divided into no stenosis (0), slight stenosis (<25%), mild stenosis (25%-49%), moderate stenosis (50%-69%), severe stenosis (70%-99%), and complete occlusion (100%).

[0089] S104. For each narrow interval, input the interval surface reconstruction map corresponding to the narrow interval in the surface reconstruction map into the pre-trained patch classification model to determine the classification result of calcified patches and non-calcified patches in the narrow interval.

[0090] In one possible implementation, step S104 may include:

[0091] S1041. For each center point located within the narrow interval among the multiple center points included in the center line, the surface reconstruction map is sampled with a sampling frame of a predetermined size centered on the center point to obtain the interval surface reconstruction map.

[0092] In this step, based on the narrow intervals obtained earlier, the start and end points in each narrow interval can be easily located. Then, the CPR map is obtained based on the effective center point between the start and end points and entered into the patch classification module.

[0093] S1042. Input the interval surface reconstruction map into the patch classification model to obtain the classification results of the calcified patches and the classification results of the non-calcified patches.

[0094] In practice, the patch classification model can be a VGG-based classification network, but the final output is changed from a network with one fully connected layer to a network with two fully connected layers.

[0095] In this plaque classification model, the first fully connected layer distinguishes calcified plaques, yielding a calcified plaque classification result; the second fully connected layer distinguishes non-calcified plaques, yielding a non-calcified plaque classification result. Specifically, the plaque classification task is broken down into two sub-tasks: Task 1: calcified plaque classification; Task 2: non-calcified plaque classification. Both tasks use the same base network with shared parameters to reduce computation and save training time. The difference between the two tasks lies in the last layer of the network. The last layer of the base network has two fully connected layers: one to distinguish between calcified and non-calcified plaques. In existing technologies, most networks may use a fully connected layer to directly classify calcified patches, non-calcified patches, mixed patches, and no patches. This classification may be effective for calcified patches, but the categories for non-calcified patches and mixed patches cannot be used directly. This is due to the nature of the patches. For calcified patches, their high brightness makes them easy to distinguish, but the latter two are very difficult to identify. Using the same processing method as for calcified patches is an unfair and inefficient solution.

[0096] S105. Based on the classification results of the calcified plaques and the classification results of the non-calcified plaques, the plaque type of the narrow interval is determined comprehensively.

[0097] The plaque types include calcified plaques, non-calcified plaques, mixed plaques, and plaque-free plaques. Mixed plaques are the most complex because they contain both calcified and non-calcified plaques. In other words, if calcified and non-calcified plaques are present in the same area of ​​a blood vessel, then that area is considered a mixed plaque.

[0098] In one possible implementation, step S105 may include:

[0099] First case: If the calcified plaque classification result indicates that the narrow region has calcified plaques, and the non-calcified plaque classification result indicates that the narrow region has non-calcified plaques, then the plaque type of the narrow region is determined to be a mixed plaque.

[0100] The second scenario: If the calcified plaque classification result indicates that the narrow region has calcified plaques, and the non-calcified plaque classification result indicates that the narrow region does not have non-calcified plaques, then the plaque type of the narrow region is determined to be calcified plaques.

[0101] The third scenario: If the calcified plaque classification result indicates that the narrow region does not have calcified plaques, and the non-calcified plaque classification result indicates that the narrow region does not have non-calcified plaques, then the plaque type of the narrow region is determined to be plaque-free.

[0102] The fourth scenario: If the calcified plaque classification result indicates that the narrow region does not have calcified plaques, and the non-calcified plaque classification result indicates that the narrow region has non-calcified plaques, then the plaque type of the narrow region is determined to be non-calcified plaque.

[0103] This application provides a method for detecting coronary artery stenosis and plaque. The method includes: inputting a coronary angiography image into a pre-trained coronary artery inner and outer diameter segmentation model to obtain a segmentation result map of the inner and outer diameters of the coronary artery in the coronary angiography image; extracting the centerline of the coronary artery and obtaining a surface reconstruction map based on the centerline; determining stenosis intervals on the coronary artery based on the inner and outer diameter segmentation result map; for each stenosis interval, inputting the corresponding interval surface reconstruction map from the surface reconstruction map into a pre-trained plaque classification model to determine the calcified plaque classification result and the non-calcified plaque classification result of the stenosis interval; and comprehensively determining the plaque type of the stenosis interval based on the calcified plaque classification result and the non-calcified plaque classification result; wherein the plaque type includes calcified plaque, non-calcified plaque, mixed plaque, and no plaque.

[0104] In this way, by using the coronary artery inner and outer diameter segmentation model to accurately segment the inner and outer diameters of the coronary arteries, the stenotic region can be located quickly and accurately, avoiding the complex process of preparing stenosis label data. By decomposing the plaque classification task into two sub-tasks and combining the calcified plaque classification results and non-calcified plaque classification results obtained from the two sub-tasks, the plaque type in the stenotic region can be determined more accurately.

[0105] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a coronary artery stenosis and plaque detection device provided in an embodiment of this application. Figure 5 As shown, the detection device 500 includes:

[0106] The segmentation module 510 is used to input the coronary angiography image into a pre-trained coronary artery inner and outer diameter segmentation model to obtain the segmentation result map of the inner and outer diameters of the coronary artery in the coronary angiography image;

[0107] The first determining module 520 is used to extract the centerline of the coronary artery and obtain a surface reconstruction map based on the centerline;

[0108] The second determining module 530 is used to determine the stenotic region on the coronary artery based on the inner and outer diameter segmentation result map;

[0109] The classification module 540 is used to input the interval surface reconstruction map corresponding to the narrow interval in the surface reconstruction map into the pre-trained patch classification model for each narrow interval, and determine the classification result of calcified patches and non-calcified patches in the narrow interval.

[0110] The integration module 550 is used to comprehensively determine the plaque type of the narrow interval based on the calcified plaque classification results and the non-calcified plaque classification results; wherein the plaque type includes calcified plaques, non-calcified plaques, mixed plaques, and no plaques.

[0111] Furthermore, the detection device also includes: a training module; during the process of training the coronary artery inner and outer diameter segmentation model, when the training module annotates the training data for training the coronary artery inner and outer diameter segmentation model, the coronary artery boundary in the training coronary angiography image is defined as the inner diameter of the coronary artery; the coronary artery boundary is extended outward by a predetermined number of pixels, and the extended coronary artery boundary is defined as the outer diameter of the coronary artery.

[0112] Furthermore, when the integration module 550 is used to comprehensively determine the plaque type of the narrow interval based on the calcified plaque classification results and the non-calcified plaque classification results, the integration module 550 is used to:

[0113] If the calcified plaque classification result indicates that the narrow region has calcified plaques, and the non-calcified plaque classification result indicates that the narrow region has non-calcified plaques, then the plaque type of the narrow region is determined to be a mixed plaque.

[0114] If the calcified plaque classification result indicates that the narrow region has calcified plaques, and the non-calcified plaque classification result indicates that the narrow region does not have non-calcified plaques, then the plaque type of the narrow region is determined to be calcified plaques.

[0115] If the calcified plaque classification result indicates that the narrow region does not have calcified plaques, and the non-calcified plaque classification result indicates that the narrow region does not have non-calcified plaques, then the plaque type of the narrow region is determined to be plaque-free.

[0116] If the calcified plaque classification result indicates that the narrow region does not have calcified plaques, and the non-calcified plaque classification result indicates that the narrow region has non-calcified plaques, then the plaque type of the narrow region is determined to be non-calcified plaque.

[0117] Furthermore, when the second determining module 530 determines the stenotic region on the coronary artery based on the inner and outer diameter segmentation result map, the second determining module 530 is used to:

[0118] For each of the multiple center points included in the centerline, based on the slice result image corresponding to each center point in the inner and outer diameter segmentation result image, the equivalent inner diameter and equivalent outer diameter at each center point are determined;

[0119] The equivalent inner diameter variation line along the center line is obtained by fitting the equivalent inner diameter at each center point.

[0120] The equivalent outer diameter variation line along the centerline is obtained by fitting the equivalent outer diameter at each center point.

[0121] Based on the equivalent outer diameter variation line along the centerline and the equivalent inner diameter variation line along the centerline, the stenotic region on the coronary artery is determined.

[0122] Furthermore, when the second determining module 530 determines the stenotic region on the coronary artery based on the inner and outer diameter segmentation result map, the second determining module 530 is used to:

[0123] For each of the multiple center points included in the centerline, the equivalent stenotic vessel diameter at that center point is determined based on the slice result image corresponding to that center point in the inner and outer diameter segmentation result image.

[0124] For multiple sampling center points located before and after the center point on the center line, the surface reconstruction map is sampled with a sampling frame of a predetermined size, centered on the center point and each sampling center point, to obtain multiple surface reconstruction sampling maps.

[0125] The multiple surface reconstruction sampling maps are input into a pre-trained recurrent neural network to regress and predict the actual diameter at the center point;

[0126] The stenosis rate at the center point is determined based on the equivalent stenosis diameter and the actual diameter of the vessel at that center point.

[0127] Based on the stenosis rate at each center point, the stenosis interval on the coronary artery is determined.

[0128] Furthermore, when the classification module 540 inputs the surface reconstruction map corresponding to the narrow interval in the surface reconstruction map into the pre-trained patch classification model for each narrow interval, and determines the classification result of calcified patches and non-calcified patches for that narrow interval, the classification module 540 is used to:

[0129] For each center point located within the narrow interval among the multiple center points included in the center line, the surface reconstruction map is sampled with a sampling frame of a predetermined size centered on the center point to obtain the interval surface reconstruction map;

[0130] The interval surface reconstruction map is input into the patch classification model to obtain the classification results of calcified patches and non-calcified patches; wherein, the first fully connected layer in the patch classification model is used to distinguish calcified patches to obtain the calcified patch classification result; the second fully connected layer in the patch classification model is used to distinguish non-calcified patches to obtain the non-calcified patch classification result.

[0131] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 600 includes a processor 610, a memory 620, and a bus 630.

[0132] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 and the memory 620 communicate via the bus 630. When the machine-readable instructions are executed by the processor 610, they can perform the operations described above. Figures 1 to 4 The steps of a method for detecting coronary artery stenosis and plaque in the illustrated embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0133] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figures 1 to 4 The steps of a method for detecting coronary artery stenosis and plaque in the illustrated embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0134] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0138] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting coronary artery stenosis and plaque, characterized in that, The detection method includes: The coronary angiography image is input into a pre-trained coronary artery inner and outer diameter segmentation model to obtain the segmentation result map of the inner and outer diameters of the coronary artery in the coronary angiography image; Extract the centerline of the coronary artery and obtain a surface reconstruction map based on the centerline; Based on the inner and outer diameter segmentation results, the stenotic region on the coronary artery is determined; For each narrow interval, the corresponding interval surface reconstruction map in the surface reconstruction map is input into a pre-trained patch classification model to determine the classification result of calcified patches and non-calcified patches for that narrow interval. The patch classification task of the model is decomposed into a calcified patch classification subtask and a non-calcified patch classification subtask. The calcified patch classification subtask and the non-calcified patch classification subtask use the same base network with shared parameters. This base network includes two fully connected layers: the calcified patch classification subtask corresponds to the first fully connected layer, used to distinguish whether a patch is calcified; the non-calcified patch classification subtask corresponds to the second fully connected layer, used to distinguish whether a patch is non-calcified. Based on the classification results of calcified plaques and non-calcified plaques, the plaque type of the narrow interval is determined comprehensively; wherein, the plaque type includes calcified plaques, non-calcified plaques, mixed plaques, and no plaques.

2. The detection method according to claim 1, characterized in that, When annotating the training data for training the coronary artery inner and outer diameter segmentation model, the coronary artery boundary in the training coronary angiography image is defined as the inner diameter of the coronary artery; the coronary artery boundary is extended outward by a predetermined number of pixels, and the extended coronary artery boundary is defined as the outer diameter of the coronary artery.

3. The detection method according to claim 1, characterized in that, The process of comprehensively determining the plaque type in the narrow region based on the classification results of calcified plaques and the classification results of non-calcified plaques includes: If the calcified plaque classification result indicates that the narrow region has calcified plaques, and the non-calcified plaque classification result indicates that the narrow region has non-calcified plaques, then the plaque type of the narrow region is determined to be a mixed plaque. If the calcified plaque classification result indicates that the narrow region has calcified plaques, and the non-calcified plaque classification result indicates that the narrow region does not have non-calcified plaques, then the plaque type of the narrow region is determined to be calcified plaques. If the calcified plaque classification result indicates that the narrow region does not have calcified plaques, and the non-calcified plaque classification result indicates that the narrow region does not have non-calcified plaques, then the plaque type of the narrow region is determined to be plaque-free. If the calcified plaque classification result indicates that the narrow region does not have calcified plaques, and the non-calcified plaque classification result indicates that the narrow region has non-calcified plaques, then the plaque type of the narrow region is determined to be non-calcified plaque.

4. The detection method according to claim 1, characterized in that, The determination of the stenotic region on the coronary artery based on the inner and outer diameter segmentation results includes: For each of the multiple center points included in the centerline, based on the slice result image corresponding to each center point in the inner and outer diameter segmentation result image, the equivalent inner diameter and equivalent outer diameter at each center point are determined; The equivalent inner diameter variation line along the center line is obtained by fitting the equivalent inner diameter at each center point. The equivalent outer diameter variation line along the centerline is obtained by fitting the equivalent outer diameter at each center point; Based on the equivalent outer diameter variation line along the centerline and the equivalent inner diameter variation line along the centerline, the stenotic region on the coronary artery is determined.

5. The detection method according to claim 1, characterized in that, The determination of the stenotic region on the coronary artery based on the inner and outer diameter segmentation results includes: For each of the multiple center points included in the centerline, the equivalent stenotic vessel diameter at that center point is determined based on the slice result image corresponding to that center point in the inner and outer diameter segmentation result image. For multiple sampling center points located before and after the center point on the center line, the surface reconstruction map is sampled with a sampling frame of a predetermined size, centered on the center point and each sampling center point, to obtain multiple surface reconstruction sampling maps. The multiple surface reconstruction sampling maps are input into a pre-trained recurrent neural network to regress and predict the actual diameter at the center point; The stenosis rate at the center point is determined based on the equivalent stenosis diameter and the actual diameter of the vessel at that center point. Based on the stenosis rate at each center point, the stenosis interval on the coronary artery is determined.

6. The detection method according to claim 1, characterized in that, For each narrow interval, the corresponding interval surface reconstruction map in the surface reconstruction map is input into a pre-trained patch classification model to determine the classification result of calcified patches and non-calcified patches for that narrow interval, including: For each center point located within the narrow interval among the multiple center points included in the center line, the surface reconstruction map is sampled with a sampling frame of a predetermined size centered on the center point to obtain the interval surface reconstruction map; The interval surface reconstruction map is input into the patch classification model to obtain the classification results of calcified patches and non-calcified patches; wherein, the first fully connected layer in the patch classification model is used to distinguish calcified patches to obtain the calcified patch classification result; the second fully connected layer in the patch classification model is used to distinguish non-calcified patches to obtain the non-calcified patch classification result.

7. A device for detecting coronary artery stenosis and plaque, characterized in that, The detection device includes: The segmentation module is used to input the coronary angiography image into a pre-trained coronary artery inner and outer diameter segmentation model to obtain the segmentation result map of the inner and outer diameters of the coronary arteries in the coronary angiography image; The first determining module is used to extract the centerline of the coronary artery and obtain a surface reconstruction map based on the centerline; The second determining module is used to determine the stenotic region on the coronary artery based on the inner and outer diameter segmentation result map; A classification module is used to input the surface reconstruction map of the corresponding narrow interval in the surface reconstruction map into a pre-trained patch classification model for each narrow interval, and determine the classification result of calcified patches and non-calcified patches for that narrow interval. The patch classification task of the patch classification model is decomposed into a calcified patch classification subtask and a non-calcified patch classification subtask. The calcified patch classification subtask and the non-calcified patch classification subtask use the same base network with shared parameters. The base network includes two fully connected layers: the calcified patch classification subtask corresponds to the first fully connected layer, used to distinguish whether a patch is calcified; the non-calcified patch classification subtask corresponds to the second fully connected layer, used to distinguish whether a patch is non-calcified. The comprehensive module is used to comprehensively determine the plaque type of the narrow interval based on the classification results of the calcified plaques and the classification results of the non-calcified plaques; wherein the plaque type includes calcified plaques, non-calcified plaques, mixed plaques, and no plaques.

8. The detection device according to claim 7, characterized in that, The detection device further includes: a training module; during the process of training the coronary artery inner and outer diameter segmentation model, when the training module annotates the training data for training the coronary artery inner and outer diameter segmentation model, the coronary artery boundary in the training coronary angiography image is defined as the inner diameter of the coronary artery; the coronary artery boundary is extended outward by a predetermined number of pixels, and the extended coronary artery boundary is defined as the outer diameter of the coronary artery.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of a method for detecting coronary artery stenosis and plaque as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a method for detecting coronary artery stenosis and plaque as described in any one of claims 1 to 6.

Images