Image segmentation method and computer-readable storage medium

Abstract

The present application provides an image segmentation method and a computer-readable storage medium. The image segmentation method includes: using a coronary artery segmentation model to segment the coronary arteries in the image to be segmented, obtaining a coronary artery segmentation result corresponding to the image to be segmented; performing surface reconstruction based on the coronary artery segmentation result corresponding to the image to be segmented, obtaining a surface reconstruction image corresponding to the image to be segmented; using a lumen and plaque segmentation model to segment the surface reconstruction image corresponding to the image to be segmented, obtaining a lumen and plaque segmentation result corresponding to the image to be segmented. Compared with traditional image segmentation methods, this method has higher accuracy, better stability, and faster speed.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to an image segmentation method and a computer-readable storage medium. Background Art

[0002] Coronary heart disease is a common cardiovascular disease. With the continuous aggravation of population aging, it seriously threatens human life health and quality of life. Coronary artery imaging (CTA) technology can non-invasively observe the morphology of blood vessels and plaques, and is an important means of clinical examination. In clinical practice, the evaluation of atherosclerotic lesions is particularly important. In order to quantitatively analyze the vascular characteristics and plaque components, doctors need to manually draw the lumen, outer wall contour lines and plaques in the blood vessels. On the one hand, this is time-consuming and laborious, greatly affecting work efficiency. On the other hand, affected by image artifacts, noise, etc., it is difficult to obtain accurate results. Especially for non-calcified plaques, due to their irregular shapes and boundaries, they are prone to misjudgment and require experienced doctors or technicians to review.

[0003] With the development of artificial intelligence in the auxiliary diagnosis of coronary CTA, it has become possible to automatically segment the lumen, plaques and blood vessel outer walls and perform quantitative analysis. This can not only provide a basis for clinically differentiating plaque types, but also provide numerical support for calculating the degree of vascular stenosis. Therefore, there is an urgent need to design an image segmentation method to meet the needs of practical applications. Summary of the Invention

[0004] The purpose of this application is to provide an image segmentation method and a computer-readable storage medium, with higher accuracy, better stability and faster speed in image segmentation.

[0005] The purpose of this application is achieved by the following technical solutions:

[0006] In the first aspect, this application provides an image segmentation method, which includes: using a coronary artery segmentation model to segment the coronary arteries in the image to be segmented, obtaining the coronary artery segmentation result corresponding to the image to be segmented; performing surface reconstruction based on the coronary artery segmentation result corresponding to the image to be segmented, obtaining the surface reconstruction image corresponding to the image to be segmented; using a lumen and plaque segmentation model to segment the surface reconstruction image corresponding to the image to be segmented, obtaining the lumen and plaque segmentation result corresponding to the image to be segmented.

[0007] The beneficial effects of this technical solution are as follows: the coronary artery segmentation model is used to segment the coronary arteries in the image to be segmented to obtain the coronary artery segmentation result, and the surface reconstruction is performed on the coronary artery segmentation result to obtain the surface reconstruction image; the lumen and plaque segmentation model is used to segment the obtained surface reconstruction image, and the lumen and plaque segmentation results corresponding to the image to be segmented can be automatically output; the lumen and plaque segmentation results corresponding to the image to be segmented are automatically obtained in an end-to-end mode, which has higher accuracy, better stability and faster speed compared with traditional image processing methods.

[0008] In some optional embodiments, the step of using the lumen and plaque segmentation model to segment the surface reconstruction image corresponding to the image to be segmented to obtain the lumen and plaque segmentation results corresponding to the image to be segmented includes: using the plaque detection network to detect the surface reconstruction image corresponding to the image to be segmented to obtain the original attention map corresponding to the image to be segmented; inputting the surface reconstruction image and the original attention map corresponding to the image to be segmented into the lumen and plaque segmentation model to obtain the lumen and plaque segmentation results corresponding to the image to be segmented.

[0009] The beneficial effects of this technical solution are as follows: by introducing the attention mechanism, the original attention map and the surface reconstruction image corresponding to the image to be segmented are used as the input of the lumen and plaque segmentation model. Compared with using only the surface reconstruction image corresponding to the segmented image as the input of the lumen and plaque segmentation model, the obtained lumen and plaque segmentation results corresponding to the image to be segmented have stronger robustness.

[0010] In some optional embodiments, the step of using the plaque detection network to detect the surface reconstruction image corresponding to the image to be segmented to obtain the original attention map corresponding to the image to be segmented includes: inputting the surface reconstruction image corresponding to the image to be segmented into the feature extraction module of the plaque detection network to obtain a multi-channel feature map; inputting the multi-channel feature map into the classifier of the plaque detection network to obtain the probability vectors of each plaque category; calculating the sensitivity weights of each plaque category to each channel based on the probability vectors of each plaque category; and obtaining the original attention map corresponding to the image to be segmented based on the multi-channel feature map and the sensitivity weights of each plaque category to each channel.

[0011] The beneficial effects of this technical solution are as follows: using the plaque detection network to detect the surface reconstruction image corresponding to the image to be segmented can obtain the original attention map corresponding to the image to be segmented, so that the obtained original attention map and the surface reconstruction image corresponding to the image to be segmented can be input into the lumen and plaque segmentation model through the attention mechanism, and then the lumen and plaque segmentation results corresponding to the image to be segmented can be obtained, improving the efficiency and accuracy of image segmentation.

[0012] In some alternative embodiments, calculating the sensitivity weights of each patch category to each channel based on the probability vectors of each patch category includes: calculating the global average value of the partial derivatives of the probability vectors of each patch category with respect to each voxel in each channel in the three dimensions of width, height, and depth as the sensitivity weights of each patch category to each channel. The beneficial effect of this technical solution is that by calculating the global average value of the partial derivatives of the probability vectors of each patch category with respect to each voxel in each channel in multiple dimensions, the sensitivity weights of each patch category to each channel can be obtained, and the original attention map corresponding to the image to be segmented ensures the consistency of the calculation results in the case of systematic errors.

[0013] In some alternative embodiments, obtaining the original attention map corresponding to the image to be segmented based on the multi-channel feature map and the sensitivity weights of each patch category to each channel includes: performing weighted summation on the multi-channel feature map based on the sensitivity weights of each patch category to each channel to obtain the weighted summation results of each patch category; inputting the weighted summation results of each patch category into an activation function layer to obtain the original attention map corresponding to the image to be segmented.

[0014] The beneficial effect of this technical solution is that by performing weighted summation on the multi-channel feature map to obtain the weighted summation results of each patch category; and inputting the weighted summation results of each patch category into an activation function layer to obtain the original attention map corresponding to the image to be segmented, which is used to input the obtained original attention map and the surface reconstruction image corresponding to the image to be segmented into the lumen and plaque segmentation model to obtain the corresponding lumen and plaque segmentation results, and the obtained lumen and plaque segmentation results can better reflect the true value of the system.

[0015] In some alternative embodiments, inputting the surface reconstruction image and the original attention map corresponding to the image to be segmented into the lumen and plaque segmentation model to obtain the lumen and plaque segmentation results corresponding to the image to be segmented includes: sequentially passing the surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain the lumen and plaque segmentation results corresponding to the image to be segmented, where N is a positive integer; wherein, for each downsampling layer, the multiplication result of the feature map to be input of the downsampling layer and the attention map of the same size is used as the input image of the downsampling layer, and the attention map of the same size is obtained by using the original attention map.

[0016] The beneficial effect of this technical solution is that while reducing the size of the feature map and increasing the number of output channels during downsampling, using the multiplication result of the feature map to be input of the downsampling layer and the attention map of the same size as the input image of the downsampling layer can reduce information loss during the downsampling process.

[0017] In some alternative embodiments, inputting the surface reconstruction image corresponding to the image to be segmented and the original attention map into the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented includes: sequentially passing the surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; wherein, for each upsampling layer, using the fusion result of the output image of the upsampling layer and the feature map to be input of the corresponding downsampling layer as the input image of the next upsampling layer.

[0018] The beneficial effect of this technical solution is that using the fusion result of the output image of the upsampling layer and the feature map to be input of the corresponding downsampling layer as the input image of the next upsampling layer can obtain a more accurate lumen and plaque segmentation result of the image to be segmented.

[0019] In some alternative embodiments, the training process of the lumen and plaque segmentation model is as follows: Obtain a sample surface reconstruction image; Based on the plaque annotation information of the sample surface reconstruction image, obtain the mean and variance of the CT values of the normal blood vessel segments in the sample surface reconstruction image, and determine the initial lumen contour, the contour of each calcified plaque, and the original contour of each non-calcified plaque in the sample surface reconstruction image based on the mean and variance of the CT values of the normal blood vessel segments; wherein, the plaque annotation information is used to indicate the plaque segmentation result; Based on the initial lumen contour, use the level set method to segment the sample surface reconstruction image to obtain the lumen contour in the sample surface reconstruction image; Based on the original contour of each non-calcified plaque, use the level set method to obtain the initial contour of each non-calcified plaque in the sample surface reconstruction image; Take the union of the contours of each calcified plaque, the initial contours of each non-calcified plaque, and the lumen contour in the sample surface reconstruction image to obtain the initial contour of the blood vessel outer wall in the sample surface reconstruction image; Based on the initial contour of the blood vessel outer wall, use the level set method to segment the sample surface reconstruction image to obtain the blood vessel outer wall contour in the sample surface reconstruction image; Based on the lumen contour and the blood vessel outer wall contour in the sample surface reconstruction image, obtain the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image; Based on the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image, obtain the gold standard annotation information of the sample surface reconstruction image; Use the sample surface reconstruction image and its gold standard annotation information to train a preset deep neural network to obtain the lumen and plaque segmentation model.

[0020] The beneficial effect of this technical solution is that by training the lumen and plaque segmentation model, the accuracy and speed of the lumen and plaque segmentation model in segmenting the contours of each calcified plaque, the initial contours of each non-calcified plaque, and the lumen contour can be improved.

[0021] In some optional embodiments, based on the original contours of the non-calcified plaques, using the level set method to obtain the initial contours of the non-calcified plaques in the sample surface reconstruction image includes: based on the original contours of the non-calcified plaques, for each non-calcified plaque, intercept the regional block image corresponding to the non-calcified plaque; obtain the ratio of the diameter of the normal blood vessel at both ends of the regional block image to the diameter of the blood vessel at the narrowest part of the lesion area of the non-calcified plaque; based on the ratio, perform dilation processing on the regional block image to obtain the dilated image corresponding to the regional block image; statistically calculate the mean and variance of the CT values in the dilated image after removing the lumen area, and perform pixel normalization of the CT values of the dilated image to [-1, 1] based on the statistical results; in the dilated image, remove the areas where the CT values are greater than 0 and equal to -1, and the remaining part is used as the initial contour of the non-calcified plaque.

[0022] The beneficial effect of this technical solution is that based on the plaque annotation information, the initial contours of the non-calcified plaques in the sample surface reconstruction image can be obtained, and the obtained initial contours of the non-calcified plaques are used to obtain the initial contour of the blood vessel outer wall, which improves the accuracy and speed of the lumen and plaque segmentation model in segmenting the initial contours of the non-calcified plaques.

[0023] In a second aspect, the present application provides an image segmentation device, and the device includes: a first segmentation module for segmenting the coronary artery blood vessels in the image to be segmented using a coronary artery segmentation model to obtain the coronary artery blood vessel segmentation result corresponding to the image to be segmented; a surface reconstruction module for performing surface reconstruction based on the coronary artery blood vessel segmentation result corresponding to the image to be segmented to obtain the surface reconstruction image corresponding to the image to be segmented; a second segmentation module for segmenting the surface reconstruction image corresponding to the image to be segmented using a lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented.

[0024] In some optional embodiments, the second segmentation module includes: an image detection sub-module for detecting the surface reconstruction image corresponding to the image to be segmented using a plaque detection network to obtain the original attention map corresponding to the image to be segmented; a model segmentation sub-module for inputting the surface reconstruction image corresponding to the image to be segmented and the original attention map into the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented.

[0025] In some alternative embodiments, the image detection sub-module includes: a feature map unit configured to input a surface reconstruction image corresponding to the image to be segmented into a feature extraction module of the patch detection network to obtain a multi-channel feature map; a classifier unit configured to input the multi-channel feature map into a classifier of the patch detection network to obtain a probability vector for each patch category; a weight unit configured to calculate a sensitivity weight of each patch category to each channel based on the probability vector for each patch category; and an attention acquisition unit configured to obtain an original attention map corresponding to the image to be segmented based on the multi-channel feature map and the sensitivity weight of each patch category to each channel.

[0026] In some alternative embodiments, the weight unit is configured to calculate a global average value of partial derivatives of the probability vector for each patch category with respect to each voxel in each channel in three dimensions of width, height, and depth as the sensitivity weight of each patch category to each channel.

[0027] In some alternative embodiments, the attention acquisition unit includes: a category weighting sub-unit configured to perform weighted summation on the multi-channel feature map based on the sensitivity weight of each patch category to each channel to obtain a weighted summation result for each patch category; and an activation function sub-unit configured to input the weighted summation result for each patch category into an activation function layer to obtain the original attention map corresponding to the image to be segmented.

[0028] In some alternative embodiments, the model segmentation sub-module includes: a first sampling unit configured to sequentially pass a surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain a lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; wherein, for each downsampling layer, a multiplication result of the input feature map to be input to the downsampling layer and an attention map of the same size is used as the input image of the downsampling layer, and the attention map of the same size is obtained by using the original attention map.

[0029] In some alternative embodiments, the model segmentation sub-module includes: a second sampling unit configured to sequentially pass a surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain a lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; wherein, for each upsampling layer, a fusion result of the output image of the upsampling layer and the input feature map to be input to the corresponding downsampling layer is used as the input image of the next upsampling layer.

[0030] In some alternative embodiments, the training process of the lumen and plaque segmentation model is as follows: Obtain a sample surface reconstruction image; Based on the plaque annotation information of the sample surface reconstruction image, obtain the mean and variance of the CT values of the normal blood vessel segments in the sample surface reconstruction image, and determine the initial lumen contour, the contours of each calcified plaque, and the original contours of each non-calcified plaque in the sample surface reconstruction image based on the mean and variance of the CT values of the normal blood vessel segments; wherein, the plaque annotation information is used to indicate the plaque segmentation result; Based on the initial lumen contour, use the level set method to segment the sample surface reconstruction image to obtain the lumen contour in the sample surface reconstruction image; Based on the original contours of each non-calcified plaque, use the level set method to obtain the initial contours of each non-calcified plaque in the sample surface reconstruction image; Take the union of the contours of each calcified plaque, the initial contours of each non-calcified plaque, and the lumen contour in the sample surface reconstruction image to obtain the initial outer wall contour of the blood vessel in the sample surface reconstruction image; Based on the initial outer wall contour of the blood vessel, use the level set method to segment the sample surface reconstruction image to obtain the outer wall contour of the blood vessel in the sample surface reconstruction image; Based on the lumen contour and the outer wall contour in the sample surface reconstruction image, obtain the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image; Based on the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image, obtain the gold standard annotation information of the sample surface reconstruction image; Use the sample surface reconstruction image and its gold standard annotation information to train a preset deep neural network to obtain the lumen and plaque segmentation model.

[0031] In some alternative embodiments, the step of obtaining the initial contours of each non-calcified plaque in the sample surface reconstruction image by using the level set method based on the original contours of each non-calcified plaque includes: Based on the original contours of each non-calcified plaque, for each non-calcified plaque, intercept the regional block image corresponding to the non-calcified plaque; Obtain the ratio of the diameter of the normal blood vessel at both ends of the regional block image to the diameter of the blood vessel at the narrowest part of the lesion area of the non-calcified plaque; Based on the ratio, perform dilation processing on the regional block image to obtain the dilated image corresponding to the regional block image; Statistically calculate the mean and variance of the CT values in the dilated image after removing the lumen area, and perform pixel normalization of the CT values of the dilated image to [-1, 1] based on the statistical results; Remove the areas where the CT values are greater than 0 and equal to -1 in the dilated image, and the remaining part is used as the initial contour of the non-calcified plaque.

[0032] In a third aspect, the present application provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the image segmentation method described in any one of the above are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present application will be further described below in conjunction with the drawings and embodiments.

[0034] Figure 1 It is a schematic flowchart of an image segmentation method provided by an embodiment of the present application;

[0035] Figure 2 It is a schematic flowchart of a process for segmenting a surface reconstruction image provided by an embodiment of the present application;

[0036] Figure 3 It is a schematic diagram of an original SCPR image and an original attention map provided by an embodiment of the present application;

[0037] Figure 4 It is a schematic flowchart of a process for detecting a plaque detection network provided by an embodiment of the present application;

[0038] Figure 5 It is a schematic flowchart of a process for obtaining an original attention map provided by an embodiment of the present application;

[0039] Figure 6 It is a schematic flowchart of a process for sampling a lumen and plaque segmentation model provided by an embodiment of the present application;

[0040] Figure 7 It is a schematic flowchart of another image segmentation method provided by an embodiment of the present application;

[0041] Figure 8 It is a schematic flowchart of a process for training a lumen and plaque segmentation model provided by an embodiment of the present application;

[0042] Figure 9 It is a schematic diagram of segmenting a calcified plaque provided by an embodiment of the present application;

[0043] Figure 10 It is a schematic diagram of an existing gold standard for manually correcting plaques;

[0044] Figure 11 It is a schematic diagram before and after lumen and plaque segmentation provided by an embodiment of the present application;

[0045] Figure 12 It is a schematic flowchart of a process for obtaining an initial contour of a non-calcified plaque provided by an embodiment of the present application;

[0046] Figure 13 It is a schematic diagram of obtaining an initial contour of a non-calcified plaque provided by an embodiment of the present application;

[0047] Figure 14 It is a schematic structural diagram of an image segmentation device provided by an embodiment of the present application;

[0048] Figure 15 It is a schematic structural diagram of a second segmentation module provided by an embodiment of the present application;

[0049] Figure 16 It is a schematic structural diagram of an image detection sub-module provided by an embodiment of the present application;

[0050] Figure 17 It is a schematic structural diagram of an attention acquisition unit provided by an embodiment of the present application;

[0051] Figure 18 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0052] Figure 19 It is a schematic structural diagram of a program product for implementing an image segmentation method provided by an embodiment of the present application. Detailed implementation manners

[0053] Next, in combination with the accompanying drawings and specific implementation manners, the present application will be further described. It should be noted that, on the premise of no conflict, any combination of the following-described embodiments or technical features can form a new embodiment.

[0054] Refer to Figure 1 , an embodiment of the present application provides an image segmentation method, and the method includes steps S101 to S103.

[0055] Step S101: Use a coronary artery segmentation model to segment the coronary artery vessels in the image to be segmented, and obtain the coronary artery vessel segmentation result corresponding to the image to be segmented. Among them, the image to be segmented is, for example, a coronary artery vessel image obtained by performing coronary computed tomography (CT) angiography (that is, performing coronary CT angiography, Computed Tomography Angiography) on the coronary artery.

[0056] Step S102: Perform surface reconstruction based on the coronary artery vessel segmentation result corresponding to the image to be segmented, and obtain the surface reconstruction image corresponding to the image to be segmented. Among them, the surface reconstruction image is, for example, a straightened surface reconstruction image.

[0057] Step S103: Use a lumen and plaque segmentation model to segment the surface reconstruction image corresponding to the image to be segmented, and obtain the lumen and plaque segmentation result corresponding to the image to be segmented.

[0058] Thus, the coronary arteries in the image to be segmented are segmented by the coronary artery segmentation model to obtain the coronary artery segmentation result, and the surface reconstruction image is obtained by performing surface reconstruction on the coronary artery segmentation result; the obtained surface reconstruction image is segmented by the lumen and plaque segmentation model, and the lumen and plaque segmentation results corresponding to the image to be segmented can be automatically output; the lumen and plaque segmentation results corresponding to the image to be segmented are automatically obtained in an end-to-end mode. Compared with traditional image processing methods, it has higher accuracy, better stability, and faster speed.

[0059] See Figure 2 , in some embodiments, step S103 may include steps S201 to S202.

[0060] Step S201: Detect the surface reconstruction image corresponding to the image to be segmented by using a plaque detection network to obtain the original attention map corresponding to the image to be segmented. Among them, the plaque detection network is, for example, a binary classification network, and the original attention map corresponding to the image to be segmented obtained by the binary classification network is, for example, a class activation map (grad-CAM).

[0061] Step S202: Input the surface reconstruction image and the original attention map corresponding to the image to be segmented into the lumen and plaque segmentation model to obtain the lumen and plaque segmentation results corresponding to the image to be segmented.

[0062] The blood vessels in the image to be segmented may have different saturations, uneven image quality, or be affected by factors such as perivascular tissues. The segmentation of plaques, especially non-calcified plaques, in the image to be segmented poses higher requirements for the image segmentation method. In a specific application, the straightened surface reconstruction image (Straightened CurvePlanar Reformation, abbreviated as SCPR) is passed through a plaque detection binary classification model to obtain a class activation map (grad-CAM) of the plaque region based on SCPR, and the class activation map of the plaque region based on SCPR is used as the attention map of the segmentation model to guide the network to focus on the plaque region, thereby improving the accuracy of segmentation. See Figure 3 , the original SCPR image is shown in Figure 3a, and the original attention map is shown in Figure 3b. Compared with simply using SCPR as the input, the introduced attention mechanism makes the result of image segmentation more robust.

[0063] Thus, by introducing the attention mechanism and using the original attention map and the surface reconstruction image corresponding to the image to be segmented as the input of the lumen and plaque segmentation model, compared with simply using the surface reconstruction image corresponding to the segmented image as the input of the lumen and plaque segmentation model, the obtained lumen and plaque segmentation results corresponding to the image to be segmented are more robust.

[0064] SeeFigure 4 , in some embodiments, step S201 may include steps S301 to S304.

[0065] Step S301: Input the surface reconstruction image corresponding to the image to be segmented into the feature extraction module of the plaque detection network to obtain a multi-channel feature map.

[0066] Step S302: Input the multi-channel feature map into the classifier of the plaque detection network to obtain a probability vector for each plaque category. Among them, the classifier of the plaque detection network is, for example, a softmax classifier.

[0067] Step S303: Based on the probability vectors of each plaque category, calculate the sensitivity weights of each plaque category for each channel.

[0068] Step S304: Based on the multi-channel feature map and the sensitivity weights of each plaque category for each channel, obtain the original attention map corresponding to the image to be segmented.

[0069] Thus, by using the plaque detection network to detect the surface reconstruction image corresponding to the image to be segmented, the original attention map corresponding to the image to be segmented can be obtained, so that the original attention map and the surface reconstruction image corresponding to the image to be segmented can be input into the lumen and plaque segmentation model through the attention mechanism, and then the lumen and plaque segmentation result corresponding to the image to be segmented can be obtained, improving the efficiency and accuracy of image segmentation.

[0070] In some embodiments, step S303 may include: calculating the global average value of the partial derivatives of the probability vectors of each plaque category with respect to each voxel in each channel in the three dimensions of width, height, and depth as the sensitivity weights of each plaque category for each channel. Thus, by calculating the global average value of the partial derivatives of the probability vectors of each plaque category with respect to each voxel in each channel in multiple dimensions, the sensitivity weights of each plaque category for each channel can be obtained, and the original attention map corresponding to the image to be segmented obtained ensures the consistency of the calculation results in the case of systematic errors.

[0071] See Figure 5 , in some embodiments, step S304 may include steps S401 to S402.

[0072] Step S401: Based on the sensitivity weights of each plaque category for each channel, perform weighted summation on the multi-channel feature map to obtain the weighted summation result of each plaque category.

[0073] Step S402: Input the weighted summation result of each patch category into the activation function layer to obtain the original attention map corresponding to the image to be segmented. The activation function is, for example, the rectified linear unit (ReLU), and the activation function layer is, for example, the ReLU activation function layer.

[0074] Thus, the multi-channel feature maps are weighted and summed to obtain the weighted summation result of each patch category; the weighted summation result of each patch category is input into the activation function layer to obtain the original attention map corresponding to the image to be segmented, which is used to input the obtained original attention map and the surface reconstruction image corresponding to the image to be segmented into the lumen and plaque segmentation model to obtain the corresponding lumen and plaque segmentation result, and the obtained lumen and plaque segmentation result can better reflect the true value of the system.

[0075] In a specific application, grad-CAM is used as the attention mechanism to calculate the partial derivative of the probability of the plaque in the output of the last softmax layer in the plaque binary classification detection network with respect to all voxels on each channel of the feature map of the last layer of the network, and the global average value in the three dimensions of width, height, and depth is taken on each channel to obtain the sensitivity weight (w) of the plaque category for each channel of the output feature map of the last convolutional layer. The specific representation is as follows:

[0076]

[0077] The sensitivity weight w is weighted and summed with the corresponding channel, and after passing through the ReLU activation function layer, grad-CAM positively correlated with the plaque category score is obtained. The specific representation is as follows:

[0078]

[0079] Among them, y is the probability vector output by softmax, c is the serial number of the plaque category, A is the feature map output by the last convolutional layer, n is the serial number of the channel dimension of the feature map, i, j, and k represent the positions of width, height, and depth on a certain channel of the feature map, and Z represents the number of voxels in the current channel.

[0080] In some embodiments, the step S202 may include: sequentially passing the surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; wherein, for each downsampling layer, the multiplication result of the feature map to be input of the downsampling layer and the attention map of the same size is used as the input image of the downsampling layer, and the attention map of the same size is obtained by using the original attention map.

[0081] Thus, while downsampling reduces the size of the feature map and increases the number of output channels, the input image of the downsampling layer is the multiplication result of the feature map to be input to the downsampling layer and the attention map of the same size, which can reduce information loss during downsampling.

[0082] In some embodiments, step S202 may include: sequentially passing the surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; for each upsampling layer, the fusion result of the output image of the upsampling layer and the feature map to be input to the corresponding downsampling layer is used as the input image of the next upsampling layer.

[0083] Thus, using the fusion result of the output image of the upsampling layer and the feature map to be input to the corresponding downsampling layer as the input image of the next upsampling layer can obtain a more accurate lumen and plaque segmentation result of the image to be segmented.

[0084] See Figure 6 , in a specific application, the lumen and plaque segmentation model includes 4 downsampling layers and 4 upsampling layers. The surface reconstruction image corresponding to the image to be segmented is sequentially passed through the 4 downsampling layers and 4 upsampling layers of the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented. Before each downsampling, the feature map extracted by the convolutional layer of the segmentation network is multiplied by the attention map (grad-CAM) at the pixel level, and the attention map is scaled according to the size of the feature map. See Figure 5 , through 4 downsampling layers (Down Block), while reducing the size of the feature map, the number of output channels is increased. The feature map after 4 downsamplings is upsampled 4 times through the upsampling layer (Up Block) to restore the feature map to the original image size. For each upsampling layer, the fusion result of the output image of the upsampling layer and the feature map to be input to the corresponding downsampling layer is used as the input image of the next upsampling layer.

[0085] Among them, the downsampling module may include a convolutional layer (Cnov) of 2×2×2, a batch normalization (Batch Normalization, BN) layer, an activation unit (ReLU), and a stride of 2. The upsampling module may include a transposed convolutional layer (transpose conv) of 2×2×2, a batch normalization (Batch Normalization, BN) layer, an activation unit (ReLU), and a stride of 2.

[0086] SeeFigure 7 , embodiments of the present application also provide an image segmentation method, which may include the following steps:

[0087] Perform coronary artery segmentation on the original image to obtain the image after coronary artery segmentation;

[0088] Extract the vascular centerline from the image after coronary artery segmentation to obtain the SCRP image after extracting the vascular centerline;

[0089] Pass the SCRP image after extracting the vascular centerline through plaque detection to obtain the original attention map (Attention Map) of the plaque region based on the SCPR image. The obtained original attention map can be a class activation map (grad-CAM);

[0090] Perform plaque and lumen segmentation on the SCRP image after extracting the vascular centerline and the original attention map obtained through plaque detection to obtain the plaque segmentation result.

[0091] In traditional image segmentation methods, the plaque gold standard of manually annotating plaques on the coronary artery surface reconstruction image (SCPR) image, obtaining the level set segmentation result through the level set image segmentation method, performing lumen and plaque segmentation on the vascular segments with plaque annotations, and only performing lumen segmentation on normal vascular segments has the disadvantage of poor robustness. It needs to be manually corrected to obtain the manually corrected result before being used as the gold standard required for deep learning training. See Figure 10 , the plaque gold standard is shown in 10a, the level set segmentation result is shown in 10b, and the manually corrected result is shown in 10c. The obtained segmentation method has poor robustness.

[0092] In some embodiments, a preset deep neural network can be trained using the sample surface reconstruction image and its gold standard annotation information, and the segmentation effect of the calcified plaque in the obtained lumen and plaque segmentation model is robust.

[0093] See Figure 8 , the training process of the lumen and plaque segmentation model may include steps S501 to S509.

[0094] Step S501: Obtain a sample surface reconstruction image.

[0095] Step S502: Based on the patch annotation information of the sample surface reconstruction image, obtain the mean and variance of the CT values of the normal blood vessel segments in the sample surface reconstruction image, and determine the initial lumen contour, the contour of each calcified plaque, and the original contour of each non-calcified plaque in the sample surface reconstruction image based on the mean and variance of the CT values of the normal blood vessel segments. Among them, the patch annotation information is used to indicate the patch segmentation result. The patch annotation information of the sample surface reconstruction image can, for example, include the contours of the patches in the manually annotated sample surface reconstruction image. The patches can include one or more of calcified plaques and non-calcified plaques.

[0096] Step S503: Based on the initial lumen contour, use the level set method to segment the sample surface reconstruction image to obtain the lumen contour in the sample surface reconstruction image.

[0097] Step S504: Based on the original contours of the non-calcified plaques, use the level set method to obtain the initial contours of the non-calcified plaques in the sample surface reconstruction image.

[0098] Step S505: Take the union of the contours of the calcified plaques, the initial contours of the non-calcified plaques, and the lumen contour in the sample surface reconstruction image to obtain the initial outer wall contour of the blood vessel in the sample surface reconstruction image.

[0099] Step S506: Based on the initial outer wall contour of the blood vessel, use the level set method to segment the sample surface reconstruction image to obtain the outer wall contour of the blood vessel in the sample surface reconstruction image.

[0100] Step S507: Based on the lumen contour and the outer wall contour in the sample surface reconstruction image, obtain the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image.

[0101] Step S508: Based on the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image, obtain the gold standard annotation information of the sample surface reconstruction image. Among them, the gold standard annotation information of the sample surface reconstruction image can, for example, include one or more of the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image. The calcified plaque segmentation result is, for example, the contour of the calcified plaque, and the non-calcified plaque segmentation result is, for example, the contour of the non-calcified plaque.

[0102] Step S509: Use the sample surface reconstruction image and its gold standard annotation information to train a preset deep neural network to obtain the lumen and plaque segmentation model.

[0103] In a specific application, the mean of the CT values of the normal vascular segments in the sample surface reconstruction image can be denoted as mean, and the variance can be denoted as std. Set the threshold as mean + 3×std. Voxels greater than this threshold are calcified plaques. Please refer to Figure 9 , as shown in 9a for the SCPR image, the mean and variance of the normal vascular segments are statistically analyzed for the SCPR image as shown in 9b. After setting the threshold mean + 3×std, it is as shown in 9c. The initial contour of the vessel wall is determined according to the threshold as shown in 9d. The segmentation effect of the calcified plaque area and the non-calcified plaque area is robust and good. In this case, the segmentation of the calcified plaque area is more accurate and the effect is robust.

[0104] Thus, on the one hand, based on the plaque annotation information of the sample surface reconstruction image, obtain the mean (mean) and variance (std) of the CT values of the normal vascular segments in the sample surface reconstruction image, and determine the initial contour of the lumen, the contour of each calcified plaque, and the original contour of each non-calcified plaque in the sample surface reconstruction image based on the mean and variance of the CT values of the normal vascular segments; based on the initial contour of the lumen, use the level set method to segment the sample surface reconstruction image to obtain the lumen contour in the sample surface reconstruction image; based on the initial contour of the vessel outer wall, use the level set method to segment the sample surface reconstruction image to obtain the vessel outer wall contour in the sample surface reconstruction image.

[0105] On the one hand, based on the original contour of each non-calcified plaque, obtain the initial contour of each non-calcified plaque in the sample surface reconstruction image, and take the union of the contour of each calcified plaque, the initial contour of each non-calcified plaque, and the lumen contour in the sample surface reconstruction image to obtain the initial contour of the vessel outer wall in the sample surface reconstruction image.

[0106] Based on the obtained vessel outer wall contour and the lumen contour in the sample surface reconstruction image, the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image can be obtained. Based on the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image, obtain the gold standard annotation information of the sample surface reconstruction image. Use the sample surface reconstruction image and its gold standard annotation information to train a preset deep neural network to obtain a lumen and plaque segmentation model.

[0107] Please refer to Figure 11 , through the training of the lumen and plaque segmentation model, before the lumen and plaque segmentation is as shown in 11a, and the lumen and plaque after the model segmentation are as shown in 11b. Thus, through the training of the lumen and plaque segmentation model, the accuracy and speed of the lumen and plaque segmentation model for segmenting the contour of each calcified plaque, the initial contour of each non-calcified plaque, and the lumen contour can be improved.

[0108] Refer to Figure 12In some embodiments, step S504 may include steps S601 to S605.

[0109] Step S601: Based on the original contours of the non-calcified plaques, for each non-calcified plaque, intercept the regional block image corresponding to the non-calcified plaque.

[0110] Step S602: Obtain the ratio of the diameters of the normal blood vessels at both ends of the regional block image to the diameter of the blood vessel at the narrowest part of the lesion area of the non-calcified plaque.

[0111] Step S603: Based on the ratio, perform dilation processing on the regional block image to obtain the dilated image corresponding to the regional block image.

[0112] Step S604: Statistically calculate the mean and variance of the CT values in the dilated image after removing the lumen area, and perform pixel normalization on the CT values of the dilated image to [-1, 1] based on the statistical results.

[0113] Step S605: Remove the areas with CT values greater than 0 and equal to -1 in the dilated image, and the remaining part is used as the initial contour of the non-calcified plaque.

[0114] See Figure 13 , the regional block image corresponding to the non-calcified plaque is shown in Fig. 13a; obtain the ratio of the diameters of the normal blood vessels at both ends of the regional block image to the diameter of the blood vessel at the narrowest part of the lesion area of the non-calcified plaque, and perform dilation processing on the regional block image according to the obtained ratio to obtain the dilated image corresponding to the regional block image as shown in Fig. 13b; statistically calculate the mean and variance of the CT values in the dilated image after removing the lumen area, and perform pixel normalization on the CT values of the dilated image to obtain the initial contour image of the non-calcified plaque as shown in Fig. 13c.

[0115] Thus, based on the original contours of the non-calcified plaques, the initial contours of the non-calcified plaques in the sample surface reconstruction image can be obtained, and the obtained initial contours of the non-calcified plaques are used to obtain the initial contour of the blood vessel outer wall, which improves the accuracy and speed of the initial contour segmentation of the non-calcified plaques by the lumen and plaque segmentation model.

[0116] See Figure 14 , the embodiment of the present application provides an image segmentation device, and its specific implementation manner is consistent with the implementation manner and the achieved technical effects recorded in the embodiment of the above image segmentation method, and some contents will not be repeated.

[0117] The device includes: a first segmentation module 101, configured to segment coronary blood vessels in the image to be segmented by using a coronary segmentation model, so as to obtain a coronary blood vessel segmentation result corresponding to the image to be segmented; a surface reconstruction module 102, configured to perform surface reconstruction based on the coronary blood vessel segmentation result corresponding to the image to be segmented, so as to obtain a surface reconstruction image corresponding to the image to be segmented; and a second segmentation module 103, configured to segment the surface reconstruction image corresponding to the image to be segmented by using a lumen and plaque segmentation model, so as to obtain a lumen and plaque segmentation result corresponding to the image to be segmented.

[0118] See Figure 15 , in some alternative embodiments, the second segmentation module 103 may include: an image detection sub-module 201, configured to detect the surface reconstruction image corresponding to the image to be segmented by using a plaque detection network, so as to obtain an original attention map corresponding to the image to be segmented; and a model segmentation sub-module 202, configured to input the surface reconstruction image corresponding to the image to be segmented and the original attention map into the lumen and plaque segmentation model, so as to obtain a lumen and plaque segmentation result corresponding to the image to be segmented.

[0119] See Figure 16 , in some alternative embodiments, the image detection sub-module 201 may include: a feature map unit 301, configured to input the surface reconstruction image corresponding to the image to be segmented into a feature extraction module of the plaque detection network, so as to obtain a multi-channel feature map; a classifier unit 302, configured to input the multi-channel feature map into a classifier of the plaque detection network, so as to obtain a probability vector of each plaque category; a weight calculation unit 303, configured to calculate a sensitivity weight of each plaque category to each channel based on the probability vector of each plaque category; and an attention acquisition unit 304, configured to obtain an original attention map corresponding to the image to be segmented based on the multi-channel feature map and the sensitivity weight of each plaque category to each channel.

[0120] In some alternative embodiments, the weight calculation unit 303 may be configured to calculate a global average value of partial derivatives of the probability vector of each plaque category with respect to each voxel in each channel in three dimensions of width, height, and depth as the sensitivity weight of each plaque category to each channel.

[0121] See Figure 17 , in some alternative embodiments, the attention acquisition unit 304 may include: a category weighting sub-unit 401, configured to perform weighted summation on the multi-channel feature map based on the sensitivity weight of each plaque category to each channel, so as to obtain a weighted summation result of each plaque category; and an activation function sub-unit 402, configured to input the weighted summation result of each plaque category into an activation function layer, so as to obtain an original attention map corresponding to the image to be segmented.

[0122] In some alternative embodiments, the model segmentation sub-module 202 may include: a first sampling unit, configured to sequentially pass the surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; wherein, for each downsampling layer, the multiplication result of the feature map to be input of the downsampling layer and an attention map of the same size is used as the input image of the downsampling layer, and the attention map of the same size is obtained by using the original attention map.

[0123] In some alternative embodiments, the model segmentation sub-module 202 may include: a second sampling unit, configured to sequentially pass the surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; wherein, for each upsampling layer, the fusion result of the output image of the upsampling layer and the feature map to be input of the corresponding downsampling layer is used as the input image of the next upsampling layer.

[0124] In some alternative embodiments, the training process of the lumen and plaque segmentation model is as follows: obtaining a sample surface reconstruction image; based on the plaque annotation information of the sample surface reconstruction image, obtaining the mean and variance of the CT values of the normal blood vessel segments in the sample surface reconstruction image, and determining the initial lumen contour, the contours of each calcified plaque, and the original contours of each non-calcified plaque in the sample surface reconstruction image based on the mean and variance of the CT values of the normal blood vessel segments; wherein, the plaque annotation information is used to indicate the plaque segmentation result; based on the initial lumen contour, using the level set method to segment the sample surface reconstruction image to obtain the lumen contour in the sample surface reconstruction image; based on the original contours of each non-calcified plaque, using the level set method to obtain the initial contours of each non-calcified plaque in the sample surface reconstruction image; taking the union of the contours of each calcified plaque, the initial contours of each non-calcified plaque, and the lumen contour in the sample surface reconstruction image to obtain the initial contour of the blood vessel outer wall in the sample surface reconstruction image; based on the initial contour of the blood vessel outer wall, using the level set method to segment the sample surface reconstruction image to obtain the blood vessel outer wall contour in the sample surface reconstruction image; based on the lumen contour and the blood vessel outer wall contour in the sample surface reconstruction image, obtaining the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image; based on the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image, obtaining the gold standard annotation information of the sample surface reconstruction image; and training a preset deep neural network by using the sample surface reconstruction image and its gold standard annotation information to obtain the lumen and plaque segmentation model.

[0125] In some alternative embodiments, obtaining the initial contour of each non-calcified plaque in the sample surface reconstruction image based on the original contour of each non-calcified plaque includes: based on the original contour of each non-calcified plaque, for each non-calcified plaque, intercepting the regional block image corresponding to the non-calcified plaque; obtaining the ratio of the diameters of the normal blood vessels at both ends of the regional block image to the diameter of the blood vessel at the narrowest part of the lesion area of the non-calcified plaque; based on the ratio, performing dilation processing on the regional block image to obtain the dilated image corresponding to the regional block image; statistically calculating the mean and variance of the CT values in the dilated image after removing the lumen area, and performing pixel normalization of the CT values of the dilated image to [-1, 1] based on the statistical results; removing the areas where the CT values are greater than 0 and equal to -1 in the dilated image, and taking the remaining part as the initial contour of the non-calcified plaque.

[0126] See Figure 18 , the embodiment of the present application further provides an electronic device 200, and the electronic device 200 includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0127] The memory 210 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 211 and / or a cache memory 212, and may further include a read-only memory (ROM) 213.

[0128] Among them, the memory 210 also stores a computer program, and the computer program can be executed by the processor 220, so that the processor 220 executes the steps of the image segmentation method in the embodiment of the present application. The specific implementation manner is the same as the implementation manner and the achieved technical effects recorded in the embodiment of the above image segmentation method, and some contents will not be elaborated.

[0129] The memory 210 may further include a utility 214 having at least one program module 215. Such program modules 215 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0130] Correspondingly, the processor 220 can execute the above computer program and can also execute the utility 214.

[0131] The bus 230 may represent one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure in a variety of bus structures.

[0132] The electronic device 200 can also communicate with one or more external devices 240 such as a keyboard, a pointing device, a Bluetooth device, etc., and can also communicate with one or more devices capable of interacting with the electronic device 200, and / or communicate with any device (such as a router, a modem, etc.) that enables the electronic device 200 to communicate with one or more other computing devices. Such communication can be carried out through the input / output interface 250. Moreover, the electronic device 200 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 260. The network adapter 260 can communicate with other modules of the electronic device 200 through the bus 230. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0133] An embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed, the steps of the image segmentation method in the embodiment of the present application are implemented. Its specific implementation manner is consistent with the implementation manner and the achieved technical effects described in the embodiment of the above image segmentation method, and some contents will not be repeated.

[0134] Figure 19 A program product 300 for implementing the above image segmentation method provided in this embodiment is shown. It can be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product 300 of the present invention is not limited thereto. In the present application, the readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device. The program product 300 can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0135] A computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable storage medium may also be any readable medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing. The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the C language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0136] This application is described from the viewpoints of purpose of use, efficacy, progress, and novelty, and has met the requirements of function improvement and use emphasized by the patent law. The above description and the accompanying drawings of this application are only preferred embodiments of this application, and do not limit this application thereto. Therefore, all those that are similar or identical to the structure, device, features, etc. of this application, that is, all equivalent substitutions or modifications made according to the scope of the patent application of this application, shall fall within the scope of protection of the patent application of this application.

Claims

1. An image segmentation method, characterized in that, The method includes: Segmenting coronary arteries in the image to be segmented using a coronary artery segmentation model to obtain a coronary artery segmentation result corresponding to the image to be segmented; Performing surface reconstruction based on the coronary artery segmentation result corresponding to the image to be segmented to obtain a surface reconstruction image corresponding to the image to be segmented; Segmenting the surface reconstruction image corresponding to the image to be segmented using a lumen and plaque segmentation model to obtain a lumen and plaque segmentation result corresponding to the image to be segmented; The segmenting the surface reconstruction image corresponding to the image to be segmented using a lumen and plaque segmentation model to obtain a lumen and plaque segmentation result corresponding to the image to be segmented includes: Detecting the surface reconstruction image corresponding to the image to be segmented using a plaque detection network to obtain an original attention map corresponding to the image to be segmented; Inputting the surface reconstruction image corresponding to the image to be segmented and the original attention map into the lumen and plaque segmentation model to obtain a lumen and plaque segmentation result corresponding to the image to be segmented.

2. The image segmentation method according to claim 1, characterized in that, The detecting the surface reconstruction image corresponding to the image to be segmented using a plaque detection network to obtain an original attention map corresponding to the image to be segmented includes: Inputting the surface reconstruction image corresponding to the image to be segmented into a feature extraction module of the plaque detection network to obtain a multi-channel feature map; Inputting the multi-channel feature map into a classifier of the plaque detection network to obtain a probability vector for each plaque category; Calculating a sensitivity weight of each plaque category to each channel based on the probability vector for each plaque category; Obtaining the original attention map corresponding to the image to be segmented based on the multi-channel feature map and the sensitivity weight of each plaque category to each channel.

3. The image segmentation method according to claim 2, wherein The calculating a sensitivity weight of each plaque category to each channel based on the probability vector for each plaque category includes: Calculating a global average value of partial derivatives of the probability vector for each plaque category with respect to each voxel in each channel in three dimensions of width, height, and depth as the sensitivity weight of each plaque category to each channel.

4. The image segmentation method according to claim 2, wherein The obtaining the original attention map corresponding to the image to be segmented based on the multi-channel feature map and the sensitivity weight of each plaque category to each channel includes: Performing weighted summation on the multi-channel feature map based on the sensitivity weight of each plaque category to each channel to obtain a weighted summation result for each plaque category; Inputting the weighted summation result for each plaque category into an activation function layer to obtain the original attention map corresponding to the image to be segmented.

5. The image segmentation method according to claim 1, wherein The inputting the surface reconstruction image corresponding to the image to be segmented and the original attention map into the lumen and plaque segmentation model to obtain a lumen and plaque segmentation result corresponding to the image to be segmented includes: Sequentially passing the surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain a lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; Wherein, for each downsampling layer, a multiplication result of the feature map to be input to the downsampling layer and an attention map of the same size is used as the input image of the downsampling layer, and the attention map of the same size is obtained using the original attention map.

6. The image segmentation method according to claim 1, wherein Feeding the surface reconstruction image and the original attention map corresponding to the image to be segmented into the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented, includes: Sequentially passing the surface reconstruction image corresponding to the image to be segmented through N downsampling layers and N upsampling layers of the lumen and plaque segmentation model to obtain the lumen and plaque segmentation result corresponding to the image to be segmented, where N is a positive integer; Wherein, for each upsampling layer, the fusion result of the output image of the upsampling layer and the input feature map to be input of the corresponding downsampling layer is used as the input image of the next upsampling layer.

7. The image segmentation method according to claim 1, wherein The training process of the lumen and plaque segmentation model is as follows: Obtaining a sample surface reconstruction image; Based on the plaque annotation information of the sample surface reconstruction image, obtaining the mean and variance of the CT values of the normal blood vessel segments in the sample surface reconstruction image, and determining the initial lumen contour, the contour of each calcified plaque, and the original contour of each non-calcified plaque in the sample surface reconstruction image based on the mean and variance of the CT values of the normal blood vessel segments; wherein, the plaque annotation information is used to indicate the plaque segmentation result; Based on the initial lumen contour, using the level set method to segment the sample surface reconstruction image to obtain the lumen contour in the sample surface reconstruction image; Based on the original contours of the non-calcified plaques, using the level set method to obtain the initial contours of the non-calcified plaques in the sample surface reconstruction image; Taking the union of the contours of the calcified plaques, the initial contours of the non-calcified plaques, and the lumen contour in the sample surface reconstruction image to obtain the initial contour of the blood vessel outer wall in the sample surface reconstruction image; Based on the initial contour of the blood vessel outer wall, using the level set method to segment the sample surface reconstruction image to obtain the blood vessel outer wall contour in the sample surface reconstruction image; Based on the lumen contour and the blood vessel outer wall contour in the sample surface reconstruction image, obtaining the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image; Based on the lumen segmentation result, the calcified plaque segmentation result, and the non-calcified plaque segmentation result of the sample surface reconstruction image, obtaining the gold standard annotation information of the sample surface reconstruction image; Training a preset deep neural network using the sample surface reconstruction image and its gold standard annotation information to obtain the lumen and plaque segmentation model.

8. The image segmentation method according to claim 7, characterized in that, The obtaining the initial contours of the non-calcified plaques in the sample surface reconstruction image based on the original contours of the non-calcified plaques using the level set method includes: Based on the original contours of the non-calcified plaques, for each non-calcified plaque, intercepting the regional block image corresponding to the non-calcified plaque; Obtaining the ratio of the diameters of the normal blood vessels at both ends of the regional block image to the diameter of the blood vessel at the narrowest part of the lesion area of the non-calcified plaque; Based on the ratio, performing dilation processing on the regional block image to obtain the dilated image corresponding to the regional block image; Statistically calculating the mean and variance of the CT values in the dilated image after removing the lumen area, and normalizing the CT values of the dilated image to [-1, 1] based on the statistical results; Remove the regions with CT values greater than 0 and equal to -1 within the dilated image, and use the remaining part as the initial contour of the non-calcified plaque.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image segmentation method according to any one of claims 1-8 are implemented.

Images