Image segmentation method and apparatus

Abstract

The specification provides an image segmentation method and device. The method comprises: acquiring a contrast image to be segmented; the contrast image contains an image region corresponding to a calcified plaque; inputting the contrast image into a pre-trained image segmentation model, identifying the image region corresponding to the calcified plaque contained in the contrast image by the image segmentation model, and performing image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque; wherein the loss function used for training the image segmentation model is a focal loss function of weighted accuracy; the focal loss function of weighted accuracy sets weights for first-class pixels identified as calcified plaques and second-class pixels identified as non-calcified plaques, and the weight of the first-class pixels is greater than the weight of the second-class pixels, so that the image segmentation model focuses on the region where the first-class pixels are located when performing image segmentation on the contrast image.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of image processing, and in particular to an image segmentation method and apparatus. Background Technology

[0002] Coronary artery disease is a common cardiovascular disease, and studies have shown that coronary atherosclerosis is the main cause of coronary artery disease. Generally, coronary atherosclerotic plaques can be classified according to their nature into non-calcified plaques, calcified plaques, and mixed plaques.

[0003] Compared to non-calcified plaques and mixed plaques, calcified plaques are generally more stable in morphology because they usually occur in the late stage of coronary atherosclerosis. Therefore, calcified plaques are often used in clinical practice to assess the degree of coronary atherosclerosis in patients.

[0004] However, calcified plaques are not prominent in the angiographic images of a patient's coronary arteries. For example, only a small percentage of angiographic images show calcified plaques, and in angiographic images that do show calcified plaques, the vast majority of the calcified plaques are very small and usually occupy only a very small percentage of the angiographic image (about 0.04%).

[0005] Therefore, how to accurately identify and segment inconspicuous calcified plaques has become an urgent problem to be solved. Summary of the Invention

[0006] This specification provides an image segmentation method and apparatus to address the problem of inaccurate segmentation of calcified plaques in coronary arteries.

[0007] According to a first aspect of the embodiments of this specification, an image segmentation method is provided, the method comprising:

[0008] Obtain the contrast image to be segmented; wherein the contrast image contains an image region corresponding to the calcified plaque;

[0009] The contrast image is input into a pre-trained image segmentation model, which identifies the image region in the contrast image that corresponds to the calcified plaque, and performs image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque.

[0010] The loss function used to train the image segmentation model is a weighted accuracy focus loss function. In the weighted accuracy focus loss function, weights are set for the first type of pixels identified as calcified patches and the second type of pixels identified as non-calcified patches, and the weight of the first type of pixels is greater than the weight of the second type of pixels, so that the image segmentation model focuses on the region where the first type of pixels are located when performing image segmentation on the imaging image.

[0011] Optionally, the first type of pixels includes true first type pixels that are correctly identified as calcified patches and false first type pixels that are incorrectly identified as calcified patches; the second type of pixels includes true second type pixels that are correctly identified as non-calcified patches and false second type pixels that are incorrectly identified as non-calcified patches.

[0012] The focus loss function for weighted accuracy also sets penalty weights for false first-class pixels and false second-class pixels respectively;

[0013] The weight of the first type of pixels is greater than the weight of the second type of pixels, including:

[0014] The weight of a true first-class pixel is greater than the weight of a true second-class pixel, and the sum of the weights of the true first-class pixels and the weights of the true second-class pixels is 1.

[0015] Optionally, the focus loss function for the weighted accuracy includes:

[0016] r is the focusing factor, AC ωt The weighted accuracy calculation index;

[0017] The weighted accuracy calculation metrics include:

[0018]

[0019] Where TP represents true first-class pixels, TN represents true second-class pixels, and F represents true first-class pixels. P FN represents a pseudo-first-class pixel, and FN represents a pseudo-second-class pixel; w P The weight coefficients of the true first-class pixels, w n α is the weight coefficient for true second-class pixels, β is the penalty weight coefficient for false first-class pixels, β is the penalty weight coefficient for false second-class pixels, and ω is other coefficients.

[0020] Optionally, the image segmentation model includes an encoder network and a decoder network, wherein the encoder network employs the focus loss function of the weighted accuracy; the method further includes:

[0021] Obtain training samples; wherein, the training samples include sample imaging images with correctly segmented calcified plaque annotation regions;

[0022] The training samples are input into the image segmentation model for supervised training. The encoder network extracts pixel features from the input sample imaging image, and the decoder network classifies the extracted pixel features.

[0023] The classification result is calculated as the difference between the predicted calcified patch region composed of first-class pixels and the labeled calcified patch region, and the difference is used to adjust the coefficients in the focus loss function of the weighted accuracy.

[0024] Optionally, the decoder network uses a preset index evaluation algorithm to evaluate the image segmentation model after adjusting the coefficients in the focus loss function with weighted accuracy;

[0025] If the evaluation result does not meet the preset conditions, the image segmentation model is iteratively trained until the evaluation result meets the preset conditions.

[0026] Optionally, the preset indicator evaluation algorithm includes:

[0027]

[0028] Where k is the number of batches set for model training, TP i For the true first-class pixels in the i-th batch, FP i For the pseudo-first class pixels in the i-th batch, FN i These are the pseudo-second-class pixels during the i-th batch processing.

[0029] Optionally, the step of inputting the contrast image into a pre-trained image segmentation model, whereby the image segmentation model identifies the image region in the contrast image corresponding to the calcified plaque, and performs image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque, includes:

[0030] The contrast image is input into the trained image segmentation model, and the encoder network of the image segmentation model extracts pixel features from the input contrast image.

[0031] The decoder network of the image segmentation model classifies the extracted pixel features, determines whether the pixel corresponding to each pixel feature belongs to the first type of pixel, and performs image segmentation on the image region composed of the first type of pixels in the imaging image to obtain the segmented image of the calcified plaque.

[0032] Optionally, the image segmentation model includes an improved model of the U-Net neural network.

[0033] Optionally, the improved model of the U-Net neural network includes the Res-Unet neural network model.

[0034] Optionally, the angiographic images include angiographic images of the coronary arteries.

[0035] According to a second aspect of the embodiments of this specification, an image segmentation apparatus is provided, the apparatus comprising:

[0036] An acquisition unit acquires an imaging image to be segmented; wherein the imaging image contains an image region corresponding to the calcified plaque;

[0037] The segmentation unit inputs the contrast image into a pre-trained image segmentation model, which identifies the image region in the contrast image that corresponds to the calcified plaque, and performs image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque.

[0038] The loss function used to train the image segmentation model is a weighted accuracy focus loss function. In the weighted accuracy focus loss function, weights are set for the first type of pixels identified as calcified patches and the second type of pixels identified as non-calcified patches, and the weight of the first type of pixels is greater than the weight of the second type of pixels, so that the image segmentation model focuses on the region where the first type of pixels are located when performing image segmentation on the imaging image.

[0039] Optionally, the first type of pixels includes true first type pixels that are correctly identified as calcified patches and false first type pixels that are incorrectly identified as calcified patches; the second type of pixels includes true second type pixels that are correctly identified as non-calcified patches and false second type pixels that are incorrectly identified as non-calcified patches.

[0040] The focus loss function for weighted accuracy also sets penalty weights for false first-class pixels and false second-class pixels respectively;

[0041] The weight of the first type of pixels is greater than the weight of the second type of pixels, including:

[0042] The weight of a true first-class pixel is greater than the weight of a true second-class pixel, and the sum of the weights of the true first-class pixels and the weights of the true second-class pixels is 1.

[0043] Optionally, the focus loss function for the weighted accuracy includes:

[0044] r is the focusing factor, AC ωt The weighted accuracy calculation index;

[0045] The weighted accuracy calculation metrics include:

[0046]

[0047] Where TP represents true first-class pixels, TN represents true second-class pixels, and F represents true first-class pixels. P FN represents a pseudo-first-class pixel, and FN represents a pseudo-second-class pixel; w P The weight coefficients of the true first-class pixels, w n α is the weight coefficient for true second-class pixels, β is the penalty weight coefficient for false first-class pixels, and ε is other coefficients.

[0048] Optionally, the image segmentation model includes an encoder network and a decoder network, wherein the encoder network employs the focus loss function of the weighted accuracy; the apparatus further includes:

[0049] The training unit acquires training samples, which include angiographic images of correctly segmented calcified plaque regions. The training samples are input into an image segmentation model for supervised training. The encoder network extracts pixel features from the input angiographic images, and the decoder network classifies the extracted pixel features. The classification result is calculated as the difference between the predicted calcified plaque region composed of first-class pixels and the labeled calcified plaque region. This difference is then used to adjust the coefficients in the focus loss function of the weighted accuracy.

[0050] Optionally, the decoder network uses a preset index evaluation algorithm to evaluate the image segmentation model after adjusting the coefficients in the focus loss function with weighted accuracy; and if the evaluation result does not meet the preset conditions, the image segmentation model is iteratively trained until the evaluation result meets the preset conditions.

[0051] Optionally, the preset indicator evaluation algorithm includes:

[0052]

[0053] Where k is the number of batches set for model training, TP i For the true first-class pixels in the i-th batch, FP i For the pseudo-first class pixels in the i-th batch, FN i These are the pseudo-second-class pixels during the i-th batch processing.

[0054] Optionally, the segmentation unit includes:

[0055] The feature extraction subunit inputs the angiography image into the trained image segmentation model, where the encoder network of the image segmentation model extracts pixel features from the input angiography image.

[0056] The feature classification subunit classifies the extracted pixel features by the decoder network of the image segmentation model to determine whether the pixel corresponding to each pixel feature belongs to the first type of pixel.

[0057] The region segmentation subunit performs image segmentation on the image region composed of the first type of pixels in the imaging image to obtain a segmented image of the calcified plaque.

[0058] Optionally, the image segmentation model includes an improved model of the U-Net neural network.

[0059] Optionally, the improved model of the U-Net neural network includes the Res-Unet neural network model.

[0060] Optionally, the angiographic images include angiographic images of the coronary arteries.

[0061] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising:

[0062] processor;

[0063] Memory used to store processor-executable instructions;

[0064] The processor is configured to use any of the methods described above.

[0065] This specification provides an image segmentation scheme. An image segmentation model is trained using a weighted accuracy focus loss function. Since the weighted accuracy focus loss function assigns weights to first-class pixels identified as calcified patches and second-class pixels identified as non-calcified patches, with the weight of the first-class pixels being greater than that of the second-class pixels, the image segmentation model can focus on the image region of the calcified patches where the first-class pixels are located when segmenting the contrast image, thereby improving the accuracy of the image segmentation model in identifying and segmenting calcified patches. Attached Figure Description

[0066] Figure 1 This is a flowchart of an image segmentation method provided in one embodiment of this specification;

[0067] Figure 2 This is a schematic diagram of the structure of an image segmentation model provided in one embodiment of this specification;

[0068] Figure 3 This is an overall flowchart provided in one embodiment of this specification;

[0069] Figure 4 This is a schematic diagram of the segmentation result of an imaging image provided in one embodiment of this specification;

[0070] Figure 5 This is a hardware structure diagram of an image segmentation apparatus provided in one embodiment of this specification;

[0071] Figure 6 This is a module of the image segmentation apparatus provided in one embodiment of this specification. Detailed Implementation

[0072] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0073] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0074] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0075] With the continuous development and application of Convolutional Neural Networks (CNN) and Deep Learning (DL) in the field of computer vision, many well-known medical image segmentation models have emerged, such as the classic U-Net neural network model and 3D U-Net, which is an improvement on U-Net.

[0076] While these medical image segmentation models perform well in common medical image segmentation, they struggle to achieve ideal segmentation results in specialized medical images such as calcified plaques in the coronary arteries.

[0077] As mentioned earlier, since calcified plaques are not prominent in the coronary angiography images of patients, they exhibit a data type distribution with maximum class imbalance. The maximum class imbalance is mainly reflected in:

[0078] 1) Only a very small percentage of contrast images show calcified plaques.

[0079] 2) In contrast images with calcified plaques, the vast majority of calcified plaques are very small. Statistics show that the area of ​​calcified plaques accounts for only about 0.04% of the total area of ​​the contrast image on average.

[0080] Due to the aforementioned extreme class imbalance, the medical image segmentation model cannot identify these small-scale calcified plaques, thus failing to accurately segment them.

[0081] To address the aforementioned issues, this specification aims to provide an image segmentation model capable of identifying calcified patches, thereby improving the accuracy of calcified patch segmentation.

[0082] Please refer to the following: Figure 1 The diagram shown illustrates an image segmentation method that may include the following steps:

[0083] Step 210: Obtain the contrast image to be segmented; wherein the contrast image contains an image region corresponding to the calcified plaque.

[0084] The angiographic images in this specification may include angiographic images of the coronary arteries, or other angiographic images of the coronary arteries that have inconspicuous calcified plaques.

[0085] Taking coronary arteries as an example, the angiographic images can refer to CT (Computed Tomography) images obtained by performing angiography on the coronary arteries using CT angiography (CTA) technology. Because CTA technology is non-invasive, provides good three-dimensional spatial structure, and contains more information about calcified plaques, it can improve the clarity and accuracy of coronary artery imaging.

[0086] Since the original contrast images may contain noise that interferes with image segmentation, it is generally advisable to preprocess the original contrast images first.

[0087] The preprocessing may include, but is not limited to:

[0088] Format conversion processing, that is, for imaging images of different data formats, the data format can be converted into a preset format for input into the image segmentation model;

[0089] Center cropping is a process that crops the input angiographic image so that the imaging content of the coronary arteries is placed in the center of the cropped angiographic image.

[0090] Image size processing refers to converting imaging images of different sizes into preset sizes for input into the image segmentation model.

[0091] Step 220: Input the contrast image into a pre-trained image segmentation model. The image segmentation model identifies the image region in the contrast image that corresponds to the calcified plaque and performs image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque.

[0092] The loss function used to train the image segmentation model is a weighted accuracy focus loss function. In the weighted accuracy focus loss function, weights are set for the first type of pixels identified as calcified patches and the second type of pixels identified as non-calcified patches, and the weight of the first type of pixels is greater than the weight of the second type of pixels, so that the image segmentation model focuses on the region where the first type of pixels are located when performing image segmentation on the imaging image.

[0093] The image segmentation model described in this specification may be an improved version of the U-Net neural network model.

[0094] For the traditional U-Net neural network model, its model structure is as follows: Figure 2 As shown, the left side can be viewed as an encoder network, and the right side as a decoder network. The encoder network can include four convolutional modules, each of which can contain two convolutional layers, followed by a downsampling layer implemented using max pooling.

[0095] like Figure 2 As shown, the input image resolution can be 572x572. After convolution processing by four convolution modules and max pooling by downsampling layers, the output resolutions are 572x572, 284x284, 140x140, 68x68 and 32x32, respectively.

[0096] The decoder network can also include four convolutional modules, each used to upsample the input. Unlike the encoder's convolutional modules, the decoder's four convolutional modules perform convolution and upsampling, and the output resolution increases sequentially until it matches the resolution of the input image.

[0097] In addition, the U-Net neural network model also uses skip connections, which connect the upsampling result to the output of the convolutional module with the same resolution in the encoder network, and use it as the input to the next convolutional module in the decoder network.

[0098] It should be noted that the number of convolutional modules and the resolution values ​​mentioned above are just examples. In actual applications, they can be flexibly configured according to actual needs.

[0099] The image segmentation model in this manual improves upon the U-Net neural network model in the following ways:

[0100] The convolutional modules in the U-Net neural network model were replaced with residual networks (ResNet), and the improved model can be called the Res-Unet neural network model. The skip connections used in the U-Net neural network model were replaced with residual connections. By using residual connections, the feature information lost during the convolution process of the convolutional modules can be supplemented.

[0101] For the improved image segmentation model described above, its performance typically depends on model training, especially the design of the loss function during training. Designing an effective loss function is crucial for model performance. Before introducing the loss function designed in this embodiment, we will first introduce the basic concepts of loss functions.

[0102] The loss function is an important concept in machine learning. It is used to measure the deviation between the predicted value (model output) and the actual value (sample label). The smaller the loss function, the closer the predicted value is to the actual value, which also means that the model performs better.

[0103] Returning to the aforementioned traditional medical image segmentation models, such as the U-Net neural network model, the loss function used during model training is generally the DICE loss function or the cross-entropy loss function, etc. Using these loss functions will cause the encoder network to be unable to effectively extract calcified plaque features during model training.

[0104] Analysis revealed that the reason why the encoder network could not effectively extract calcified plaque features during model training was that the aforementioned calcified plaques were not significant in the angiographic images of the patient's coronary arteries, exhibiting a maximally imbalanced data type distribution.

[0105] Traditional medical image segmentation models use loss functions that assign equal penalty weights to false second-class pixels and false first-class pixels, thus failing to reflect the importance of first-class pixels, such as calcified plaques, which constitute a minority.

[0106] In related technologies, there is also the use of focus loss function to focus on pixels that are not easy to classify. However, when the focus loss function is designed, the focus object is true first-class pixels. Calcified plaques are not significant in the angiographic images of patients' coronary arteries. Most angiographic images do not contain first-class pixels, so they cannot reflect the calcified plaque features of the coronary arteries and cannot effectively train the encoder network to extract coronary plaque features.

[0107] Therefore, while improving the model architecture, this manual also designs a brand-new weighted accuracy focus loss function, which is then used to train the improved model.

[0108] It should be noted that the focus loss function for weighted accuracy in this specification is different from the traditional focus loss function. Specifically, the focus loss function for weighted accuracy assigns weights to the first type of pixels identified as calcified patches and the second type of pixels identified as non-calcified patches, and the weight of the first type of pixels is greater than the weight of the second type of pixels. This allows the image segmentation model to focus on the calcified patch region where the first type of pixels are located when segmenting calcified patches in the imaging image.

[0109] In one exemplary embodiment, the first type of pixels includes true first type pixels that are correctly identified as calcified patches and false first type pixels that are incorrectly identified as calcified patches; the second type of pixels includes true second type pixels that are correctly identified as non-calcified patches and false second type pixels that are incorrectly identified as non-calcified patches.

[0110] Accordingly, the weight of the first type of pixels is greater than the weight of the second type of pixels, including:

[0111] The weight of a true first-class pixel is greater than the weight of a true second-class pixel, and the sum of the weights of the true first-class pixels and the weights of the true second-class pixels is 1.

[0112] This setting allows the weights of true first-class pixels and true second-class pixels to be inversely proportional. Even if first-class pixels are not significant in the imaging image, they can be focused with a larger weight.

[0113] Furthermore, the focus loss function for weighted accuracy also incorporates penalty weights for false first-class pixels and false second-class pixels.

[0114] The following section will detail the Focal weighted intersection loss function provided in this manual.

[0115] The accuracy loss (FAL) is shown in Formula 1 below:

[0116]

[0117] Where r is the focusing factor, AC ωt The focus factor is a weighted accuracy calculation metric, where t represents the time series. For example, the numerical range of the focus factor can be [1, 5].

[0118] The weighted accuracy calculation index is shown in Formula 2 below:

[0119]

[0120] Where TP represents true first-class pixels, TN represents true second-class pixels, and F represents true first-class pixels. PFN represents a pseudo-first-class pixel, and FN represents a pseudo-second-class pixel; w P The weight coefficients of the true first-class pixels, w n α is the weight coefficient for true second-class pixels, β is the penalty weight coefficient for false first-class pixels, β is the penalty weight coefficient for false second-class pixels, and ω is other coefficients.

[0121] In formula 2 above, w is set to be inversely proportional. P and w n This can increase the weight of the first type of pixel and satisfy w P +w n The relationship is 1. If there are only second-class pixels, then since the first-class pixels are zero, the accuracy of the all-negative input image is 1, which can be called weighted accuracy. However, if the first-class pixels are included, then since the weight of the first-class pixels is very large, the effect of the second-class pixels can be ignored. Therefore, Equation 1, which contains Equation 2 above, better reflects the distribution of calcified plaques in the coronary arteries and is more suitable for training the aforementioned improved image segmentation image (the preset image model).

[0122] After designing the weighted accuracy focus loss function shown in Formula 1 above, this weighted accuracy focus loss function can be used to train an image segmentation model; the image segmentation model, as previously described, may include an encoder network and a decoder network, wherein the encoder network adopts the weighted accuracy focus loss function; in the above... Figure 1 The illustrated embodiment may also include:

[0123] Obtain training samples; wherein, the training samples include sample imaging images with correctly segmented calcified plaque annotation regions;

[0124] The training samples are input into the image segmentation model for supervised training. The encoder network extracts pixel features from the input sample imaging image, and the decoder network classifies the extracted pixel features.

[0125] The classification result is calculated as the difference between the predicted calcified patch region composed of first-class pixels and the labeled calcified patch region, and the difference is used to adjust the coefficients (such as weight coefficients, penalty weight coefficients and other coefficients) in the focus loss function of the weighted accuracy.

[0126] It should be noted that, in adjusting the coefficients of the focus loss function for the weighted accuracy in the decoder network, optimization algorithms such as gradient descent can be used to minimize the loss function value and thus optimize the adjustment of the coefficients.

[0127] During the training process, the decoder network can also use a preset index evaluation algorithm to evaluate the image segmentation model after adjusting the focus loss function with weighted accuracy.

[0128] If the evaluation result does not meet the preset conditions, the image segmentation model is iteratively trained until the evaluation result meets the preset conditions.

[0129] It should be noted that traditional loss functions assign equal weights to false second-class pixels and false first-class pixels, which fails to reflect the importance of calcified patches, which are a minority of first-class pixels. This not only causes the encoder network to be unable to effectively extract calcified patch features, as mentioned earlier, but also makes it impossible for the decoder network to evaluate the model.

[0130] Here, due to design flaws in the metric evaluation algorithm used by the decoder network in traditional medical image segmentation models, please refer to Equation 3 for the metric evaluation algorithm used by the encoder network in traditional medical image segmentation models (such as U-Net):

[0131]

[0132] Where k is the number of batches set for model training, DICE i Let DICE be the DICE coefficient for the i-th batch. The problem with this metric evaluation algorithm is:

[0133] The DICE coefficient only focuses on the first type of pixels. Due to the insignificance of calcified patches, most batches lack the first type of pixels, resulting in a DICE of zero. After averaging, the evaluation index calculated by Formula 3 will be flooded with zero noise, thus failing to effectively evaluate the correctness of the output segmentation results during model training.

[0134] Therefore, this specification designs a novel index evaluation algorithm (i.e., the aforementioned preset index evaluation algorithm) for the decoder network of the aforementioned improved image segmentation model.

[0135] In one exemplary embodiment, the novel metric evaluation algorithm is shown in Formula 4 below:

[0136]

[0137] Where k is the number of batches set for model training, TP i For the true first-class pixels in the i-th batch, FP i For the pseudo-first class pixels in the i-th batch, FN i These are the pseudo-second-class pixels during the i-th batch processing.

[0138] The metric evaluation algorithm shown in Equation 4 is not limited to focusing on the first type of pixels during batch processing, but also considers the pseudo-first type pixels and pseudo-second type pixels during batch processing to comprehensively evaluate the metrics. This avoids the problem of the evaluation metrics being overwhelmed by zero noise, thus failing to effectively evaluate the correctness of the output segmentation results during model training.

[0139] Please refer to the following. Figure 3 The overall flowchart of the image segmentation scheme shown is as follows: Figure 3 The diagram can be divided into 5 parts.

[0140] The first part is the preprocessing of the contrast imaging image. By preprocessing the input contrast imaging image, some interference noise is removed. The preprocessed contrast imaging image can then be saved.

[0141] The second part is the annotation of calcified regions. Automatic annotation algorithms or manual annotation can be used to determine the annotated regions of calcified patches in the sample imaging images used as training samples.

[0142] The third part is the image segmentation model training. The image segmentation model is trained using the labeled training samples from the second part. Specifically, the encoder network of the image segmentation model can be trained using a focus loss function with weighted accuracy, and the decoder network can be trained using an improved metric evaluation algorithm.

[0143] The fourth part is calcified plaque segmentation. This fourth part may correspond to the aforementioned step 220, and may include, for example, the following:

[0144] The contrast image (after the first part of preprocessing) is input into the image segmentation model trained in the third part, and the encoder network of the image segmentation model extracts pixel features from the input contrast image.

[0145] Then, the decoder network of the image segmentation model classifies the extracted pixel features, determines whether the pixel corresponding to each pixel feature belongs to the first type of pixel, and performs image segmentation on the image region composed of the first type of pixels in the imaging image to obtain the segmented image of the calcified plaque.

[0146] The fifth part is the segmentation results presentation. Using visualization technology, the segmented images of the calcified plaques output from the fourth part are presented in a visual manner.

[0147] Please refer to the following. Figure 4The schematic diagram of the contrast-enhanced image shown illustrates that traditional medical segmentation models using conventional loss functions (such as U-Net) cannot identify the tiny calcified plaques 41 in the contrast-enhanced image. However, the image segmentation model trained using the focus loss function with weighted accuracy provided in this specification (i.e., the improved image segmentation model) can identify and segment these tiny calcified plaques 42. Figure 4 (Middle red area).

[0148] Corresponding to the foregoing embodiments of the image segmentation method, this specification also provides embodiments of an image segmentation apparatus. These apparatus embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logically defined apparatus, it is formed by the processor of its device reading the corresponding computer program from non-volatile memory into memory and running it. From a hardware perspective, such as... Figure 5 The diagram shown is a hardware structure diagram of the device containing the image segmentation apparatus described in this specification. (Except for...) Figure 5 In addition to the processor, network interface, memory, and non-volatile memory shown, the device in the embodiment may also include other hardware depending on the actual communication function, which will not be described in detail here.

[0149] Please see Figure 6 This is a block diagram of an image segmentation apparatus provided in one embodiment of this specification, the apparatus corresponding to... Figure 1 The illustrated embodiment shows that the apparatus includes:

[0150] The acquisition unit 610 acquires the contrast image to be segmented; wherein the contrast image contains an image region corresponding to the calcified plaque;

[0151] The segmentation unit 620 inputs the contrast image into a pre-trained image segmentation model, which identifies the image region in the contrast image that corresponds to the calcified plaque, and performs image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque.

[0152] The loss function used to train the image segmentation model is a weighted accuracy focus loss function. In the weighted accuracy focus loss function, weights are set for the first type of pixels identified as calcified patches and the second type of pixels identified as non-calcified patches, and the weight of the first type of pixels is greater than the weight of the second type of pixels, so that the image segmentation model focuses on the region where the first type of pixels are located when performing image segmentation on the imaging image.

[0153] In one exemplary embodiment, the first type of pixels includes true first type pixels that are correctly identified as calcified patches and false first type pixels that are incorrectly identified as calcified patches; the second type of pixels includes true second type pixels that are correctly identified as non-calcified patches and false second type pixels that are incorrectly identified as non-calcified patches.

[0154] The focus loss function for weighted accuracy also sets penalty weights for false first-class pixels and false second-class pixels respectively;

[0155] The weight of the first type of pixels is greater than the weight of the second type of pixels, including:

[0156] The weight of a true first-class pixel is greater than the weight of a true second-class pixel, and the sum of the weights of the true first-class pixels and the weights of the true second-class pixels is 1.

[0157] In one exemplary embodiment, the focus loss function for weighted accuracy includes:

[0158] Where r is the focusing factor, AC ωt The weighted accuracy calculation index;

[0159] The weighted accuracy calculation metrics include:

[0160]

[0161] Where TP represents true first-class pixels, TN represents true second-class pixels, and F represents true first-class pixels. P FN represents a pseudo-first-class pixel, and FN represents a pseudo-second-class pixel; w P The weight coefficients of the true first-class pixels, w n α is the weight coefficient for true second-class pixels, β is the penalty weight coefficient for false first-class pixels, and ε is other coefficients.

[0162] In one exemplary embodiment, the image segmentation model includes an encoder network and a decoder network, the encoder network employing the focus loss function of the weighted accuracy; the apparatus further includes:

[0163] The training unit acquires training samples, which include angiographic images of correctly segmented calcified plaque regions. The training samples are input into an image segmentation model for supervised training. The encoder network extracts pixel features from the input angiographic images, and the decoder network classifies the extracted pixel features. The classification result is calculated as the difference between the predicted calcified plaque region composed of first-class pixels and the labeled calcified plaque region. This difference is then used to adjust the coefficients in the focus loss function of the weighted accuracy.

[0164] In an exemplary embodiment, the decoder network uses a preset index evaluation algorithm to evaluate the image segmentation model after adjusting the coefficients in the focus loss function with weighted accuracy; and if the evaluation result does not meet the preset conditions, the image segmentation model is iteratively trained until the evaluation result meets the preset conditions.

[0165] In one exemplary embodiment, the preset indicator evaluation algorithm includes:

[0166]

[0167] Where k is the number of batches set for model training, TP i For the true first-class pixels in the i-th batch, FP i For the pseudo-first class pixels in the i-th batch, FN i These are the pseudo-second-class pixels during the i-th batch processing.

[0168] In one exemplary embodiment, the segmentation unit 620 includes:

[0169] The feature extraction subunit inputs the angiography image into the trained image segmentation model, where the encoder network of the image segmentation model extracts pixel features from the input angiography image.

[0170] The feature classification subunit classifies the extracted pixel features by the decoder network of the image segmentation model to determine whether the pixel corresponding to each pixel feature belongs to the first type of pixel.

[0171] The region segmentation subunit performs image segmentation on the image region composed of the first type of pixels in the imaging image to obtain a segmented image of the calcified plaque.

[0172] In one exemplary embodiment, the image segmentation model includes an improved model of the U-Net neural network.

[0173] In one exemplary embodiment, the improved model of the U-Net neural network includes the Res-Unet neural network model.

[0174] In one exemplary embodiment, the angiographic images include angiographic images of the coronary arteries.

[0175] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0176] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0177] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0178] above Figure 6 The internal functional modules and structural diagram of the image segmentation device are described. Its essential execution entity can be an electronic device, including:

[0179] processor;

[0180] Memory used to store processor-executable instructions;

[0181] The processor is configured to execute any of the above-described image segmentation methods.

[0182] In the embodiments of the above-described electronic device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor, and the aforementioned memory can be read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or solid-state drive. The steps of the method disclosed in the embodiments of this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0183] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the electronic device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0184] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.

[0185] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.

Claims

1. An image segmentation method, characterized in that, The method includes: Obtain the contrast image to be segmented; wherein the contrast image contains an image region corresponding to the calcified plaque; The contrast image is input into a pre-trained image segmentation model, which identifies the image region in the contrast image that corresponds to the calcified plaque, and performs image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque. The loss function used to train the image segmentation model is a focus loss function with weighted accuracy; the focus loss function with weighted accuracy includes: , As a focusing factor, The weighted accuracy calculation index is t, where t is the time series point. The weighted accuracy calculation metrics include: in, To correctly identify true Class I pixels as calcified plaques, To correctly identify true Type II pixels as non-calcified patches, For false Class I pixels that were mistakenly identified as calcified plaques, These are false Class II pixels that were mistakenly identified as non-calcified patches; The weight coefficients of true first-class pixels, The weighting coefficients of true second-class pixels, The penalty weight coefficient for false first-class pixels, The penalty weight coefficient for pseudo-second type pixels, Other coefficients; the weight of the true first type pixel is greater than the weight of the true second type pixel, and the sum of the weights of the true first type pixel and the true second type pixel is 1.

2. The method according to claim 1, characterized in that, The image segmentation model includes an encoder network and a decoder network, wherein the encoder network employs the focus loss function based on the weighted accuracy; the method further includes: Obtain training samples; wherein, the training samples include sample imaging images with correctly segmented calcified plaque annotation regions; The training samples are input into the image segmentation model for supervised training. The encoder network extracts pixel features from the input sample imaging image, and the decoder network classifies the extracted pixel features. The classification result is calculated as the difference between the predicted calcified patch region composed of first-class pixels and the labeled calcified patch region, and the difference is used to adjust the coefficients in the focus loss function of the weighted accuracy.

3. The method according to claim 2, characterized in that, The decoder network uses a preset index evaluation algorithm to evaluate the image segmentation model after adjusting the coefficients in the focus loss function with weighted accuracy. If the evaluation result does not meet the preset conditions, the image segmentation model is iteratively trained until the evaluation result meets the preset conditions.

4. The method according to claim 3, characterized in that, The preset indicator evaluation algorithm includes: Where k is the number of batches set for model training. For the true first-class pixels in the i-th batch, For the false first-class pixels in the i-th batch, These are the pseudo-second-class pixels during the i-th batch processing.

5. The method according to claim 2, characterized in that, The step of inputting the contrast image into a pre-trained image segmentation model, whereby the image segmentation model identifies the image region in the contrast image corresponding to the calcified plaque, and performs image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque, includes: The contrast image is input into the trained image segmentation model, and the encoder network of the image segmentation model extracts pixel features from the input contrast image. The decoder network of the image segmentation model classifies the extracted pixel features, determines whether the pixel corresponding to each pixel feature belongs to the first type of pixel, and performs image segmentation on the image region composed of the first type of pixels in the imaging image to obtain the segmented image of the calcified plaque.

6. The method according to claim 1, characterized in that, The image segmentation model includes an improved version of the U-Net neural network.

7. The method according to claim 6, characterized in that, The improved model of the U-Net neural network includes the Res-Unet neural network model.

8. The method according to claim 1, characterized in that, The angiographic images include angiographic images of the coronary arteries.

9. An image segmentation apparatus, characterized in that, The device includes: An acquisition unit acquires an imaging image to be segmented; wherein the imaging image contains an image region corresponding to the calcified plaque; The segmentation unit inputs the contrast image into a pre-trained image segmentation model, which identifies the image region in the contrast image that corresponds to the calcified plaque, and performs image segmentation on the image region corresponding to the calcified plaque to obtain a segmented image of the calcified plaque. The loss function used to train the image segmentation model is a focus loss function with weighted accuracy; the focus loss function with weighted accuracy includes: , As a focusing factor, The weighted accuracy calculation index is t, where t is the time series point. The weighted accuracy calculation metrics include: in, To correctly identify true Class I pixels as calcified plaques, To correctly identify true Type II pixels as non-calcified patches, For false Class I pixels that were mistakenly identified as calcified plaques, These are false Class II pixels that were mistakenly identified as non-calcified patches; The weight coefficients of true first-class pixels, The weighting coefficients of true second-class pixels, The penalty weight coefficient for false first-class pixels, The penalty weight coefficient for pseudo-second type pixels, Other coefficients; the weight of the true first type pixel is greater than the weight of the true second type pixel, and the sum of the weights of the true first type pixel and the true second type pixel is 1.

10. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method according to any one of claims 1-8.

Images