Coronary CT calcification integral determination method and device and electronic equipment

Abstract

The invention discloses a coronary CT calcification integral determination method and device and electronic equipment. The method comprises the following steps: acquiring a coronary CT image to be processed; a multi-class segmentation model is adopted to process the coronary CT image to obtain a target mask, the multi-class segmentation model is obtained by adopting the coronary CT feature map based on calcification second-class segmentation tasks and calcification multi-class segmentation tasks, and a framework of the multi-class segmentation model comprises a representation learning encoder, a binary classification task branch and a multi-classification task branch; and determining a coronary CT calcification integral corresponding to the coronary CT image according to the target mask. According to the method and the device, the technical problem of relatively high optimization cost of a coronary CT calcification integral calculation method in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the medical field, and in particular, to a method, device and electronic equipment for determining coronary CT calcification score. Background Art

[0002] Cardiovascular disease has become one of the major health challenges worldwide. One of its main forms is coronary artery disease, which is manifested by atherosclerotic plaques inside the coronary arteries, which lead to blood flow obstruction, thereby causing serious complications such as myocardial ischemia, angina pectoris, and myocardial infarction. Therefore, early detection and assessment of the extent of atherosclerosis are crucial to preventing the development of cardiovascular disease. In clinical practice, coronary CT (computed tomography) calcium score has become an important non-invasive diagnostic tool, providing an objective method for assessing the risk of coronary artery disease. In recent years, due to the development and implementation of deep learning, the current automatic calculation schemes for coronary CT calcium score have been continuously proposed and applied. However, the accuracy of these technologies still has certain problems, and the optimization cost is high.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method, device and electronic device for determining a coronary CT calcification score, so as to at least solve the technical problem of high optimization cost of the calculation method of the coronary CT calcification score in the related art.

[0005] According to one aspect of an embodiment of the present application, a method for determining a coronary CT calcification score is provided, comprising: acquiring a coronary CT image to be processed; processing the coronary CT image using a multi-class segmentation model to obtain a target mask, wherein the multi-class segmentation model is trained based on a calcification binary segmentation task and a calcification multi-class segmentation task using a coronary CT feature map, and the framework of the multi-class segmentation model includes a representation learning encoder, a binary classification task branch, and a multi-class classification task branch; determining the coronary CT calcification score corresponding to the coronary CT image based on the target mask.

[0006] Optionally, the multi-class segmentation model is trained in the following manner: obtaining historical coronary CT images, wherein the historical coronary CT images include calcification category annotation information, and the calcification category annotation information is used to indicate whether there is calcification in the historical coronary CT images; determining historical matrix data corresponding to the historical coronary CT images; randomly dividing the historical matrix data to obtain a first data set, and determining the data in the first data set that contains calcification category annotation information and is labeled as having calcification as a second data set; training a representation learning encoder based on the first data set and the second data set; and constructing a multi-class segmentation model based on the representation learning encoder.

[0007] Optionally, training a representation learning encoder based on a first data set and a second data set includes: inputting the first data set into a framework model where the representation learning encoder to be trained is located to obtain a first feature representation and a second feature representation; determining a first loss value based on the first feature representation and the second feature representation; when the rate of change of the first loss value is less than a first threshold, inputting the second data set into the framework model to obtain a third feature representation and a fourth feature representation; determining a second loss value based on the third feature representation and the fourth feature representation; when the rate of change of the second loss value is less than a second threshold, obtaining a trained representation learning encoder from the framework model, wherein the first threshold is greater than the second threshold.

[0008] Optionally, the framework model includes a first encoder, a second encoder, a first multi-layer perceptron module and a second multi-layer perceptron module, and the first data set is input into the framework model where the representation learning encoder to be trained is located to obtain a first feature representation and a second feature representation, including: performing two data enhancement processes on the first data set to obtain a first image pair, wherein when the first image pair comes from the same image processed by different data enhancements, the first image pair is a positive sample, and when the first image pair comes from different images, the first image pair is a negative sample; the first image pair is input into the first encoder and the second encoder respectively to obtain a first image feature representation and a second image feature representation; the first image feature representation and the second image feature representation are input into the first multi-layer perceptron module and the second multi-layer perceptron module respectively to obtain a first feature representation and a second feature representation.

[0009] Optionally, the second data set is input into the framework model to obtain a third feature representation and a fourth feature representation, including: performing two data enhancement processes on the second data set to obtain a second image pair, wherein when the second image pair comes from the same image processed by different data enhancement processes, the second image pair is a positive sample, and when the second image pair comes from different images, the second image pair is a negative sample; inputting the second image pair into the first encoder and the second encoder respectively to obtain a third image feature representation and a fourth image feature representation; inputting the third image feature representation and the fourth image feature representation into the first multi-layer perceptron module and the second multi-layer perceptron module respectively to obtain a third feature representation and a fourth feature representation.

[0010] Optionally, a multi-class segmentation model is constructed based on a representation learning encoder, including: obtaining a third data set consisting of an image block corresponding to the second data set and a calcification segmentation mask pair corresponding to the second data set, wherein the calcification segmentation mask pair includes a multi-class segmentation mask and a two-class segmentation mask; performing feature extraction on the third data set through the representation learning encoder to obtain a coronary CT feature map; determining a first output based on the coronary CT feature map and the calcification two-class segmentation task in the multi-class segmentation model; determining a second output based on the coronary CT feature map and the calcification multi-class segmentation task in the multi-class segmentation model; determining a first loss function based on the first output and the two-class segmentation mask in the second data set, and determining a second loss function based on the second output and the multi-class segmentation mask in the second data set; determining a total loss function based on the first loss function and the second loss function, adjusting the parameters of the total loss function, and stopping the training until the total loss function meets a preset condition to obtain a trained multi-class segmentation model.

[0011] Optionally, the calcification two-class segmentation task includes a first adapter, a first decoder and a calcification two-class segmentation head, and the calcification multi-class segmentation task includes a second adapter, a second decoder and a calcification multi-class segmentation head. The first output is determined based on the coronary CT feature map and the calcification two-class segmentation task in the multi-class segmentation model, and the second output is determined based on the coronary CT feature map and the calcification multi-class segmentation task in the multi-class segmentation model, including: inputting the coronary CT feature map into the first adapter and the second adapter respectively for feature extraction to obtain a first coronary CT feature map and a second coronary CT feature map; inputting the first coronary CT feature map into the second adapter for feature extraction, and fusing the result of the feature extraction with the second coronary CT feature map to obtain a third coronary CT feature map; inputting the first coronary CT feature map into the first decoder to obtain a fourth coronary CT feature map; inputting the fourth coronary CT feature map into the calcification two-class segmentation head to obtain the first output; inputting the third coronary CT feature map into the second decoder to obtain a fifth coronary CT feature map; and inputting the fifth coronary CT feature map into the calcification multi-class segmentation head to obtain the second output.

[0012] Optionally, when the representation learning encoder is Resnet, the method also includes: dividing the coronary CT feature map into multiple windows, and determining the correlation between the features of each window through a self-attention mechanism, and determining the correlation between the features of different windows.

[0013] Optionally, determining the coronary CT calcification score corresponding to the coronary CT image based on the target mask includes: performing pixel processing on the target mask to obtain target pixels; determining the density score corresponding to each pixel in the target pixels, and determining the maximum density score of each layer of the coronary CT image; determining the calcification score of each layer of the coronary CT image based on the maximum density score of each layer and the calcification area of ​​each layer, wherein the calcification area is determined based on the number of pixels in the calcification region of each layer and the pixel spacing of the coronary CT image; determining the coronary CT calcification score based on the calcification score of each layer and the layer spacing of the coronary CT image.

[0014] According to another aspect of an embodiment of the present application, a device for determining a coronary CT calcification score is also provided, including: an acquisition module, used to acquire a coronary CT image to be processed; a classification module, used to process the coronary CT image using a multi-class segmentation model to obtain a target mask, wherein the multi-class segmentation model is trained based on a calcification binary segmentation task and a calcification multi-class segmentation task using a coronary CT feature map, and the framework of the multi-class segmentation model includes a representation learning encoder, a binary classification task branch, and a multi-class classification task branch; a determination module, used to determine the coronary CT calcification score corresponding to the coronary CT image based on the target mask.

[0015] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including: a memory and a processor, the memory being used to store program instructions; the processor being connected to the memory and being used to execute the above-mentioned method for determining the coronary CT calcification score.

[0016] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned method for determining the coronary CT calcification score by running the computer program.

[0017] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which, when executed by a processor, implement the above-mentioned method for determining the coronary CT calcification score.

[0018] In an embodiment of the present application, a coronary CT image to be processed is obtained; and a multi-class segmentation model is used to process the coronary CT image to obtain a target mask, wherein the multi-class segmentation model is trained based on a coronary CT feature map based on a calcification binary segmentation task and a calcification multi-class segmentation task, and the framework of the multi-class segmentation model includes a representation learning encoder, a binary classification task branch, and a multi-class classification task branch; the coronary CT calcification score corresponding to the coronary CT image is determined based on the target mask, which can solve the technical problem of high optimization cost of the calculation method of the coronary CT calcification score in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a method for determining a coronary CT calcification score according to an embodiment of the present application; Figure 2 is a schematic diagram of a process of training a representation learning encoder according to an embodiment of the present application; Figure 3 is a schematic diagram of a process of training a multi-class segmentation model according to an embodiment of the present application; Figure 4 is a structural diagram of a device for determining coronary CT calcification score according to an embodiment of the present application; Figure 5 It is a hardware structure block diagram of a computer terminal for implementing a method for determining coronary CT calcification score according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0023] With the rise of deep learning, computer vision and artificial intelligence technologies, researchers have begun to explore the application of these advanced technologies to coronary CT calcification scoring. By training neural networks, automatic calcification area detection, segmentation and classification can be achieved, thereby reducing manual intervention and improving the consistency and accuracy of analysis. For example, in the field of medical imaging, the U-net network (U-Net) based on CNN (convolutional neural network) is widely used for lesion or organ segmentation. The model structure is simple and easy to train. In order to solve specific problems, variants based on the U-net model are also widely used, such as U-net++, Unet3+, etc. In addition, the Transformer architecture originated from the field of natural language processing has gradually become popular. Compared with convolutional neural networks, the transformer-based self-attention mechanism can capture the long-distance dependency of images and obtain more accurate results. However, due to the huge number of pixels in the image, the time complexity of the algorithm will be very high if the self-attention weights are directly learned in units of pixels. Therefore, the Transformer structure combining image characteristics has also been developed, such as SwinTransforme, and TransUnet, which combines CNN and Transformer.

[0024] In addition, due to the high cost of obtaining labeled data, especially in the field of medical imaging, unsupervised training methods based on contrastive learning have also developed rapidly and become popular. Related frameworks include MOCO, SimCLR, SimSiam, BYOL, SwAV, etc. Contrastive learning does not require labeled data, but mines its intrinsic representation based on the similarity of images. It can use a large amount of unlabeled data for pre-training, and then fine-tune on a small amount of labeled data to achieve or even exceed the effect of supervised models.

[0025] There are two mainstream algorithm frameworks in related technologies, with the following specific disadvantages: (1) Based on convolutional neural networks, a two-class segmentation model or a multi-class segmentation model is learned. This method lacks long-distance dependence, that is, the existing convolutional neural network cannot capture the correlation between pixels with a long distance, and can only capture local information of the image. The ability to capture global information of the image is limited, which easily leads to many false positives on the aorta; (2) Using a coarse segmentation model and a false positive removal model, the output of the first model is the input of the second model. This method makes model training not flexible enough and often requires the joint optimization of the two models. In addition, due to the differences in imaging equipment and parameters, the trained model is often not robust enough in practical applications, that is, the model generalization is not enough, and the model is not stable enough when there are many image interference factors. To solve this problem, a large amount of high-quality data needs to be collected and labeled, which will result in too high a time and labor cost.

[0026] The automatic calculation technology of calcification score based on coronary CT can greatly improve the efficiency of radiologists in manually marking calcification foci and improve the consistency of results. However, the existing coronary calcification score technology has the problems of false positives and insufficient robustness, which leads to insufficient accuracy; and the existing model training is not flexible, which leads to high optimization costs. In order to solve the above problems, the embodiment of the present application provides a method for determining the coronary CT calcification score.

[0027] Figure 1 is a flow chart of a method for determining a coronary CT calcification score according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps: Step S202: Acquire the coronary CT image to be processed.

[0028] Step S204, use a multi-class segmentation model to process the coronary CT image to obtain a target mask, wherein the multi-class segmentation model is trained based on the calcification binary segmentation task and the calcification multi-class segmentation task using the coronary CT feature map, and the framework of the multi-class segmentation model includes a representation learning encoder, a binary classification task branch, and a multi-class classification task branch.

[0029] In the above step S204, the trained multi-class segmentation model is applied to a new coronary CT image (i.e., the above coronary CT image to be processed), and the model will automatically identify and segment different categories of calcification areas in the image. Through the segmentation results output by the model, the target mask of each category of calcification area can be obtained.

[0030] Step S206, determining the coronary CT calcification score corresponding to the coronary CT image according to the target mask.

[0031] In the above step S206, the coronary CT calcification score is an indicator for quantitatively evaluating the overall calcification of the coronary artery. It is scored based on the area and density of the calcified area detected in the CT image, reflecting the severity of coronary artery disease and the level of cardiovascular risk of the patient. The target mask is obtained from the coronary CT image through a multi-class segmentation model, which identifies the different categories of calcified areas in the image. Using the target mask, the area and density information of each calcified area can be accurately extracted. Substituting this information into the calculation formula of the calcification score, the calcification score of the coronary CT image can be obtained.

[0032] In step S202 of the above-mentioned method for determining the coronary CT calcification score, after obtaining the coronary CT image to be processed, the method further includes: determining a preset CT value range of the coronary CT image, wherein the preset CT value range includes a preset maximum value and a preset minimum value; replacing CT values ​​in the coronary CT image that are less than or equal to the preset minimum value with the preset minimum value, and replacing CT values ​​in the coronary CT image that are greater than or equal to the preset maximum value with the preset maximum value, to obtain a processed coronary CT image; and resampling the processed coronary CT image to obtain matrix data of the coronary CT image.

[0033] In an embodiment of the present application, the CT value (Hounsfield Units, HU) is a unit that describes the degree of X-ray absorption by different tissues in a CT image. In a coronary CT image, different CT value ranges usually correspond to different tissue types. In order to better highlight the calcified area or other areas of interest and reduce the impact of irrelevant tissues, it is necessary to set a preset CT value range, that is, a preset maximum value and a preset minimum value. In an optional embodiment, the preset CT value range can be [-800, 1200], wherein -800 is the above-mentioned preset minimum value, and 1200 is the above-mentioned preset maximum value. The CT values ​​of the coronary CT image that are less than or equal to -800 are unified to -800, and those that are greater than or equal to 1200 are unified to 1200. This can improve the contrast of the CT image and reduce the image of the non-target area. The coronary CT image after the above preprocessing is resampled. Resampling is the process of changing the pixel spacing (or resolution) of the image to a new value. After resampling, the coronary CT image will be converted into matrix data with a specified pixel spacing. For example, the coronary CT image is resampled using bilinear interpolation to set the axial spacing of the CT image to 1.5 mm, thereby obtaining the matrix data of the coronary CT image.

[0034] In the above-mentioned method for determining the coronary CT calcification score, the multi-class segmentation model is trained in the following manner: obtaining historical coronary CT images, wherein the historical coronary CT images include calcification category annotation information, and the calcification category annotation information is used to indicate whether calcification exists in the historical coronary CT images, and the location of the calcification; determining the historical matrix data corresponding to the historical coronary CT images; randomly dividing the historical matrix data to obtain a first data set, and determining the data in the first data set that contains calcification category annotation information and the annotation information indicates that there is calcification as a second data set; training a representation learning encoder based on the first data set and the second data set; and constructing a multi-class segmentation model based on the representation learning encoder.

[0035] In an embodiment of the present application, it is necessary to first collect a large number of historical coronary CT images, which contain key information such as calcification areas. By annotating these historical coronary CT images, calcification category annotation information is obtained. For example, the imaging physician annotates the calcification category on the aorta in the historical coronary CT images. There are a total of 7 vascular location categories (left main trunk, anterior descending branch, circumflex branch, right coronary artery, posterior descending branch, ascending aorta, and descending aorta). When the model is subsequently trained, an auxiliary location information can be given to the model to guide the model to distinguish calcifications in different locations. The annotation ratio is 10-30%. After annotation, the calcification area and the background area can be distinguished, thereby obtaining the segmentation mask of the calcification area corresponding to the historical coronary CT image. The segmentation mask can be a grayscale image. It should be noted that when annotating, two types of segmentation annotations are performed simultaneously when annotating multiple types of segmentation masks, namely, branch arteries and aorta.

[0036] By annotating the historical coronary images and then preprocessing them, for example, limiting the CT values ​​in the historical coronary CT images to [-800, 1200], and resampling the preprocessed historical coronary CT images, the historical matrix data corresponding to the historical coronary CT images are obtained. The historical matrix data is randomly divided into image blocks of size [48, 128, 128] to obtain a first data set, which can be represented by data set A, wherein the historical matrix data includes annotated data and unannotated data, and the random division means that the number and position of the image blocks are random. It should be noted that the size of the above image blocks is only for example and does not represent a limitation, and the specific size can be adjusted according to the actual situation. In the first data set, the image blocks with annotated information and annotated information as calcified are used as the second data set, which can be represented by data set B. The representation learning encoder is trained by the first data set and the second data set obtained, for example, unsupervised training can be performed by contrast learning to obtain a representation learning encoder, and a multi-headed supervised multi-task model is trained according to the trained representation learning encoder, thereby constructing a multi-class segmentation model. Using contrastive learning for pre-training reduces the need for labeled data and can achieve better results with less detailed annotations. In addition, a large number of data enhancement techniques are used in contrastive learning to make the model more robust. The following is an explanation.

[0037] In the above steps, the representation learning encoder is trained based on the first data set and the second data set, including: inputting the first data set into the framework model where the representation learning encoder to be trained is located to obtain the first feature representation and the second feature representation; determining the first loss value based on the first feature representation and the second feature representation; when the rate of change of the first loss value is less than a first threshold, inputting the second data set into the framework model to obtain the third feature representation and the fourth feature representation; determining the second loss value based on the third feature representation and the fourth feature representation; when the rate of change of the second loss value is less than the second threshold, obtaining the trained representation learning encoder from the framework model, wherein the first threshold is greater than the second threshold.

[0038] In the above steps, the framework model includes a first encoder, a second encoder, a first multi-layer perceptron module and a second multi-layer perceptron module, and the first data set is input into the framework model where the representation learning encoder to be trained is located to obtain a first feature representation and a second feature representation, including: performing two data enhancement processes on the first data set to obtain a first image pair, wherein when the first image pair comes from the same image block processed by different data enhancements, the first image pair is a positive sample, and when the first image pair comes from different image blocks, the first image pair is a negative sample; the first image pair is input into the first encoder and the second encoder respectively to obtain a first image feature representation and a second image feature representation; the first image feature representation and the second image feature representation are input into the first multi-layer perceptron module and the second multi-layer perceptron module respectively to obtain a first feature representation and a second feature representation.

[0039] In the above steps, the second data set is input into the framework model to obtain the third feature representation and the fourth feature representation, including: the second data set is subjected to two data enhancement processes to obtain the second image pair, wherein when the second image pair originates from the same image block subjected to different data enhancement processes, the second image pair is a positive sample, and when the second image pair originates from different image blocks, the second image pair is a negative sample; the second image pair is input into the first encoder and the second encoder respectively to obtain the third image feature representation and the fourth image feature representation; the third image feature representation and the fourth image feature representation are input into the first multi-layer perceptron module and the second multi-layer perceptron module respectively to obtain the third feature representation and the fourth feature representation.

[0040] In the embodiments of the present application, Figure 2 This is a flow chart of a training representation learning encoder according to an embodiment of the present application. Figure 2 The above training process of the representation learning encoder is explained. The model structure of the representation learning encoder adopts the MoCo framework, and the encoder is not restricted (Resnet or Transformer). The MoCo framework includes the first encoder, the second encoder and two multi-layer perceptron (MLP) modules. Figure 2 In the example, the first multilayer perceptron module is connected to the first encoder, and the second multilayer perceptron module is connected to the second encoder. The input is a historical coronary CT image, and then a pair of images is obtained after two data enhancements A and B. If the pair of images comes from the same image with different data enhancement schemes, the image pair is a positive sample, and if the pair of images comes from different images, the image pair is a negative sample. The data enhancement scheme includes translation, rotation, brightness shift, and scaling. The pair of images can obtain the first image feature representation and the second image feature representation by passing through the first encoder and the second encoder respectively, and then obtain the first feature representation and the second feature representation by passing through the first and second multilayer perceptron modules respectively.

[0041] Specifically, in the training phase, the dataset A (i.e., the first dataset) is first used as the input of the MoCo framework model. After two data enhancements (the above-mentioned data enhancement A and data enhancement B), the first image pair is obtained. The first image pair is respectively input into the first encoder and the second encoder to obtain the first image feature representation and the second image feature representation. Then, the first and second multi-layer perceptron modules are respectively input into the first and second multi-layer perceptron modules to obtain the first feature representation and the second feature representation. The contrast loss is used for training, and the first loss value is determined based on the first and second feature representations. In the above process, the formula of the contrast loss function is as follows:

[0042] Among them, q represents the first feature representation, k + The second feature representation when the first input image pair is a positive sample, k _ The second feature representation when the first input image pair is a negative sample, is a temperature hyperparameter, Lloss represents the contrast loss function, the numerator of the loss function is the similarity of the positive sample pairs, the denominator is the similarity of all sample pairs, and it is normalized with softmax, so that the model can learn the feature representation of the image to improve the similarity of the positive samples.

[0043] When the first loss value gradually converges to a change rate of less than 0.1% (i.e., the first threshold value) for 5 consecutive epochs (one generation of training), the data set B (i.e., the second data set) is used as the input of the MoCo framework model to make the model pay more attention to the characteristics of calcification. Specifically, the data set B is subjected to two types of data enhancement to obtain a second image pair, and the second image pair is respectively input into the first encoder and the second encoder to obtain a third image feature representation and a fourth image feature representation. The third image feature representation and the fourth image feature representation are respectively input into the first multi-layer perceptron module and the second multi-layer perceptron module to obtain a third feature representation and a fourth feature representation. The contrast loss is also used for training, and the second loss value is determined based on the third feature representation and the fourth feature representation. When the second loss gradually converges to a change rate of less than 0.001% (i.e., the second threshold value) for 5 consecutive epochs, the pre-training is terminated, and the trained representation learning encoder is obtained from the MoCo framework model, and the representation learning encoder may be the trained first encoder.

[0044] In the above steps, a multi-class segmentation model is constructed based on the representation learning encoder, including: obtaining a third data set consisting of an image block corresponding to the second data set and a calcification segmentation mask pair corresponding to the second data set, wherein the calcification segmentation mask pair includes a multi-class segmentation mask and a two-class segmentation mask; performing feature extraction on the third data set through the representation learning encoder to obtain a coronary CT feature map; determining a first output based on the coronary CT feature map and the calcification two-class segmentation task in the multi-class segmentation model; determining a second output based on the coronary CT feature map and the calcification multi-class segmentation task in the multi-class segmentation model; determining a first loss function based on the first output and the two-class segmentation mask in the second data set, and determining a second loss function based on the second output and the multi-class segmentation mask in the second data set; determining a total loss function based on the first loss function and the second loss function, adjusting the parameters of the total loss function, and stopping the training until the total loss function meets the preset conditions to obtain a trained multi-class segmentation model.

[0045] In the above steps, the calcification two-class segmentation task includes a first adapter, a first decoder and a calcification two-class segmentation head, the calcification multi-class segmentation task includes a second adapter, a second decoder and a calcification multi-class segmentation head, and the first output is determined based on the coronary CT feature map and the calcification two-class segmentation task in the multi-class segmentation model, and the second output is determined based on the coronary CT feature map and the calcification multi-class segmentation task in the multi-class segmentation model, including: inputting the coronary CT feature map into the first adapter and the second adapter respectively for feature extraction to obtain a first coronary CT feature map and a second coronary CT feature map; inputting the first coronary CT feature map into the second adapter for feature extraction, and fusing the result of the feature extraction with the second coronary CT feature map to obtain a third coronary CT feature map; inputting the first coronary CT feature map into the first decoder to obtain a fourth coronary CT feature map; inputting the fourth coronary CT feature map into the calcification two-class segmentation head to obtain the first output; inputting the third coronary CT feature map into the second decoder to obtain a fifth coronary CT feature map; and inputting the fifth coronary CT feature map into the calcification multi-class segmentation head to obtain the second output.

[0046] In the embodiment of the present application, the data blocks corresponding to the second data set and the calcification segmentation mask pair are combined into a third data set, and the third data set can be represented by data set C. The calcification segmentation mask pair includes a multi-class segmentation mask and a two-class segmentation mask. In the embodiment of the present application, multi-class segmentation refers to the division into 7 vascular categories (left main trunk, anterior descending branch, circumflex branch, right coronary artery, posterior descending branch, ascending aorta, descending aorta); the two-class segmentation is to divide the above 7 vascular categories into 2 categories, including branch arteries and aorta. Data set C is used as the input of the multi-class segmentation model to be trained, and the following data enhancements are added during training: random cropping, flipping, rotation, cutmix, mixup, random erasing, so that the model can learn more robust information with a small amount of labeled data.

[0047] To train a multi-class segmentation model, a supervised model framework is constructed. When the representation learning encoder is Transformer, such as Figure 3 As shown, Figure 3 is a flow chart of a training multi-class segmentation model according to an embodiment of the present application. Figure 3 The above-mentioned process of training a multi-class segmentation model is explained. The trained representation learning encoder is used as the backbone network of the supervised model (i.e., the multi-class segmentation model), and is divided into two networks according to two tasks (calcification binary classification and calcification multi-classification). Each network consists of an adapter, a decoder, and a segmentation head. Specifically, the first network where the calcification binary classification segmentation task is located includes a first adapter, a first decoder, and a calcification two-class segmentation head, and the second network where the calcification multi-class segmentation task is located includes a second adapter, a second decoder, and a calcification multi-class segmentation head. During training, the parameters of the representation learning encoder are frozen, and only the adapter, decoder, and segmentation head are learned to adapt to downstream tasks, while preventing catastrophic forgetting of the pre-trained model, reducing training parameters, and improving training efficiency.

[0048] exist Figure 3 In the embodiment, the representation learning encoder extracts features from the coronary CT data in the third data set to obtain a coronary CT feature map; the coronary CT feature map output by the representation learning encoder is input into an adapter, and the adapter architecture includes: a normalization layer (Layer Norm), a first fully connected layer (Linear1), an activation layer (ReLU layer), and a second fully connected layer (Linear2). The first adapter processes the coronary CT feature map (including feature extraction and conversion into the same format to facilitate fusion between features) to obtain the first coronary CT feature map, and transmits it to the first decoder; the second adapter processes the coronary CT feature map to obtain the second coronary CT feature map, and transmits it to the second decoder; at the same time, the first adapter transmits the first coronary CT feature map to the second adapter, and the second adapter extracts features from the first coronary CT feature map, adds and fuses the feature extraction result with the second coronary CT feature map, and obtains the third coronary CT feature map.

[0049] The decoder can be a Transformer or CNN architecture. The decoder receives the coronary feature map transmitted by the adapter through the upsampling layer (i.e., the first decoder receives the first coronary CT feature map, and the second decoder receives the third coronary CT feature map), gradually restores the coronary feature map to the original image size, and the decoders at each level are connected to the corresponding encoder through residual connections to fuse the low-level detail information and high-level semantic information, thereby obtaining a learned fine segmentation mask. Specifically, the first coronary CT feature map is input into the first decoder to obtain the fourth coronary CT feature map, and the third coronary CT feature map is input into the second decoder to obtain the fifth coronary CT feature map. The fourth coronary CT feature map is input into the calcification two-class segmentation head to obtain the first output; the fifth coronary CT feature map is input into the calcification multi-class segmentation head to obtain the second output, wherein the output1 (i.e., the first output) output by the two-class segmentation head is a two-class segmentation mask with a size of [1, 48, 128, 128], and the output of the multi-class segmentation head is a multi-class segmentation mask output2 (i.e., the second output) with a size of [8, 48, 128, 128]. It should be noted that the size of the mask can also be adjusted according to actual needs.

[0050] Output1 and the manually annotated two-class segmentation mask are input into the two-class cross entropy loss function L1 (i.e., the first loss function mentioned above), and output2 and the manually annotated multi-class segmentation mask are input into the multi-class loss function L2 (i.e., the second loss function mentioned above). Since the multi-classification head is an 8-class task (7 vascular categories plus background), and the actual probabilities of the 7 vascular categories are different, there will be a serious class imbalance problem. In order to solve this problem, a multi-classification loss function for the segmentation head is designed. Based on the weighted cross entropy loss, a penalty factor is introduced according to prior knowledge. According to the vascular distribution of the coronary arteries, the difficulty of confusion between different blood vessels can be scored (the scores from extremely easy to be confused to not confused are from low to high), thereby obtaining a KxK relationship matrix (K is the number of categories) and normalize it. When calculating the WCE (Weighted Cross Entropy) loss, the annotation mask is processed into one-hot encoding. Unlike the original multi-category WCE, which only calculates the pixel loss of the channel where the Label category is located, the designed loss function calculates the loss of K channels, so it has more guidance for the training direction of the model. The formulas for L1 and L2 are as follows:

[0051] Where N is the number of pixels in the segmentation mask, K is the number of categories, w is the weight of K categories (a 1*k vector), and y is i represents the manually annotated calcification segmentation mask, pi Represents the output of the segmentation head. The total loss L is the weighted average of the losses (L1, L2) of the two-class segmentation head and the multi-class segmentation head. In the early stage of training, the weight ɑ of L1 is 1 and the weight of L2 is 0, so that the model can focus on the calcification area first. When the threshold of the two-classification head reaches a high value (dice is 0.8), the multi-task network learning is assisted. At this time, the weight of L1 is set to 0.1 and the weight of L2 is set to 0.9, focusing on training the multi-class segmentation head, and finally obtaining a trained multi-class segmentation model.

[0052] It should be noted that the representation learning encoder used is the backbone network of the supervised model. By setting two task branches, each branch includes an adapter, a decoder, and a segmentation head. During training, the representation learning encoder is frozen, and the adapter, decoder, and segmentation head are learned to reduce training parameters, improve training efficiency, and prevent catastrophic forgetting of the representation learning encoder. The multi-class segmentation model has two segmentation heads, one for the two-class segmentation head and the other for the multi-class segmentation head. Through the connection between joint training and calcification classification, the training difficulty of the multi-class segmentation model can be reduced, the convergence of the model can be accelerated, and the classification error rate can be reduced. The above steps enable the feature information obtained from the two-class segmentation task to be integrated into the multi-classification task, so that the multi-classification task can focus on the features of the two-class segmentation task, and the model is more robust.

[0053] In the above steps, when the representation learning encoder is Resnet, the method also includes: dividing the coronary CT feature map into multiple windows, and determining the correlation between the features of each window through a self-attention mechanism, and determining the correlation between the features of different windows.

[0054] In an embodiment of the present application, the difference between the multi-class segmentation model when the representation learning encoder is Resnet and the multi-class segmentation model when the representation learning encoder is Transformer is that a swin transformer block module is arranged between the representation learning encoder and the first adapter and the second adapter. After the swin transformer block module processes the coronary CT feature map, the processed coronary CT feature map is input into the first adapter (Adapter1) and the second adapter (Adapter1). The self-attention mechanism of the swin transformer block can capture long-distance dependencies, so that the model pays attention to more overall location information and improves the accuracy of calcification classification.

[0055] In step S206 of the above-mentioned method for determining the coronary CT calcification score, the coronary CT calcification score corresponding to the coronary CT image is determined based on the target mask, including: performing pixel processing on the target mask to obtain target pixels; determining the density score corresponding to each pixel in the target pixels, and determining the maximum density score of each layer of the coronary CT image; determining the calcification score of each layer of the coronary CT image based on the maximum density score of each layer and the calcification area of ​​each layer, wherein the calcification area is determined based on the number of pixels in the calcification area of ​​each layer and the pixel spacing of the coronary CT image; determining the coronary CT calcification score based on the calcification score of each layer and the layer spacing of the coronary CT image.

[0056] In the above steps, pixel processing is performed on the target mask to obtain target pixels, including: assigning pixels on the ascending aorta and the descending aorta in the target mask to 0; removing pixels on the coronary CT image in the target mask whose HU is less than a first value; dividing the target mask into lesions according to connected domains, and removing lesions whose volume is less than a second value or greater than a third value, wherein the second value is less than the third value.

[0057] In the embodiment of the present application, the coronary CT image is classified using the trained multi-class segmentation model, the output2 corresponding to the coronary CT image is used as the above-mentioned target mask, and subsequent processing is performed, including: (1) Remove the calcification on the ascending aorta and descending aorta in the target mask, that is, assign 0 to the pixels with the category number of the ascending aorta and descending aorta injury in the target mask; (2) removing pixels on the original coronary CT image corresponding to the target mask whose HU values ​​are less than a first value (i.e., assigning a value of 0). The first value may be, for example, 130, but this value is only an example and is not intended to be limiting, and may be adjusted according to actual conditions; (3) Divide the target mask into lesions according to the connected domains, and remove the lesions whose volume is smaller than the second value or larger than the third value. The second value may be 5, and the third value may be 100000, for example. It should be noted that the second value and the third value are only examples and do not represent limitations, and can be adjusted according to actual conditions.

[0058] After the target mask is processed as above, the target pixel is obtained, and the density score corresponding to each pixel in the target pixel is determined. Specifically, the pixel is assigned a density score according to the HU value, where HU values ​​between 130 and 199 are 1 point, 200 and 299 are 2 points, 300 and 399 are 3 points, and 400 and above are 4 points. The maximum density score of each layer of the coronary CT image is multiplied by the calcification area of ​​each layer to determine the calcification score of each layer of the coronary CT image, where the calcification area is equal to the product of the number of pixels in the calcification area of ​​each layer and the pixel spacing of the coronary CT image. Finally, the calcification integral values ​​of all layers are added and multiplied by the inter-layer spacing of the coronary CT image to obtain the overall coronary CT calcification score.

[0059] The method for determining the coronary CT calcification score provided in the embodiment of the present application can be applied to the early screening system of cardiovascular diseases in hospitals and physical examination institutions. The patient only needs to do a non-invasive chest or coronary CT scan to quickly obtain a disease level classification based on the technology, and can perform intervention treatment as early as possible based on the type. Specifically, in the CT image analysis system, the patient's CT image is uploaded to the system, and the calcification score can be automatically segmented and the calcification score value can be calculated. It can also be subdivided into the calcification score of each coronary vessel, and the patient type can be further classified according to the calcification score and recorded in the report.

[0060] Figure 4 is a structural diagram of a device for determining coronary CT calcification score according to an embodiment of the present application, such as Figure 4 As shown, the device comprises: An acquisition module 40 is used to acquire a coronary CT image to be processed; A classification module 42 is used to process the coronary CT image using a multi-class segmentation model to obtain a target mask, wherein the multi-class segmentation model is trained based on a calcification binary segmentation task and a calcification multi-class segmentation task using a coronary CT feature map, and the framework of the multi-class segmentation model includes a representation learning encoder, a binary classification task branch, and a multi-class classification task branch; The determination module 44 is used to determine the coronary CT calcification score corresponding to the coronary CT image according to the target mask.

[0061] Through the acquisition module 40, classification module 42 and determination module 44 in the above-mentioned coronary CT calcification score determination device, the purpose of accurately assessing the risk of coronary artery disease is achieved, thereby achieving the technical effect of improving the accuracy of calcification score calculation, and further solving the technical problem of high optimization cost of the coronary CT calcification score calculation method in the related technology.

[0062] The above-mentioned device for determining the coronary CT calcification score also includes a training module 46 for training a multi-class segmentation model. Specifically, the training module is used to obtain historical coronary CT images, wherein the historical coronary CT images include calcification category annotation information, and the calcification category annotation information is used to indicate whether there is calcification in the historical coronary CT images; determine the historical matrix data corresponding to the historical coronary CT images; randomly divide the historical matrix data to obtain a first data set, and determine the data in the first data set that contains calcification category annotation information and the annotation information indicates that there is calcification as a second data set; train a representation learning encoder based on the first data set and the second data set; and construct a multi-class segmentation model based on the representation learning encoder.

[0063] In the training module in the above-mentioned device for determining the coronary CT calcification score, the training module is also used to input the first data set into the framework model where the representation learning encoder to be trained is located to obtain the first feature representation and the second feature representation; determine the first loss value based on the first feature representation and the second feature representation; when the rate of change of the first loss value is less than the first threshold, input the second data set into the framework model to obtain the third feature representation and the fourth feature representation; determine the second loss value based on the third feature representation and the fourth feature representation; when the rate of change of the second loss value is less than the second threshold, obtain the trained representation learning encoder from the framework model, wherein the first threshold is greater than the second threshold.

[0064] In the training module in the above-mentioned device for determining coronary CT calcification scores, the framework model includes a first encoder, a second encoder, a first multi-layer perceptron module and a second multi-layer perceptron module. The training module is also used to obtain a first image pair after performing two data enhancement processes on the first data set, wherein when the first image pair comes from the same image processed by different data enhancements, the first image pair is a positive sample, and when the first image pair comes from different images, the first image pair is a negative sample; the first image pair is input into the first encoder and the second encoder respectively to obtain a first image feature representation and a second image feature representation; the first image feature representation and the second image feature representation are input into the first multi-layer perceptron module and the second multi-layer perceptron module respectively to obtain a first feature representation and a second feature representation.

[0065] In the training module in the above-mentioned device for determining the coronary CT calcification score, the training module is also used to obtain a second image pair after performing two data enhancement processes on the second data set, wherein when the second image pair comes from the same image processed by different data enhancement processes, the second image pair is a positive sample, and when the second image pair comes from different images, the second image pair is a negative sample; the second image pair is input into the first encoder and the second encoder respectively to obtain a third image feature representation and a fourth image feature representation; the third image feature representation and the fourth image feature representation are input into the first multi-layer perceptron module and the second multi-layer perceptron module respectively to obtain a third feature representation and a fourth feature representation.

[0066] In the training module in the above-mentioned device for determining the coronary CT calcification score, the training module is also used to obtain a third data set consisting of an image block corresponding to the second data set and a calcification segmentation mask pair corresponding to the second data set, wherein the calcification segmentation mask pair includes a multi-class segmentation mask and a two-class segmentation mask; extract features of the third data set through a representation learning encoder to obtain a coronary CT feature map; determine a first output based on the coronary CT feature map and the calcification two-class segmentation task in the multi-class segmentation model; determine a second output based on the coronary CT feature map and the calcification multi-class segmentation task in the multi-class segmentation model; determine a first loss function based on the first output and the two-class segmentation mask in the second data set, and determine a second loss function based on the second output and the multi-class segmentation mask in the second data set; determine a total loss function based on the first loss function and the second loss function, adjust the parameters of the total loss function, and stop training when the total loss function meets a preset condition to obtain a trained multi-class segmentation model.

[0067] In the training module in the above-mentioned device for determining the coronary CT calcification score, the calcification two-class segmentation task includes a first adapter, a first decoder and a calcification two-class segmentation head, and the calcification multi-class segmentation task includes a second adapter, a second decoder and a calcification multi-class segmentation head. The training module is also used to input the coronary CT feature map into the first adapter and the second adapter respectively for feature extraction to obtain a first coronary CT feature map and a second coronary CT feature map; input the first coronary CT feature map into the second adapter for feature extraction, and fuse the result of the feature extraction with the second coronary CT feature map to obtain a third coronary CT feature map; input the first coronary CT feature map into the first decoder to obtain a fourth coronary CT feature map; input the fourth coronary CT feature map into the calcification two-class segmentation head to obtain a first output; input the third coronary CT feature map into the second decoder to obtain a fifth coronary CT feature map; input the fifth coronary CT feature map into the calcification multi-class segmentation head to obtain a second output.

[0068] In the training module in the above-mentioned device for determining the coronary CT calcification score, the training module is also used to divide the coronary CT feature map into multiple windows when the representation learning encoder is Resnet, and determine the correlation between the features of each window through the self-attention mechanism, as well as determine the correlation between the features of different windows.

[0069] In the determination module in the above-mentioned device for determining the coronary CT calcification score, the determination module is also used to perform pixel processing on the target mask to obtain the target pixel; determine the density score corresponding to each pixel in the target pixel, and determine the maximum density score of each layer of the coronary CT image; determine the calcification score of each layer of the coronary CT image based on the maximum density score of each layer and the calcification area of ​​each layer, wherein the calcification area is determined based on the number of pixels in the calcification area of ​​each layer and the pixel spacing of the coronary CT image; determine the coronary CT calcification score based on the calcification score of each layer and the layer spacing of the coronary CT image.

[0070] It should be noted that Figure 4 The apparatus for determining the coronary CT calcium score is used to perform Figure 1 The method for determining the coronary CT calcification score is shown in the figure. Therefore, the relevant explanations in the above-mentioned method for determining the coronary CT calcification score are also applicable to the device for determining the coronary CT calcification score, and will not be repeated here.

[0071] The method for determining the coronary CT calcification score provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 5 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a method for determining coronary CT calcification score. Figure 5 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art will understand that Figure 5 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 5 More or fewer components as shown, or with Figure 5 Different configurations are shown.

[0072] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0073] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for determining the coronary CT calcification score in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned method for determining the coronary CT calcification score is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0074] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0075] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0076] It should be noted that, in some optional embodiments, the above Figure 5 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 5 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0077] Under the above operating environment, an embodiment of the present application provides an embodiment of a method for determining coronary CT calcification scores. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0078] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned method for determining the coronary CT calcification score.

[0079] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned method for determining the coronary CT calcification score by running the computer program.

[0080] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the method for determining the coronary CT calcification score in each embodiment of the present application.

[0081] An embodiment of the present application also provides a computer program, which, when executed by a processor, implements the steps of the method for determining the coronary CT calcification score in each embodiment of the present application.

[0082] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0083] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0085] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0087] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0088] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining coronary CT calcification score, characterized in that: include: Acquire a coronary CT image to be processed; The coronary CT image is processed using a multi-class segmentation model to obtain a target mask, wherein the multi-class segmentation model is trained based on a calcification binary segmentation task and a calcification multi-class segmentation task using a coronary CT feature map, and the framework of the multi-class segmentation model includes a representation learning encoder, a binary classification task branch, and a multi-class classification task branch; A coronary CT calcification score corresponding to the coronary CT image is determined according to the target mask.

2. The method according to claim 1, characterized in that: The multi-class segmentation model is trained in the following way: Acquire a historical coronary CT image, wherein the historical coronary CT image includes calcification category annotation information, and the calcification category annotation information is used to indicate whether calcification exists in the historical coronary CT image; Determining historical matrix data corresponding to the historical coronary CT image; Randomly divide the historical matrix data to obtain a first data set, and determine the data in the first data set that contains the calcification category labeling information and has the labeling information that calcification exists as a second data set; Training the representation learning encoder based on the first data set and the second data set; The multi-class segmentation model is constructed based on the representation learning encoder.

3. The method according to claim 2, characterized in that Training the representation learning encoder according to the first data set and the second data set includes: Inputting the first data set into a framework model where a representation learning encoder to be trained is located to obtain a first feature representation and a second feature representation; determining a first loss value according to the first feature representation and the second feature representation; When the rate of change of the first loss value is less than a first threshold, inputting the second data set into the framework model to obtain a third feature representation and a fourth feature representation; Determining a second loss value according to the third feature representation and the fourth feature representation; When the rate of change of the second loss value is less than a second threshold, a trained representation learning encoder is obtained from the framework model, wherein the first threshold is greater than the second threshold.

4. The method according to claim 3, characterized in that The framework model includes a first encoder, a second encoder, a first multi-layer perceptron module and a second multi-layer perceptron module. The first data set is input into the framework model where the representation learning encoder to be trained is located to obtain a first feature representation and a second feature representation, including: After performing two data enhancement processes on the first data set, a first image pair is obtained, wherein when the first image pair is derived from the same image subjected to different data enhancement processes, the first image pair is a positive sample, and when the first image pair is derived from different images, the first image pair is a negative sample; Inputting the first image pair into the first encoder and the second encoder respectively to obtain a first image feature representation and a second image feature representation; The first image feature representation and the second image feature representation are input into the first multi-layer perceptron module and the second multi-layer perceptron module respectively to obtain the first feature representation and the second feature representation.

5. The method according to claim 4, characterized in that Inputting the second data set into the framework model to obtain a third feature representation and a fourth feature representation, including: The second data set is subjected to two data enhancement processes to obtain a second image pair, wherein when the second image pair is derived from the same image subjected to different data enhancement processes, the second image pair is a positive sample, and when the second image pair is derived from different images, the second image pair is a negative sample; Inputting the second image pair into the first encoder and the second encoder respectively to obtain a third image feature representation and a fourth image feature representation; The third image feature representation and the fourth image feature representation are respectively input into the first multi-layer perceptron module and the second multi-layer perceptron module to obtain the third feature representation and the fourth feature representation.

6. The method according to claim 2, characterized in that Constructing the multi-class segmentation model according to the representation learning encoder includes: Acquire a third data set consisting of an image block corresponding to the second data set and a calcification segmentation mask pair corresponding to the second data set, wherein the calcification segmentation mask pair includes a multi-class segmentation mask and a two-class segmentation mask; Performing feature extraction on the third data set by the representation learning encoder to obtain a coronary CT feature map; Determining a first output according to the coronary CT feature map and the calcification two-class segmentation task in the multi-class segmentation model; Determining a second output according to the coronary CT feature map and the calcification multi-class segmentation task in the multi-class segmentation model; Determining a first loss function based on the first output and the two-class segmentation mask in the second data set, and determining a second loss function based on the second output and the multi-class segmentation mask in the second data set; A total loss function is determined according to the first loss function and the second loss function, and the parameters of the total loss function are adjusted until the total loss function meets a preset condition, and the training is stopped to obtain a trained multi-class segmentation model.

7. The method according to claim 6, characterized in that The calcification two-class segmentation task includes a first adapter, a first decoder and a calcification two-class segmentation head, the calcification multi-class segmentation task includes a second adapter, a second decoder and a calcification multi-class segmentation head, and the first output is determined according to the coronary CT feature map and the calcification two-class segmentation task in the multi-class segmentation model, and the second output is determined according to the coronary CT feature map and the calcification multi-class segmentation task in the multi-class segmentation model, including: Inputting the coronary CT feature map into the first adapter and the second adapter respectively for feature extraction to obtain a first coronary CT feature map and a second coronary CT feature map; Inputting the first coronary CT feature map into the second adapter for feature extraction, and fusing the feature extraction result with the second coronary CT feature map to obtain a third coronary CT feature map; Inputting the first coronary CT characteristic map into the first decoder to obtain a fourth coronary CT characteristic map; Inputting the fourth coronary CT characteristic map into the calcification type 2 segmentation head to obtain the first output; Inputting the third coronary artery CT characteristic map into the second decoder to obtain a fifth coronary artery CT characteristic map; The fifth coronary CT feature map is input into the calcification multi-class segmentation head to obtain the second output.

8. The method according to claim 7, characterized in that In the case where the representation learning encoder is Resnet, the method further includes: The coronary CT feature map is divided into multiple windows, and the correlation between the features of each window and the correlation between the features of different windows are determined through a self-attention mechanism.

9. The method according to claim 1, characterized in that: Determining the coronary CT calcification score corresponding to the coronary CT image according to the target mask includes: Performing pixel processing on the target mask to obtain target pixels; Determine a density score corresponding to each pixel in the target pixel, and determine a maximum density score of each layer of the coronary CT image; Determining the calcification score of each layer of the coronary CT image according to the maximum density score of each layer and the calcification area of ​​each layer, wherein the calcification area is determined according to the number of pixels in the calcification region of each layer and the pixel spacing of the coronary CT image; The coronary CT calcification score is determined according to the calcification score of each layer and the inter-layer spacing of the coronary CT image.

10. A device for determining coronary CT calcification score, characterized in that: include: An acquisition module, used for acquiring a coronary CT image to be processed; A classification module, used for processing the coronary CT image using a multi-class segmentation model to obtain a target mask, wherein the multi-class segmentation model is trained based on a calcification binary segmentation task and a calcification multi-class segmentation task using a coronary CT feature map, and the framework of the multi-class segmentation model includes a representation learning encoder, a binary classification task branch, and a multi-class classification task branch; A determination module is used to determine the coronary CT calcification score corresponding to the coronary CT image according to the target mask.

11. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and is used to execute the method for determining the coronary CT calcification score as described in any one of claims 1 to 9.

12. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the method for determining the coronary CT calcification score according to any one of claims 1 to 9 by running the computer program.

13. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method for determining the coronary CT calcification score according to any one of claims 1 to 9 is implemented.

Images