Plaque self-supervised segmentation method and system

Abstract

The invention relates to a plaque self-supervised segmentation method and system. The method comprises the following steps: acquiring a high-resolution magnetic resonance blood vessel wall image of a sample and preprocessing the high-resolution magnetic resonance blood vessel wall image; carrying out image enhancement processing on each preprocessed image; constructing a self-supervised model, and inputting all the images after image enhancement processing into the self-supervised model for self-supervised pre-training; transferring parameters of all the self-supervised models after self-supervised pre-training to a downstream segmentation network model, and performing fine tuning training on the segmentation network model by using a sample high-resolution magnetic resonance blood vessel wall image marked with plaques in advance; and inputting a target high-resolution magnetic resonance blood vessel wall image into the fine-tuning trained segmentation network model for segmentation to obtain a segmentation prediction map. The method does not need to depend on a tedious manual labeling process, reduces the cost and time consumption caused by manual labeling, is efficient and feasible, and provides a more accurate and more convenient plaque positioning and evaluation tool for doctors.

Description

Technical Field

[0001] The present invention relates to the technical field of cardiovascular image processing, and in particular to a plaque self-supervised segmentation method and system. Background Art

[0002] Plaques, usually atherosclerotic plaques, are thick protrusions on the inner wall of blood vessels formed by the deposition of lipids, calcium, blood cells and fibrous tissue. In cerebrovascular diseases, the presence and stability of plaques are crucial. Stable plaques may grow locally, while unstable plaques are prone to rupture, leading to platelet aggregation and thrombus formation, which may cause myocardial infarction or stroke. Therefore, detecting and evaluating the stability of vascular plaques is the key to preventing cardiovascular events.

[0003] Traditional vascular imaging techniques, such as CTA (computed tomography angiography) and MRA (magnetic resonance angiography), have certain limitations in determining the pathological characteristics of the vascular wall. These methods cannot accurately provide detailed information about the vascular wall itself, and it is difficult to comprehensively evaluate the stability of the plaque and related pathological characteristics. In recent years, high-resolution magnetic resonance vascular wall imaging (HRMR-VWI) has developed rapidly and has become an effective imaging examination method. HRMR-VWI can clearly display the details of the vascular wall, accurately segment and evaluate the location, morphology and size of the plaque, and has been widely used in clinical practice.

[0004] Although HRMR-VWI has made significant progress in the detection and assessment of vascular plaques, the existing technology still has some shortcomings: (1) Manual labeling is complex and time-consuming. Plaque segmentation requires labeled samples to train the deep learning network. Plaques in images have diverse shapes and sizes, and highly professional medical knowledge is required to accurately label them. This makes manual labeling of plaque samples a complex and time-consuming task. (2) Subjectivity and inconsistency in labeling results. Due to the complexity and diversity of medical imaging data, there may be subjectivity and inconsistency in the labeling process. These factors will affect the reliability and consistency of the labeling results, thereby affecting the training effect and segmentation accuracy of the deep learning network. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a plaque self-supervised segmentation method and system in view of the deficiencies of the above-mentioned prior art.

[0006] The technical solution of the present invention to solve the above technical problem is as follows: a plaque self-supervised segmentation method, comprising the following steps:

[0007] Acquiring a sample high-resolution magnetic resonance vascular wall image, and preprocessing the sample high-resolution magnetic resonance vascular wall image;

[0008] performing image enhancement processing on each of the preprocessed high-resolution magnetic resonance vascular wall images of the sample;

[0009] Constructing a self-supervised model, and inputting all the sample high-resolution magnetic resonance vascular wall images after image enhancement processing into the self-supervised model for self-supervised pre-training;

[0010] Migrating all parameters of the self-supervised model after self-supervised pre-training to a downstream segmentation network model, and fine-tuning the segmentation network model using sample high-resolution magnetic resonance vascular wall images with pre-labeled plaque annotations;

[0011] The target high-resolution magnetic resonance vascular wall image is input into the fine-tuned and trained segmentation network model for segmentation to obtain a segmentation prediction map.

[0012] On the basis of the above technical solution, the present invention can also be improved as follows:

[0013] Further: the image enhancement processing of the pre-processed sample high-resolution magnetic resonance vascular wall image specifically includes the following steps:

[0014] Each of the preprocessed sample high-resolution magnetic resonance vascular wall images is subjected to two different image enhancement operations, wherein the image enhancement operations at least include random rotation, random cropping, and noise addition. The specific processing formula is:

[0015] x′=T(x)

[0016] x′1=T1(x)

[0017] x′2=T2(x)

[0018] Among them, x is the original unlabeled sample high-resolution magnetic resonance vascular wall image, T is the image enhancement operation, x′ is the sample high-resolution magnetic resonance vascular wall image after the enhancement operation, T1 is the first image enhancement method, T2 is the second image enhancement method, x1′ is the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and x2′ is the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method.

[0019] The beneficial effect of the above further solution is that the high-resolution magnetic resonance vascular wall images can be processed by the image enhancement process to enable the self-supervised model to learn a more robust feature representation. The model can more effectively capture the fine structure and key patterns of the vascular wall, thereby showing better performance in subsequent tasks. Further: the step of inputting all the sample high-resolution magnetic resonance vascular wall images after the image enhancement process into the self-supervised model for self-supervised pre-training specifically includes the following steps:

[0020] The same sample high-resolution magnetic resonance vascular wall image after two different image enhancement operations is input into the self-supervised model to extract feature information, wherein the feature information extraction formula is:

[0021] feature=F(x)

[0022] feature1=F(x′1)

[0023] feature2=F(x′2)

[0024] Among them, feature is the feature information extracted from the original unlabeled sample high-resolution magnetic resonance vascular wall image by the self-supervised model, feature1 is the first feature information obtained by extracting the feature information of the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and feature2 is the second feature information obtained by extracting the feature information of the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method;

[0025] A first loss function is constructed, and a first similarity between the first feature information and the second feature information is calculated. The first loss function is:

[0026] L1=|feature1-feature2|

[0027] Wherein, L1 is the first similarity between the first feature information and the second feature information;

[0028] An image reconstruction function is constructed, and based on the image reconstruction function, the sample high-resolution magnetic resonance vascular wall image after image enhancement is restored to obtain a reconstructed image, wherein the image reconstruction function is:

[0029] y=G(x′)

[0030] y1′=G(x1′)

[0031] y2′=G(x2′)

[0032] Wherein, G is an image reconstruction function, y is an original unlabeled sample high-resolution magnetic resonance vascular wall image after enhancement operation, y1′ is a first reconstructed image restored from the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and y2′ is a second reconstructed image restored from the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method;

[0033] A second loss function is constructed, and second similarities between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the first reconstructed image, between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the second reconstructed image, and between the first reconstructed image and the second reconstructed image are calculated respectively. The second loss function is:

[0034]

[0035]

[0036]

[0037] Among them, L2 01 is the second similarity between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the first reconstructed image, L2 02 is the second similarity between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the second reconstructed image, L2 12 is a second similarity between the first reconstructed image and the second reconstructed image.

[0038] The beneficial effect of the above further scheme is: by inputting the same sample high-resolution magnetic resonance vascular wall image after two different image enhancement operations into the self-supervised model for training, the first feature information and the second feature information are obtained, and the first similarity between the first feature information and the second feature information is calculated by the first loss function. At the same time, the sample high-resolution magnetic resonance vascular wall image after the image enhancement is restored by the image reconstruction function to obtain the first reconstructed image and the second reconstructed image, and the second similarity between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the first reconstructed image, between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the second reconstructed image, and between the first reconstructed image and the second reconstructed image is calculated by the second loss function. By training under different enhancement conditions, the self-supervised model can extract richer and more robust feature representations, thereby enhancing the understanding ability and sensitivity of the vascular wall image. In addition, the introduction of image reconstruction loss enables the model to better optimize the ability to restore image details. Through this multi-level feature learning, the model obtains rich feature information in the feature extraction stage and migrates these features to the segmentation network, thereby improving the accuracy of the segmentation task and reducing the reliance of the segmentation network on a large number of labeled images during the training process.

[0039] Further: the step of migrating all the parameters of the self-supervised model after self-supervised pre-training to the downstream segmentation network model, and fine-tuning the segmentation network model using the sample high-resolution magnetic resonance vascular wall images with pre-labeled plaque annotations specifically includes the following steps:

[0040] Initializing the segmentation network model using all parameters of the self-supervised model after self-supervised pre-training;

[0041] Label the pre-marked patches Y = {y1, y2, ..., y n} of sample high-resolution MRI vessel wall images X = {x1, x2, …, x n} is input into the segmentation network model, and the segmentation network model is trained using the segmentation loss function and cross entropy. The segmentation network model is:

[0042] L3=αDL+(1-α)CE

[0043]

[0044]

[0045] Among them, α is the adjustment coefficient, DL is the segmentation loss function, CE is the cross entropy loss, N is the number of samples, and y i is the true label of the i-th sample, p i is the prediction result of the i-th sample.

[0046] The beneficial effect of the above further scheme is: the segmentation network model is trained by using sample high-resolution magnetic resonance vascular wall images with pre-marked plaque annotations to determine the model parameters, thereby ensuring that the trained segmentation network model has segmentation recognition accuracy.

[0047] Further: the step of inputting the target high-resolution magnetic resonance vascular wall image into the fine-tuned and trained segmentation network model for segmentation to obtain the segmentation prediction map specifically includes the following steps:

[0048] Segmenting the target high-resolution magnetic resonance vascular wall image into blocks according to preset specifications;

[0049] The target high-resolution magnetic resonance vascular wall image segmented into blocks is input into the segmentation network model for segmentation to obtain a segmentation prediction map.

[0050] The beneficial effect of the above further scheme is that by segmenting the target high-resolution magnetic resonance vascular wall image into blocks according to preset specifications, memory limitations and deficiencies in the network architecture can be avoided, thereby improving the performance of the segmentation algorithm.

[0051] Further: the plaque segmentation method based on the self-supervisory model also includes the following steps:

[0052] The Dice coefficient and the maximum surface distance MSD are calculated according to the segmentation prediction map, and the segmentation quality of the segmentation prediction map is evaluated according to the Dice coefficient and the maximum surface distance MSD.

[0053] The beneficial effect of the above further scheme is that by calculating the Dice coefficient and the maximum surface distance MSD, the quality of the segmentation result can be evaluated more accurately, which is convenient and intuitive.

[0054] Further: the method for calculating the Dice coefficient and the maximum surface distance MSD according to the segmentation prediction graph comprises the following steps:

[0055] Calculate the voxel V in the segmentation prediction map A and the voxel V of the ground truth B , and calculate the Dice coefficient, the calculation formula is:

[0056]

[0057] The surface S formed by the segmentation prediction map A The maximum surface distance MSD is calculated based on the surface composed of the reference true value, and the calculation formula is:

[0058] MSD=max{maxd(S A ,S B ),max d(S B ,S A )}

[0059] Among them, d(S B ,S A ) is the surface S formed by the segmentation prediction map A The surface S formed by any point in B The closest Euclidean distance, d(S A ,S B ) is any point in the surface formed by the reference true value to the surface S formed by the segmentation prediction map A The closest Euclidean distance.

[0060] The beneficial effect of the above further scheme is: by calculating the Dice coefficient, the overlap of the two segmentation results can be indicated as a criterion for evaluating the segmentation quality. By calculating the maximum surface distance MSD, another criterion for evaluating the segmentation quality is used. The smaller the maximum surface distance MSD, the higher the segmentation quality.

[0061] The present invention also provides a plaque segmentation system based on a self-supervisory model, comprising a preprocessing module, an image enhancement module, a self-supervisory module and a segmentation network module;

[0062] The preprocessing module is used to obtain a sample high-resolution magnetic resonance vascular wall image and preprocess the sample high-resolution magnetic resonance vascular wall image;

[0063] The image enhancement module is used to perform image enhancement processing on each of the pre-processed sample high-resolution magnetic resonance vascular wall images;

[0064] The self-supervision module is used to construct a self-supervision model, and input all the sample high-resolution magnetic resonance vascular wall images after image enhancement processing into the self-supervision model for self-supervision pre-training;

[0065] The segmentation network module is used to transfer all parameters of the self-supervised model after self-supervised pre-training to the downstream segmentation network model, and use the sample high-resolution magnetic resonance vascular wall image with pre-labeled plaque annotations to fine-tune the segmentation network model;

[0066] The segmentation network module is also used to input the target high-resolution magnetic resonance vascular wall image into the fine-tuned and trained segmentation network model for segmentation to obtain a segmentation prediction map.

[0067] The present invention also provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the plaque segmentation method based on the self-supervisory model is implemented.

[0068] The present invention also provides a plaque segmentation device based on a self-supervisory model, comprising a communication interface, a memory, a communication bus and a processor, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0069] The memory is used to store computer programs;

[0070] The processor is used to implement the steps of the blob segmentation method based on the self-supervisory model when executing the program stored in the memory.

[0071] The beneficial effects of the present invention are as follows: the plaque self-supervised segmentation method and system of the present invention performs image enhancement processing on the sample high-resolution magnetic resonance vascular wall image, then uses the inherent information in the vascular wall image to train the self-supervised model, and migrates the parameters of the self-supervised model to the downstream segmentation network model, and fine-tunes the segmentation network model in combination with the sample high-resolution magnetic resonance vascular wall image with pre-labeled plaque annotations, so as to accurately segment the target high-resolution magnetic resonance vascular wall image without relying on cumbersome manual annotation processes, reducing the cost and time consumption caused by manual annotation, and is efficient and feasible, providing doctors with a more accurate and convenient plaque positioning and evaluation tool. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A schematic diagram of a process of a plaque segmentation method based on a self-supervised model according to an embodiment of the present invention;

[0073] Figure 2 A schematic diagram of the overall structure of a self-supervisory model according to an embodiment of the present invention;

[0074] Figure 3 It is a schematic diagram of the results of a plaque segmentation system based on a self-supervisory model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0076] like Figure 1 As shown, a patch segmentation method based on a self-supervised model includes the following steps:

[0077] S1: acquiring a sample high-resolution magnetic resonance vascular wall image, and preprocessing the sample high-resolution magnetic resonance vascular wall image;

[0078] S2: performing image enhancement processing on each of the preprocessed high-resolution magnetic resonance vascular wall images of the sample;

[0079] S3: constructing a self-supervised model, and inputting all the sample high-resolution magnetic resonance vascular wall images after image enhancement processing into the self-supervised model for self-supervised pre-training;

[0080] S4: migrating all parameters of the self-supervised model after self-supervised pre-training to a downstream segmentation network model, and fine-tuning the segmentation network model using sample high-resolution magnetic resonance vascular wall images with pre-labeled plaque annotations;

[0081] S5: Input the target high-resolution magnetic resonance vascular wall image into the fine-tuned and trained segmentation network model for segmentation to obtain a segmentation prediction map.

[0082] In one or more embodiments of the present invention, in S1, preprocessing the sample high-resolution magnetic resonance vascular wall image is specifically performed as follows:

[0083] The maximum and minimum normalization preprocessing operation is performed on the sample high-resolution magnetic resonance vascular wall image, which can scale the plaque feature values ​​in different ranges to a common range, usually between 0 and 1. This helps model learning because it ensures that all features are at the same magnitude in terms of value, avoiding some features from dominating the training process due to their large values ​​when calculating the gradient. The normalization formula is:

[0084]

[0085] Among them, x is the original image, xmin is the minimum pixel value of the original image, and xmax is the maximum pixel value of the original image.

[0086] In one or more embodiments of the present invention, the image enhancement processing of the pre-processed sample high-resolution magnetic resonance vascular wall image specifically comprises the following steps:

[0087] S21: performing two different image enhancement operations on each of the pre-processed sample high-resolution magnetic resonance vascular wall images, wherein the image enhancement operations at least include random rotation, random cropping, and noise addition, and the specific processing formula is:

[0088] x′=T(x)

[0089] x′1=T1(x)

[0090] x′2=T2(x)

[0091] Among them, x is the original unlabeled sample high-resolution magnetic resonance vascular wall image, T is the image enhancement operation, x′ is the sample high-resolution magnetic resonance vascular wall image after the enhancement operation, T1 is the first image enhancement method, T2 is the second image enhancement method, x1′ is the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and x2′ is the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method.

[0092] After the image enhancement processing of the high-resolution magnetic resonance vascular wall image, the self-supervised model can learn a more robust feature representation. The model can more effectively capture the fine structure and key patterns of the vascular wall, thereby showing better performance in subsequent tasks.

[0093] It should be noted that, here, each of the pre-processed sample high-resolution magnetic resonance vascular wall images can be subjected to two arbitrary different image enhancement operations, such as any two of random rotation, random cropping and noise addition.

[0094] like Figure 2 As shown, in one or more embodiments of the present invention, the step of inputting all the sample high-resolution magnetic resonance vascular wall images after image enhancement processing into the self-supervised model for self-supervised pre-training specifically includes the following steps:

[0095] S31: Inputting the same sample high-resolution magnetic resonance vascular wall image processed by two different image enhancement operations into the self-supervised model to extract feature information, wherein the feature information extraction formula is:

[0096] feature=F(x)

[0097] feature1=F(x′1)

[0098] feature2=F(x′2)

[0099] Among them, feature is the feature information extracted from the original unlabeled sample high-resolution magnetic resonance vascular wall image by the self-supervised model, feature1 is the first feature information obtained by extracting the feature information of the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and feature2 is the second feature information obtained by extracting the feature information of the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method;

[0100] Since the images after the two enhancement methods come from the same image, they have similarities in the feature space, and the L1 loss function is used to measure their similarity.

[0101] S32: Construct a first loss function and calculate a first similarity between the first feature information and the second feature information. The first loss function is:

[0102] L1=|feature1-feature2|

[0103] Wherein, L1 is the first similarity between the first feature information and the second feature information;

[0104] Next, the encoder-decoder of the self-supervised model constructs an image reconstruction function to restore the enhanced image to the original image.

[0105] S33: constructing an image reconstruction function, and restoring the sample high-resolution magnetic resonance vascular wall image after image enhancement processing based on the image reconstruction function to obtain a reconstructed image, wherein the image reconstruction function is:

[0106] y=G(x′)

[0107] y1′=G(x1′)

[0108] y2′=G(x2′)

[0109] Wherein, G is an image reconstruction function, y is an original unlabeled sample high-resolution magnetic resonance vascular wall image after enhancement operation, y1′ is a first reconstructed image restored from the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and y2′ is a second reconstructed image restored from the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method;

[0110] S34: constructing a second loss function, and respectively calculating second similarities between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the first reconstructed image, between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the second reconstructed image, and between the first reconstructed image and the second reconstructed image, wherein the second loss function is:

[0111]

[0112]

[0113]

[0114] Among them, L2 01 is the second similarity between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the first reconstructed image, L2 02 is the second similarity between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the second reconstructed image, L2 12 is a second similarity between the first reconstructed image and the second reconstructed image.

[0115] The same sample high-resolution magnetic resonance vascular wall image after two different image enhancement operations is input into the self-supervised model for training, and the first feature information and the second feature information are obtained, and the first similarity of the first feature information and the second feature information is calculated by the first loss function. At the same time, the sample high-resolution magnetic resonance vascular wall image after the image enhancement is restored by the image reconstruction function to obtain the first reconstructed image and the second reconstructed image, and the second similarity between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the first reconstructed image, between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the second reconstructed image, and between the first reconstructed image and the second reconstructed image is calculated by the second loss function. By training under different enhancement conditions, the self-supervised model can extract richer and more robust feature representations, thereby enhancing the understanding ability and sensitivity of the vascular wall image. In addition, the introduction of image reconstruction loss enables the model to better optimize the restoration ability of image details. Through this multi-level feature learning, the model obtains rich feature information in the feature extraction stage, and migrates these features to the segmentation network, thereby improving the accuracy of the segmentation task, while reducing the segmentation network's dependence on a large number of labeled images during the training process.

[0116] In one or more embodiments of the present invention, the steps of transferring all parameters of the self-supervised model after self-supervised pre-training to a downstream segmentation network model and fine-tuning the segmentation network model using sample high-resolution magnetic resonance vascular wall images with pre-labeled plaque annotations specifically include the following steps:

[0117] S41: Initializing the segmentation network model using all parameters of the self-supervised model after self-supervised pre-training;

[0118] Here, the optimal parameters of the trained self-supervised model are directly migrated to the downstream segmentation network model. Specifically, the segmentation network model can adopt the Unet network, but other methods can also be used, such as SwinUnet, DeepLab and other segmentation algorithm models.

[0119] S42: label the pre-marked patch Y = {y1, y2, ..., y n} of sample high-resolution MRI vessel wall images X = {x1, x2, …, x n} is input into the segmentation network model, and the segmentation network model is trained using the segmentation loss function and cross entropy. The segmentation network model is:

[0120] L3=αDL+(1-α)CE

[0121]

[0122]

[0123] Among them, α is the adjustment coefficient, which is set to 0.5 in this embodiment, DL is the segmentation loss function, CE is the cross entropy loss, N is the number of samples, and y i is the true label of the i-th sample, p i is the prediction result of the i-th sample.

[0124] The segmentation network model is trained by pre-marking the sample high-resolution magnetic resonance vascular wall images with plaque annotations, and the model prediction results (P = {p1, p2, ..., p n}) and the difference between the labeled results, so as to realize automatic and accurate segmentation of the plaque and ensure that the trained segmentation network model has the segmentation and recognition accuracy.

[0125] In one or more embodiments of the present invention, the step of inputting the target high-resolution magnetic resonance vascular wall image into the fine-tuned and trained segmentation network model for segmentation to obtain a segmentation prediction map specifically includes the following steps:

[0126] S51: Segmenting the target high-resolution magnetic resonance vascular wall image into blocks according to preset specifications;

[0127] S52: Inputting the target high-resolution magnetic resonance vascular wall image segmented into blocks into the segmentation network model for segmentation to obtain a segmentation prediction map.

[0128] By segmenting the target high-resolution magnetic resonance vascular wall image into blocks according to preset specifications, memory limitations and deficiencies in the network architecture can be avoided and the performance of the segmentation algorithm can be improved.

[0129] It should be pointed out that in order to reduce the boundary effect, the overlapping area is patched, and the overlapping area is usually half the size of the patch. In addition, for the edge part, because the network has a lower prediction accuracy at the edge of the patch, when merging the prediction results, a higher weight is given to the central area and a lower weight is given to the edge part, which can improve the segmentation accuracy.

[0130] Optionally, in one or more embodiments of the present invention, the blob segmentation method based on the self-supervisory model further includes the following steps:

[0131] S6: Calculate the Dice coefficient and the maximum surface distance MSD according to the segmentation prediction map, and evaluate the segmentation quality of the segmentation prediction map according to the Dice coefficient and the maximum surface distance MSD.

[0132] By calculating the Dice coefficient and the maximum surface distance MSD, the quality of the segmentation results can be evaluated more accurately, which is convenient and intuitive.

[0133] Specifically, in one or more embodiments of the present invention, the method for calculating the Dice coefficient and the maximum surface distance MSD according to the segmentation prediction map comprises the following steps:

[0134] S61: Calculate the voxel V in the segmentation prediction map A and the voxel V of the ground truth B , and calculate the Di ce coefficient, the calculation formula is:

[0135]

[0136] S62: Surface S formed according to the segmentation prediction map A The maximum surface distance MSD is calculated based on the surface composed of the reference true value, and the calculation formula is:

[0137] MSD=max{maxd(S A ,S B ),max d(S B ,S A )}

[0138] Among them, d(S B ,S A ) is the surface S formed by the segmentation prediction map A The surface S formed by any point in B The closest Euclidean distance, d(SA ,S B ) is any point in the surface formed by the reference true value to the surface S formed by the segmentation prediction map A The closest Euclidean distance.

[0139] The Dice coefficient can be calculated to show the overlap between the two segmentation results. It is used as a criterion for evaluating the segmentation quality. The maximum surface distance MSD is calculated as another criterion for evaluating the segmentation quality. The smaller the maximum surface distance MSD, the higher the segmentation quality.

[0140] like Figure 3 As shown, the present invention also provides a plaque segmentation system based on a self-supervisory model, including a preprocessing module, an image enhancement module, a self-supervisory module and a segmentation network module;

[0141] The preprocessing module is used to obtain a sample high-resolution magnetic resonance vascular wall image and preprocess the sample high-resolution magnetic resonance vascular wall image;

[0142] The image enhancement module is used to perform image enhancement processing on each of the pre-processed sample high-resolution magnetic resonance vascular wall images;

[0143] The self-supervision module is used to construct a self-supervision model, and input all the sample high-resolution magnetic resonance vascular wall images after image enhancement processing into the self-supervision model for self-supervision pre-training;

[0144] The segmentation network module is used to transfer all parameters of the self-supervised model after self-supervised pre-training to the downstream segmentation network model, and use the sample high-resolution magnetic resonance vascular wall image with pre-labeled plaque annotations to fine-tune the segmentation network model;

[0145] The segmentation network module is also used to input the target high-resolution magnetic resonance vascular wall image into the fine-tuned and trained segmentation network model for segmentation to obtain a segmentation prediction map.

[0146] The present invention also provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the plaque segmentation method based on the self-supervisory model is implemented.

[0147] The present invention also provides a plaque segmentation device based on a self-supervisory model, comprising a communication interface, a memory, a communication bus and a processor, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;

[0148] The memory is used to store computer programs;

[0149] The processor is used to implement the steps of the blob segmentation method based on the self-supervisory model when executing the program stored in the memory.

[0150] The plaque self-supervised segmentation method and system of the present invention performs image enhancement processing on the sample high-resolution magnetic resonance vascular wall image, then uses the inherent information in the vascular wall image to train the self-supervised model, and migrates the parameters of the self-supervised model to the downstream segmentation network model, and fine-tunes the segmentation network model in combination with the sample high-resolution magnetic resonance vascular wall image with pre-labeled plaque annotations, so as to accurately segment the target high-resolution magnetic resonance vascular wall image, and realizes automatic segmentation of plaques in the image through a deep learning method. This method overcomes the challenge of difficult sample annotation in traditional algorithms, and provides a more efficient and accurate solution for plaque segmentation. It does not need to rely on cumbersome manual annotation processes, reduces the cost and time consumption caused by manual annotation, is efficient and feasible, and provides doctors with a more accurate and convenient plaque positioning and evaluation tool.

[0151] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A plaque segmentation method based on a self-supervised model, characterized in that: The steps include: Acquiring a sample high-resolution magnetic resonance vascular wall image, and preprocessing the sample high-resolution magnetic resonance vascular wall image; performing image enhancement processing on each of the preprocessed high-resolution magnetic resonance vascular wall images of the sample; Constructing a self-supervised model, and inputting all the sample high-resolution magnetic resonance vascular wall images after image enhancement processing into the self-supervised model for self-supervised pre-training; Migrating all parameters of the self-supervised model after self-supervised pre-training to a downstream segmentation network model, and fine-tuning the segmentation network model using sample high-resolution magnetic resonance vascular wall images with pre-labeled plaque annotations; The target high-resolution magnetic resonance vascular wall image is input into the fine-tuned and trained segmentation network model for segmentation to obtain a segmentation prediction map.

2. The plaque segmentation method based on the self-supervised model according to claim 1, characterized in that: The image enhancement processing of the pre-processed sample high-resolution magnetic resonance vascular wall image specifically comprises the following steps: Each of the preprocessed sample high-resolution magnetic resonance vascular wall images is subjected to two different image enhancement operations, wherein the image enhancement operations at least include random rotation, random cropping, and noise addition. The specific processing formula is: x′=T(x) x′1=T1(x) x′2=T2(x) Among them, x is the original unlabeled sample high-resolution magnetic resonance vascular wall image, T is the image enhancement operation, x′ is the sample high-resolution magnetic resonance vascular wall image after the enhancement operation, T1 is the first image enhancement method, T2 is the second image enhancement method, x1′ is the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and x2′ is the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method.

3. The plaque segmentation method based on the self-supervised model according to claim 2, characterized in that: The step of inputting all the sample high-resolution magnetic resonance vascular wall images after image enhancement processing into the self-supervised model for self-supervised pre-training specifically comprises the following steps: The same sample high-resolution magnetic resonance vascular wall image after two different image enhancement operations is input into the self-supervised model to extract feature information, wherein the feature information extraction formula is: feature=F(x) feature1=F(x′1) feature2=F(x′2) Among them, feature is the feature information extracted from the original unlabeled sample high-resolution magnetic resonance vascular wall image by the self-supervised model, feature1 is the first feature information obtained by extracting the feature information of the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and feature2 is the second feature information obtained by extracting the feature information of the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method; A first loss function is constructed, and a first similarity between the first feature information and the second feature information is calculated. The first loss function is: L1=|feature1-feature2| Wherein, L1 is the first similarity between the first feature information and the second feature information; An image reconstruction function is constructed, and based on the image reconstruction function, the sample high-resolution magnetic resonance vascular wall image after image enhancement is restored to obtain a reconstructed image, wherein the image reconstruction function is: y=G(x′) y1′=G(x1′) y2′=G(x2′) Wherein, G is an image reconstruction function, y is an original unlabeled sample high-resolution magnetic resonance vascular wall image after enhancement operation, y1′ is a first reconstructed image restored from the sample high-resolution magnetic resonance vascular wall image processed by the first image enhancement method, and y2′ is a second reconstructed image restored from the sample high-resolution magnetic resonance vascular wall image processed by the second image enhancement method; A second loss function is constructed, and second similarities between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the first reconstructed image, between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the second reconstructed image, and between the first reconstructed image and the second reconstructed image are calculated respectively. The second loss function is: Among them, L2 01 is the second similarity between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the first reconstructed image, L2 02 is the second similarity between the original unlabeled sample high-resolution magnetic resonance vascular wall image and the second reconstructed image, L2 12 is a second similarity between the first reconstructed image and the second reconstructed image.

4. The plaque segmentation method based on the self-supervised model according to claim 1, characterized in that: The method of transferring all the parameters of the self-supervised model after the self-supervised pre-training to the downstream segmentation network model and fine-tuning the segmentation network model using the sample high-resolution magnetic resonance vascular wall image with pre-labeled plaque annotations specifically includes the following steps: Initializing the segmentation network model using all parameters of the self-supervised model after self-supervised pre-training; Label the pre-marked patches Y = {y1, y2, ..., y n } of sample high-resolution MRI vessel wall images X = {x1, x2, …, x n } is input into the segmentation network model, and the segmentation network model is trained using the segmentation loss function and cross entropy. The segmentation network model is: L3=αDL+(1-α)CE Among them, α is the adjustment coefficient, DL is the segmentation loss function, CE is the cross entropy loss, N is the number of samples, and y i is the true label of the i-th sample, p i is the prediction result of the i-th sample.

5. The plaque segmentation method based on the self-supervised model according to claim 1, characterized in that: The step of inputting the target high-resolution magnetic resonance vascular wall image into the fine-tuned and trained segmentation network model for segmentation to obtain a segmentation prediction map specifically includes the following steps: Segmenting the target high-resolution magnetic resonance vascular wall image into blocks according to preset specifications; The target high-resolution magnetic resonance vascular wall image segmented into blocks is input into the segmentation network model for segmentation to obtain a segmentation prediction map.

6. The plaque segmentation method based on the self-supervised model according to claim 1, characterized in that: The following steps are also included: The Dice coefficient and the maximum surface distance MSD are calculated according to the segmentation prediction map, and the segmentation quality of the segmentation prediction map is evaluated according to the Dice coefficient and the maximum surface distance MSD.

7. The plaque segmentation method based on the self-supervised model according to claim 6, characterized in that: The method for calculating the Dice coefficient and the maximum surface distance MSD according to the segmentation prediction graph comprises the following steps: Calculate the voxel V in the segmentation prediction map A and the voxel V of the ground truth B , and calculate the Dice coefficient, the calculation formula is: The surface S formed by the segmentation prediction map A The maximum surface distance MSD is calculated based on the surface composed of the reference true value, and the calculation formula is: MSD=max{maxd(S A ,S B ),max d(S B ,S A )} Among them, d(S B ,S A ) is the surface S formed by the segmentation prediction map A The surface S formed by any point in B The closest Euclidean distance, d(S A ,S B ) is any point in the surface formed by the reference true value to the surface S formed by the segmentation prediction map A The closest Euclidean distance.

8. A plaque segmentation system based on a self-supervised model, characterized in that: It includes preprocessing module, image enhancement module, self-supervision module and segmentation network module; The preprocessing module is used to obtain a sample high-resolution magnetic resonance vascular wall image and preprocess the sample high-resolution magnetic resonance vascular wall image; The image enhancement module is used to perform image enhancement processing on each of the preprocessed sample high-resolution magnetic resonance vascular wall images; The self-supervision module is used to construct a self-supervision model, and input all the sample high-resolution magnetic resonance vascular wall images after image enhancement processing into the self-supervision model for self-supervision pre-training; The segmentation network module is used to transfer all parameters of the self-supervised model after self-supervised pre-training to the downstream segmentation network model, and use the sample high-resolution magnetic resonance vascular wall image with pre-labeled plaque annotations to fine-tune the segmentation network model; The segmentation network module is also used to input the target high-resolution magnetic resonance vascular wall image into the fine-tuned and trained segmentation network model for segmentation to obtain a segmentation prediction map.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the plaque segmentation method based on the self-supervisory model described in any one of claims 1 to 7 is implemented.

10. A plaque segmentation device based on a self-supervised model, characterized in that: It includes a communication interface, a memory, a communication bus and a processor, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the steps of the blob segmentation method based on the self-supervised model described in any one of claims 1 to 7 when executing the program stored in the memory.

Images