Deep learning segmentation model training method, and vascular image segmentation method and apparatus

Abstract

Disclosed is a deep learning segmentation model training method, comprising: acquiring a three-dimensional vascular image containing a vascular segment of interest; processing the three-dimensional vascular image by means of a deep learning segmentation model to be trained, so as to obtain a tunica media segmentation result image and a tunica intima segmentation result image; determining a loss function according to a tunica media segmentation result label, a tunica intima segmentation result label, the tunica media segmentation result image, and the tunica intima segmentation result image, and updating a parameter of the deep learning segmentation model to be trained so as to obtain a trained deep learning segmentation model, wherein a tunica media segmentor and a tunica intima segmentor of the deep learning segmentation model use a cross-attention mechanism to segment the three-dimensional vascular image to obtain the tunica media segmentation result image and the tunica intima segmentation result image. The described method allows for the accurate segmentation of tunica intimae and tunica mediae of blood vessels. Further disclosed are a deep learning segmentation model training apparatus, a vascular image segmentation method and apparatus, an electronic device, and a readable storage medium.

Description

Training method of deep learning segmentation model, and blood vessel image segmentation method and device TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a training method of a deep learning segmentation model, a training device of the deep learning segmentation model, a blood vessel image segmentation method, a blood vessel image segmentation device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] Cardiovascular disease is the disease with the highest incidence and mortality in China. Coronary computed tomography angiography (CTA) is an important technology for detecting heart vessel diseases. At present, many methods automatically perform blood vessel segmentation and center line extraction at the same time to further calculate quantitative parameters to assist doctors in diagnosis.

[0003] In the prior art, the coronary CTA image processing technology usually only focuses on the single-layer endometrial segmentation of the blood vessel, aiming to segment the endometrium and the blood vessel lumen, and then directly observing the blood vessel condition according to the segmentation result of the endometrium, and then making a diagnosis accordingly. However, in the area with high plaque accumulation or severe stenosis, the endometrial structure is complex and changes greatly, the accuracy of the single endometrial segmentation method of the prior art which only segments the endometrium and the blood vessel lumen is significantly reduced, the potential value of the blood vessel media is ignored, the relationship between the media and the endometrium is not considered during segmentation, the segmentation result is not accurate enough, and the blood vessel center line extraction is not accurate enough, which will adversely affect the subsequent clinical diagnosis and treatment decision. SUMMARY

[0004] The present application provides a training method of a deep learning segmentation model to solve the above problems.

[0005] In a first aspect, an embodiment of the present application discloses a training method of a deep learning segmentation model, the deep learning segmentation model comprising a media segmenter and an endometrial segmenter, the media segmenter being configured to segment a media segmentation result image, and the endometrial segmenter being configured to segment an endometrial segmentation result image, the training method comprising:

[0006] obtaining a three-dimensional blood vessel image containing a blood vessel segment of interest, and obtaining a media segmentation result label and an endometrial segmentation result label of the blood vessel segment of interest;

[0007] processing the three-dimensional blood vessel image containing the blood vessel segment of interest by using the deep learning segmentation model to be trained, to obtain the media segmentation result image and the endometrial segmentation result image;

[0008] determining a loss function according to the media segmentation result label, the endometrial segmentation result label, the media segmentation result image and the endometrial segmentation result image of the blood vessel segment of interest;

[0009] The deep learning segmentation model to be trained is updated in parameters according to the three-dimensional blood vessel image containing the blood vessel segment of interest, the media layer segmentation result image, the intima layer segmentation result image, the media layer segmentation result label, the intima layer segmentation result label, and the loss function until the deep learning segmentation model to be trained meets a preset training condition, so as to obtain the trained deep learning segmentation model.

[0010] The media layer segmenter and the intima layer segmenter segment the three-dimensional blood vessel image containing the blood vessel segment of interest by using the cross attention mechanism to obtain the media layer segmentation result image and the intima layer segmentation result image.

[0011] According to the technical solution, the training method of the deep learning segmentation model combines the cross attention mechanism, the cross attention mechanism enables the media layer features and the intima layer features to interact, and the deep learning segmentation model trained by the method can more accurately segment the media layer segmentation result image and the intima layer segmentation result image of the blood vessel segment of interest, thereby improving the segmentation accuracy of the media layer and the intima layer.

[0012] According to another specific embodiment of the present application, the process of segmenting the three-dimensional blood vessel image containing the blood vessel segment of interest by using the cross attention mechanism specifically includes:

[0013] The media layer features of the three-dimensional blood vessel image containing the blood vessel segment of interest are extracted by the media layer segmenter to obtain media layer feature data;

[0014] The intima layer features of the three-dimensional blood vessel image containing the blood vessel segment of interest are extracted by the intima layer segmenter to obtain intima layer feature data;

[0015] The media layer segmenter performs feature fusion on the media layer feature data and the intima layer feature data by using the cross attention mechanism to obtain the media layer segmentation result image;

[0016] The intima layer segmenter performs feature fusion on the media layer feature data and the intima layer feature data by using the cross attention mechanism to obtain the intima layer segmentation result image.

[0017] According to another specific embodiment of the present application, the loss function includes a cross entropy loss function, a Dice loss function, and a consistency loss function.

[0018] According to another specific embodiment of the present application, the consistency loss function is obtained in the following manner:

[0019] The media layer skeleton result data of the media layer segmentation result image is obtained by using a soft skeleton extraction method according to the media layer segmentation result image, and the intima layer skeleton result data of the intima layer segmentation result image is obtained by using a soft skeleton extraction method according to the intima layer segmentation result image;

[0020] The consistency loss function is determined according to the media layer skeleton result data and the intima layer skeleton result data.

[0021] According to another specific embodiment of the present application, the loss function is specifically: Total Loss = Dice Lumen + Dice Wall + CE Lumen + CE Wall + Consistencylumen-wall

[0022] wherein Total Loss represents the loss function, Dice Lumen + Dice Wall represents the Dice loss function, CE Lumen + CE Wall represents the cross-entropy loss function, Consistencylumen-wall represents the consistency loss function, G L represents the intima segmentation result label, P L represents the intima segmentation result image, S L represents the intima skeletonization result data, G W represents the media segmentation result label, P W represents the media segmentation result image, S W represents the media skeletonization result data.

[0023] According to another specific embodiment of the present application, the consistency loss function is specifically: S l = SoftSkeleton (P l ) S w = SoftSkeleton (P w )

[0024] wherein P I represents the intima segmentation result image, S I represents the intima skeletonization result data, P W represents the media segmentation result image, S W represents the media skeletonization result data, SoftSkeleton represents the soft skeleton extraction method, and Consistencylumen-wall represents the consistency loss function.

[0025] According to another specific embodiment of the present application, a three-dimensional blood vessel image containing a blood vessel segment of interest is obtained, specifically comprising:

[0026] obtaining an untreated three-dimensional blood vessel image of the blood vessel segment of interest;

[0027] performing normalization processing and data enhancement processing on the untreated three-dimensional blood vessel image to obtain the three-dimensional blood vessel image.

[0028] According to another specific embodiment of the present application, the deep learning segmentation model further comprises a feature extractor, and the loss function is further used for training the feature extractor, and the three-dimensional blood vessel image containing the blood vessel segment of interest is processed by the deep learning segmentation model to be trained, specifically comprising:

[0029] The three-dimensional blood vessel image containing the blood vessel segment of interest is subjected to blood vessel feature extraction by the feature extractor through the attention mechanism, to obtain three-dimensional blood vessel image feature data of the blood vessel segment of interest;

[0030] The attention mechanism comprises a spatial attention mechanism and / or a channel attention mechanism.

[0031] The spatial attention mechanism is used for extracting blood vessel spatial feature data of the blood vessel segment of interest in the three-dimensional blood vessel image containing the blood vessel segment of interest.

[0032] The channel attention mechanism is used for extracting blood vessel channel feature data of the blood vessel segment of interest in the three-dimensional blood vessel image containing the blood vessel segment of interest.

[0033] In a second aspect, embodiments of the present application disclose a training device of a deep learning segmentation model, comprising: the deep learning segmentation model comprises a media segmentation device and an intima segmentation device, the media segmentation device is used for segmenting a media segmentation result image, the intima segmentation device is used for segmenting an intima segmentation result image, and the training device comprises:

[0034] An image acquisition module is configured to acquire a three-dimensional blood vessel image containing a blood vessel segment of interest, and acquire a labeled media segmentation result label and an intima segmentation result label of the blood vessel segment of interest.

[0035] A pre-segmentation module is configured to process the three-dimensional blood vessel image containing the blood vessel segment of interest by the deep learning segmentation model to be trained, to obtain a media segmentation result image and an intima segmentation result image.

[0036] A loss function determination module is configured to determine a loss function according to the media segmentation result label, the intima segmentation result label, the media segmentation result image and the intima segmentation result image of the blood vessel segment of interest.

[0037] A training module is configured to perform parameter updating on the deep learning segmentation model to be trained according to the three-dimensional blood vessel image containing the blood vessel segment of interest, the media segmentation result image, the intima segmentation result image, the media segmentation result label, the intima segmentation result label and the loss function, until the deep learning segmentation model to be trained meets a preset training condition, to obtain a trained deep learning segmentation model.

[0038] The media layer separator and the intima layer separator adopt a cross attention mechanism to segment the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain a media layer segmentation result image and an intima layer segmentation result image.

[0039] With the technical solution, the training device of the deep learning segmentation model combines the cross attention mechanism, and the cross attention mechanism enables the media layer features and the intima layer features to interact, so that the deep learning segmentation model is trained to more accurately simultaneously segment the media layer segmentation result image and the intima layer segmentation result image of the blood vessel segment of interest, and the segmentation accuracy of the media layer and the intima layer is improved.

[0040] According to another specific embodiment of the present application, in the process of processing the three-dimensional blood vessel image containing the blood vessel segment of interest by the deep learning segmentation model to be trained, the media layer separator and the intima layer separator adopt a cross attention mechanism to segment the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain a media layer segmentation result image and an intima layer segmentation result image.

[0041] In the process of segmenting the three-dimensional blood vessel image containing the blood vessel segment of interest by the cross attention mechanism, the process specifically includes:

[0042] The media layer separator extracts the media layer features of the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain media layer feature data;

[0043] The intima layer separator extracts the intima layer features of the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain intima layer feature data;

[0044] The media layer separator performs feature fusion on the media layer feature data and the intima layer feature data by the cross attention mechanism to obtain a media layer segmentation result image;

[0045] The intima layer separator performs feature fusion on the media layer feature data and the intima layer feature data by the cross attention mechanism to obtain an intima layer segmentation result image.

[0046] In a third aspect, the embodiments of the present application disclose a blood vessel image segmentation method for segmenting a blood vessel image by a deep learning segmentation model, the deep learning segmentation model including a media layer separator and an intima layer separator, the media layer separator being used to segment a media layer segmentation result image, and the intima layer separator being used to segment an intima layer segmentation result image, the blood vessel image segmentation method including:

[0047] Obtaining three-dimensional blood vessel image feature data of a blood vessel segment of interest;

[0048] Segmenting the three-dimensional blood vessel image feature data of the blood vessel segment of interest by the deep learning segmentation model by a cross attention mechanism to obtain a media layer segmentation result image and an intima layer segmentation result image of the blood vessel segment of interest.

[0049] By adopting the technical scheme, the blood vessel image segmentation method can improve the segmentation accuracy of the media and the intima by fully utilizing the interaction between the media and the intima through the cross attention mechanism.

[0050] According to another specific embodiment of the present application, the process of segmenting the three-dimensional blood vessel image feature data by adopting the cross attention mechanism specifically includes:

[0051] The media feature of the three-dimensional blood vessel image feature data is extracted by the media segmenter to obtain media feature data;

[0052] The intima feature of the three-dimensional blood vessel image feature data is extracted by the intima segmenter to obtain intima feature data;

[0053] The media segmenter performs feature fusion on the media feature data and the intima feature data through the cross attention mechanism to obtain a media segmentation result image;

[0054] The intima segmenter performs feature fusion on the media feature data and the intima feature data through the cross attention mechanism to obtain an intima segmentation result image.

[0055] According to another specific embodiment of the present application, the deep learning segmentation model further includes a feature extractor for obtaining three-dimensional blood vessel image feature data of a blood vessel segment of interest, specifically including:

[0056] Obtaining a three-dimensional blood vessel image containing a blood vessel segment of interest;

[0057] According to another specific embodiment of the present application, the deep learning segmentation model further includes a feature extractor for obtaining three-dimensional blood vessel image feature data of a blood vessel segment of interest, specifically including:

[0058] The attention mechanism includes a spatial attention mechanism and / or a channel attention mechanism;

[0059] The spatial attention mechanism is used to extract blood vessel spatial feature data of the blood vessel segment of interest in the three-dimensional blood vessel image;

[0060] The channel attention mechanism is used to extract blood vessel channel feature data of the blood vessel segment of interest in the three-dimensional blood vessel image.

[0061] According to another specific embodiment of the present application, the three-dimensional blood vessel image containing the blood vessel segment of interest is obtained, specifically including:

[0062] Obtaining an untreated three-dimensional blood vessel image of the blood vessel segment of interest;

[0063] The three-dimensional blood vessel image of the interest blood vessel segment is obtained by normalizing the untreated three-dimensional blood vessel image.

[0064] According to another specific embodiment of the present application, the image enhancement optimization operation is performed on the media segmentation result image and the intima segmentation result image to obtain the enhanced final media segmentation result image and the enhanced final intima segmentation result image.

[0065] The image enhancement optimization operation includes a morphological operation and / or a connectivity analysis, the morphological operation is selected from one or more of the following: an erosion operation, a dilation operation, an opening operation, a closing operation, a morphological gradient, a top-hat, and a black-hat, and the connectivity analysis is selected from one or more of the following: connected component labeling, region growing, boundary tracing, hole filling, skeleton extraction, and region segmentation.

[0066] In a fourth aspect, embodiments of the present application disclose a blood vessel image segmentation device, which segments a blood vessel image through a deep learning segmentation model, the deep learning segmentation model includes a media segmenter and an intima segmenter, the media segmenter is used to segment a media segmentation result image, and the intima segmenter is used to segment an intima segmentation result image, and the blood vessel image segmentation device includes:

[0067] a feature acquisition module, configured to acquire three-dimensional blood vessel image feature data of the interest blood vessel segment;

[0068] an image segmentation module, configured to segment the three-dimensional blood vessel image feature data of the interest blood vessel segment through the deep learning segmentation model by using a cross-attention mechanism to obtain the media segmentation result image and the intima segmentation result image of the interest blood vessel segment.

[0069] By using the above technical solution, the blood vessel image segmentation device of the present application can improve the segmentation accuracy of the media and the intima by fully utilizing the interaction between the media and the intima through the cross-attention mechanism.

[0070] According to another specific embodiment of the present application, the process of segmenting the three-dimensional blood vessel image feature data by the image segmentation module through the cross-attention mechanism specifically includes:

[0071] the media segmenter extracts media features of the three-dimensional blood vessel image feature data to obtain media feature data;

[0072] the intima segmenter extracts intima features of the three-dimensional blood vessel image feature data to obtain intima feature data;

[0073] the media segmenter performs feature fusion on the media feature data and the intima feature data through the cross-attention mechanism to obtain the media segmentation result image;

[0074] The intima segmenter fuses the media feature data and the intima feature data through a cross-attention mechanism to obtain an intima segmentation result image.

[0075] In a fifth aspect, an embodiment of the present application discloses an electronic device including a processor and a memory, the memory storing at least one instruction, the at least one instruction, when executed by the processor, implementing the training method of the deep learning segmentation model in any of the preceding embodiments, and / or the blood vessel image segmentation method in any of the preceding embodiments.

[0076] With the above technical solution, the electronic device can combine the cross-attention mechanism, interact the media feature data and the intima feature data, train the deep learning segmentation model to more accurately simultaneously segment the media segmentation result image and the intima segmentation result image of the blood vessel segment of interest, and improve the segmentation accuracy of the media and the intima.

[0077] In a sixth aspect, an embodiment of the present application discloses a computer-readable storage medium, the computer-readable storage medium storing at least one instruction, the at least one instruction, when executed, implementing the training method of the deep learning segmentation model in any of the preceding embodiments, and / or the blood vessel image segmentation method in any of the preceding embodiments.

[0078] With the above technical solution, the computer-readable storage medium can combine the cross-attention mechanism, interact the media feature data and the intima feature data, train the deep learning segmentation model to more accurately simultaneously segment the media segmentation result image and the intima segmentation result image of the blood vessel segment of interest, and improve the segmentation accuracy of the media and the intima.

[0079] It should be noted that the training method of the deep learning segmentation model and the blood vessel image segmentation method of the present application do not involve surgical methods, but are based on processing of medical images, i.e., the direct object is a medical image, such as a three-dimensional coronary CTA image. The training method of the deep learning segmentation model and the blood vessel image segmentation method of the present application have a non-therapeutic purpose.

[0080] The training device of the deep learning segmentation model, the blood vessel image segmentation device, the electronic device and the computer-readable storage medium of the present application are also not devices related to surgical methods, but are devices related to processing of medical images, i.e., the direct object is a medical image, such as a three-dimensional coronary CTA image. The training device of the deep learning segmentation model, the blood vessel image segmentation device, the electronic device and the computer-readable storage medium of the present application all have a non-therapeutic purpose.

[0081] The beneficial effects of the present application are as follows:

[0082] 1.The training method of a deep learning segmentation model and the training device of the deep learning segmentation model are provided, a cross-attention mechanism is introduced into the deep learning segmentation model, the interaction between the media feature data and the intima feature data is promoted, the media segmentation and the intima segmentation are promoted, the deep learning segmentation model can segment the media segmentation result and the intima segmentation result of the interested blood vessel segment, and the accuracy of the media and intima segmentation is improved.

[0083] In the training process of the deep learning segmentation model, a consistency loss function is also designed, the relationship between the media and the intima is established through the consistency loss function, and the coordination between the intima segmentation result and the media segmentation result is ensured.

[0084] In addition, the unprocessed three-dimensional blood vessel image containing the interested blood vessel segment in the training process of the deep learning segmentation model is also normalized and data enhanced, the data amount available during training can be expanded, the data diversity is improved, and the deep learning segmentation model has better generalization ability.

[0085] 2.The blood vessel image segmentation method and the blood vessel image segmentation device are provided, the cross-attention mechanism is used for feature fusion of the media feature data and the intima feature data, the interaction between the media feature data and the intima feature data is promoted, the media segmentation and the intima segmentation are promoted, and the segmentation accuracy of the media and the intima is improved.

[0086] In addition, the unprocessed three-dimensional blood vessel image containing the interested blood vessel segment in the training process of the deep learning segmentation model is also normalized and data enhanced, the data amount available during training can be expanded, the data diversity is improved, and the deep learning segmentation model has better generalization ability.

[0087] In addition, the unprocessed three-dimensional blood vessel image containing the interested blood vessel segment in the training process of the deep learning segmentation model is also normalized and data enhanced, the data amount available during training can be expanded, the data diversity is improved, and the deep learning segmentation model has better generalization ability. BRIEF DESCRIPTION OF DRAWINGS

[0088] Fig. 1 shows a flowchart of the training method of the deep learning segmentation model in the embodiment of the present application;

[0089] Fig. 2 shows a flowchart of the training method of the deep learning segmentation model in the embodiment of the present application;

[0090] Figure 3 shows a flowchart of a training method of a deep learning segmentation model according to an embodiment of the present application;

[0091] Figure 4 shows a flowchart of a training method of a deep learning segmentation model according to an embodiment of the present application;

[0092] Figure 5 shows a flowchart of a blood vessel image segmentation method according to an embodiment of the present application;

[0093] Figure 6 shows a flowchart of a blood vessel image segmentation method according to an embodiment of the present application;

[0094] Figure 7 shows a flowchart of a blood vessel image segmentation method according to an embodiment of the present application;

[0095] Figure 8 shows a flowchart of a blood vessel image segmentation method according to an embodiment of the present application;

[0096] Figure 9 shows a structural diagram of a training device of a deep learning segmentation model according to an embodiment of the present application;

[0097] Figure 10 shows a structural diagram of a blood vessel image segmentation device according to an embodiment of the present application;

[0098] Figure 11 shows a structural diagram of an electronic device according to an embodiment of the present application;

[0099] Figure 12 shows a structural diagram of a deep learning segmentation model according to an embodiment of the present application;

[0100] Figure 13 shows a raw three-dimensional coronary CTA cross-sectional image according to an embodiment of the present application;

[0101] Figure 14 shows a three-dimensional coronary CTA cross-sectional image obtained after normalization processing of a raw three-dimensional coronary CTA cross-sectional image according to an embodiment of the present application;

[0102] Figure 15 shows an image obtained after data enhancement of a normalized three-dimensional coronary CTA cross-sectional image according to an embodiment of the present application;

[0103] Figure 16 shows a result image in which a media segmentation result image and an intima segmentation result image are superimposed on a raw three-dimensional blood vessel image according to an embodiment of the present application;

[0104] Figure 17 shows a schematic diagram of a three-dimensional visualization result of a segmented media and a segmented intima according to an embodiment of the present application;

[0105] Figure 18 shows a plan view along a blood vessel centerline according to an embodiment of the present application. DETAILED DESCRIPTION

[0106] The advantages and effects of the present application can be easily understood by those skilled in the art from the content disclosed in the specification. Although the description of the present application will be introduced in combination with the preferred embodiments, it does not mean that the features of the present application are limited to the embodiments. On the contrary, the purpose of introducing the present application in combination with the embodiments is to cover other options or modifications that can be extended based on the claims of the present application. In order to provide a deep understanding of the present application, many specific details will be included in the following description. The present application can also be implemented without using these details. In addition, in order to avoid confusion or obscure the focus of the present application, some specific details will be omitted in the description. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0107] It should be noted that in the specification, similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0108] The terms "first", "second", and the like are only used for differentiation description, and cannot be understood as indicating or implying relative importance.

[0109] In the description of the present embodiment, it should also be noted that unless otherwise explicitly specified and limited, the terms "set", "connected", "connected" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be connected inside two elements. For those skilled in the art, the specific meaning of the above terms in the present embodiment can be understood according to the specific circumstances.

[0110] In order to make the purpose, technical scheme and advantages of the present application more clear, the embodiments of the present application will be further described in detail below in combination with the drawings.

[0111] The inventors found that the prior art only focuses on the segmentation between the intima and the lumen of the blood vessel, and does not focus on the segmentation between the intima and the media of the blood vessel, and the segmentation of the prior art is not accurate enough, which is not conducive to the accurate extraction of the center line, and will affect the subsequent clinical diagnosis of heart blood vessel diseases. In addition, since the media and the intima are closely connected in the tissue structure, the boundary between the media and the intima is not easy to distinguish on the image, and different regions of the blood vessel intima and the blood vessel media may also have different thicknesses, and the blood vessel morphology may also cause the media and the intima to be more difficult to distinguish. The above situations will increase the complexity of image segmentation, and the existing blood vessel image segmentation algorithm is difficult to accurately segment the blood vessel intima and the blood vessel media. The inventors hope to solve the above problems.

[0112] In a first aspect, embodiments of the present application disclose a training method of a deep learning segmentation model, the deep learning segmentation model comprising a middle membrane segmenter and an inner membrane segmenter, the middle membrane segmenter being configured to segment a middle membrane segmentation result image, and the inner membrane segmenter being configured to segment an inner membrane segmentation result image.

[0113] Referring to FIG. 1 and in combination with FIG. 12, the training method of the deep learning segmentation model specifically comprises the following steps:

[0114] S1: obtaining a three-dimensional blood vessel image containing a blood vessel segment of interest, and obtaining a middle membrane segmentation result label and an inner membrane segmentation result label of the blood vessel segment of interest.

[0115] Specifically, the three-dimensional blood vessel image is a three-dimensional coronary CTA image, and the three-dimensional coronary CTA image can clearly observe the condition of the heart blood vessels. The middle membrane segmentation result label of the blood vessel segment of interest is, for example, a result of a middle membrane segmentation labeled by a person in advance, and the inner membrane segmentation result label is, for example, a result of an inner membrane segmentation labeled by a person in advance.

[0116] S2: processing the three-dimensional blood vessel image containing the blood vessel segment of interest by using the deep learning segmentation model to be trained to obtain the middle membrane segmentation result image and the inner membrane segmentation result image.

[0117] In the training method of the deep learning segmentation model, the middle membrane segmenter and the inner membrane segmenter are configured to segment the three-dimensional blood vessel image containing the blood vessel segment of interest by using a cross-attention mechanism to obtain the middle membrane segmentation result image and the inner membrane segmentation result image.

[0118] More specifically, in the process of segmenting the three-dimensional blood vessel image containing the blood vessel segment of interest by using the cross-attention mechanism, the middle membrane segmenter is configured to extract middle membrane features of the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain middle membrane feature data, the inner membrane segmenter is configured to extract inner membrane features of the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain inner membrane feature data, the middle membrane segmenter is configured to perform feature fusion on the middle membrane feature data and the inner membrane feature data by using the cross-attention mechanism to obtain the middle membrane segmentation result image, and the inner membrane segmenter is configured to perform feature fusion on the middle membrane feature data and the inner membrane feature data by using the cross-attention mechanism to obtain the inner membrane segmentation result image.

[0119] The cross-attention mechanism is a mechanism used in deep learning neural networks for processing sequence-to-sequence (Seq2Seq) tasks, especially in the interaction between the encoder and the decoder of the deep learning neural network. The core idea of the cross-attention mechanism is that the cross-attention mechanism can be used in both the encoder and the decoder, and the function is to interact, fuse and update the features from different objects (in the present application, the features are specifically the middle membrane features and the inner membrane features).

[0120] In the present application, the application mode of cross attention is further improved. Specifically, the media membrane segmenter and the intima membrane segmenter are both decoders, and the present application segments the three-dimensional blood vessel image through a cross attention mechanism. The cross attention mechanism can make the media membrane segmenter and the intima membrane segmenter learn from each other when segmenting the media membrane and the intima membrane, respectively, and establish a mutual dependence relationship between the intima membrane and the media membrane. The media membrane segmenter can combine the intima membrane features during intima membrane segmentation to learn and decode the blood vessel media membrane features, and the intima membrane segmenter can combine the blood vessel media membrane features during media membrane segmentation to learn and decode the blood vessel intima membrane features. Then, the media membrane segmenter and the intima membrane segmenter can simultaneously segment the media membrane segmentation result and the intima membrane segmentation result, respectively. In this way, the mutual relationship between the blood vessel media membrane and the blood vessel intima membrane can be fully utilized, efficient feature interaction between the blood vessel media membrane and the blood vessel intima membrane can be realized, the expression ability of the deep learning segmentation model can be improved, and the segmentation accuracy of the media membrane segmenter and the intima membrane segmenter for the blood vessel media membrane and the blood vessel intima membrane can be improved.

[0121] S3: determining a loss function according to the media membrane segmentation result label, the intima membrane segmentation result label, the media membrane segmentation result image, and the intima membrane segmentation result image of the blood vessel segment of interest.

[0122] S4: performing parameter updating on the deep learning segmentation model to be trained according to the three-dimensional blood vessel image containing the blood vessel segment of interest, the media membrane segmentation result image, the intima membrane segmentation result image, the media membrane segmentation result label, the intima membrane segmentation result label, and the loss function, until the deep learning segmentation model to be trained meets a preset training condition, to obtain a trained deep learning segmentation model.

[0123] During the entire training process, the media membrane segmentation result label and the intima membrane segmentation result label are segmentation labels. The media membrane segmentation result label is a standard media membrane segmentation result, i.e., an expected media membrane segmentation result. The intima membrane segmentation result label is a standard intima membrane segmentation result, i.e., an expected intima membrane segmentation result. During training, the three-dimensional blood vessel image and its corresponding media membrane segmentation result label and intima membrane segmentation result label are input into the deep learning segmentation model, so that the parameters of the deep learning segmentation model are updated, the segmentation process is learned, and the preset training condition is met.

[0124] The preset training condition refers to that the loss function is reduced to convergence (i.e., the loss function does not continue to decrease during continuous training). When the preset training condition is met, it is considered that the deep learning segmentation model can accurately segment the media membrane segmentation result and the intima membrane segmentation result. The preset training condition can be a specific number of training times, and the number of training times can be determined according to the amount of data used for training and the specific strategy during training.

[0125] With the technical scheme, the training method of the deep learning segmentation model combines the cross attention mechanism, and through the interaction of the media feature data and the intima feature data, the deep learning segmentation model trained can more accurately simultaneously segment the media segmentation result image and the intima segmentation result image of the blood vessel segment of interest. The deep learning segmentation model with an end-to-end processing framework is trained, and the three-dimensional blood vessel image containing the blood vessel segment of interest is directly input into the deep learning segmentation model, so that the media and intima of the blood vessel can be processed at the same time, and the corresponding media segmentation result and intima segmentation result can be obtained. In the above embodiments, the deep learning segmentation model of the application can optimize and guide the accurate segmentation of the intima by using the relative stability of the media and the potential topological relationship between the media and the intima, and combining the cross attention mechanism, and can also promote the accurate segmentation of the media at the same time, thereby improving the segmentation accuracy of the media and the intima. In addition, the deep learning segmentation model with an end-to-end deep learning framework can avoid complex artificial feature design and multi-stage processing, and simplify the workflow.

[0126] Further, in the above embodiments, the loss function in step S3 includes a cross-entropy loss function, a Dice loss function and a consistency loss function.

[0127] Specifically, the cross-entropy loss function is used to measure the difference between the actual output (from the model) and the target output (the true label). In the present application, it is specifically used to measure the difference between the media segmentation result and the intima segmentation result actually segmented by the deep learning segmentation model, and the media segmentation label and the intima segmentation label (i.e. the expected media segmentation result and intima segmentation result).

[0128] The Dice loss function can better handle the problem of large difference in the number of pixels between the foreground and the background in the image segmentation task, and improve the segmentation accuracy. In the present application, the foreground is the blood vessel media and the blood vessel intima, and the background is the surrounding environment of the blood vessel. The Dice loss function is beneficial to accurately distinguishing the blood vessel intima, the blood vessel media and the environment where the blood vessel is located, and improving the segmentation accuracy of the blood vessel intima and the blood vessel media.

[0129] Fig. 18 shows a plan view along the center line of the blood vessel, in which the inner contour line (i.e. the contour line closer to the center line of the blood vessel) corresponds to the intima of the blood vessel, and the outer contour line corresponds to the media of the blood vessel. As can be seen from Fig. 18, the media is thicker than the intima, the area displayed on the image is larger and more continuous, and it is more conducive to segmentation.

[0130] The media is relatively easy to maintain its segmentation result because the lumen is relatively thick and the diameter changes little, and the stability of the media segmentation result is relatively high. The consistency loss function (i.e., consistency constraint) is proposed in the process of simultaneously segmenting the media and intima, in order to constrain the intima continuity by using the continuity of the media. Due to the relatively high stability of the media segmentation, the relationship between the media and the intima established by the consistency constraint helps to ensure the coordination between the intima segmentation result and the media segmentation result. Referring to FIG. 2, the consistency loss function is obtained by the following steps:

[0131] S310: obtaining the media skeletonization result data of the media segmentation result image by a soft skeleton extraction method according to the media segmentation result image, and obtaining the intima skeletonization result data of the intima segmentation result image by a soft skeleton extraction method according to the intima segmentation result image.

[0132] The soft skeleton extraction method is a soft-skeleton algorithm. The soft-skeleton algorithm is a differentiable method for calculating the skeletonization result of the segmentation result. The result obtained by the soft-skeleton is similar to the conventional skeletonization algorithm, but the soft-skeleton adopts a differentiable operation, which is convenient for calculating the loss function.

[0133] S320: determining the consistency loss function according to the media skeletonization result data and the intima skeletonization result data.

[0134] The consistency loss function is more specifically determined as follows: the dot product is calculated by the media skeletonization result data and the intima skeletonization result data, and then divided by the sum of the pixel points of the media skeletonization result data, so that the continuity of the intima relative to the media can be calculated, and the continuity of the intima relative to the media is the consistency loss function.

[0135] The media is relatively thick, and the media segmentation result is relatively easy to maintain continuity, so the consistency loss function can be used to make the segmentation results of the media and the intima consistent in continuity, so as to promote the continuity of the intima segmentation result.

[0136] The consistency loss function is determined by the above method. First, the media centerline is obtained by soft-skeleton, and the consistency of the connectivity of the intima and the media is calculated by the consistency loss function. This way is not easily affected by the different diameters of the blood vessels in places where the blood vessels are different in thickness, which helps to more accurately segment the media and the intima.

[0137] The consistency loss function can be specifically S l =SoftSkeleton(P l ) (Formula 1) S w= SoftSkeleton(P w ) (Formula 2)

[0138] wherein, P I denotes the intima segmentation result image, S I denotes the intima skeletonization result data, P W denotes the media segmentation result image, S W denotes the media skeletonization result data, SoftSkeleton denotes the soft skeleton extraction method (i.e. the function of the aforementioned soft-skeleton algorithm), and Consistencylumen-wall denotes the consistency loss function.

[0139] In each of the above embodiments, the loss function can be specifically: Total Loss = Dice Lumen + Dice Wall + CE Lumen + CE Wall + Consistencylumen-wall (Formula 4)

[0140] wherein, Total Loss denotes the loss function, Dice Lumen + Dice Wall denotes the Dice loss function, CE Lumen + CE Wall denotes the cross-entropy loss function, Consistencylumen-wall denotes the consistency loss function, G L denotes the intima segmentation result label, P L denotes the intima segmentation result image, S L denotes the intima skeletonization result data, G W denotes the media segmentation result label, P W denotes the media segmentation result image, S W denotes the media skeletonization result data.

[0141] Further, referring to FIG. 3, the aforementioned step S1 specifically includes the following steps:

[0142] S11: obtaining an unprocessed three-dimensional blood vessel image of a blood vessel segment of interest.

[0143] FIG. 13 shows an original three-dimensional coronary CTA cross-sectional image, which is an example of an unprocessed three-dimensional blood vessel image.

[0144] S12: performing normalization processing and data enhancement processing on the unprocessed three-dimensional blood vessel image to obtain a three-dimensional blood vessel image.

[0145] The normalization processing and the data enhancement processing can improve the image quality and the data diversity. Specifically, the normalization processing refers to scaling the image pixel value to the interval of 0-1, which is more conducive to the training and prediction of the model. The data enhancement processing includes random rotation, random noise addition and the like. The data enhancement processing is only used in the model training process. Through the data enhancement processing, the data can be expanded as much as possible, i.e., through random changes in the contrast of the image, random cropping, scaling, grayscale transformation and noise addition and the like, the changed data is equivalent to a new data for the deep learning segmentation model, so that the data diversity can be improved, and thus the trained model has better generalization performance.

[0146] FIG. 15 shows a three-dimensional blood vessel image obtained after normalization and data enhancement processing of the original three-dimensional coronary CTA cross-sectional image. In this embodiment, the data increase specifically includes random noise and random contrast transformation in addition to the normalized original three-dimensional coronary CTA cross-sectional image.

[0147] The deep learning segmentation model further includes a feature extractor, and the loss function is also used to train the feature extractor. Further, with reference to FIG. 4 and in combination with FIG. 12, the foregoing step S2 further includes the following steps:

[0148] S21: performing blood vessel feature extraction on the three-dimensional blood vessel image containing the blood vessel segment of interest by the feature extractor through the attention mechanism, to obtain three-dimensional blood vessel image feature data of the blood vessel segment of interest.

[0149] The attention mechanism includes a spatial attention mechanism and / or a channel attention mechanism.

[0150] The spatial attention mechanism is performed in the three-dimensional spatial dimension of the image, which can enable the deep learning segmentation model to focus on a specific key region in the image and ignore unimportant parts. The spatial attention mechanism can enable the deep learning segmentation model to automatically learn the importance of different spatial positions in the image, and improve the representation ability of the deep learning segmentation model for spatial information. In the present application, the spatial attention mechanism is used to extract the blood vessel spatial feature data of the blood vessel segment of interest in the three-dimensional blood vessel image containing the blood vessel segment of interest, which is conducive to improving the segmentation accuracy of the media and intima.

[0151] The channel attention mechanism can assign different weights to different feature channels of an image, so that the deep learning segmentation model can pay more attention to the feature channels with more important information and suppress the feature channels with less important information. In the present application, the channel attention mechanism is used to extract the feature channel data (i.e., feature channel) of the blood vessel segment of interest in the three-dimensional blood vessel image containing the blood vessel segment of interest. The weight of each feature channel can be adaptively adjusted to improve the representation ability of the deep learning segmentation model for the key features of the target blood vessel, thereby improving the performance of the deep learning segmentation model and facilitating the improvement of the segmentation accuracy of the intima and media.

[0152] Specifically, with reference to FIG. 12, the feature extractor is a shared encoder for feature encoding of the three-dimensional blood vessel image to obtain a feature vector F1. The feature vector F1 includes an intima feature vector for representing intima features and a media feature vector for representing media features. Then, the feature vector F1 is subjected to channel attention mechanism and spatial attention mechanism calculation, so that the feature vector F1 is further encoded into a feature vector F2. The feature vector F2 is the three-dimensional blood vessel image feature data. The feature vector F2 is further decoded by the intima segmenter and the media segmenter to obtain the media features and the intima features, respectively. The intima segmenter and the media segmenter promote each other to learn the intima features and the media features, respectively, and finally obtain the intima segmentation result image and the media segmentation result image with high segmentation accuracy.

[0153] Further, in the above embodiments, the feature extractor, the intima segmenter and the media segmenter can all be any backbone. The backbone refers to the main feature extraction part of the neural network, which can be a model pre-trained based on a large image dataset (such as ImageNet) and can extract specific features of an image from input data. The backbone can be composed of a series of convolutional layers, pooling layers, normalization layers and activation function layers.

[0154] In the above embodiments, the present application can achieve higher accuracy of intima segmentation of three-dimensional coronary CTA images, especially three-dimensional coronary CTA images with high plaque load, by using the topology of the media as a guide. The present application improves the segmentation accuracy of the intima and the media by using the channel attention, spatial attention and cross-attention mechanisms to realize efficient feature interaction between the intima and the media. The present application can also optimize the continuity and accuracy of intima segmentation by using the above deep learning segmentation model. Since the extraction of the blood vessel centerline is obtained by skeletonization of the intima segmentation result, the present application can also improve the accuracy of the subsequent extraction of the blood vessel centerline and optimize the extraction quality of the blood vessel centerline.

[0155] In addition, by means of the optimized inner membrane segmentation and center line extraction technology, the risk assessment and pathological analysis of heart diseases can be more accurately performed, the misdiagnosis rate and missed diagnosis rate are reduced, the clinical application value of CCTA (coronary computed tomography angiography) analysis is optimized, the accurate assessment of heart diseases and subsequent treatment planning are facilitated, and significant social and economic benefits are achieved.

[0156] In a second aspect, referring to FIG. 9, an embodiment of the present application discloses a training device 1 of a deep learning segmentation model, which is trained by the training method of the deep learning segmentation model in any of the preceding embodiments. The deep learning segmentation model comprises a media membrane segmenter and an inner membrane segmenter, the media membrane segmenter is used to segment a media membrane segmentation result image, and the inner membrane segmenter is used to segment an inner membrane segmentation result image. The training device 1 of the deep learning segmentation model comprises an image acquisition module 11, a pre-segmentation module 12, a loss function determination module 13, and a training module 14.

[0157] The image acquisition module 11 is connected with the pre-segmentation module 12, the pre-segmentation module 12 is connected with the loss function determination module 13, and the loss function determination module 13 is connected with the training module 14.

[0158] Continuing to refer to FIG. 9, the image acquisition module 11 is used to acquire a three-dimensional blood vessel image containing a blood vessel segment of interest, and acquire a media membrane segmentation result label and an inner membrane segmentation result label of the blood vessel segment of interest.

[0159] The pre-segmentation module 12 is used to process the three-dimensional blood vessel image containing the blood vessel segment of interest by the deep learning segmentation model to be trained, to obtain a media membrane segmentation result image and an inner membrane segmentation result image.

[0160] The loss function determination module 13 is used to determine a loss function according to the three-dimensional blood vessel image of the blood vessel segment of interest, the media membrane segmentation result image, and the inner membrane segmentation result image.

[0161] The training module 14 is used to update parameters of the deep learning segmentation model to be trained according to the three-dimensional blood vessel image containing the blood vessel segment of interest, the media membrane segmentation result image, the inner membrane segmentation result image, the media membrane segmentation result label, the inner membrane segmentation result label, and the loss function, until the deep learning segmentation model to be trained meets a preset training condition, to obtain a trained deep learning segmentation model.

[0162] By adopting the above technical solution, the training device 1 of the deep learning segmentation model in the embodiment combines the cross-attention mechanism, the media membrane features and the inner membrane features are interacted by the cross-attention mechanism, the deep learning segmentation model trained can more accurately segment the media membrane segmentation result image and the inner membrane segmentation result image of the blood vessel segment of interest at the same time, and the segmentation accuracy of the media membrane and the inner membrane is improved.

[0163] In the process of the pre-segmentation module 12 processing the three-dimensional blood vessel image containing the blood vessel segment of interest by the deep learning segmentation model to be trained, specifically, the three-dimensional blood vessel image containing the blood vessel segment of interest is segmented by the media layer segmenter and the intima layer segmenter to obtain the media layer segmentation result image and the intima layer segmentation result image by using the cross-attention mechanism. In the process of segmenting the three-dimensional blood vessel image containing the blood vessel segment of interest by using the cross-attention mechanism, specifically, the media layer features of the three-dimensional blood vessel image containing the blood vessel segment of interest are extracted by the media layer segmenter to obtain the media layer feature data; the intima layer features of the three-dimensional blood vessel image containing the blood vessel segment of interest are extracted by the intima layer segmenter to obtain the intima layer feature data; the media layer feature data and the intima layer feature data are fused by the media layer segmenter by using the cross-attention mechanism to obtain the media layer segmentation result image; and the media layer feature data and the intima layer feature data are fused by the intima layer segmenter by using the cross-attention mechanism to obtain the intima layer segmentation result image.

[0164] By using the above technical solution, the training device 1 of the deep learning segmentation model of the embodiment combines the cross-attention mechanism, and through the interaction of the media layer feature data and the intima layer feature data, the deep learning segmentation model trained can more accurately segment the media layer segmentation result image and the intima layer segmentation result image of the blood vessel segment of interest at the same time, and the segmentation accuracy of the media layer and the intima layer is improved.

[0165] Further, the image acquisition module 11 can also be used to perform the foregoing steps S11 and S12. The pre-segmentation module 12 can also be used to perform the foregoing step S21. The loss function determination module 13 can also be used to perform the foregoing steps S310 and S320.

[0166] In a third aspect, an embodiment of the present application discloses a blood vessel image segmentation method. The blood vessel image is segmented by a deep learning segmentation model. The deep learning segmentation model includes a media layer segmenter and an intima layer segmenter. The media layer segmenter is used to segment a media layer segmentation result image. The intima layer segmenter is used to segment an intima layer segmentation result image. The deep learning segmentation model is trained by the deep learning segmentation model training method in any of the foregoing embodiments.

[0167] Referring to FIG. 5, the blood vessel image segmentation method specifically includes the following steps:

[0168] S01: Obtain three-dimensional blood vessel image feature data of a blood vessel segment of interest.

[0169] The three-dimensional blood vessel image feature data can be the feature vector F2 in FIG. 12.

[0170] S02: segmenting the three-dimensional blood vessel image feature data of the blood vessel segment of interest by using a cross-attention mechanism of a deep learning segmentation model to obtain a media segmentation result image and an intima segmentation result image of the blood vessel segment of interest.

[0171] By using the above technical solution, the blood vessel image segmentation method can fully utilize the interaction between the media and the intima by using the cross-attention mechanism, thereby improving the segmentation accuracy of the media and the intima.

[0172] In the process of segmenting the three-dimensional blood vessel image feature data by using the cross-attention mechanism, the process specifically includes: extracting media features of the three-dimensional blood vessel image feature data by using a media segmenter to obtain media feature data; and extracting intima features of the three-dimensional blood vessel image feature data by using an intima segmenter to obtain intima feature data. The media segmenter performs feature fusion on the media feature data and the intima feature data by using the cross-attention mechanism to obtain a media segmentation result image. The intima segmenter performs feature fusion on the media feature data and the intima feature data by using the cross-attention mechanism to obtain an intima segmentation result image.

[0173] FIG. 16 shows a schematic diagram in which the media segmentation result image and the intima segmentation result image are superimposed on the original three-dimensional blood vessel image, and FIG. 17 shows a three-dimensional visualization result of the corresponding media segmentation result and intima segmentation result. In FIG. 16, the area indicated by the white arrow in the white dashed line box and in FIG. 17, the relatively bright part (i.e., the part with a color closer to white) represents the media segmentation result, and the relatively dark part represents the intima segmentation result. It should be noted that the blue part is also part of the media. Since it is a superimposed image, part of the green color has been covered by the blue color.

[0174] By using the above technical solution, the blood vessel image segmentation method can make the media features and the intima features interact by using the cross-attention mechanism, so that the deep learning segmentation model can more accurately segment the media segmentation result image and the intima segmentation result image of the blood vessel segment of interest at the same time, thereby improving the segmentation accuracy of the media and the intima.

[0175] Further, with reference to FIG. 6, the foregoing step S01 specifically includes the following steps:

[0176] S011: acquiring a three-dimensional blood vessel image containing a blood vessel segment of interest.

[0177] Specifically, the three-dimensional blood vessel image is a three-dimensional coronary CTA image.

[0178] S012: obtaining three-dimensional vessel image feature data of the vessel segment of interest from the three-dimensional vessel image containing the vessel segment of interest, comprising: extracting vessel features of the three-dimensional vessel image of the vessel segment of interest by a feature extractor using an attention mechanism to obtain the three-dimensional vessel image feature data of the vessel segment of interest.

[0179] The attention mechanism includes a spatial attention mechanism and / or a channel attention mechanism. The spatial attention mechanism is used to extract spatial feature data of the vessel segment of interest in the three-dimensional vessel image. The channel attention mechanism is used to extract channel feature data of the vessel segment of interest in the three-dimensional vessel image. For more specific content, refer to the foregoing description of the spatial attention mechanism and the channel attention mechanism, which will not be repeated here.

[0180] Further, referring to FIG. 7, the foregoing step S011 specifically includes the following steps:

[0181] S0111: obtaining an untreated three-dimensional vessel image of the vessel segment of interest.

[0182] The untreated three-dimensional vessel image is, for example, the original three-dimensional coronary CTA cross-sectional image shown in FIG. 13.

[0183] S0112: performing normalization processing on the untreated three-dimensional vessel image to obtain a three-dimensional vessel image of the vessel segment of interest.

[0184] The three-dimensional vessel image obtained after normalization is, for example, the three-dimensional coronary CTA cross-sectional image shown in FIG. 14.

[0185] It should be noted that when using the trained deep learning segmentation model in practice, data enhancement processing is not required on the untreated three-dimensional vessel image during preprocessing. Only normalization processing is required.

[0186] Referring to FIG. 8, after the foregoing steps S01 and S02 are performed, step S03 can be further performed: performing image enhancement optimization operation on the media segmentation result image and the intima segmentation result image to obtain the enhanced final media segmentation result image and the final intima segmentation result image.

[0187] The image enhancement optimization operation includes morphological operation and / or connectivity analysis. The morphological operation is selected from one or more of the following: erosion operation, dilation operation, opening operation, closing operation, morphological gradient, top hat, and black hat. The connectivity analysis is selected from one or more of the following: connected component labeling, region growing, boundary tracing, hole filling, skeleton extraction, and region segmentation. The above operations can fill the holes inside the blood vessels in the segmentation image, or remove noise generated during segmentation, etc., all of which can achieve the purpose of optimizing the image. The appropriate operation can be selected for segmentation image optimization according to the actual segmentation situation.

[0188] By further image enhancement optimization operation on the media split result image and the intima split result image, the accuracy and robustness of the media and intima split can be further improved.

[0189] In a fourth aspect, referring to FIG. 10, an embodiment of the present application discloses a blood vessel image segmentation device 2, which segments a blood vessel image by a deep learning segmentation model, the deep learning segmentation model comprising a media splitter and an intima splitter, the media splitter being used to split a media split result image, and the intima splitter being used to split an intima split result image. The blood vessel image segmentation device 2 comprises a feature acquisition module 21 and an image segmentation module 22. The feature acquisition module 21 is connected with the image segmentation module 22.

[0190] The feature acquisition module 21 is used to acquire three-dimensional blood vessel image feature data of a blood vessel segment of interest.

[0191] The image segmentation module 22 is used to segment the three-dimensional blood vessel image feature data of the blood vessel segment of interest by the deep learning segmentation model using a cross-attention mechanism, to obtain the media split result image and the intima split result image of the blood vessel segment of interest.

[0192] By using the above technical solution, the blood vessel image segmentation device 2 of the present application can improve the segmentation accuracy of the media and intima by using the cross-attention mechanism and fully utilizing the interaction between the media and intima.

[0193] In the process of segmenting the three-dimensional blood vessel image feature data by the image segmentation module using the cross-attention mechanism, specifically comprising: extracting the media feature of the three-dimensional blood vessel image feature data by the media splitter to obtain media feature data; extracting the intima feature of the three-dimensional blood vessel image feature data by the intima splitter to obtain intima feature data; the media splitter performing feature fusion on the media feature data and the intima feature data by the cross-attention mechanism to obtain the media split result image; and the intima splitter performing feature fusion on the media feature data and the intima feature data by the cross-attention mechanism to obtain the intima split result image.

[0194] By using the above technical solution, the blood vessel image segmentation device 2 of the present application can make the media feature and the intima feature interact by using the cross-attention mechanism, so that the deep learning segmentation model can more accurately simultaneously segment the media split result image and the intima split result image of the blood vessel segment of interest, and improve the segmentation accuracy of the media and intima.

[0195] Further, the feature acquisition module 21 can also be used to perform the aforementioned steps S011, S012, S0111 and S0112. The image segmentation module 22 can also be used to perform the aforementioned step S03.

[0196] In a fifth aspect, referring to FIG. 11, the embodiments of the present application further disclose an electronic device 3 comprising a memory 31 and a processor 32, the memory 31 storing at least one instruction, the at least one instruction being executed by the processor 32 to implement the training method of the deep learning segmentation model in any one of the preceding embodiments, and / or the blood vessel image segmentation method in any one of the preceding embodiments. The memory 31 may, for example, comprise a system memory, a fixed non-volatile storage medium, etc. The system memory may, for example, store an operating system, an application program, a Boot Loader, and other programs, etc.

[0197] In the present embodiment, the electronic device 3 can combine the cross-attention mechanism, interact the media feature data and the intima feature data, and make the trained deep learning segmentation model more accurately simultaneously segment the media segmentation result image and the intima segmentation result image of the blood vessel segment of interest, thereby improving the segmentation accuracy of the media and the intima.

[0198] In a sixth aspect, the embodiments of the present application further disclose a computer readable storage medium, the computer readable storage medium storing at least one instruction, the at least one instruction being executed to implement the training method of the deep learning segmentation model in any one of the preceding embodiments, and / or the blood vessel image segmentation method in any one of the preceding embodiments.

[0199] With the above technical solution, the computer readable storage medium can combine the cross-attention mechanism, interact the media feature data and the intima feature data, and make the trained deep learning segmentation model more accurately segment the media segmentation result image and the intima segmentation result image of the blood vessel segment of interest, thereby improving the segmentation accuracy of the media and the intima.

[0200] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0201] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks or in conjunction with the flowchart blocks.

[0202] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks or in conjunction with the flowchart blocks.

[0203] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks or in conjunction with the flowchart blocks.

[0204] Although the present application has been described in terms of certain preferred embodiments, the skilled artisan will appreciate that various modifications, alterations, permutations, and substitutions can be made without departing from the spirit and scope of the application. Accordingly, the application is not limited by the foregoing description, but is only limited by the scope of the patent.

Claims

1. A method for training a deep learning segmentation model, characterized in that, The deep learning segmentation model comprises a media layer segmenter and an intima layer segmenter, the media layer segmenter is used for segmenting a media layer segmentation result image, the intima layer segmenter is used for segmenting an intima layer segmentation result image, and the training method comprises the following steps: obtaining a three-dimensional blood vessel image containing a blood vessel segment of interest; obtaining a media layer segmentation result label and an intima layer segmentation result label of the blood vessel segment of interest; processing the three-dimensional blood vessel image containing the blood vessel segment of interest by the deep learning segmentation model to be trained to obtain the media layer segmentation result image and the intima layer segmentation result image; determining a loss function according to the media layer segmentation result label, the intima layer segmentation result label, the media layer segmentation result image and the intima layer segmentation result image of the blood vessel segment of interest; updating parameters of the deep learning segmentation model to be trained according to the three-dimensional blood vessel image containing the blood vessel segment of interest, the media layer segmentation result image, the intima layer segmentation result image, the media layer segmentation result label, the intima layer segmentation result label and the loss function until the deep learning segmentation model to be trained meets a preset training condition, thereby obtaining the trained deep learning segmentation model; wherein the media layer segmenter and the intima layer segmenter adopt a cross-attention mechanism to segment the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain the media layer segmentation result image and the intima layer segmentation result image. 2.The method of claim 1, wherein, In the process of segmenting the three-dimensional blood vessel image containing the blood vessel segment of interest by using the cross-attention mechanism, the following steps are specifically included: extracting media layer features of the three-dimensional blood vessel image containing the blood vessel segment of interest by the media layer segmenter to obtain media layer feature data; extracting intima layer features of the three-dimensional blood vessel image containing the blood vessel segment of interest by the intima layer segmenter to obtain intima layer feature data; the media layer segmenter performs feature fusion on the media layer feature data and the intima layer feature data by the cross-attention mechanism to obtain the media layer segmentation result image; the intima layer segmenter performs feature fusion on the media layer feature data and the intima layer feature data by the cross-attention mechanism to obtain the intima layer segmentation result image. 3.The method of claim 1, wherein, The loss function comprises a cross-entropy loss function, a Dice loss function and a consistency loss function. 4.The method of claim 3, wherein, The consistency loss function is obtained by the following method: obtaining media layer skeletonization result data of the media layer segmentation result image by a soft skeleton extraction method according to the media layer segmentation result image, and obtaining intima layer skeletonization result data of the intima layer segmentation result image by the soft skeleton extraction method according to the intima layer segmentation result image; determining the consistency loss function according to the media layer skeletonization result data and the intima layer skeletonization result data. 5.The method of claim 3, wherein, The loss function is specifically: Total Loss = Dice Lumen + Dice Wall + CE Lumen + CE Wall + Consistency lumen-wall wherein Total Loss denotes the loss function, Dice Lumen + Dice Wall denotes the Dice loss function, CE Lumen + CE Wall denotes the cross-entropy loss function, Consistency lumen-wall denotes the consistency loss function, G L denotes the lumen segmentation result label, P L denotes the lumen segmentation result image, S L denotes the lumen skeletonization result data, G W denotes the media segmentation result label, P W denotes the media segmentation result image, S W denotes the media skeletonization result data. 6.The method of claim 4, wherein, The consistency loss function is specifically: S l = SoftSkeleton(P l ) S w = SoftSkeleton(P w ) wherein P I denotes the intima segmentation result image, S I denotes the intima skeletonization result data, P W denotes the media segmentation result image, S W denotes the media skeletonization result data, SoftSkeleton denotes the soft skeleton extraction method, Consistencylumen-wall denotes the consistency loss function. 7.The method of claim 1, wherein, The three-dimensional blood vessel image containing the blood vessel segment of interest is obtained by the following steps: obtaining an untreated three-dimensional blood vessel image of the blood vessel segment of interest; performing normalization processing and data enhancement processing on the untreated three-dimensional blood vessel image to obtain the three-dimensional blood vessel image. 8.The method of claim 1, wherein, The deep learning segmentation model further comprises a feature extractor, and the loss function is further used to train the feature extractor, and the processing of the three-dimensional blood vessel image containing the blood vessel segment of interest by the deep learning segmentation model to be trained specifically comprises: adopting an attention mechanism to extract blood vessel features of the three-dimensional blood vessel image containing the blood vessel segment of interest by the feature extractor to obtain three-dimensional blood vessel image feature data of the blood vessel segment of interest; wherein the attention mechanism comprises a spatial attention mechanism and / or a channel attention mechanism; the spatial attention mechanism is used to extract blood vessel spatial feature data of the blood vessel segment of interest in the three-dimensional blood vessel image containing the blood vessel segment of interest; the channel attention mechanism is used to extract blood vessel channel feature data of the blood vessel segment of interest in the three-dimensional blood vessel image containing the blood vessel segment of interest. 9.A device for training a deep learning segmentation model, characterized in that, The deep learning segmentation model comprises a media segmentation device and an intima segmentation device, the media segmentation device is used to segment a media segmentation result image, the intima segmentation device is used to segment an intima segmentation result image, and the training device comprises: an image acquisition module, configured to acquire a three-dimensional blood vessel image containing a blood vessel segment of interest, and acquire a labeled media segmentation result label and an intima segmentation result label of the blood vessel segment of interest; a pre-segmentation module, configured to process the three-dimensional blood vessel image containing the blood vessel segment of interest by the deep learning segmentation model to be trained to obtain the media segmentation result image and the intima segmentation result image; a loss function determination module, configured to determine a loss function according to the media segmentation result label, the intima segmentation result label, the media segmentation result image and the intima segmentation result image of the blood vessel segment of interest; a training module, configured to perform parameter update on the deep learning segmentation model to be trained according to the three-dimensional blood vessel image containing the blood vessel segment of interest, the media segmentation result image, the intima segmentation result image, the media segmentation result label, the intima segmentation result label and the loss function, until the deep learning segmentation model to be trained meets a preset training condition, to obtain the trained deep learning segmentation model; wherein the media segmentation device and the intima segmentation device adopt a cross-attention mechanism to segment the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain the media segmentation result image and the intima segmentation result image. 10.The device of claim 9, wherein In the process of processing the three-dimensional blood vessel image containing the blood vessel segment of interest by the deep learning segmentation model to be trained by the pre-segmentation module, the media segmentation device and the intima segmentation device adopt the cross-attention mechanism to segment the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain the media segmentation result image and the intima segmentation result image; in the process of segmenting the three-dimensional blood vessel image containing the blood vessel segment of interest by the cross-attention mechanism, specifically comprising: extracting, by the middle membrane segmenter, middle membrane features of the three-dimensional blood vessel image containing the blood vessel segment of interest, to obtain middle membrane feature data; extracting, by the inner membrane segmenter, inner membrane features of the three-dimensional blood vessel image containing the blood vessel segment of interest, to obtain inner membrane feature data; the middle membrane segmenter performs feature fusion on the middle membrane feature data and the inner membrane feature data through the cross-attention mechanism, to obtain the middle membrane segmentation result image; the inner membrane segmenter performs feature fusion on the middle membrane feature data and the inner membrane feature data through the cross-attention mechanism, to obtain the inner membrane segmentation result image.

11. A blood vessel image segmentation method characterized by comprising: The deep learning segmentation model includes a middle membrane segmenter and an inner membrane segmenter, the middle membrane segmenter is used for segmenting a middle membrane segmentation result image, and the inner membrane segmenter is used for segmenting an inner membrane segmentation result image. obtaining three-dimensional blood vessel image feature data of a blood vessel segment of interest; segmenting, by the deep learning segmentation model, the three-dimensional blood vessel image feature data of the blood vessel segment of interest through a cross-attention mechanism, to obtain the middle membrane segmentation result image and the inner membrane segmentation result image of the blood vessel segment of interest.

12. The blood vessel image segmentation method of claim 11, wherein, In the process of segmenting the three-dimensional blood vessel image feature data through the cross-attention mechanism, the process specifically includes: extracting, by the middle membrane segmenter, middle membrane features of the three-dimensional blood vessel image feature data, to obtain middle membrane feature data; extracting, by the inner membrane segmenter, inner membrane features of the three-dimensional blood vessel image feature data, to obtain inner membrane feature data; the middle membrane segmenter performs feature fusion on the middle membrane feature data and the inner membrane feature data through the cross-attention mechanism, to obtain the middle membrane segmentation result image; the inner membrane segmenter performs feature fusion on the middle membrane feature data and the inner membrane feature data through the cross-attention mechanism, to obtain the inner membrane segmentation result image.

13. The blood vessel image segmentation method of claim 11, wherein, The deep learning segmentation model further includes a feature extractor, and the obtaining of the three-dimensional blood vessel image feature data of the blood vessel segment of interest specifically includes: obtaining a three-dimensional blood vessel image containing the blood vessel segment of interest; obtaining the three-dimensional blood vessel image feature data of the blood vessel segment of interest from the three-dimensional blood vessel image containing the blood vessel segment of interest, including: extracting blood vessel features of the blood vessel segment of interest from the three-dimensional blood vessel image of the blood vessel segment of interest through the feature extractor by using an attention mechanism, to obtain the three-dimensional blood vessel image feature data of the blood vessel segment of interest; wherein the attention mechanism includes a spatial attention mechanism and / or a channel attention mechanism; the spatial attention mechanism is used for extracting blood vessel spatial feature data of the blood vessel segment of interest in the three-dimensional blood vessel image; the channel attention mechanism is used for extracting blood vessel channel feature data of the blood vessel segment of interest in the three-dimensional blood vessel image.

14. The blood vessel image segmentation method of claim 11, wherein, The obtaining of the three-dimensional blood vessel image containing the blood vessel segment of interest specifically includes: obtaining an untreated three-dimensional blood vessel image of the blood vessel segment of interest; The three-dimensional blood vessel image is normalized to obtain the three-dimensional blood vessel image of the blood vessel segment of interest.

15. The blood vessel image segmentation method of claim 11, wherein, Further comprising: An image enhancement optimization operation is performed on the media layer segmentation result image and the intima layer segmentation result image to obtain an enhanced final media layer segmentation result image and a final intima layer segmentation result image. The image enhancement optimization operation includes a morphological operation and / or a connectivity analysis, the morphological operation being selected from one or more of the following: an erosion operation, a dilation operation, an opening operation, a closing operation, a morphological gradient, a top hat, and a black hat; and the connectivity analysis being selected from one or more of the following: connected component labeling, region growing, boundary tracing, hole filling, skeleton extraction, and region segmentation.

16. A blood vessel image segmentation apparatus characterized by comprising: The blood vessel image is segmented by a deep learning segmentation model, the deep learning segmentation model including a media layer segmenter and an intima layer segmenter, the media layer segmenter being configured to segment a media layer segmentation result image, and the intima layer segmenter being configured to segment an intima layer segmentation result image, the blood vessel image segmentation device comprising: a feature acquisition module configured to acquire three-dimensional blood vessel image feature data of a blood vessel segment of interest; an image segmentation module configured to segment the three-dimensional blood vessel image feature data of the blood vessel segment of interest by the deep learning segmentation model using a cross-attention mechanism to obtain the media layer segmentation result image and the intima layer segmentation result image of the blood vessel segment of interest.

17. The blood vessel image segmentation apparatus of claim 16, wherein In the process of segmenting the three-dimensional blood vessel image feature data by the image segmentation module using the cross-attention mechanism, specifically comprising: extracting media layer features of the three-dimensional blood vessel image feature data by the media layer segmenter to obtain media layer feature data; extracting intima layer features of the three-dimensional blood vessel image feature data by the intima layer segmenter to obtain intima layer feature data; the media layer segmenter performing feature fusion on the media layer feature data and the intima layer feature data by the cross-attention mechanism to obtain the media layer segmentation result image; the intima layer segmenter performing feature fusion on the media layer feature data and the intima layer feature data by the cross-attention mechanism to obtain the intima layer segmentation result image.

18. An electronic device, comprising: The electronic device includes a processor and a memory, the memory storing at least one instruction, the at least one instruction being executed by the processor to implement the training method of the deep learning segmentation model according to any one of claims 1-8, and / or the blood vessel image segmentation method according to any one of claims 11-15.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, the at least one instruction being executed to implement the training method of the deep learning segmentation model according to any one of claims 1-8, and / or the blood vessel image segmentation method according to any one of claims 11-15.

Images