Training methods for deep learning segmentation models and methods and devices for blood vessel image segmentation

Abstract

This invention discloses a training method for a deep learning segmentation model, comprising: acquiring a three-dimensional blood vessel image containing a segment of interest; processing the three-dimensional blood vessel image using a deep learning segmentation model to be trained to obtain media segmentation result images and intima segmentation result images; determining a loss function based on the media segmentation result labels, intima segmentation result labels, the media segmentation result images, and the intima segmentation result images; updating the parameters of the deep learning segmentation model to be trained to obtain a trained deep learning segmentation model; wherein the media segmenter and intima segmenter of the deep learning segmentation model employ a cross-attention mechanism to segment the three-dimensional blood vessel image, obtaining the media segmentation result images and intima segmentation result images. This invention can accurately segment the intima and media of blood vessels using the above method. This invention also discloses a training device for the deep learning segmentation model, a blood vessel image segmentation method and device, an electronic device, and a readable storage medium.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a training method for a deep learning segmentation model, a training device for a deep learning segmentation model, a blood vessel image segmentation method, a blood vessel image segmentation device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Coronary computed tomography (CTA) is an important technique for detecting cardiovascular diseases. Currently, there are many methods that can simultaneously and automatically perform vessel segmentation and centerline extraction to further calculate quantitative parameters to assist doctors in diagnosis.

[0003] In existing technologies, coronary CTA image processing techniques typically focus only on single-layer intimal segmentation, aiming to segment the vascular intima and lumen, and then directly observe the vascular condition based on the segmentation results to make a corresponding diagnosis. However, in areas with high plaque accumulation or severe stenosis, the intimal structure is complex and highly variable. The accuracy of this single-layer intimal segmentation method, which only segments the intima and lumen, is significantly reduced. It ignores the potential value of the vascular media, does not consider the relationship between the media and intima during segmentation, and the segmentation results are not accurate enough, leading to inaccurate extraction of the vascular centerline, which can adversely affect subsequent clinical diagnosis and treatment decisions. Summary of the Invention

[0004] This invention proposes a training method for a deep learning segmentation model to solve the above-mentioned problems.

[0005] In a first aspect, embodiments of the present invention disclose a training method for a deep learning segmentation model. The deep learning segmentation model includes a middle membrane segmenter and an inner membrane segmenter. The middle membrane segmenter is used to segment the middle membrane segmentation result image, and the inner membrane segmenter is used to segment the inner membrane segmentation result image. The training method includes:

[0006] Obtain a 3D vascular image containing the segment of interest; obtain the media segmentation result labels and intima segmentation result labels of the labeled segment of interest;

[0007] The deep learning segmentation model to be trained is used to process the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain the media segmentation result image and the intima segmentation result image.

[0008] The loss function is determined based on the media segmentation result labels, endothelial segmentation result labels, media segmentation result images, and endothelial segmentation result images of the vessel segment of interest.

[0009] The parameters of the deep learning segmentation model to be trained are updated based on the three-dimensional blood vessel image containing the segment of interest, the media segmentation result image, the intima segmentation result image, the media segmentation result label, the intima segmentation result label, and the loss function, until the deep learning segmentation model to be trained meets the preset training conditions, and the trained deep learning segmentation model is obtained.

[0010] The media segmenter and the endothelial segmenter use a cross-attention mechanism to segment the three-dimensional vascular image containing the segment of interest, resulting in media segmentation image and endothelial segmentation image.

[0011] Using the above technical solution, the training method of the deep learning segmentation model of the present invention combines a cross-attention mechanism. Through the cross-attention mechanism, the features of the media and the intima can interact, so that the trained deep learning segmentation model can more accurately segment the media and intima segments of the vessel segment of interest at the same time, thereby improving the segmentation accuracy of the media and intima.

[0012] According to another specific embodiment of the present invention, the process of segmenting a three-dimensional vascular image containing a vascular segment of interest using a cross-attention mechanism specifically includes:

[0013] The medial features of a 3D vascular image containing the segment of interest are extracted using a media segmenter to obtain medial feature data.

[0014] Intima feature data is obtained by extracting intima features from a three-dimensional vascular image containing the segment of interest using an intima segmenter.

[0015] The middle membrane segmenter uses a cross-attention mechanism to fuse the features of the middle membrane and the inner membrane to obtain the segmented image of the middle membrane.

[0016] The endometrial segmenter uses a cross-attention mechanism to fuse the features of the middle lining and the endometrial features to obtain the endometrial segmentation result image.

[0017] According to another specific embodiment of the present invention, the loss function includes the cross-entropy loss function, the Dice loss function, and the consistency loss function.

[0018] According to another specific embodiment of the present invention, the consistency loss function is obtained in the following manner:

[0019] Based on the segmentation result image of the middle tunicae, the skeletonization result data of the middle tunicae segmentation result image is obtained by the soft skeleton extraction method; based on the segmentation result image of the inner tunicae, the skeletonization result data of the inner tunicae segmentation result image is obtained by the soft skeleton extraction method.

[0020] The consistency loss function was determined based on the results of the middle membrane skeletonization and the endometrial skeletonization.

[0021] According to another specific embodiment of the present invention, the loss function is specifically as follows:

[0022] Total Loss = Dice Lumen +Dice Wall +CE Lumen +CE Wall +Consistency lumen-wall

[0023]

[0024] Where Total Loss represents the loss function, and Dice Lumen +Dice Wall Denotes the Dice loss function, CE Lumen +CE Wall Represents the cross-entropy loss function, Consistency lumen-wall Let G represent the consistency loss function. L Indicates the label for endometrial segmentation results, P L S represents the image of the endometrial segmentation result. L G represents the data of endometrial skeletalization results. W P represents the label for the middle membrane segmentation result. W S represents the segmentation result image of the middle membrane. W This represents the data on the results of the medial membrane skeletonization.

[0025] According to another specific embodiment of the present invention, the consistency loss function is specifically as follows:

[0026] S l =SoftSkeleton(P l )

[0027] S w =SoftSkeleton(P w )

[0028]

[0029] Among them, P I S represents the image of the endometrial segmentation result. I P represents the data on endometrial skeletalization results. W S represents the segmentation result image of the middle membrane. W This represents the results of the middle membrane skeletonization; SoftSkeleton represents the soft skeleton extraction method; Consistency lumen-wall This represents the consistency loss function.

[0030] According to another specific embodiment of the present invention, acquiring a three-dimensional vascular image containing a vascular segment of interest specifically includes:

[0031] Acquire unprocessed 3D vascular images of the segment of interest;

[0032] The unprocessed 3D vascular images were normalized and data augmented to obtain 3D vascular images.

[0033] According to another specific embodiment of the present invention, the deep learning segmentation model further includes a feature extractor, and the loss function is further used to train the feature extractor. The deep learning segmentation model to be trained processes a three-dimensional vascular image containing a vascular segment of interest, specifically including:

[0034] An attention mechanism is used to extract vascular features from a 3D vascular image containing the segment of interest using a feature extractor, thereby obtaining the 3D vascular image feature data of the segment of interest.

[0035] Among them, attention mechanisms include spatial attention mechanisms and / or channel attention mechanisms;

[0036] Spatial attention mechanism is used to extract spatial feature data of the vessel segment of interest in a 3D vascular image containing the vessel segment of interest;

[0037] The channel attention mechanism is used to extract channel feature data of the vessel segment of interest in a 3D vascular image containing the vessel segment of interest.

[0038] Secondly, embodiments of the present invention disclose a training apparatus for a deep learning segmentation model, comprising: the deep learning segmentation model including a middle membrane segmenter and an inner membrane segmenter, the middle membrane segmenter being used to segment a middle membrane segmentation result image, and the inner membrane segmenter being used to segment an inner membrane segmentation result image; the training apparatus comprising:

[0039] The image acquisition module is used to acquire a three-dimensional vascular image containing the vascular segment of interest, as well as to acquire the labeled media segmentation result labels and intima segmentation result labels of the labeled vascular segment of interest.

[0040] The pre-segmentation module is used to process the three-dimensional blood vessel image containing the blood vessel segment of interest through the deep learning segmentation model to be trained, and to obtain the media segmentation result image and the intima segmentation result image.

[0041] The loss function determination module is used to determine the loss function based on the media segmentation result label, the endothelial segmentation result label, the media segmentation result image, and the endothelial segmentation result image of the blood vessel segment of interest.

[0042] The training module is used to update the parameters of the deep learning segmentation model to be trained based on the three-dimensional blood vessel image containing the blood vessel segment of interest, the media segmentation result image, the intima segmentation result image, the media segmentation result label, the intima segmentation result label, and the loss function, until the deep learning segmentation model to be trained meets the preset training conditions, and the trained deep learning segmentation model is obtained.

[0043] The media segmenter and the endothelial segmenter use a cross-attention mechanism to segment the three-dimensional vascular image containing the segment of interest, resulting in media segmentation image and endothelial segmentation image.

[0044] Using the above technical solution, the training device of the deep learning segmentation model of the present invention combines a cross-attention mechanism, which enables the media and endothelial features to interact, so that the trained deep learning segmentation model can more accurately segment the media and endothelial segments of the vessel segment of interest at the same time, thereby improving the segmentation accuracy of the media and endothelial segments.

[0045] According to another specific embodiment of the present invention, during the process of the pre-segmentation module processing the three-dimensional blood vessel image containing the blood vessel segment of interest through the deep learning segmentation model to be trained, the three-dimensional blood vessel image containing the blood vessel segment of interest is segmented by the media segmenter and the endothelial segmenter using a cross-attention mechanism to obtain the media segmentation result image and the endothelial segmentation result image.

[0046] The process of segmenting a 3D vascular image containing a segment of interest using a cross-attention mechanism specifically includes:

[0047] The medial features of a 3D vascular image containing the segment of interest are extracted using a media segmenter to obtain medial feature data.

[0048] Intima feature data is obtained by extracting intima features from a three-dimensional vascular image containing the segment of interest using an intima segmenter.

[0049] The middle membrane segmenter uses a cross-attention mechanism to fuse the features of the middle membrane and the inner membrane to obtain the segmented image of the middle membrane.

[0050] The endometrial segmenter uses a cross-attention mechanism to fuse the features of the middle lining and the endometrial features to obtain the endometrial segmentation result image.

[0051] Thirdly, embodiments of the present invention disclose a blood vessel image segmentation method, which segments blood vessel images using a deep learning segmentation model. The deep learning segmentation model includes a media segmenter and an intima segmenter. The media segmenter is used to segment the media segmentation result image, and the intima segmenter is used to segment the intima segmentation result image. The blood vessel image segmentation method includes:

[0052] Obtain 3D vascular image feature data of the vascular segment of interest;

[0053] The 3D vascular image feature data of the vascular segment of interest is segmented by using a deep learning segmentation model with a cross-attention mechanism, resulting in segmentation images of the media and intima of the vascular segment of interest.

[0054] By employing the above technical solution, the vascular image segmentation method of the present invention, through the use of a cross-attention mechanism, fully utilizes the interaction between the media and the endothelium, thereby improving the segmentation accuracy of the media and the endothelium.

[0055] According to another specific embodiment of the present invention, the process of segmenting three-dimensional vascular image feature data using a cross-attention mechanism specifically includes:

[0056] The medial features are extracted from the three-dimensional vascular image feature data using a medial segmenter to obtain medial feature data;

[0057] Intima features are extracted from the three-dimensional vascular image feature data using an intima segmenter to obtain intima feature data;

[0058] The middle membrane segmenter uses a cross-attention mechanism to fuse the features of the middle membrane and the inner membrane to obtain the segmented image of the middle membrane.

[0059] The endometrial segmenter uses a cross-attention mechanism to fuse the features of the middle lining and the endometrial features to obtain the endometrial segmentation result image.

[0060] According to another specific embodiment of the present invention, the deep learning segmentation model further includes a feature extractor to obtain three-dimensional vascular image feature data of the vascular segment of interest, specifically including:

[0061] Obtain a three-dimensional vascular image containing the vascular segment of interest;

[0062] The method involves obtaining 3D vascular image feature data of a segment of interest from a 3D vascular image containing the segment of interest, including: using an attention mechanism to extract vascular features from the 3D vascular image of the segment of interest through a feature extractor, thereby obtaining 3D vascular image feature data of the segment of interest.

[0063] Among them, attention mechanisms include spatial attention mechanisms and / or channel attention mechanisms;

[0064] Spatial attention mechanisms are used to extract spatial feature data of blood vessels of interest in three-dimensional vascular images;

[0065] Channel attention mechanism is used to extract channel feature data of blood vessel segments of interest in 3D blood vessel images.

[0066] According to another specific embodiment of the present invention, acquiring a three-dimensional vascular image containing a vascular segment of interest specifically includes:

[0067] Acquire unprocessed 3D vascular images of the segment of interest;

[0068] The unprocessed 3D vascular image is normalized to obtain a 3D vascular image of the segment of interest.

[0069] According to another specific embodiment of the present invention, image enhancement and optimization operations are performed on the middle membrane segmentation result image and the end membrane segmentation result image to obtain the enhanced final middle membrane segmentation result image and the final end membrane segmentation result image;

[0070] The image enhancement and optimization operations include morphological operations and / or connectivity analysis. The morphological operations are selected from one or more of the following: erosion, dilation, opening, closing, morphological gradient, top hat and black hat. The connectivity analysis is selected from one or more of the following: connected component labeling, region growing, boundary tracing, hole filling, skeleton extraction and region segmentation.

[0071] Fourthly, embodiments of the present invention disclose a blood vessel image segmentation device that segments blood vessel images using a deep learning segmentation model. The deep learning segmentation model includes a media segmenter and an intima segmenter. The media segmenter is used to segment the media segmentation result image, and the intima segmenter is used to segment the intima segmentation result image. The blood vessel image segmentation device includes:

[0072] The feature acquisition module is used to acquire 3D vascular image feature data of the vascular segment of interest;

[0073] The image segmentation module is used to segment the 3D vascular image feature data of the vascular segment of interest using a deep learning segmentation model with a cross-attention mechanism, so as to obtain the media segmentation result image and the intima segmentation result image of the vascular segment of interest.

[0074] By employing the above technical solution, the vascular image segmentation device of the present invention can improve the segmentation accuracy of the media and endothelium by making full use of the interaction between the media and endothelium through the use of a cross-attention mechanism.

[0075] According to another specific embodiment of the present invention, the image segmentation module employs a cross-attention mechanism to segment the feature data of a three-dimensional blood vessel image, specifically including:

[0076] The medial features are extracted from the three-dimensional vascular image feature data using a medial segmenter to obtain medial feature data;

[0077] Intima features are extracted from the three-dimensional vascular image feature data using an intima segmenter to obtain intima feature data;

[0078] The middle membrane segmenter uses a cross-attention mechanism to fuse the features of the middle membrane and the inner membrane to obtain the segmented image of the middle membrane.

[0079] The endometrial segmenter uses a cross-attention mechanism to fuse the features of the middle lining and the endometrial features to obtain the endometrial segmentation result image.

[0080] Fifthly, embodiments of the present invention disclose an electronic device, including a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction, when executed by the processor, implements the training method of the deep learning segmentation model in any of the foregoing embodiments, and / or the blood vessel image segmentation method in any of the foregoing embodiments.

[0081] By adopting the above technical solution, the electronic device can combine the cross-attention mechanism and, through the interaction of medial feature data and endothelial feature data, enable the trained deep learning segmentation model to more accurately segment the medial segmentation result image and the endothelial segmentation result image of the blood vessel segment of interest at the same time, thereby improving the segmentation accuracy of the medial and endothelial.

[0082] In a sixth aspect, embodiments of the present invention disclose a computer-readable storage medium storing at least one instruction, which, when executed, implements the training method of the deep learning segmentation model in any of the foregoing embodiments, and / or the blood vessel image segmentation method in any of the foregoing embodiments.

[0083] By adopting the above technical solution, the computer-readable storage medium can combine the cross-attention mechanism and, through the interaction of medial feature data and endothelial feature data, enable the trained deep learning segmentation model to more accurately segment the medial segmentation result image and the endothelial segmentation result image of the blood vessel segment of interest at the same time, thereby improving the segmentation accuracy of the medial and endothelial.

[0084] It should be noted that the deep learning segmentation model training method and vascular image segmentation method of the present invention do not involve surgical procedures, but are based on the processing of medical images; that is, their direct object is medical images, such as three-dimensional coronary CTA images. The deep learning segmentation model training method and vascular image segmentation method of the present invention have non-therapeutic purposes.

[0085] The training device, vascular image segmentation device, electronic device, and computer-readable storage medium of the deep learning segmentation model of this invention are not devices related to surgical procedures, but rather devices related to the processing of medical images; that is, their direct object is medical images, such as three-dimensional coronary CTA images. The training device, vascular image segmentation device, electronic device, and computer-readable storage medium of the deep learning segmentation model of this invention are all for non-therapeutic purposes.

[0086] The beneficial effects of this invention are as follows:

[0087] 1. The training method and device for the deep learning segmentation model provided by this invention introduce a cross-attention mechanism into the deep learning segmentation model. Through the interaction between the medial feature data and the endothelial feature data, the medial and endothelial segments are mutually promoted. The trained deep learning segmentation model can simultaneously segment the medial and endothelial segments of the vessel segment of interest, thereby improving the accuracy of medial and endothelial segmentation.

[0088] In the training process of the deep learning segmentation model, this invention also designs a consistency loss function to establish the relationship between the middle membrane and the inner membrane, so as to ensure the consistency between the subsequent segmentation results of the inner membrane and the middle membrane.

[0089] Furthermore, this invention normalizes and augments the unprocessed 3D blood vessel images containing the blood vessel segments of interest used in the training of the deep learning segmentation model. This can increase the amount of data available during training, improve data diversity, and enable the trained deep learning segmentation model to have better generalization ability.

[0090] 2. The vascular image segmentation method and vascular image segmentation device provided by the present invention perform feature fusion on media feature data and endothelial feature data through a cross-attention mechanism, enabling the media feature data and endothelial feature data to interact and mutually promote media and endothelial segmentation, thereby improving the segmentation accuracy of the media and endothelial membranes.

[0091] This invention introduces spatial attention and / or channel attention mechanisms when extracting 3D vascular image feature data of the vascular segment of interest. This can more accurately locate the spatial position of the vascular segment of interest in the image, increase the weight of the feature channels corresponding to important vascular features, highlight the vascular segment of interest, improve the deep learning segmentation model's ability to represent the key features of the target blood vessel, and help improve the accuracy of media and intima segmentation.

[0092] In addition, after obtaining the segmentation results images of the intima and the media, the present invention also performs image enhancement and optimization operations. By using morphological operations and / or connectivity analysis, it can fill in any voids that may exist inside the segmented blood vessels and remove noise generated during segmentation, thereby further improving the accuracy and robustness of the segmentation of the media and intima. Attached Figure Description

[0093] Figure 1 This diagram illustrates the training method of the deep learning segmentation model in an embodiment of the present invention. Figure 1 ;

[0094] Figure 2 This diagram illustrates the training method of the deep learning segmentation model in an embodiment of the present invention. Figure 2 ;

[0095] Figure 3 This diagram illustrates the training method of the deep learning segmentation model in an embodiment of the present invention. Figure 3 ;

[0096] Figure 4 This diagram illustrates the training method of the deep learning segmentation model in an embodiment of the present invention. Figure 4 ;

[0097] Figure 5 This diagram illustrates the flow chart of the blood vessel image segmentation method in an embodiment of the present invention. Figure 1 ;

[0098] Figure 6 This diagram illustrates the flow chart of the blood vessel image segmentation method in an embodiment of the present invention. Figure 2 ;

[0099] Figure 7 This diagram illustrates the flow chart of the blood vessel image segmentation method in an embodiment of the present invention. Figure 3 ;

[0100] Figure 8 This diagram illustrates the flow chart of the blood vessel image segmentation method in an embodiment of the present invention. Figure 4 ;

[0101] Figure 9 A schematic diagram of the structure of a training device for a deep learning segmentation model in an embodiment of the present invention is shown;

[0102] Figure 10 This diagram illustrates the structure of the blood vessel image segmentation device in an embodiment of the present invention.

[0103] Figure 11 A schematic diagram of the structure of the electronic device in an embodiment of the present invention is shown;

[0104] Figure 12 This diagram illustrates the structure of a deep learning segmentation model in an embodiment of the present invention.

[0105] Figure 13 This shows the original three-dimensional coronary CTA cross-sectional image in an embodiment of the present invention;

[0106] Figure 14 This illustrates a three-dimensional coronary CTA cross-sectional image obtained after normalization processing of the original three-dimensional coronary CTA cross-sectional image in an embodiment of the present invention;

[0107] Figure 15 This illustrates an image obtained after enhancing normalized three-dimensional coronary CTA cross-sectional image data in an embodiment of the present invention;

[0108] Figure 16 This shows the result image of the middle membrane segmentation result image and the intima segmentation result image superimposed on the original three-dimensional blood vessel image in an embodiment of the present invention;

[0109] Figure 17 A schematic diagram showing the three-dimensional visualization results of the segmented middle membrane and segmented inner membrane in an embodiment of the present invention;

[0110] Figure 18 This is a plan view taken along the center line of the blood vessel in an embodiment of the present invention. Detailed Implementation

[0111] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention is presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a deep understanding of the invention, many specific details will be included in the following description. The invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0112] It should be noted that in this specification, similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0113] The terms “first”, “second”, etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0114] In the description of this embodiment, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set up," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this embodiment based on the specific circumstances.

[0115] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0116] Through careful study of existing technologies, the inventors discovered that current techniques only focus on segmenting the vascular intima and lumen, neglecting the segmentation between the vascular intima and media. This insufficient segmentation hinders accurate extraction of the centerline, impacting subsequent clinical diagnosis of cardiovascular diseases. Furthermore, because the media and intima are structurally closely connected, their boundaries are difficult to distinguish in images. Different regions of the intima and media may also exhibit varying thicknesses, and vascular morphology can further complicate their differentiation. All these factors increase the complexity of image segmentation, making it difficult for existing vascular image segmentation algorithms to accurately segment the vascular intima and media. The inventors aim to address these issues.

[0117] In a first aspect, embodiments of the present invention disclose a training method for a deep learning segmentation model. The deep learning segmentation model includes a middle membrane segmenter and an inner membrane segmenter. The middle membrane segmenter is used to segment the middle membrane segmentation result image, and the inner membrane segmenter is used to segment the inner membrane segmentation result image.

[0118] refer to Figure 1 and combined Figure 12 The training method for deep learning segmentation models specifically includes the following steps:

[0119] S1: Obtain a 3D vascular image containing the segment of interest, and obtain the labeled media segmentation result labels and intima segmentation result labels of the labeled segment of interest.

[0120] Specifically, the three-dimensional vascular image is a three-dimensional coronary CTA image, which can clearly show the condition of the heart's blood vessels. The labels for the media segmentation results of the segment of interest are, for example, pre-annotated manually labeled media segmentation results, and the labels for the intima segmentation results are, for example, pre-annotated manually labeled intima segmentation results.

[0121] S2: The three-dimensional vascular image containing the vascular segment of interest is processed by the deep learning segmentation model to be trained to obtain the media segmentation result image and the intima segmentation result image.

[0122] The media segmenter and the endothelial segmenter use a cross-attention mechanism to segment the three-dimensional vascular image containing the segment of interest, resulting in media segmentation image and endothelial segmentation image.

[0123] More specifically, in the process of segmenting a 3D vascular image containing the segment of interest using a cross-attention mechanism, the media segmenter extracts the media features of the 3D vascular image containing the segment of interest to obtain media feature data; the endothelial segmenter extracts the endothelial features of the 3D vascular image containing the segment of interest to obtain endothelial feature data; the media segmenter performs feature fusion of the media feature data and endothelial feature data through a cross-attention mechanism to obtain the media segmentation result image; the endothelial segmenter performs feature fusion of the media feature data and endothelial feature data through a cross-attention mechanism to obtain the endothelial segmentation result image.

[0124] Cross-attention is a mechanism used in deep learning neural networks to handle sequence-to-sequence (Seq2Seq) tasks, particularly in the interaction between the encoder and decoder. Its core idea is that cross-attention can be used in both the encoder and decoder to interact, fuse, and update features from different sources (specifically, the middle membrane and inner membrane features in this invention).

[0125] In this invention, the application of cross-attention is further improved. Specifically, both the media segmenter and the endothelial segmenter are decoders. This invention segments 3D vascular images through a cross-attention mechanism. This mechanism enables the media and endothelial segmenters to mutually learn and establish a dependency relationship between the endothelial and media when segmenting the media and endothelial tissues, respectively. This allows the media segmenter to learn and decode media features by incorporating endothelial features from endothelial segmentation, and vice versa. Consequently, the media and endothelial segmenters can simultaneously obtain media and endothelial segmentation results, respectively. This approach fully utilizes the interrelationship between the media and endothelial tissues, achieving efficient feature interaction between them. This improves the expressive power of the deep learning segmentation model and enhances the segmentation accuracy of the media and endothelial segmenters for the media and endothelial tissues, respectively.

[0126] S3: Determine the loss function based on the media segmentation result label, endothelial segmentation result label, media segmentation result image, and endothelial segmentation result image of the vessel segment of interest.

[0127] S4: Update the parameters of the deep learning segmentation model to be trained based on the 3D blood vessel image containing the segment of interest, the media segmentation result image, the intima segmentation result image, the media segmentation result label, the intima segmentation result label, and the loss function, until the deep learning segmentation model to be trained meets the preset training conditions, and obtain the trained deep learning segmentation model.

[0128] Throughout the training process, the segmentation results labels for the media and endothelium serve as the segmentation labels. The media segmentation result label represents the standard media segmentation result, i.e., the desired media segmentation result. The endothelium segmentation result label represents the standard endothelium segmentation result, i.e., the desired endothelium segmentation result. During training, the deep learning segmentation model is input with a 3D blood vessel image and its corresponding media and endothelium segmentation result labels. This updates the parameters of the deep learning segmentation model, allowing it to learn the segmentation process until the preset training conditions are met.

[0129] The preset training condition refers to the loss function decreasing to convergence (i.e., the loss function no longer decreases during further training). When the preset training condition is met, the deep learning segmentation model is considered to be able to accurately segment the middle membrane and the inner membrane. The preset training condition can be a specific number of training iterations, which can be determined based on the amount of data used for training and the specific training strategy.

[0130] By employing the above technical solution, the training method of the deep learning segmentation model of this invention combines a cross-attention mechanism. Through the interaction of medial and endothelial feature data, the trained deep learning segmentation model can more accurately segment both the medial and endothelial segments of the vessel segment of interest simultaneously. This invention trains a deep learning segmentation model with an end-to-end processing framework. By directly inputting a 3D vessel image containing the vessel segment of interest into the deep learning segmentation model, it can simultaneously process the media and endothelial segments of the vessel, obtaining corresponding medial and endothelial segmentation results. In the above embodiments, the deep learning segmentation model of this invention, by utilizing the relative stability of the medial and the potential topological relationship between the medial and endothelial segments, combined with the cross-attention mechanism, can optimize and guide the accurate segmentation of the endothelial segment, while simultaneously promoting the accurate segmentation of the medial segment, thus improving the segmentation accuracy of both the medial and endothelial segments. Furthermore, the deep learning segmentation model with an end-to-end deep learning framework of this invention avoids complex manual feature design and multi-stage processing, simplifying the workflow.

[0131] Furthermore, in the above embodiments, the loss function in step S3 includes the cross-entropy loss function, the Dice loss function, and the consistency loss function.

[0132] Specifically, the cross-entropy loss function is used to measure the degree of difference between the actual output (from the model) and the target output (true label). In this invention, it specifically measures the degree of difference between the actual segmentation results of the middle membrane and the inner membrane produced by the deep learning segmentation model and the respective middle membrane segmentation labels and inner membrane segmentation labels (i.e., the expected middle membrane segmentation results and inner membrane segmentation results).

[0133] In image segmentation tasks, the Dice loss function can better handle the problem of large differences in the number of pixels between the foreground and background, thus improving segmentation accuracy. In this invention, the foreground refers to the tunica media and tunica intima of the blood vessel, while the background refers to the surrounding environment of the blood vessel. The Dice loss function helps to accurately distinguish the tunica intima, tunica media, and the environment in which the blood vessel is located, thereby improving the segmentation accuracy of the tunica intima and tunica media.

[0134] Figure 18 This shows a plan view taken along the centerline of the blood vessel, where the yellow portion represents the intima (inner lining) and the orange portion represents the media (middle lining). Figure 18 As can be seen, compared with the inner membrane, the middle membrane is thicker, and the area displayed on the image is larger and more continuous, which is more conducive to segmentation.

[0135] The median membrane, due to its larger lumen and smaller diameter variation, is relatively easy to maintain its segmentation results, resulting in relatively high stability. The consistency loss function (i.e., consistency constraint) is proposed during the simultaneous segmentation of the median and endothelial membranes to constrain the continuity of the endothelial membrane using the continuity of the median. Because of the relatively high stability of median segmentation, establishing the relationship between the median and endothelial membranes through consistency constraints helps ensure the consistency between the segmentation results of the endothelial membrane and the median. (Referencing...) Figure 2 The consistency loss function is obtained through the following steps:

[0136] S310: Obtain the middle tunica skeletonization result data of the middle tunica segmentation result image through the soft skeleton extraction method based on the middle tunica segmentation result image, and obtain the endotunica skeletonization result data of the endotunica segmentation result image through the soft skeleton extraction method based on the endotunica segmentation result image.

[0137] The soft skeleton extraction method is the soft-skeleton algorithm. The soft-skeleton algorithm is a differentiable method for calculating the skeletonization result of the segmentation result. The result obtained by soft-skeleton is similar to that of conventional skeletonization algorithms, but soft-skeleton uses differentiable operations, which facilitates the calculation of the loss function.

[0138] S320: Determine the consistency loss function based on the results of the middle membrane skeletonization and the endometrial skeletonization.

[0139] The consistency loss function is determined more specifically by calculating the dot product of the middle membrane skeletonization result data and the inner membrane skeletonization result data, and then dividing it by the sum of the pixel points of the middle membrane skeletonization result data. This yields the continuity of the inner membrane relative to the middle membrane, which is the consistency loss function.

[0140] Since the middle membrane is relatively thick, it is easier to maintain the continuity of the segmentation results of the middle membrane. Therefore, the consistency loss function can be used to keep the segmentation results of the middle membrane and the inner membrane consistent in terms of continuity, thereby promoting the continuity of the segmentation results of the inner membrane.

[0141] The consistency loss function is determined using the above method. First, the media centerline is obtained using soft-skeleton, and then the consistency loss function is used to calculate the consistency of connectivity between the intima and media. This method is not easily affected by the difference in vessel diameter in places with different vessel thicknesses, which helps to more accurately segment the media and intima.

[0142] Specifically, the consistency loss function can be:

[0143] S l =SoftSkeleton(P l )(Formula 1)

[0144] S w =SoftSkeleton(P w )(Formula 2)

[0145]

[0146] Among them, P I S represents the image of the endometrial segmentation result. I P represents the data on endometrial skeletalization results. W S represents the segmentation result image of the middle membrane. W This represents the results of the middle membrane skeletonization. SoftSkeleton represents the soft skeleton extraction method (i.e., the function of the aforementioned soft-skeleton algorithm). Consistency lumen-wall This represents the consistency loss function.

[0147] In the above embodiments, the loss function may specifically be:

[0148] Total Loss = Dice Lumen +Dice Wall +CELumen +CE Wall +Consistency lumen-wall (male

[0149] Equation 4)

[0150]

[0151] Where Total Loss represents the loss function, and Dice Lumen +Dice Wall Denotes the Dice loss function, CE Lumen +CE Wall Represents the cross-entropy loss function, Consistency lumen-wall Let G represent the consistency loss function. L Indicates the label for endometrial segmentation results, P L S represents the image of the endometrial segmentation result. L G represents the data of endometrial skeletalization results. W P represents the label for the middle membrane segmentation result. W S represents the segmentation result image of the middle membrane. W This represents the data on the results of the medial membrane skeletonization.

[0152] Further, refer to Figure 3 The aforementioned step S1 specifically includes the following steps:

[0153] S11: Obtain an unprocessed 3D vascular image of the segment of interest.

[0154] Figure 13 The image shown is a raw 3D coronary CTA cross-sectional image, an example of an unprocessed 3D vascular image.

[0155] S12: Normalize and augment the unprocessed 3D vascular image to obtain a 3D vascular image.

[0156] Normalization and data augmentation can improve image quality and data diversity. Specifically, normalization involves scaling image pixel values ​​to the 0-1 range, which is more beneficial for model training and prediction. Data augmentation includes random rotation and the addition of random noise. Data augmentation is only used during model training. It maximizes data augmentation by randomly altering image contrast, cropping, scaling, grayscale transformations, and adding noise. These altered data are treated as new data for the deep learning segmentation model, thus increasing data diversity and leading to better generalization performance in the trained model.

[0157] Figure 15The diagram illustrates a three-dimensional vascular image obtained after normalizing and data augmentation of the original three-dimensional coronary CTA cross-sectional image. In this embodiment, the data augmentation specifically involves adding random noise and random contrast transformation to the normalized original three-dimensional coronary CTA cross-sectional image.

[0158] Deep learning segmentation models also include feature extractors, and the loss function is used to train the feature extractors. Further, refer to... Figure 4 and combined Figure 12 The aforementioned step S2 further includes the following steps:

[0159] S21: An attention mechanism is used to extract vascular features from a 3D vascular image containing the vascular segment of interest using a feature extractor, thereby obtaining the 3D vascular image feature data of the vascular segment of interest.

[0160] Among them, attention mechanisms include spatial attention mechanisms and / or channel attention mechanisms.

[0161] Spatial attention mechanisms operate within the three-dimensional space of an image, enabling deep learning segmentation models to focus on specific key regions while ignoring less important parts. This mechanism allows deep learning segmentation models to automatically learn the importance of different spatial locations within an image, improving their ability to represent spatial information. In this invention, the spatial attention mechanism is used to extract spatial feature data of the vessel segment of interest within a three-dimensional vascular image containing that segment, which is beneficial for improving the segmentation accuracy of the media and intima.

[0162] Channel attention mechanisms can assign different weights to different feature channels of an image, enabling deep learning segmentation models to focus more on feature channels with more information content while suppressing less important feature channels. In this invention, the channel attention mechanism is used to extract the feature data (i.e., feature channels) of the blood vessel segment of interest in a 3D blood vessel image containing the blood vessel segment of interest. It can adaptively adjust the weights of different feature channels, improve the deep learning segmentation model's ability to represent the key features of the target blood vessel, thereby improving the performance of the deep learning segmentation model and improving the segmentation accuracy of the media and intima in subsequent segments.

[0163] Specifically, continue to refer to Figure 12The feature extractor is a shared encoder used to encode features from the 3D vascular image, obtaining a feature vector F1. Feature vector F1 includes an intima feature vector representing intimal features and a media feature vector representing media features. Then, channel attention and spatial attention mechanisms are applied to feature vector F1, further encoding it into feature vector F2. Feature vector F2 represents the 3D vascular image feature data. The intima segmenter and media segmenter then decode feature vector F2, obtaining media and intima features respectively. A cross-attention mechanism is applied to the media and intima features, allowing the intima and media segmenters to mutually promote their respective learning of intima and media features, ultimately resulting in highly accurate intima and media segmentation images.

[0164] Furthermore, in the above embodiments, the feature extractor, the inner membrane segmenter, and the middle membrane segmenter can all be arbitrary backbones. Here, backbone refers to the main feature extraction part of the neural network, which can be a pre-trained model based on a large image dataset (such as ImageNet) capable of extracting specific features of the image from the input data. The backbone can consist of a series of convolutional layers, pooling layers, normalization layers, and activation function layers.

[0165] In the above embodiments, this invention, guided by the topological structure of the media, achieves higher intimal segmentation accuracy in three-dimensional coronary CTA images, particularly those with high plaque burden. This invention utilizes channel attention, spatial attention, and cross-attention mechanisms to achieve efficient feature interaction between the intima and media, improving segmentation accuracy. Furthermore, this invention, through the aforementioned deep learning segmentation model, can optimize the continuity and accuracy of intimal segmentation. Since the extraction of the vascular centerline is obtained by skeletonizing the intimal segmentation results, this invention can also improve the accuracy and quality of subsequent vascular centerline extraction.

[0166] Furthermore, through optimized intimal segmentation and centerline extraction techniques, risk assessment and pathological analysis of cardiac diseases can be performed more accurately, reducing misdiagnosis and missed diagnosis rates. This optimizes the clinical application value of CCTA (coronary computed tomography angiography) analysis, which is beneficial for accurately assessing cardiac lesions and subsequent treatment planning, and has significant social and economic benefits.

[0167] Secondly, refer to Figure 9This invention discloses a training device 1 for a deep learning segmentation model, which trains the deep learning segmentation model using the training method described in any of the foregoing embodiments. The deep learning segmentation model includes a middle membrane segmenter and an inner membrane segmenter. The middle membrane segmenter is used to segment the middle membrane segmentation result image, and the inner membrane segmenter is used to segment the inner membrane segmentation result image. The training device 1 for the deep learning segmentation model includes an image acquisition module 11, a pre-segmentation module 12, a loss function determination module 13, and a training module 14.

[0168] The image acquisition module 11 is connected to the pre-segmentation module 12, the pre-segmentation module 12 is connected to the loss function determination module 13, and the loss function determination module 13 is connected to the training module 14.

[0169] Continue to refer to Figure 9 The image acquisition module 11 is used to acquire a three-dimensional vascular image containing the vascular segment of interest, and to acquire the media segmentation result label and intima segmentation result label of the labeled vascular segment of interest.

[0170] The pre-segmentation module 12 is used to process the three-dimensional blood vessel image containing the blood vessel segment of interest through the deep learning segmentation model to be trained, and to obtain the media segmentation result image and the intima segmentation result image.

[0171] The loss function determination module 13 is used to determine the loss function based on the three-dimensional vascular image of the segment of interest, the media segmentation result image, and the intima segmentation result image.

[0172] The training module 14 is used to update the parameters of the deep learning segmentation model to be trained based on the three-dimensional blood vessel image containing the blood vessel segment of interest, the media segmentation result image, the intima segmentation result image, the media segmentation result label, the intima segmentation result label, and the loss function, until the deep learning segmentation model to be trained meets the preset training conditions, and the trained deep learning segmentation model is obtained.

[0173] Using the above technical solution, the training device 1 of the deep learning segmentation model in this embodiment combines a cross-attention mechanism. Through the cross-attention mechanism, the media features and endothelial features interact, enabling the trained deep learning segmentation model to more accurately segment the media segmentation result image and the endothelial segmentation result image of the blood vessel segment of interest at the same time, thereby improving the segmentation accuracy of the media and endothelial.

[0174] In the pre-segmentation module 12, during the processing of the 3D vascular image containing the segment of interest using the deep learning segmentation model to be trained, specifically, the 3D vascular image containing the segment of interest is segmented using a cross-attention mechanism through a media segmenter and an intima segmenter to obtain media segmentation result images and intima segmentation result images. The process of segmenting the 3D vascular image containing the segment of interest using the cross-attention mechanism specifically includes: extracting media features from the 3D vascular image containing the segment of interest using the media segmenter to obtain media feature data; extracting intima features from the 3D vascular image containing the segment of interest using the intima segmenter to obtain intima feature data; fusing the media feature data and intima feature data using the cross-attention mechanism through the media segmenter to obtain a media segmentation result image; and fusing the media feature data and intima feature data using the cross-attention mechanism through the intima segmenter to obtain an intima segmentation result image.

[0175] Using the above technical solution, the training device 1 of the deep learning segmentation model in this embodiment combines a cross-attention mechanism. Through the interaction of medial feature data and endothelial feature data, the trained deep learning segmentation model can more accurately segment the medial segmentation result image and the endothelial segmentation result image of the blood vessel segment of interest at the same time, thereby improving the segmentation accuracy of the medial and endothelial membranes.

[0176] Furthermore, the image acquisition module 11 can also be used to perform the aforementioned steps S11 and S12. The pre-segmentation module 12 can also be used to perform the aforementioned step S21. The loss function determination module 13 can also be used to perform the aforementioned steps S310 and S320.

[0177] Thirdly, embodiments of the present invention disclose a blood vessel image segmentation method, which segments blood vessel images using a deep learning segmentation model. The deep learning segmentation model includes a media segmenter and an endothelial segmenter. The media segmenter is used to segment the media segmentation result image, and the endothelial segmenter is used to segment the endothelial segmentation result image. The deep learning segmentation model is trained using the training method for the deep learning segmentation model in any of the foregoing embodiments.

[0178] refer to Figure 5 The blood vessel image segmentation method specifically includes the following steps:

[0179] S01: Obtain the 3D vascular image feature data of the vascular segment of interest.

[0180] Among them, the three-dimensional vascular image feature data can be Figure 12 The eigenvector F2 in the model.

[0181] S02: The cross-attention mechanism is used to segment the 3D vascular image feature data of the vascular segment of interest using a deep learning segmentation model, resulting in the segmentation results of the media and intima of the vascular segment of interest.

[0182] By employing the above technical solution, the vascular image segmentation method of the present invention, through the use of a cross-attention mechanism, fully utilizes the interaction between the media and the endothelium, thereby improving the segmentation accuracy of the media and the endothelium.

[0183] The process of segmenting 3D vascular image feature data using a cross-attention mechanism specifically includes: extracting medial features from the 3D vascular image feature data using a media segmenter to obtain medial feature data; and extracting endothelial features from the 3D vascular image feature data using an intima segmenter to obtain intima feature data. The media segmenter then fuses the medial and intima feature data using a cross-attention mechanism to obtain the medial segmentation result image. Similarly, the intima segmenter fuses the medial and intima feature data using a cross-attention mechanism to obtain the intima segmentation result image.

[0184] Figure 16 This diagram illustrates the overlay of the medial segmentation image and the intima segmentation image onto the original 3D vascular image. Figure 17 The corresponding three-dimensional visualization results of the medial and endothelial segmentation are shown. Figure 16 and Figure 17 In the image, the green portion represents the middle tunica media segmentation result, and the blue portion represents the inner tunica media segmentation result. It's important to note that the blue portion is also part of the middle tunica media; however, due to the superimposed image, some of the green portion is obscured by the blue.

[0185] By employing the above technical solution, the vascular image segmentation method of the present invention uses a cross-attention mechanism to enable the interaction between the media and endothelium features, allowing the deep learning segmentation model to more accurately segment the media and endothelium segments of the vascular segment of interest simultaneously, thereby improving the segmentation accuracy of the media and endothelium.

[0186] Further, refer to Figure 6 The aforementioned step S01 specifically includes the following steps:

[0187] S011: Obtain a three-dimensional vascular image containing the vascular segment of interest.

[0188] Specifically, three-dimensional vascular images are three-dimensional coronary CTA images.

[0189] S012: Obtain the three-dimensional vascular image feature data of the vascular segment of interest from the three-dimensional vascular image containing the vascular segment of interest, including: using an attention mechanism to extract vascular features from the three-dimensional vascular image of the vascular segment of interest through a feature extractor to obtain the three-dimensional vascular image feature data of the vascular segment of interest.

[0190] The attention mechanisms include spatial attention and / or channel attention. Spatial attention is used to extract spatial feature data of the vessel segment of interest in a 3D vascular image. Channel attention is used to extract channel feature data of the vessel segment of interest in a 3D vascular image. For more detailed information, please refer to the foregoing descriptions of spatial and channel attention mechanisms; they will not be repeated here.

[0191] Further, refer to Figure 7 The aforementioned step S011 specifically includes the following steps:

[0192] S0111: Obtain an unprocessed 3D vascular image of the segment of interest.

[0193] Unprocessed 3D vascular images, for example, are Figure 13 The original three-dimensional coronary CTA cross-sectional image is shown.

[0194] S0112: Normalize the unprocessed 3D vascular image to obtain a 3D vascular image of the segment of interest.

[0195] The normalized three-dimensional blood vessel image is, for example, Figure 14 The image shown is a three-dimensional coronary CTA cross-sectional image.

[0196] It is important to note that when using a pre-trained deep learning segmentation model, there is no need to perform data augmentation on the unprocessed 3D blood vessel images during preprocessing; only normalization is required.

[0197] refer to Figure 8 After completing the aforementioned steps S01 and S02, step S03 can be performed: image enhancement and optimization operations are performed on the middle membrane segmentation result image and the endometrial segmentation result image to obtain the enhanced final middle membrane segmentation result image and the final endometrial segmentation result image.

[0198] The image enhancement and optimization operations include morphological operations and / or connectivity analysis. Morphological operations are selected from one or more of the following: erosion, dilation, opening, closing, morphological gradient, top-hat and black-hat morphology. Connectivity analysis is selected from one or more of the following: connected component labeling, region growing, boundary tracing, hole filling, skeleton extraction, and region segmentation. These operations can fill holes inside blood vessels in the segmented image or remove noise generated during segmentation, all achieving the goal of image optimization. Appropriate operations can be selected based on the actual segmentation situation for image optimization.

[0199] Further image enhancement and optimization operations on the segmentation results of the middle tunica and the endocrine tunica can improve the accuracy and robustness of the segmentation of the middle tunica and the endocrine tunica.

[0200] Fourthly, refer to Figure 10 This invention discloses a blood vessel image segmentation device 2, which segments blood vessel images using a deep learning segmentation model. The deep learning segmentation model includes a media segmenter and an endothelial segmenter. The media segmenter is used to segment the media into a segmented image, and the endothelial segmenter is used to segment the endothelial into a segmented image. The blood vessel image segmentation device 2 includes a feature acquisition module 21 and an image segmentation module 22. The feature acquisition module 21 is connected to the image segmentation module 22.

[0201] The feature acquisition module 21 is used to acquire the three-dimensional vascular image feature data of the vascular segment of interest.

[0202] The image segmentation module 22 is used to segment the three-dimensional vascular image feature data of the vascular segment of interest using a cross-attention mechanism through a deep learning segmentation model, so as to obtain the media segmentation result image and the intima segmentation result image of the vascular segment of interest.

[0203] By adopting the above technical solution, the vascular image segmentation device 2 of the present invention can improve the segmentation accuracy of the media and endothelium by making full use of the interaction between the media and endothelium through the use of cross-attention mechanism.

[0204] Specifically, the image segmentation module employs a cross-attention mechanism to segment the 3D vascular image feature data. This process includes: extracting medial features from the 3D vascular image feature data using a media segmenter; extracting endothelial features from the 3D vascular image feature data using an intima segmenter; fusing the medial and endothelial feature data using a cross-attention mechanism to obtain a medial segmentation result image; and fusing the medial and endothelial feature data using a cross-attention mechanism to obtain an intima segmentation result image.

[0205] By adopting the above technical solution, the vascular image segmentation device 2 of the present invention uses a cross-attention mechanism to enable the media features and intima features to interact, so that the deep learning segmentation model can more accurately segment the media segmentation result image and the intima segmentation result image of the vascular segment of interest at the same time, thereby improving the segmentation accuracy of the media and intima.

[0206] Furthermore, the feature acquisition module 21 can also be used to perform the aforementioned steps S011, S012, S0111, and S0112. The image segmentation module 22 can also be used to perform the aforementioned step S03.

[0207] Fifthly, refer to Figure 11 The present invention also discloses an electronic device 3, including a memory 31 and a processor 32. The memory 31 stores at least one instruction, which, when executed by the processor 32, implements the training method of the deep learning segmentation model in any of the foregoing embodiments, and / or the blood vessel image segmentation method in any of the foregoing embodiments. The memory 31 may include, for example, system memory, fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs.

[0208] In this embodiment, the electronic device 3 can combine a cross-attention mechanism to enable the deep learning segmentation model trained by the interaction of medial feature data and endothelial feature data to more accurately segment the medial segmentation result image and endothelial segmentation result image of the blood vessel segment of interest at the same time, thereby improving the segmentation accuracy of the medial and endothelial.

[0209] In a sixth aspect, embodiments of the present invention also disclose a computer-readable storage medium storing at least one instruction, which, when executed, implements the training method of the deep learning segmentation model in any of the foregoing embodiments, and / or the blood vessel image segmentation method in any of the foregoing embodiments.

[0210] By adopting the above technical solution, the computer-readable storage medium can combine the cross-attention mechanism and, through the interaction of medial feature data and endothelial feature data, enable the trained deep learning segmentation model to more accurately segment the medial segmentation result image and endothelial segmentation result image of the blood vessel segment of interest, thereby improving the segmentation accuracy of the medial and endothelial.

[0211] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0212] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0213] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0214] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0215] While the present invention has been illustrated and described with reference to certain preferred embodiments, those skilled in the art should understand that the above description is a further detailed explanation of the invention in conjunction with specific embodiments, and should not be construed as limiting the specific implementation of the invention to these descriptions. Various changes in form and detail can be made by those skilled in the art, including several simple deductions or substitutions, without departing from the spirit and scope of the invention.

Claims

1. A training method for a deep learning segmentation model, characterized in that, The deep learning segmentation model includes a middle tunica segmenter and an inner tunica segmenter. The middle tunica segmenter is used to segment the middle tunica segmentation result image, and the inner tunica segmenter is used to segment the inner tunica segmentation result image. The training method includes: Obtain a three-dimensional vascular image containing the segment of interest; obtain the labeled media segmentation result labels and intima segmentation result labels of the labeled segment of interest; The deep learning segmentation model to be trained is used to process the three-dimensional blood vessel image containing the blood vessel segment of interest to obtain the media segmentation result image and the intima segmentation result image. The loss function is determined based on the media segmentation result label, the endothelial segmentation result label, the media segmentation result image, and the endothelial segmentation result image of the vessel segment of interest; The parameters of the deep learning segmentation model to be trained are updated based on the three-dimensional blood vessel image containing the blood vessel segment of interest, the media segmentation result image, the intima segmentation result image, the media segmentation result label, the intima segmentation result label, and the loss function until the deep learning segmentation model to be trained meets the preset training conditions, thus obtaining the trained deep learning segmentation model. The media segmenter and the endothelial segmenter use a cross-attention mechanism to segment the three-dimensional vascular image containing the vascular segment of interest, thereby obtaining the media segmentation result image and the endothelial segmentation result image. The loss functions include the cross-entropy loss function, the Dice loss function, and the consistency loss function; The consistency loss function is obtained in the following way: Based on the segmented middle membrane image, the soft skeleton extraction method is used to obtain the middle membrane skeletonization result data of the segmented middle membrane image; based on the segmented inner membrane image, the soft skeleton extraction method is used to obtain the inner membrane skeletonization result data of the segmented inner membrane image. The consistency loss function is determined based on the middle membrane skeletonization result data and the endometrial skeletonization result data.

2. The training method for the deep learning segmentation model as described in claim 1, characterized in that, The process of segmenting the three-dimensional blood vessel image containing the blood vessel segment of interest using the cross-attention mechanism specifically includes: The medial features of the three-dimensional vascular image containing the segment of interest are extracted using the medial segmenter to obtain medial feature data. The intima features of the three-dimensional vascular image containing the segment of interest are extracted using the intima segmenter to obtain intima feature data. The middle membrane segmenter performs feature fusion on the middle membrane feature data and the inner membrane feature data through the cross-attention mechanism to obtain the middle membrane segmentation result image; The endometrial segmenter performs feature fusion on the middle membrane feature data and the endometrial feature data through the cross-attention mechanism to obtain the endometrial segmentation result image.

3. The training method for the deep learning segmentation model as described in claim 1, characterized in that, The loss function is specifically as follows: Total Loss=Dice Lumen +Dice Wall +CE Lumen +CE Wall +Consistency lumen-wall Where Total Loss represents the loss function, Dice Lumen +Dice Wall Denotes the Dice loss function, CE Lumen +CE Wall Represents the cross-entropy loss function, Consistency lumen-wall Let G represent the consistency loss function. L The label representing the endometrial segmentation result, P L S represents the image of the endometrial segmentation result. L G represents the data indicating the intima skeletonization results. W The label representing the mid-membrane segmentation result, P W S represents the image of the middle membrane segmentation result. W This represents the data indicating the results of the middle membrane skeletonization.

4. The training method for the deep learning segmentation model as described in claim 1, characterized in that, The consistency loss function is specifically as follows: S l =SoftSkeleton(P l ) S w =SoftSkeleton(P w ) Among them, P I S represents the image of the endometrial segmentation result. I P represents the data indicating the intima skeletonization results. W S represents the image of the middle membrane segmentation result. W This represents the results of the middle membrane skeletonization; SoftSkeleton represents the soft skeleton extraction method; Consistency lumen-wall This represents the consistency loss function.

5. The training method for the deep learning segmentation model as described in claim 1, characterized in that, The acquisition of a three-dimensional vascular image containing the vascular segment of interest specifically includes: Obtain an unprocessed three-dimensional vascular image of the vascular segment of interest; The unprocessed three-dimensional blood vessel image is normalized and data augmented to obtain the three-dimensional blood vessel image.

6. The training method for the deep learning segmentation model as described in claim 1, characterized in that, The deep learning segmentation model further includes a feature extractor, and the loss function is used to train the feature extractor. The processing of the three-dimensional blood vessel image containing the blood vessel segment of interest using the deep learning segmentation model to be trained specifically includes: An attention mechanism is used to extract vascular features from the three-dimensional vascular image containing the vascular segment of interest using the feature extractor, thereby obtaining the three-dimensional vascular image feature data of the vascular segment of interest. The attention mechanism includes a spatial attention mechanism and / or a channel attention mechanism; The spatial attention mechanism is used to extract the spatial feature data of the blood vessel segment of interest in the three-dimensional blood vessel image containing the blood vessel segment of interest; The channel attention mechanism is used to extract the channel feature data of the blood vessel segment of interest in the three-dimensional blood vessel image containing the blood vessel segment of interest.

7. A training device for a deep learning segmentation model, characterized in that, The deep learning segmentation model includes a middle membrane segmenter and an inner membrane segmenter. The middle membrane segmenter is used to segment the middle membrane segmentation result image, and the inner membrane segmenter is used to segment the inner membrane segmentation result image. The training device includes: The image acquisition module is used to acquire a three-dimensional vascular image containing the vascular segment of interest, and to acquire the labeled media segmentation result label and intima segmentation result label of the vascular segment of interest. The pre-segmentation module is used to process the three-dimensional blood vessel image containing the blood vessel segment of interest using the deep learning segmentation model to be trained, so as to obtain the media segmentation result image and the intima segmentation result image. The loss function determination module is used to determine the loss function based on the media segmentation result label, the endothelial segmentation result label, the media segmentation result image, and the endothelial segmentation result image of the blood vessel segment of interest; The training module is used to update the parameters of the deep learning segmentation model to be trained based on the three-dimensional blood vessel image containing the blood vessel segment of interest, the media segmentation result image, the intima segmentation result image, the media segmentation result label, the intima segmentation result label, and the loss function, until the deep learning segmentation model to be trained meets the preset training conditions, and thus obtain the trained deep learning segmentation model. The media segmenter and the endothelial segmenter use a cross-attention mechanism to segment the three-dimensional vascular image containing the vascular segment of interest, thereby obtaining the media segmentation result image and the endothelial segmentation result image. The loss functions include the cross-entropy loss function, the Dice loss function, and the consistency loss function; The consistency loss function is obtained in the following way: Based on the segmented middle membrane image, the soft skeleton extraction method is used to obtain the middle membrane skeletonization result data of the segmented middle membrane image; based on the segmented inner membrane image, the soft skeleton extraction method is used to obtain the inner membrane skeletonization result data of the segmented inner membrane image. The consistency loss function is determined based on the middle membrane skeletonization result data and the endometrial skeletonization result data.

8. The training apparatus for the deep learning segmentation model as described in claim 7, characterized in that, During the process of the pre-segmentation module processing the three-dimensional blood vessel image containing the blood vessel segment of interest through the deep learning segmentation model to be trained, the three-dimensional blood vessel image containing the blood vessel segment of interest is segmented by the media segmenter and the endothelial segmenter using a cross-attention mechanism to obtain the media segmentation result image and the endothelial segmentation result image. The process of segmenting the three-dimensional blood vessel image containing the blood vessel segment of interest using the cross-attention mechanism specifically includes: The medial features of the three-dimensional vascular image containing the segment of interest are extracted using the medial segmenter to obtain medial feature data. The intima features of the three-dimensional vascular image containing the segment of interest are extracted using the intima segmenter to obtain intima feature data. The middle membrane segmenter performs feature fusion on the middle membrane feature data and the inner membrane feature data through the cross-attention mechanism to obtain the middle membrane segmentation result image; The endometrial segmenter performs feature fusion on the middle membrane feature data and the endometrial feature data through the cross-attention mechanism to obtain the endometrial segmentation result image.

9. A method for segmenting blood vessel images, characterized in that, The blood vessel image segmentation method utilizes a deep learning segmentation model, comprising a media segmenter and an endothelial segmenter. The media segmenter is used to segment the media into a segmented image, and the endothelial segmenter is used to segment the endothelial into a segmented image. Obtain 3D vascular image feature data of the vascular segment of interest; The deep learning segmentation model employs a cross-attention mechanism to segment the 3D vascular image feature data of the vascular segment of interest, thereby obtaining the media segmentation result image and the intima segmentation result image of the vascular segment of interest. The deep learning segmentation model is obtained by training the deep learning segmentation model according to any one of claims 1-6.

10. The blood vessel image segmentation method as described in claim 9, characterized in that, The process of segmenting the three-dimensional blood vessel image feature data using the cross-attention mechanism specifically includes: The medial features of the three-dimensional blood vessel image feature data are extracted by the medial segmenter to obtain medial feature data; The intima features are extracted from the three-dimensional vascular image feature data using the intima segmenter to obtain intima feature data. The middle membrane segmenter performs feature fusion on the middle membrane feature data and the inner membrane feature data through the cross-attention mechanism to obtain the middle membrane segmentation result image; The endometrial segmenter performs feature fusion on the middle membrane feature data and the endometrial feature data through the cross-attention mechanism to obtain the endometrial segmentation result image.

11. The blood vessel image segmentation method as described in claim 9, characterized in that, The deep learning segmentation model further includes a feature extractor, and the acquisition of three-dimensional vascular image feature data of the vascular segment of interest specifically includes: Obtain a three-dimensional vascular image containing the vascular segment of interest; Obtaining the three-dimensional vascular image feature data of the vascular segment of interest from the three-dimensional vascular image containing the vascular segment of interest includes: using an attention mechanism to extract vascular features from the three-dimensional vascular image of the vascular segment of interest through the feature extractor to obtain the three-dimensional vascular image feature data of the vascular segment of interest; The attention mechanism includes a spatial attention mechanism and / or a channel attention mechanism; The spatial attention mechanism is used to extract the spatial feature data of the blood vessel segment of interest in the three-dimensional blood vessel image. The channel attention mechanism is used to extract the channel feature data of the blood vessel segment of interest in the three-dimensional blood vessel image.

12. The blood vessel image segmentation method as described in claim 9, characterized in that, The acquisition of a three-dimensional vascular image containing the vascular segment of interest specifically includes: Obtain an unprocessed three-dimensional vascular image of the vascular segment of interest; The unprocessed three-dimensional vascular image is normalized to obtain the three-dimensional vascular image of the vascular segment of interest.

13. The blood vessel image segmentation method as described in claim 9, characterized in that, Also includes: Image enhancement and optimization operations are performed on the middle membrane segmentation result image and the end membrane segmentation result image to obtain the enhanced final middle membrane segmentation result image and the final end membrane segmentation result image; The image enhancement and optimization operations include morphological operations and / or connectivity analysis. The morphological operations are selected from one or more of the following: erosion, dilation, opening, closing, morphological gradient, top hat, and black hat. The connectivity analysis is selected from one or more of the following: connected component labeling, region growing, boundary tracking, hole filling, skeleton extraction, and region segmentation.

14. A blood vessel image segmentation device, characterized in that, The blood vessel image segmentation device uses a deep learning segmentation model to segment blood vessel images. The deep learning segmentation model includes a media segmenter and an endothelial segmenter. The media segmenter is used to segment the media into a segmented image, and the endothelial segmenter is used to segment the endothelial into a segmented image. The blood vessel image segmentation device includes: The feature acquisition module is used to acquire 3D vascular image feature data of the vascular segment of interest; The image segmentation module is used to segment the three-dimensional vascular image feature data of the vascular segment of interest using the deep learning segmentation model with a cross-attention mechanism, so as to obtain the media segmentation result image and the intima segmentation result image of the vascular segment of interest. The deep learning segmentation model is obtained by training the deep learning segmentation model according to any one of claims 1-6.

15. The blood vessel image segmentation apparatus as described in claim 14, characterized in that, The image segmentation module, using the cross-attention mechanism, specifically includes the following steps during the segmentation of the three-dimensional blood vessel image feature data: The medial features of the three-dimensional blood vessel image feature data are extracted by the medial segmenter to obtain medial feature data; The intima features are extracted from the three-dimensional vascular image feature data using the intima segmenter to obtain intima feature data. The middle membrane segmenter performs feature fusion on the middle membrane feature data and the inner membrane feature data through the cross-attention mechanism to obtain the middle membrane segmentation result image; The endometrial segmenter performs feature fusion on the middle membrane feature data and the endometrial feature data through the cross-attention mechanism to obtain the endometrial segmentation result image.

16. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction, which, when executed by the processor, implements the training method of the deep learning segmentation model as described in any one of claims 1-6, and / or the blood vessel image segmentation method as described in any one of claims 9-13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed, implements the training method of the deep learning segmentation model as described in any one of claims 1-6, and / or the blood vessel image segmentation method as described in any one of claims 9-13.

Images