Three-dimensional semi-supervised segmentation method and device for stent in coronary artery

Abstract

The invention provides a three-dimensional semi-supervised segmentation method and device for a stent in a coronary artery. The method comprises the steps that an initial image to be segmented is input into a three-dimensional semi-supervised segmentation deep learning network model trained in advance, the three-dimensional semi-supervised segmentation deep learning network model comprises a shared encoder, a BiFormer module and a decoding module which are connected in sequence, and the decoding module comprises a plurality of decoders adopting different attention mechanisms; the average value of result prediction probabilities output by all decoders in the decoding module is obtained, a prediction result is obtained, and three-dimensional segmentation of the stent in the coronary artery is achieved; in the model training process, model parameter optimization is carried out through a plurality of generated prediction output results with perception deviation and a cross pseudo-supervised training strategy. According to the three-dimensional semi-supervised segmentation deep learning network model designed by the invention, the problem that model training lacks annotation supervision is solved, model parameters are optimized through a cross pseudo-supervised training strategy, and the problem that segmentation of a support depends on the number of annotated data is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more particularly, to a three-dimensional semi-supervised segmentation method and device for stents based on intravascular optical coherence tomography (IV-OCT) images. Background Art

[0002] Coronary artery disease (CAD) is a common and serious cardiovascular disease. Atherosclerosis of the coronary arteries causes stenosis or occlusion of blood vessels, leading to insufficient blood supply to the myocardium. The typical symptoms of CAD include angina pectoris, chest tightness, and myocardial ischemia, and in severe cases, it may lead to myocardial infarction or heart failure. With the influence of aging, unhealthy lifestyles, and chronic diseases (such as hypertension, diabetes, and hyperlipidemia), the incidence of CAD has been increasing year by year. As part of percutaneous coronary intervention (PCI), stent implantation is an effective means for treating CAD. Stent segmentation can accurately extract the stent and vascular morphology from OCT images, helping doctors locate the stent implantation position, ensure effective dilation and maintain vascular patency. At the same time, it can be used for postoperative evaluation to detect stent displacement, restenosis, or thrombosis, timely discover problems, and avoid complications.

[0003] Optical coherence tomography (OCT) is a non-invasive, high-resolution three-dimensional imaging technology and has important application value in the diagnosis and treatment of coronary artery disease (CAD). Intravascular optical coherence tomography (IV-OCT) can provide cross-sectional images of blood vessels at the micron level, helping doctors accurately evaluate the degree of vascular stenosis, plaque properties, lesion area characteristics, and the effect after stent implantation. However, the manual analysis of IV-OCT images is time-consuming and laborious, which hinders the rapid diagnosis of IV-OCT in clinical practice. Traditional deep learning methods for analyzing IV-OCT images rely on manual data annotation, and the in-depth study of IV-OCT images is limited by the excessive workload of manual annotation. In addition, traditional analysis methods are limited to the evaluation of two-dimensional IV-OCT images. Currently, there is a lack of a more intuitive IV-OCT image evaluation method. Summary of the Invention

[0004] In view of the limitations of the prior art, the present invention proposes a three-dimensional semi-supervised segmentation method and device for coronary stents, aiming to achieve high-precision and automated three-dimensional semi-supervised segmentation of stents in IV-OCT images without relying on a large amount of manual annotation.

[0005] To achieve the above object, the technical solutions adopted in the embodiments of the present invention are as follows:

[0006] In a first aspect, the present invention provides a three-dimensional semi-supervised segmentation method for coronary stents, including the following steps:

[0007] Input the initial image to be segmented into a pre-trained 3D semi-supervised segmentation deep learning network model. Among them, the 3D semi-supervised segmentation deep learning network model includes a shared encoder, a BiFormer module, and a decoding module connected in sequence. The decoding module includes multiple decoders using different attention mechanisms.

[0008] Take the average of the predicted probabilities of the results output by each decoder in the decoding module to obtain the prediction result, and realize the 3D segmentation of the intracoronary stent.

[0009] During the training process of the 3D semi-supervised segmentation deep learning network model, the BatchSize of the model input data is set to at least 2, and each group of input data contains labeled data and unlabeled data. For the unlabeled data, use the cross-pseudo-supervised training strategy to perform pseudo-label mutual supervision on the multiple predicted output results with perceptual bias generated by the decoding module to obtain the unsupervised loss. The unsupervised loss is combined with the supervised loss of the predicted result of the labeled data and the true label to jointly optimize the model parameters to obtain the 3D semi-supervised segmentation deep learning model.

[0010] The image dataset used in the training process of the 3D semi-supervised segmentation deep learning network model comes from the IV-OCT image dataset of patients who have received PCI treatment and a small amount of manually labeled datasets, and the images contain stent regions.

[0011] The shared encoder receives the initial image output by the intravascular optical coherence tomography system and performs downsampling on the image, and transfers the feature map after the downsampling process to the BiFormer module at the backend of the encoder. The BiFormer module receives the feature map transferred by the shared encoder, extracts the global context information in the feature map, and inputs the feature map after the information extraction to multiple independent decoders using different attention mechanisms. The decoders with multiple different attention mechanisms independently perform upsampling and feature fusion on the input feature map to generate multiple prediction results corresponding to their respective decoders.

[0012] Specifically, the shared encoder includes 5 downsampling layers, and each layer includes a 3D convolutional block with a stride of 2, a downsampling module, and a Dropout layer.

[0013] Specifically, the BiFormer module is composed of a double-layer routing attention mechanism.

[0014] Specifically, the decoding module includes three decoders using different attention mechanisms, including a decoder based on channel attention mechanism, a decoder based on spatial attention mechanism, and a decoder based on hybrid attention mechanism. The three decoders with different attention mechanisms use the softmax activation function to normalize and sharpen their respective output results to generate three pseudo-labels corresponding to their respective decoders.

[0015] Specifically, for the unlabeled data, the cross pseudo-supervision training strategy is used to perform pseudo-label mutual supervision on multiple predicted output results with perceptual biases generated by the decoding module, and an unsupervised loss is obtained; the unsupervised loss is combined with the supervised loss of the predicted result and the true label of the labeled data to jointly optimize the model parameters to obtain a three-dimensional semi-supervised segmentation deep learning model. Specifically:

[0016] In each group of input data, for the unlabeled data, the output results of the three predicted output results with perceptual biases generated by the three decoders of the decoding module are used as pseudo-label one, pseudo-label two, and pseudo-label three. Pseudo-label one calculates the loss function with pseudo-label two and pseudo-label three respectively to obtain the first loss value and the second loss value. Pseudo-label two calculates the loss function with pseudo-label one and three respectively to obtain the third loss value and the fourth loss value. Pseudo-label three calculates the loss function with pseudo-label one and pseudo-label two respectively to obtain the fifth loss value and the sixth loss value; the average value of the six loss values is used as the unsupervised loss in this group of data. The calculation method of the six loss values all uses the Tversky loss function; for the labeled data, the output results generated by the three decoders of the decoding module are respectively calculated with the true label to obtain the Dice loss value and then averaged as the supervised loss in this group of data;

[0017] The total loss of this group of input data is calculated through a composite loss function, including the Tversky loss calculated based on the predicted result of the unlabeled data and the pseudo-label, and the Dice loss calculated based on the predicted result of the labeled data and the true label. The formula for the total loss L is as follows:

[0018] L = λ(L D1 + L D2 +... + L Dm ) + μ(L T1 + L T2 +... + L Tn )

[0019] Among them, λ = 0.5, μ = 0.1 are hyperparameters, L Di is the Dice loss value of the labeled data in this group of data, L Tj is the Tversky loss value of the unlabeled data in this group of data, i ∈ (1, m), j ∈ (1, n), m is the number of labeled data in this group of data, n is the number of unlabeled data in this group of data, and m + n = batchsize; the network parameters are dynamically adjusted through the stochastic gradient descent algorithm; the model continuously calculates the loss value of the model and continuously optimizes the parameters and weights of the model under the supervision of the pseudo-label. Finally, the output with a higher confidence after cross pseudo-supervision training is selected as the final segmentation result.

[0020] Further, this method further includes: performing three-dimensional visualization reconstruction based on the three-dimensional segmentation result, where the three-dimensional visualization reconstruction result is used to evaluate the morphological characteristics of the stent in the coronary artery.

[0021] On the other hand, the present invention provides a three-dimensional semi-supervised segmentation device for an intracoronary stent, including: an imaging module for performing intravascular optical coherence tomography to obtain an image of the coronary artery;

[0022] a stent recognition module for using a pre-trained three-dimensional semi-supervised segmentation deep learning network model to recognize the image of the coronary artery to obtain a stent recognition result;

[0023] a three-dimensional modeling module for performing three-dimensional visualization reconstruction based on the stent recognition result to facilitate the user to evaluate the morphological characteristics of the intracoronary stent;

[0024] wherein, the three-dimensional semi-supervised segmentation deep learning network model includes a shared encoder, a BiFormer module, and a decoding module connected in sequence, and the decoding module includes a plurality of decoders adopting different attention mechanisms.

[0025] After adopting the above solution, the beneficial effects of the present invention are as follows:

[0026] A three-dimensional semi-supervised segmentation method and device for an intracoronary stent are provided. By obtaining the OCT image of the coronary stent region output by the optical coherence tomography system and inputting it into a pre-trained three-dimensional semi-supervised stent segmentation network model for three-dimensional segmentation, a target image containing only the stent region is obtained. The three-dimensional semi-supervised stent segmentation network model can achieve high-precision and automated three-dimensional segmentation under the condition of a small amount of labeled data, and generate a high-quality three-dimensional segmentation result of the coronary stent region. This technology can be applied to PCI treatment and postoperative evaluation, providing doctors with accurate lesion localization and stent evaluation tools, thereby significantly improving the accuracy and efficiency of CAD diagnosis and treatment decisions.

[0027] To make the above objects and features of the present invention more obvious and understandable, the following specific preferred embodiments are given in conjunction with the accompanying drawings and described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0029] Figure 1Schematic diagram of the three-dimensional semi-supervised stent segmentation network model structure provided by the embodiments of the present invention;

[0030] Figure 2 Schematic diagram of the three-dimensional semi-supervised segmentation scenario of the coronary artery stent provided by the embodiments of the present invention;

[0031] Figure 3 Flowchart of the three-dimensional semi-supervised segmentation method of the coronary artery stent provided by the embodiments of the present invention;

[0032] Figure 4 Flowchart of the construction process of the dataset for training the three-dimensional semi-supervised segmentation deep learning network model of the present invention provided by the embodiments of the present invention;

[0033] Figure 5 Detailed flowchart of the three-dimensional semi-supervised segmentation method of the coronary artery stent provided by the embodiments of the present invention;

[0034] Figure 6 Schematic diagram of the three-dimensional visualization reconstruction of the coronary artery stent provided by the embodiments of the present invention;

[0035] Figure 7 Block diagram of the structure of an electronic device provided by the embodiments of the present invention. Detailed implementation manners

[0036] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It should be understood that the accompanying drawings in the present invention only serve the purpose of illustration and description, and are not used to limit the protection scope of the present invention. In addition, it should be understood that the schematic drawings are not drawn to actual scale. The flowcharts used in the present invention show the operations implemented according to some embodiments of the present invention. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present invention.

[0037] In addition, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the present invention claimed, but only represents the selected embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0038] It should be noted that the term "including" will be used in the embodiments of the present invention to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.

[0039] Figure 2 This is a schematic diagram of the scenario for three-dimensional semi-supervised segmentation of an intracoronary stent provided by an embodiment of the present invention. The method implementation system includes an IV-OCT imaging system and a supporting data processing device. In the specific implementation process, the IV-OCT system performs three-dimensional scanning on the target vessel area of patients who have undergone percutaneous coronary intervention (PCI) to generate an original IV-OCT image sequence containing the stent structure. This image data is input into a computing device configured with a three-dimensional semi-supervised segmentation deep learning network model through a digital transmission interface. After the algorithm performs three-dimensional spatial feature extraction and collaborative segmentation processing on the continuous tomographic images, a three-dimensional lesion area reconstruction model with anatomical significance is finally output. In particular, the three-dimensional semi-supervised segmentation deep learning network model effectively improves the recognition accuracy of the stent edge and apposition state through a collaborative training mechanism that combines labeled samples and unlabeled samples.

[0040] The IV-OCT system collaboratively realizes tomographic imaging of biological tissues through the following components: The laser light source module generates a broadband optical signal, which is split by an optical fiber coupler into a sample arm and a reference arm; a rotating and retracting catheter is configured at the end of the sample arm, and intravascular spiral scanning is realized through motor drive; the reference arm is provided with an adjustable mirror to form a Michelson interference structure; after the two reflected lights interfere at the coupler, the interference signal is collected by a spectral detector and transmitted to a computer system, and after being processed by a Fourier transform and a gray-scale reconstruction algorithm, the original OCT image data containing the stent structure is generated.

[0041] Figure 3 This is a schematic flowchart of a method for three-dimensional semi-supervised segmentation of an intracoronary stent provided by an embodiment of the present invention. As Figure 3 shown, the method of the present invention is deployed in a computing device with image processing capabilities. The computing device includes, but is not limited to, a desktop computer, a portable terminal device, or a cloud server system, and specifically includes the following processing steps:

[0042] S101. Obtain the initial image data output by the intravascular optical coherence tomography system as the image to be segmented, and input the image to be segmented into a pre-trained three-dimensional semi-supervised segmentation deep learning network model; wherein, the three-dimensional semi-supervised segmentation deep learning network model includes a shared encoder, a BiFormer module, and a decoding module connected in sequence, and the decoding module includes multiple decoders using different attention mechanisms; the three-dimensional semi-supervised segmentation deep learning network model can perform three-dimensional voxel-level segmentation on the image to be segmented.

[0043] S102. Take the average of the result prediction probabilities output by each decoder of the decoding module to obtain the final prediction result, and achieve the three-dimensional segmentation of the intracoronary stent.

[0044] S103. Perform three-dimensional visualization reconstruction based on the three-dimensional segmentation result, and the three-dimensional visualization reconstruction result is used to evaluate the morphological characteristics of the stent in the coronary artery.

[0045] During the training process of the three-dimensional semi-supervised segmentation deep learning network model, the BatchSize of the model input data is set to at least 2, and each group of input data contains labeled data and unlabeled data; for the unlabeled data, the cross-pseudo-supervised training strategy is used to perform pseudo-label mutual supervision on multiple prediction output results with perceptual biases generated by the decoding module to obtain the unsupervised loss; the unsupervised loss is combined with the supervised loss of the prediction result of the labeled data and the true label to jointly optimize the model parameters to obtain the three-dimensional semi-supervised segmentation deep learning model.

[0046] This pre-trained three-dimensional semi-supervised stent segmentation network model can still achieve high-precision and automated three-dimensional optical coherence tomography image segmentation of intracoronary stents under the condition of only having a small amount of limited labeled data, so as to obtain high-quality and automatically labeled three-dimensional segmentation results of intracoronary stents.

[0047] The following embodiments will detail the specific structure and training method of the three-dimensional semi-supervised segmentation vascular stent network model.

[0048] Figure 1 It is a schematic diagram of the structure of the three-dimensional semi-supervised segmentation deep learning network model provided by the embodiment of the present invention. As shown in the figure, this model is based on the BiFormer architecture, multi-attention mechanism encoder, and cross-pseudo-supervised training strategy.

[0049] The three-dimensional semi-supervised segmentation deep learning network model of this specific implementation includes: a shared feature encoder, a BiFormer attention module, and a decoder module. The shared encoder includes a three-dimensional convolution module, a custom downsampling layer, and a Dropout layer, and is used to extract multi-scale features of the input image. The shared encoder receives the initial image output by the intravascular optical coherence tomography imaging system and performs downsampling processing on the image, and transmits the feature map after the downsampling processing to the BiFormer module at the backend of the encoder.

[0050] The BiFormer module adopts a two-layer routing attention mechanism, including a regional-level attention branch and a fine-grained attention branch: the regional-level branch captures long-range dependence relationships by dividing feature regions and calculating the relevance between regions; the fine-grained branch performs local feature refinement within the key regions to achieve efficient extraction of detailed information.

[0051] The multi-branch decoder in this specific implementation includes three branches, namely the channel attention mechanism, the spatial attention mechanism, and the hybrid attention mechanism: The channel attention enhances the key feature expression ability by dynamically adjusting the feature channel weights; the spatial attention focuses on the spatial position information of the feature map to improve the target localization accuracy; the hybrid attention collaboratively optimizes the features in the channel and spatial dimensions to achieve a more comprehensive feature expression. The BiFormer module receives the feature map transmitted by the shared encoder, extracts the global context information in the feature map, and inputs the feature map after extracting the information into multiple independent decoders using different attention mechanisms. It should be understood that the multi-branch decoder in this specific implementation uses three branches, and these three branches can also use other attention mechanisms, such as common attention mechanisms like the temporal attention mechanism and the memory attention mechanism, and the number of branches is not necessarily 3, and can also be 4, 5, etc., but the preferred number of branches is 3.

[0052] For unlabeled data, the three-dimensional semi-supervised stent segmentation network model generates three pseudo-labels corresponding to their respective decoders, and uses the cross-pseudo-supervision training strategy to train the three-dimensional semi-supervised stent segmentation network model according to the predicted output results with perceptual biases generated by the three decoders. The output results are regarded as three pseudo-labels, and each pseudo-label serves as the supervision signal for the other two decoders. Under the supervision of the pseudo-labels and together with the supervision of the labeled data, the model continuously calculates the loss value of the model and continuously optimizes the parameters and weights of the model. Finally, the output with a higher confidence after cross-pseudo-supervision training is selected as the final segmentation result.

[0053] The three-dimensional semi-supervised stent segmentation network model achieves high-precision segmentation in the following ways: 1) The three decoders generate prediction results with perceptual differences based on different attention mechanisms; 2) Use the cross-pseudo-supervision strategy to generate pseudo-labels for the prediction results and use them as the supervision signals for other decoders; 3) Gradually reduce the perceptual bias between the models through iterative optimization, and finally select the prediction result with the highest confidence as the three-dimensional stent segmentation output. This technical solution significantly improves the segmentation accuracy and robustness in complex scenarios while ensuring the computational efficiency.

[0054] Figure 4 This is a schematic flowchart of the construction process of the dataset for training the three-dimensional semi-supervised segmentation deep learning network model provided by the embodiments of the present invention. As Figure 4 shown, the specific steps for constructing the sample dataset include:

[0055] S201. Use an IV-OCT imaging system to perform three-dimensional scanning on the coronary artery samples of PCI postoperative patients to generate an original OCT image sequence containing stent structures.

[0056] S202. Respectively take each initial image slice of the original OCT image sequence as the initial image slice in the image pair, and respectively combine it with the reference annotation image of the stent area manually marked to form an image pair. Specifically, for the image slices in the initial OCT image sequence of each sample, select the target segmentation area of the vascular stent for manual annotation: the clinician accurately outlines the stent contour in the image to generate a reference annotation image that is spatially aligned with the initial image. Each initial OCT image and its corresponding reference annotation image form an independent data pair, and multiple tomographic images of a single sample can generate the corresponding number of data pairs.

[0057] S203. Preprocess each combined image pair to generate an initial coronary artery stent sample data set. Execute the standard preprocessing process on the data set to complete data augmentation. The preprocessing process includes image size normalization, isotropic cropping, random rotation, and contrast transformation. After data preprocessing, divide the complete data set into a training set, a test set, and a validation set at a ratio of 8:1:1 at the case level. Exemplarily, when there are 100 cases of patient data, 80 cases are used for model training, 10 cases are used for model testing, and the remaining 10 cases are used to verify the clinical applicability of the model to ensure the scientific nature of data distribution and the reliability of model evaluation.

[0058] The specific implementation process of training the three-dimensional semi-supervised vascular stent segmentation network model based on the sample data set of the present invention includes:

[0059] S301. Set the proportion of labeled samples; specifically, set 10%-50% of the data as labeled data, and the remaining 50-90% as unlabeled data. Set the Batchsize. In this specific implementation, the Batchsize is taken as 4, and the ratio of labeled data to unlabeled data in each Batchsize is 1:3, so as to construct a collaborative training mechanism for labeled and unlabeled data. Input a set of training set sample data into the shared encoder according to the three-dimensional voxel size of 224×224×64.

[0060] S302. The shared encoder extracts the multi-scale spatial features of the input data through the three-dimensional convolutional layer and the downsampling module, and then inputs the feature map into the BiFormer module for feature extraction. The BiFormer module adopts a regional division strategy and synchronously enhances the representation capabilities of global semantic features and local detail features through a double-layer routing attention mechanism.

[0061] S303. Input the features output by the BiFormer module into three decoder branches respectively: the channel attention decoder, the spatial attention decoder, and the hybrid attention decoder. For unlabeled data, the probability maps output by each decoder are normalized by softmax and sharpened to generate pseudo-labels for unsupervised training; for labeled data, the probability maps output by each decoder are normalized by softmax and sharpened for supervised training.

[0062] S304. Use a composite loss function to optimize the model: (1) Calculate the Tversky loss based on the prediction results of unlabeled data and pseudo-labels to optimize the segmentation accuracy of the stent edge; (2) Calculate the Dice loss based on the prediction results of labeled data and true labels to constrain the global segmentation performance. Specifically, in each group of input data, for unlabeled data, take the output results of the three prediction outputs with perceptual biases generated by the three decoders of the decoding module as pseudo-label one, pseudo-label two, and pseudo-label three. Calculate the loss function between pseudo-label one and pseudo-label two, and pseudo-label three respectively to obtain the first loss value and the second loss value. Calculate the loss function between pseudo-label two and pseudo-label one, and three respectively to obtain the third loss value and the fourth loss value. Calculate the loss function between pseudo-label three and pseudo-label one, and pseudo-label two respectively to obtain the fifth loss value and the sixth loss value; take the average of the six loss values as the unsupervised loss in this group of data, and the calculation method of the six loss values all uses the Tversky loss function; for labeled data, calculate the Dice loss values between the output results generated by the three decoders of the decoding module and the true labels respectively, and then take the average as the supervised loss of this group of data.

[0063] Calculate the total loss of this group of input data through the composite loss function, including the Tversky loss calculated based on the prediction results of unlabeled data and pseudo-labels, and the Dice loss calculated based on the prediction results of labeled data and true labels. The formula for the total loss L is as follows:

[0064] L = λ(L D1 + L D2 +... + L Dm ) + μ(L T1 + L T2 +... + L Tn )

[0065] where λ = 0.5, μ = 0.1 are hyperparameters, L Di is the Dice loss value of the labeled data in this group of data, L TjIt is the Tversky loss value of the unlabeled data in this group of data. i ∈ (1, m), j ∈ (1, n), mm is the number of labeled data in this group of data, n is the number of unlabeled data in this group of data, and m + n = batchsize; the network parameters are dynamically adjusted by the stochastic gradient descent algorithm; the model continuously calculates the loss value of the model under the supervision of the pseudo-labels and continuously optimizes the parameters and weights of the model. Finally, the output with a higher confidence after cross-pseudo-supervised training is selected as the final segmentation result.

[0066] S305. Repeat steps S301 - S304 for iterative training. After each epoch, evaluate the model performance using the test set. When the Dice coefficient of the validation set reaches the preset threshold, save the current optimal network weight parameters to obtain the final three-dimensional semi-supervised vascular stent segmentation network model.

[0067] Figure 5 It is a detailed process schematic diagram of the three-dimensional semi-supervised segmentation method for coronary artery stents provided by an embodiment of the present invention. An IV-OCT image dataset of patients who have received PCI treatment and a small amount of manual annotations are obtained using an intravascular optical coherence tomography system and preprocessed to obtain a dataset. The dataset is input into a three-dimensional semi-supervised stent segmentation deep learning network for training, and three prediction output results with perceptual biases are generated during training. For unlabeled data, the cross-pseudo-supervised training strategy is used to perform pseudo-label mutual supervision on multiple prediction output results with perceptual biases generated by the decoding module to obtain an unsupervised loss; the unsupervised loss is combined with the supervised loss of the prediction result of the labeled data and the true label to jointly optimize the model parameters to obtain a three-dimensional semi-supervised segmentation deep learning model. The model continuously calculates the loss value of the model in the semi-supervised mechanism and continuously optimizes the parameters and weights of the model through backpropagation. Finally, the output with a higher confidence after training is selected as the final desirable model weight.

[0068] Figure 6 It is a schematic diagram of three-dimensional visualization reconstruction of a stent provided by an embodiment of the present invention. The segmentation result of the stent is obtained using the three-dimensional semi-supervised segmentation method. The stent is combined with the vascular contour of the original data for three-dimensional visualization reconstruction, and the three-dimensional visualization reconstruction result shows the morphological characteristics of the stent.

[0069] Figure 7 It shows a structural block diagram of an electronic device for implementing the method. The electronic device includes a processor 101, a memory 102, and a data bus 103 connecting each component. The memory stores executable instructions. When the device runs, the processor calls the instructions through the bus to execute the entire processing flow of the three-dimensional semi-supervised segmentation method.

[0070] An embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the instructions are executed by a processor, the entire processing flow of the three-dimensional semi-supervised segmentation method described in the foregoing embodiment is implemented.

[0071] It should be understood that those skilled in the art can directly derive the implementation manners of related systems and devices based on the teachings of the method embodiments of the present invention. The division of each functional module in the system embodiments is only based on logical function requirements. In actual implementation, different architectures can be adopted, including but not limited to: multiple functional modules integrated into a single processing unit, each module independently deployed as a physical entity, or some modules combined to form a new functional unit. The communication connection between the modules can be realized through electrical interfaces, mechanical interfaces or other forms of communication protocols, and can be flexibly selected according to the system design requirements during specific implementation.

[0072] Furthermore, when the technical solution is implemented in software form, it can be stored in a non-volatile storage medium as an independent computer program product, including program codes that enable a computer device to execute the steps described in the claims of the method of the present invention. The storage medium includes but is not limited to: USB flash drives, solid-state drives, read-only memories (ROM), programmable read-only memories (PROM), erasable programmable read-only memories (EPROM), random access memories (RAM), disk memories, and optical media, etc. The computer device covers electronic devices with data processing capabilities such as personal computers, medical imaging servers, and cloud computing platforms. By loading and executing the instructions in the storage medium, the automated processing of three-dimensional segmentation of coronary stents is realized.

[0073] The above are only specific implementation manners of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all should be covered by the protection scope of the present invention.

Claims

1. A three-dimensional semi-supervised segmentation method for intracoronary stents, characterized in that The method includes: Inputting the initial image to be segmented into a pre-trained 3D semi-supervised segmentation deep learning network model; wherein, the 3D semi-supervised segmentation deep learning network model includes a shared encoder, a BiFormer module, and a decoding module connected in sequence, and the decoding module includes multiple decoders using different attention mechanisms; Taking the average value of the predicted probabilities of the results output by each decoder in the decoding module to obtain a prediction result, and realizing the 3D segmentation of the stent in the coronary artery; During the training process of the 3D semi-supervised segmentation deep learning network model, the BatchSize of the model input data is set to at least 2, and each group of input data contains labeled data and unlabeled data; for the unlabeled data, the cross pseudo-supervision training strategy is used to perform pseudo-label mutual supervision on multiple predicted output results with perceptual bias generated by the decoding module to obtain an unsupervised loss; the unsupervised loss is combined with the supervised loss of the predicted result of the labeled data and the true label to jointly optimize the model parameters to obtain a 3D semi-supervised segmentation deep learning model.

2. The three-dimensional semi-supervised segmentation method for an intracoronary stent according to claim 1, wherein The shared encoder includes 5 downsampling layers, and each layer includes a 3D convolutional block with a stride of 2, a downsampling module, and a Dropout layer.

3. A three-dimensional semi-supervised segmentation method for an intracoronary stent according to claim 1, characterized in that The BiFormer module is composed of a double-layer routing attention mechanism.

4. A three-dimensional semi-supervised segmentation method for an intracoronary stent according to claim 1, characterized in that The decoding module includes three decoders using different attention mechanisms, including a decoder based on channel attention mechanism, a decoder based on spatial attention mechanism, and a decoder based on hybrid attention mechanism. The three decoders with different attention mechanisms use the softmax activation function to normalize and sharpen their respective output results to generate three pseudo-labels corresponding to their respective decoders.

5. A three-dimensional semi-supervised segmentation method for an intracoronary stent according to claim 4, characterized in that, For the unlabeled data, the cross pseudo-supervision training strategy is used to perform pseudo-label mutual supervision on multiple predicted output results with perceptual bias generated by the decoding module to obtain an unsupervised loss; the unsupervised loss is combined with the supervised loss of the predicted result of the labeled data and the true label to jointly optimize the model parameters to obtain a 3D semi-supervised segmentation deep learning model. Specifically: In each group of input data, for the unlabeled data, the output results of the three predicted output results with perceptual bias generated by the three decoders of the decoding module are used as pseudo-label one, pseudo-label two, and pseudo-label three. Pseudo-label one calculates the loss function with pseudo-label two and pseudo-label three respectively to obtain the first loss value and the second loss value. Pseudo-label two calculates the loss function with pseudo-label one and three respectively to obtain the third loss value and the fourth loss value. Pseudo-label three calculates the loss function with pseudo-label one and pseudo-label two respectively to obtain the fifth loss value and the sixth loss value; taking the average value of the six loss values as the unsupervised loss in this group of data, and the calculation method of the six loss values all uses the Tversky loss function; for the labeled data, the output results generated by the three decoders of the decoding module are respectively calculated with the true label to obtain the Dice loss value and then take the average value as the supervised loss in this group of data; Calculate the total loss of this group of input data through a composite loss function, including the Tversky loss calculated based on the prediction results of unlabeled data and pseudo-labels, and the Dice loss calculated based on the prediction results of labeled data and true labels. The formula for the total loss L is as follows: L = λ(L D1 + L D2 +... + L Dm ) + μ(L T1 + L T2 +... + L Tn ) Among them, λ = 0.5 and μ = 0.1 are hyperparameters, and L Di is the Dice loss value of the labeled data in this group of data, and L Tj is the Tversky loss value of the unlabeled data in this group of data. i ∈ (1, m), j ∈ (1, n), m is the number of labeled data in this group of data, n is the number of unlabeled data in this group of data, and m + n = batchsize; the network parameters are dynamically adjusted by the stochastic gradient descent algorithm; the model continuously calculates the loss value of the model and continuously optimizes the parameters and weights of the model under the supervision of the pseudo-labels, and finally selects the output with a higher confidence after cross-pseudo-supervision training as the final segmentation result.

6. The three-dimensional semi-supervised segmentation method for an intracoronary stent according to claim 1, characterized in that It also includes: Perform three-dimensional visualization reconstruction based on the three-dimensional segmentation result, where the three-dimensional visualization reconstruction result is used to evaluate the morphological characteristics of the stent in the coronary artery.

7. A three-dimensional semi-supervised segmentation device for an intracoronary stent, characterized in that The device includes: An imaging module for performing intravascular optical coherence tomography to obtain an image of the coronary artery; A stent recognition module for using a pre-trained three-dimensional semi-supervised segmentation deep learning network model to recognize the image of the coronary artery and obtain a stent recognition result; A three-dimensional modeling module for performing three-dimensional visualization reconstruction based on the stent recognition result to facilitate the user to evaluate the morphological characteristics of the stent in the coronary artery; Among them, the three-dimensional semi-supervised segmentation deep learning network model includes a shared encoder, a BiFormer module, and a decoding module connected in sequence, and the decoding module includes multiple decoders using different attention mechanisms.

Images