Method, device and electronic device for segmenting fibrous plaque in IVUS images

Abstract

This invention proposes a method, apparatus, and electronic device for fibrous plaque segmentation in IVUS images, belonging to the field of medical imaging technology, specifically involving the processing technology of intravascular ultrasound images. It solves the problems existing in current IVUS image fibrous plaque segmentation techniques: when using manual segmentation, the workload is enormous, subjectivity is high, the results are inconsistent, reproducibility is poor, it is easily influenced by the operator's experience, and it is prone to missed or misdiagnosis; and when using segmentation methods based on traditional machine learning or general neural networks, the segmentation results are not accurate enough and are prone to false positives. The proposed IVUS image fibrous plaque segmentation method uses a trained continuous-frame fibrous plaque segmentation network to process the dataset to be segmented to obtain fibrous plaque segmentation result images. It is mainly used for segmenting fibrous plaques in IVUS images and for providing reliable diagnostic evidence for the diagnosis of vascular plaque diseases.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, specifically to the processing technology of intravascular ultrasound images. Background Technology

[0002] Atherosclerosis (AS) is a systemic chronic inflammatory disease that begins in childhood. Before plaque rupture, it can undergo a long silent phase, making early screening of high-risk populations difficult. This often results in excessive and unnecessary screening of low-risk individuals, while many patients are already in advanced stages of the disease at their first diagnosis, such as the end-stage of coronary artery disease—ischemic cardiomyopathy and heart failure. Even when AS plaques are detected, determining whether intervention is necessary and whether the plaque is a vulnerable plaque prone to rupture becomes a challenge in clinical practice. Consequently, many patients with stable plaques receive unnecessary interventional stent treatment.

[0003] One commonly used interventional imaging technique for analyzing vascular morphology and detecting intravascular fibrous plaques (such as atherosclerotic plaques) is intravascular ultrasound (IVUS). IVUS utilizes a tiny ultrasound probe integrated into the tip of a catheter to obtain cross-sectional images of the blood vessel lumen. These images, known as IVUS images, can fully display a cross-section of the blood vessel, including the boundaries between the adventitia and media, the intima (lumen), and the plaque load between them. This allows for the acquisition of various information, including lumen size, plaque location, and plaque morphology. Using IVUS to examine the cardiovascular system can help detect cardiovascular lesions and reduce patient deaths caused by cardiovascular diseases.

[0004] Currently, the identification and segmentation (or classification) of plaques in IVUS images are mainly done manually. Due to the low signal-to-noise ratio of IVUS images and the diverse characteristics of intravascular fibrous plaques, manual identification and segmentation of plaques by humans (such as doctors) is not only extremely labor-intensive, but also results in highly subjective and unrepeatable outcomes. It is easily influenced by the operator's clinical experience and professional knowledge, which can easily lead to missed diagnoses and misdiagnoses.

[0005] Therefore, the automatic detection and segmentation of plaque tissue in IVUS images has significant clinical application value.

[0006] One existing method for fibrous plaque segmentation is based on traditional machine learning. However, due to the complex morphology and structure of fibrous plaques, their similarity to blood texture features, and the blurred boundaries between them, traditional machine learning-based segmentation methods cannot achieve ideal results, and the segmentation results are not accurate enough.

[0007] Meanwhile, with the development of neural network technology, numerous techniques have emerged to assist doctors in medical diagnosis. However, methods for segmenting fibrous plaques in IVUS images based on existing general-purpose neural networks, while performing similarly to traditional machine learning-based segmentation methods, still yield inaccurate results and are prone to false positives.

[0008] Currently, there is no dedicated specialized neural network for fiber patch segmentation in IVUS images. Summary of the Invention

[0009] This invention proposes a method, apparatus, and electronic device for segmenting fibrous plaques in IVUS images, which solves the problems existing in the current fibrous plaque segmentation technology in IVUS images: when using manual segmentation, the workload is huge, the subjectivity is strong, the consistency of results is low, the repeatability is poor, it is easily affected by the operator's experience, and it is easy to miss or misdiagnose; and when using segmentation methods based on traditional machine learning or general neural networks, the segmentation results are not accurate enough and false positive results are easy to occur.

[0010] The method for segmenting fiber patches in IVUS images according to the present invention has the following technical solution:

[0011] The method specifically includes:

[0012] S1. Obtain the set of original IVUS images to be segmented;

[0013] S2. Preprocess the original IVUS image set to obtain a dataset to be segmented, consisting of IVUS images;

[0014] S3. Obtain the trained continuous frame fiber patch segmentation network; the trained continuous frame fiber patch segmentation network is obtained by training using transfer learning; the trained continuous frame fiber patch segmentation network includes a module for enhancing features in the IVUS image that are strongly correlated with fiber patch information, and a module for improving the IVUS image's attention to fiber patch information.

[0015] S4. Using the trained continuous frame fiber patch segmentation network, process the dataset to be segmented to obtain the fiber patch segmentation result map.

[0016] Furthermore, a preferred embodiment is provided, in which the trained continuous-frame fiber patch segmentation network is used to process the dataset to be segmented to obtain a fiber patch segmentation result map, specifically:

[0017] S4.1 The trained continuous frame fiber patch segmentation network sequentially extracts one IVUS image from the dataset to be segmented.

[0018] S4.2 After each capture, the current frame IVUS image and the previous frame IVUS image are used together as the input of the trained continuous frame fiber patch segmentation network; if the current frame IVUS image is the first frame image, then the current frame IVUS image is used as its previous frame IVUS image, and together they are used as the input of the trained continuous frame fiber patch segmentation network.

[0019] S4.3. Encoding operations are performed on the current frame IVUS image and the previous frame IVUS image respectively to obtain multi-level current frame encoded feature maps and previous frame encoded feature maps; wherein, during the encoding operation, the trained continuous frame fiber patch segmentation network is used to obtain the correlation weight between each position point in the current frame IVUS image and the corresponding position point in the previous frame IVUS image, and according to the correlation weight, the position features with strong correlation in the multi-level current frame encoded feature maps are enhanced, and the position features with weak correlation in the multi-level current frame encoded feature maps are suppressed.

[0020] S4.4 Perform multi-level decoding operation on the multi-level current frame encoded feature map to obtain a multi-level decoded feature map; wherein, during the decoding operation, the trained continuous frame fiber patch segmentation network is used to obtain the attention weight of each channel in the current frame IVUS image, and according to the attention weight, enhance the channels with strong attention in the multi-level current frame encoded feature map and suppress the channels with weak attention in the multi-level current frame encoded feature map.

[0021] S4.5. Based on the feature map obtained from the last decoding in the multi-level decoding feature map, obtain the fiber patch segmentation result map.

[0022] Furthermore, in a preferred embodiment, step S2 involves preprocessing the original IVUS image set, specifically:

[0023] S2.1, Perform image augmentation on the original IVUS image set;

[0024] S2.2. Perform center cropping on each IVUS original image in the set of original IVUS images after image amplification, and crop out the surrounding low signal and featureless regions to obtain the IVUS image;

[0025] S2.3. Combine all IVUS images into a set as the dataset to be segmented.

[0026] Furthermore, in a preferred embodiment, in step S3, the trained continuous frame fiber patch segmentation network is obtained using a transfer learning training method, specifically:

[0027] S3.1 Construct a single-frame fiber patch segmentation network;

[0028] S3.2. Using the training dataset, train the constructed single-frame fiber patch segmentation network to obtain pre-trained weights; wherein, the training dataset is obtained in the following manner:

[0029] Obtain the original IVUS image set for training; preprocess the original IVUS image set for training to obtain a continuous sequence consisting of at least one IVUS image arranged in chronological order, which serves as the training dataset;

[0030] S3.3 Construct a continuous frame fiber patch segmentation network; the continuous frame fiber patch segmentation network includes the network structure of the single frame fiber patch segmentation network;

[0031] During the construction process, the pre-trained weights are used as the initial weights of the continuous frame fiber patch segmentation network; and the continuous frame fiber patch segmentation network includes the first current frame coding module RB1 and the previous frame coding module RB. * 1. The first current frame encoding module RB1 and the previous frame encoding module RB * 1. Shared parameters;

[0032] S3.4. Using the training dataset, train the constructed continuous frame fiber patch segmentation network to obtain the trained continuous frame fiber patch segmentation network.

[0033] Furthermore, a preferred embodiment is provided, wherein the continuous frame fiber patch segmentation network includes: n cascaded current frame coding modules and previous frame coding modules (RBs). * 1. End Encoding Module RB n+1 The system consists of a correlation weight acquisition module (NCC), n-1 cascaded downsampling modules, n cascaded decoding modules, n attention weight acquisition modules (ECA), and an output module (OP); where n = 4.

[0034] The i-th current frame coding module RB in the n cascaded current frame coding modules i This is used to encode the result of multiplying the input (i-1)th level current frame encoded feature map with the (i-1)th level correlation weight map, and output the i-th level current frame encoded feature map; where i∈[1,n], and when i=1, the 0th level current frame encoded feature map is the current frame IVUS image; the (i-1)th level correlation weight map is obtained by the correlation weight acquisition module NCC and n-1 cascaded downsampling modules;

[0035] The previous frame encoding module RB *1. Used to encode the previous frame IVUS image as input and output the encoded feature map of the previous frame;

[0036] The end encoding module RB n+1 This is used to encode the result of multiplying the input nth level current frame encoded feature map with the nth level correlation weight map, and output the (n+1)th level current frame encoded feature map;

[0037] The input to the correlation weight acquisition module NCC is the first-level current frame coding feature map and the previous frame coding feature map, and its output is the first-level correlation weight map.

[0038] The j-th downsampling module Ds in the n-1 cascaded downsampling modules j , is used to downsample the input j-th level correlation weight map and output the (j+1)-th level correlation weight map, where j∈[1, n-1];

[0039] The i-th decoding module Up in the n cascaded decoding modules i This is used to perform decoding operations on the input i-th level attention weight map and the (i+1)-th level current frame decoding feature map, and output the i-th level current frame decoding feature map; wherein, when i=n, ​​the (n+1)-th level current frame decoding feature map is the (n+1)-th level current frame encoding feature map; the i-th level attention weight map is obtained by the n attention weight acquisition modules ECA;

[0040] The i-th attention weight acquisition module among the n attention weight acquisition modules ECA i The input is the i-th level current frame encoded feature map, and the output is the i-th level attention weight map;

[0041] The input to the output module OP is the first-level current frame decoding feature map, and the output is the fiber patch segmentation result map.

[0042] Furthermore, a preferred embodiment is provided, wherein the processing procedure of the correlation weight acquisition module NCC is as follows:

[0043] The correlation weight acquisition module NCC takes the first-level current frame encoded feature map α obtained by the first current frame encoding module RB1, and performs spatial flattening processing on it to obtain a two-dimensional vector corresponding to the current frame; similarly, it takes the feature map α obtained by the previous frame encoding module RB1... * The encoded feature map β of the previous frame is obtained by spatial flattening to obtain a two-dimensional vector corresponding to the previous frame.

[0044] The feature size of the first-level current frame encoded feature map α and the previous frame encoded feature map β is C1×H1×W1; where C1 represents the number of feature channels, H1 represents the feature map height, and W1 represents the feature map width.

[0045] The feature size of both two two-dimensional vectors is C1×(H1×W1); each two-dimensional vector consists of H1×W1 column vectors of C1×1 arranged sequentially after being flattened in space;

[0046] Using the NCC calculation formula, along the width direction of the two two-dimensional vectors, the normalized cross-correlation coefficients of the two C1×1 column vectors at corresponding positions are obtained, resulting in a 1×(H1×W1) one-dimensional row vector.

[0047] The NCC calculation formula is as follows:

[0048]

[0049] Where f1(x,y) represents the first image being compared; f2(x,y) represents the second image being compared; μ1 represents the average value of the first image; μ2 represents the average value of the first image; σ1 represents the standard deviation of the first image; σ2 represents the standard deviation of the first image; the first image refers to the image represented by the C1×1 column vector at the corresponding position in the two-dimensional vector corresponding to the current frame; the second image refers to the image represented by the C1×1 column vector at the corresponding position in the two-dimensional vector corresponding to the previous frame;

[0050] The sigmoid function is used to normalize the data in the 1×(H1×W1) one-dimensional row vector to [0, 1];

[0051] The normalized 1×(H1×W1) one-dimensional row vector is recombined to obtain a two-dimensional feature vector φ with feature size H1×W1; the two-dimensional feature vector φ is the first-level correlation weight map.

[0052] Furthermore, a preferred embodiment is provided, wherein the i-th attention weight acquisition module ECA i The processing procedure is as follows:

[0053] Obtain the i-th level current frame encoded feature map F, with C channels. i ;

[0054] For the i-th level current frame encoded feature map F, perform global average pooling along its channel dimension to obtain a feature map with C i There are 1×1 feature vectors M with each channel having a feature size of 1×1.

[0055] Based on the number of channels C of the feature vector M iThe size k of the adaptive convolution kernel is obtained using the following formula:

[0056]

[0057] Where γ is 2 and b is 1;

[0058] The adaptive convolution kernel of size k is applied to the channel dimension of the feature vector M to obtain the output N;

[0059] The resulting output N is activated using the sigmoid function to obtain the i-th level attention weight map; the i-th level attention weight map contains the attention weight S for each channel.

[0060] Furthermore, a preferred embodiment is provided, wherein the n cascaded current frame coding modules and previous frame coding modules RB * 1 and the end encoding module RB n+1 Each encoding module consists of at least two stacked encoding sub-modules; where:

[0061] The first current frame encoding module RB1, the second current frame encoding module RB2, and the previous frame encoding module RB * 1. Each is composed of two stacked encoding sub-modules;

[0062] The third current frame encoding module RB3 and the fourth current frame encoding module RB4 are both composed of three stacked encoding sub-modules;

[0063] The end encoding module RB n+1 It consists of 4 stacked encoding sub-modules;

[0064] The encoding submodule includes: two 3×3 convolutions, a BN layer, a ReLU activation layer, and an output; the output of the encoding submodule is further enhanced with a Dropout regularization.

[0065] This invention also proposes a fiber patch segmentation device for IVUS images, the technical solution of which is as follows:

[0066] The device specifically includes:

[0067] The acquisition module is used to acquire the set of original IVUS images to be segmented;

[0068] The preprocessing module is used to preprocess the original IVUS image set to obtain a dataset to be segmented, consisting of IVUS images.

[0069] A loading module is used to acquire a trained continuous frame fiber patch segmentation network; the trained continuous frame fiber patch segmentation network is obtained by training using transfer learning; the trained continuous frame fiber patch segmentation network includes a module for enhancing features in the IVUS image that are strongly correlated with fiber patch information, and a module for improving the IVUS image's attention to fiber patch information.

[0070] The segmentation module is used to process the dataset to be segmented using the trained continuous frame fiber patch segmentation network to obtain a fiber patch segmentation result map.

[0071] The present invention also proposes an electronic device, the technical solution of which is as follows:

[0072] An electronic device includes a processor and a memory, the memory storing executable instructions of the processor, the processor being configured to perform the aforementioned fiber patch segmentation method in an IVUS image by executing the executable instructions.

[0073] The present invention has the following beneficial effects:

[0074] 1. The fiber patch segmentation method or apparatus in IVUS images described in this invention enables the network to learn the image information of both the current frame and the previous frame simultaneously by using the current frame and the previous frame IVUS image as input to the network, thus no longer being limited to the features of a single frame IVUS image.

[0075] 2. The IVUS image fiber patch segmentation method or apparatus of the present invention, by employing a module that enhances features in the IVUS image that are strongly correlated with fiber patch information, can enable the obtained feature map to retain the fiber patch information with strong correlation, while suppressing irrelevant false positive patch information.

[0076] 3. The IVUS image fiber patch segmentation method or apparatus of the present invention, by employing a module that enhances the attention of the IVUS image to fiber patch information, can improve the network's attention to the fiber patch region, making the segmentation result more accurate, and suppressing false positive fiber patches.

[0077] The method, apparatus, and electronic device for segmenting fibrous plaques in IVUS images described in this invention are applicable to segmenting fibrous plaques in IVUS images and can be used to provide reliable diagnostic evidence for the diagnosis of vascular plaque diseases. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 A structural diagram of a continuous frame fiber patch segmentation network is shown in a preferred embodiment of the present invention.

[0080] Figure 2 In a preferred embodiment of the present invention, a structural diagram of the correlation weight acquisition module NCC is shown.

[0081] Figure 3 In a preferred embodiment of the present invention, a structural diagram of the attention weight acquisition module ECA is shown.

[0082] Figure 4 A structural diagram of the encoding submodule is shown in a preferred embodiment of the present invention.

[0083] Figure 5 In a preferred embodiment of the present invention, the structure diagram of the pre-trained network of the single-frame fiber patch segmentation network is shown. Detailed Implementation

[0084] To make the technical solutions and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Implementation Method 1: Combination Figure 1 -5. This embodiment describes a method for segmenting fiber patches in IVUS images. The specific implementation details are as follows:

[0086] The method specifically includes:

[0087] S1. Obtain the set of original IVUS images to be segmented;

[0088] S2. Preprocess the original IVUS image set to obtain a dataset to be segmented, consisting of IVUS images;

[0089] S3. Obtain the trained continuous frame fiber patch segmentation network; the trained continuous frame fiber patch segmentation network is obtained by training using transfer learning; the trained continuous frame fiber patch segmentation network includes a module for enhancing features in the IVUS image that are strongly correlated with fiber patch information, and a module for improving the IVUS image's attention to fiber patch information.

[0090] S4. Using the trained continuous frame fiber patch segmentation network, process the dataset to be segmented to obtain the fiber patch segmentation result map.

[0091] In this embodiment, the input to the continuous frame fiber patch segmentation network is the current frame IVUS image to be segmented and its previous frame IVUS image. The continuous frame fiber patch segmentation network sequentially captures one IVUS image from the dataset (training, testing, or the dataset to be segmented), with the currently captured IVUS image being the current frame IVUS image, and the previously captured IVUS image being the previous frame IVUS image. If the previous frame IVUS image for the current frame IVUS image does not exist, meaning this is the first time the continuous frame fiber patch segmentation network is capturing an IVUS image from the dataset (training, testing, or the dataset to be segmented), then the first captured current frame IVUS image, i.e., the first frame IVUS image, is used as its previous frame IVUS image and input into the continuous frame fiber patch segmentation network.

[0092] Implementation Method Two: Combination Figure 1 -5. This embodiment further defines the fiber patch segmentation method in IVUS images described in Embodiment 1. The specific implementation details are as follows:

[0093] The trained continuous-frame fiber patch segmentation network is used to process the dataset to be segmented to obtain fiber patch segmentation result images, specifically:

[0094] S4.1 The trained continuous frame fiber patch segmentation network sequentially extracts one IVUS image from the dataset to be segmented.

[0095] S4.2 After this capture, the current frame IVUS image and the previous frame IVUS image are used together as the input of the trained continuous frame fiber patch segmentation network; if the current frame IVUS image is the first frame image, then the current frame IVUS image is used as its previous frame IVUS image, and together they are used as the input of the trained continuous frame fiber patch segmentation network.

[0096] S4.3. Encoding operations are performed on the current frame IVUS image and the previous frame IVUS image respectively to obtain multi-level current frame encoded feature maps and previous frame encoded feature maps; wherein, during the encoding operation, the trained continuous frame fiber patch segmentation network is used to obtain the correlation weight between each position point in the current frame IVUS image and the corresponding position point in the previous frame IVUS image, and according to the correlation weight, the position features with strong correlation in the multi-level current frame encoded feature maps are enhanced, and the position features with weak correlation in the multi-level current frame encoded feature maps are suppressed.

[0097] S4.4 Perform multi-level decoding operation on the multi-level current frame encoded feature map to obtain a multi-level decoded feature map; wherein, during the decoding operation, the trained continuous frame fiber patch segmentation network is used to obtain the attention weight of each channel in the current frame IVUS image, and according to the attention weight, enhance the channels with strong attention in the multi-level current frame encoded feature map and suppress the channels with weak attention in the multi-level current frame encoded feature map.

[0098] S4.5. Based on the feature map obtained from the last decoding in the multi-level decoding feature map, obtain the fiber patch segmentation result map.

[0099] In this embodiment, the trained continuous frame fiber patch segmentation network sequentially captures one IVUS image from the dataset to be segmented; wherein, the IVUS image captured this time is the current frame IVUS image, and the IVUS image captured previously is the previous frame IVUS image.

[0100] In this embodiment, to prevent data leakage during deep learning, the dataset to be segmented and the pre-set training dataset are obtained from sets of original IVUS images from different individuals. When training the continuous frame fiber patch segmentation network, the input images required by the network are extracted from the training dataset. In a further embodiment, the training process also includes a testing step, where the input images are extracted from the test dataset. Whether it's the training dataset, the test dataset, or the dataset to be segmented, the method for obtaining them is the same: preprocessing the corresponding sets of original IVUS images.

[0101] In this embodiment, by obtaining the correlation weight between each position point in the current frame IVUS image and the corresponding position point in the previous frame IVUS image during the encoding operation, and thereby enhancing the strongly correlated position features in the encoded feature map while suppressing the weakly correlated position features in the encoded feature map, the resulting feature map can retain the strongly correlated fiber plaque information while suppressing irrelevant false positive plaque information.

[0102] In this embodiment, by obtaining the attention weight of each channel in the current frame IVUS image during the decoding operation, and enhancing the channels with strong attention (contributing greatly to accurate segmentation) in the encoded feature map and suppressing the channels with weak attention (contributing little to accurate segmentation) in the encoded feature map according to the attention weight, the network's attention to the fiber patch region can be improved, making the segmentation result more accurate and suppressing false positive fiber patches.

[0103] Implementation Method 3: Combination Figure 1 -5. This embodiment further defines step S2 in the fiber patch segmentation method for IVUS images described in Embodiment 1. The specific implementation details are as follows:

[0104] In step S2, the original IVUS image set is preprocessed, specifically:

[0105] S2.1, Perform image augmentation on the original IVUS image set;

[0106] S2.2. For each IVUS original image in the set of original IVUS images after image amplification, perform center cropping to remove the surrounding low-signal, featureless regions to obtain the IVUS image.

[0107] S2.3. Combine all IVUS images into a set as the dataset to be segmented.

[0108] In this embodiment, image augmentation of the original image set not only increases the network's generalization ability but also reduces overfitting. The specific image augmentation method is as follows:

[0109] (1) Spatial flipping method: flip the original IVUS image horizontally or vertically; or rotate the original IVUS image at a random angle; or flip or rotate the position of the pixels in the original IVUS image in space.

[0110] (2) Brightness Modification Method: Randomly enhance or darken the brightness of the original IVUS image. This method allows the network to learn the features of the same image under different brightness levels by changing the overall brightness of the image.

[0111] (3) Image cropping method: A non-fixed-size black mask is randomly generated on the original IVUS image to block a part of the original IVUS image. This allows the network to learn all the features even when the image is incomplete, thus enhancing the network's feature extraction ability.

[0112] (4) Image translation method: The original IVUS image is randomly translated in four directions: upper left, lower left, upper right, or lower right, and the translation distance is uncertain. This method changes the spatial position of the pixels, which may make the image not fully displayed. It plays a role similar to the image cropping method, which can enable the network to learn all the features when the image is incomplete, thereby enhancing the feature extraction ability of the network.

[0113] In this embodiment, the labels of the original IVUS images also need to be adjusted accordingly as the original IVUS images change. Amplifying each original IVUS image using the above method can increase the number of images in the original IVUS image set by 10 times, thus greatly enriching the features of the images in the image set and increasing the knowledge that the network can learn.

[0114] In this embodiment, the original IVUS image is center-cropped to remove low-signal, featureless regions surrounding the original IVUS image. The original IVUS image has a size of 512×512 pixels, and the center-cropped IVUS image has a size of 448×448 pixels. By center-cropping, the computational load of subsequent networks can be reduced.

[0115] Implementation Method 4: Combination Figure 1 -5. This embodiment further defines step S3 in the fiber patch segmentation method for IVUS images described in Embodiment 1. The specific implementation details are as follows:

[0116] In step S3, the trained continuous frame fiber patch segmentation network is obtained using a transfer learning training method, specifically:

[0117] S3.1 Construct a single-frame fiber patch segmentation network;

[0118] S3.2. Using the training dataset, train the constructed single-frame fiber patch segmentation network to obtain pre-trained weights; wherein, the training dataset is obtained in the following manner:

[0119] Obtain the original IVUS image set for training; preprocess the original IVUS image set for training to obtain a continuous sequence consisting of at least one IVUS image arranged in chronological order, which serves as the training dataset;

[0120] S3.3 Construct a continuous frame fiber patch segmentation network; the continuous frame fiber patch segmentation network includes the network structure of the single frame fiber patch segmentation network;

[0121] During the construction process, the pre-trained weights are used as the initial weights of the continuous frame fiber patch segmentation network; and the continuous frame fiber patch segmentation network includes the first current frame coding module RB1 and the previous frame coding module RB. * 1. The first current frame encoding module RB1 and the previous frame encoding module RB * 1. Shared parameters;

[0122] S3.4. Using the training dataset, train the constructed continuous frame fiber patch segmentation network to obtain the trained continuous frame fiber patch segmentation network.

[0123] In this embodiment, the training method based on transfer learning is a learning paradigm of "pre-trained network + fine-tuning". That is, an existing pre-trained network is used for initial learning, and the learning results are transferred to another different but related network as the starting point for formal training.

[0124] In this embodiment, the training dataset is obtained in the same way as the dataset to be segmented, that is, by preprocessing the corresponding original IVUS image set. The preprocessing includes image augmentation and center cropping steps.

[0125] Implementation Method 5: Combination Figure 1 -5. This embodiment further defines the continuous frame fiber patch segmentation network in the IVUS image fiber patch segmentation method described in Embodiment 4. The specific implementation details are as follows:

[0126] The continuous frame fiber patch segmentation network includes: n cascaded current frame coding modules and previous frame coding modules (RBs). * 1. End Encoding Module RB n+1 The system consists of a correlation weight acquisition module (NCC), n-1 cascaded downsampling modules, n cascaded decoding modules, n attention weight acquisition modules (ECA), and an output module (OP); where n = 4.

[0127] The i-th current frame coding module RB in the n cascaded current frame coding modules iThis is used to encode the result of multiplying the input (i-1)th level current frame encoded feature map with the (i-1)th level correlation weight map, and output the i-th level current frame encoded feature map; where i∈[1,n], and when i=1, the 0th level current frame encoded feature map is the current frame IVUS image; the (i-1)th level correlation weight map is obtained by the correlation weight acquisition module NCC and n-1 cascaded downsampling modules;

[0128] The previous frame encoding module RB * 1. Used to encode the previous frame IVUS image as input and output the encoded feature map of the previous frame;

[0129] The end encoding module RB n+1 This is used to encode the result of multiplying the input nth level current frame encoded feature map with the nth level correlation weight map, and output the (n+1)th level current frame encoded feature map;

[0130] The input to the correlation weight acquisition module NCC is the first-level current frame coding feature map and the previous frame coding feature map, and its output is the first-level correlation weight map.

[0131] The j-th downsampling module Ds in the n-1 cascaded downsampling modules j , is used to downsample the input j-th level correlation weight map and output the (j+1)-th level correlation weight map, where j∈[1, n-1];

[0132] The i-th decoding module Up in the n cascaded decoding modules i This is used to perform decoding operations on the input i-th level attention weight map and the (i+1)-th level current frame decoding feature map, and output the i-th level current frame decoding feature map; wherein, when i=n, ​​the (n+1)-th level current frame decoding feature map is the (n+1)-th level current frame encoding feature map; the i-th level attention weight map is obtained by the n attention weight acquisition modules ECA;

[0133] The i-th attention weight acquisition module among the n attention weight acquisition modules ECA i The input is the i-th level current frame encoded feature map, and the output is the i-th level attention weight map;

[0134] The input to the output module OP is the first-level current frame decoding feature map, and the output is the fiber patch segmentation result map.

[0135] In this embodiment, the correlation weight acquisition module NCC and the n-1 cascaded downsampling modules are modules that enhance features in the IVUS image that are strongly correlated with fiber patch information.

[0136] In this embodiment, the n attention weight acquisition module (ECA) is a module that improves the attention of the IVUS image to fiber patch information.

[0137] In this embodiment, the single-frame fiber patch segmentation network includes: n cascaded current frame coding modules and an end coding module RB. n+1 The system consists of n cascaded decoding modules, n attention weight acquisition modules (ECA), and an output module (OP); where n = 4.

[0138] The i-th current frame coding module RB in the n cascaded current frame coding modules i , is used to encode the input i-1 level current frame coding feature map and output the i level current frame coding feature map; where i∈[1,n], and when i=1, the 0 level current frame coding feature map is the current frame IVUS image;

[0139] The end encoding module RB n+1 This is used to encode the input nth level current frame encoded feature map and output the (n+1)th level current frame encoded feature map;

[0140] The i-th decoding module Up in the n cascaded decoding modules i This is used to perform decoding operations on the input i-th level attention weight map and the (i+1)-th level current frame decoding feature map, and output the i-th level current frame decoding feature map; wherein, when i=n, ​​the (n+1)-th level current frame decoding feature map is the (n+1)-th level current frame encoding feature map; the i-th level attention weight map is obtained by the n attention weight acquisition modules ECA;

[0141] The i-th attention weight acquisition module among the n attention weight acquisition modules ECA i The input is the i-th level current frame encoded feature map, and the output is the i-th level attention weight map;

[0142] The input to the output module OP is the current frame decoding feature map of level 1, and the output is the pre-training result map.

[0143] In this embodiment, as described above, the continuous frame fiber patch segmentation network includes the network structure of the single frame fiber patch segmentation network. The continuous frame fiber patch segmentation network, compared to the single frame fiber patch segmentation network, further includes a previous frame encoding module (RB). *1. The network consists of a correlation weight acquisition module (NCC) and n-1 cascaded downsampling modules. Due to the similarity in these network structures, the pre-trained weights obtained by training the single-frame fiber patch segmentation network can be used as the initial weights for the continuous-frame fiber patch segmentation network, ensuring the accuracy of the training results.

[0144] Implementation Method Six: Combination Figure 1 -5. This embodiment further defines the correlation weight acquisition module NCC in the IVUS image fiber patch segmentation method described in Embodiment 5. The specific implementation details are as follows:

[0145] The processing procedure of the correlation weight acquisition module NCC is as follows:

[0146] The correlation weight acquisition module NCC takes the first-level current frame encoded feature map α obtained by the first current frame encoding module RB1, and performs spatial flattening processing on it to obtain a two-dimensional vector corresponding to the current frame; similarly, it takes the feature map α obtained by the previous frame encoding module RB1... * The encoded feature map β of the previous frame is obtained by spatial flattening to obtain a two-dimensional vector corresponding to the previous frame.

[0147] The feature size of the first-level current frame encoded feature map α and the previous frame encoded feature map β is C1×H1×W1; where C1 represents the number of feature channels, H1 represents the feature map height, and W1 represents the feature map width.

[0148] The feature size of both two two-dimensional vectors is C1×(H1×W1); each two-dimensional vector consists of H1×W1 column vectors of C1×1 arranged sequentially after being flattened in space;

[0149] Using the NCC calculation formula, along the width direction of the two two-dimensional vectors, the normalized cross-correlation coefficients of the two C1×1 column vectors at corresponding positions are obtained, resulting in a 1×(H1×W1) one-dimensional row vector.

[0150] The NCC calculation formula is as follows:

[0151]

[0152] Where f1(x,y) represents the first image being compared; f2(x,y) represents the second image being compared; μ1 represents the average value of the first image; μ2 represents the average value of the first image; σ1 represents the standard deviation of the first image; σ2 represents the standard deviation of the first image; the first image refers to the image represented by the C1×1 column vector at the corresponding position in the two-dimensional vector corresponding to the current frame; the second image refers to the image represented by the C1×1 column vector at the corresponding position in the two-dimensional vector corresponding to the previous frame;

[0153] The sigmoid function is used to normalize the data in the 1×(H1×W1) one-dimensional row vector to [0, 1];

[0154] The normalized 1×(H1×W1) one-dimensional row vector is recombined to obtain a two-dimensional feature vector φ with feature size H1×W1; the two-dimensional feature vector φ is the first-level correlation weight map.

[0155] In this embodiment, the sigmoid function is a common S-shaped function in biology, also known as an S-shaped growth curve. In information science, due to its monotonically increasing properties and the monotonically increasing properties of its inverse function, the sigmoid function is often used as an activation function for neural networks, mapping variables to the range of 0 and 1.

[0156] In this embodiment, the data value of a certain row and column in the two-dimensional feature vector φ represents the correlation strength between the features at a certain corresponding position in the first-level current frame coding feature map α and the previous frame coding feature map β.

[0157] Implementation Method Seven: Combination Figure 1 -5 This embodiment describes the i-th attention weight acquisition module ECA in the fiber patch segmentation method for IVUS images described in Embodiment Six. i Further restrictions are specified, and the specific implementation details are as follows:

[0158] The i-th attention weight acquisition module ECA i The processing procedure is as follows:

[0159] Obtain the i-th level current frame encoded feature map F, with C channels. i ;

[0160] For the i-th level current frame encoded feature map F, perform global average pooling along its channel dimension to obtain a feature map with C i There are 1×1 feature vectors M with each channel having a feature size of 1×1.

[0161] Based on the number of channels C of the feature vector M i The size k of the adaptive convolution kernel is obtained using the following formula:

[0162]

[0163] Where γ is 2 and b is 1;

[0164] The adaptive convolution kernel of size k is applied to the channel dimension of the feature vector M to obtain the output N;

[0165] The resulting output N is activated using the sigmoid function to obtain the i-th level attention weight map; the i-th level attention weight map contains the attention weight S for each channel.

[0166] In this embodiment, the English name for global average pooling is Global Average Pooling, abbreviated as GAP.

[0167] In this embodiment, the attention weight S is multiplied with the i-th level current frame coding feature map F to increase the channel weights in the i-th level current frame coding feature map F that contribute more to accurate segmentation, while suppressing the channel weights that contribute less to accurate segmentation.

[0168] Implementation Method 8: Combination Figure 1 -5. This embodiment further defines the continuous frame fiber patch segmentation network in the IVUS image fiber patch segmentation method described in Embodiment 7. The specific implementation details are as follows:

[0169] The n cascaded current frame encoding modules and previous frame encoding modules RB * 1 and the end encoding module RB n+1 Each encoding module consists of at least two stacked encoding sub-modules; where:

[0170] The first current frame encoding module RB1, the second current frame encoding module RB2, and the previous frame encoding module RB * 1. Each is composed of two stacked encoding sub-modules;

[0171] The third current frame encoding module RB3 and the fourth current frame encoding module RB4 are both composed of three stacked encoding sub-modules;

[0172] The end encoding module RB n+1 It consists of 4 stacked encoding sub-modules;

[0173] The encoding submodule includes: two 3×3 convolutions, a BN layer, a ReLU activation layer, and an output; the output of the encoding submodule is further enhanced with a Dropout regularization.

[0174] In this embodiment, Dropout regularization is added to the output of the encoding submodule to prevent overfitting.

[0175] In a further embodiment, each of the n cascaded decoding modules includes a bilinear interpolation unit, an upsampling unit, and two convolution operations with a kernel size of 3 and a stride size of 1.

[0176] In a further embodiment, the output module OP includes two convolutional operation structures with 3 convolution kernels and a sliding stride of 1; the fiber patch segmentation result map output by the output module OP has 2 channels.

[0177] In a further embodiment, each of the n-1 cascaded downsampling modules includes a convolutional kernel with a size of 2 and a stride of 2.

[0178] Implementation Method Nine: Combination Figure 1 -5. This embodiment describes a fiber patch segmentation device for IVUS images. The specific implementation details are as follows:

[0179] The device specifically includes:

[0180] The acquisition module is used to acquire the set of original IVUS images to be segmented;

[0181] The preprocessing module is used to preprocess the original IVUS image set to obtain a dataset to be segmented, consisting of IVUS images.

[0182] A loading module is used to acquire a trained continuous frame fiber patch segmentation network; the trained continuous frame fiber patch segmentation network is obtained by training using transfer learning; the trained continuous frame fiber patch segmentation network includes a module for enhancing features in the IVUS image that are strongly correlated with fiber patch information, and a module for improving the IVUS image's attention to fiber patch information.

[0183] The segmentation module is used to process the dataset to be segmented using the trained continuous frame fiber patch segmentation network to obtain a fiber patch segmentation result map.

[0184] In this embodiment, the device is used to implement the method described in Embodiment 1.

[0185] Implementation Method 10: Combination Figure 1-5 To further verify the fiber patch segmentation method in the IVUS image and the continuous frame fiber patch segmentation network, this embodiment tested the continuous frame fiber patch segmentation network.

[0186] This test was conducted using 3D Slicer software (a medical image processing and analysis software). Specifically:

[0187] (1) Install PyTorch (an open-source Python machine learning library) in 3D Slicer;

[0188] (2) In 3D Slicer, develop a fiber patch segmentation module for IVUS images, which has network model loading and IVUS image segmentation functions;

[0189] (3) In the fiber patch segmentation module for IVUS images, the trained continuous frame fiber patch segmentation network is loaded.

[0190] (4) Load the original IVUS image set into the 3D Slicer and preprocess the original IVUS image set to obtain the test dataset;

[0191] (5) The fiber patch segmentation module for IVUS images uses the fiber patch segmentation module for IVUS images and the test dataset to obtain a fiber patch segmentation result image.

[0192] The hyperparameters of the continuous frame fiber patch segmentation network constructed in this test are as follows:

[0193] Learning rate = 1e-6; batch size = 16; loss function: CE Loss; optimization function: RMSprop.

[0194] In this test, the evaluation metrics for the continuous frame fiber patch segmentation network or the fiber patch segmentation method in the IVUS image were the Dice coefficient and the Hausdorff distance, as detailed below:

[0195] (1) Dice coefficient, the formula is as follows:

[0196]

[0197] Where X represents the network output segmentation result, and Y represents the real label.

[0198] (2) Hausdorff distance, as follows:

[0199] Suppose we have two sets A = {a1, ..., a2} p B = {b1,…,b} p}, then the Hausdorff distance between these two sets of points is defined as:

[0200] H(A,B)=max(h(A,B),h(B,A));

[0201] Where h(A,B)=max a∈A {min b∈B ||ab||};h(B,A)=max b∈B {min a∈A ‖ba‖};‖·‖ is the distance paradigm between point sets A and B, such as Euclidean distance.

[0202] The Dice coefficients and Hausdorff distance can be used to evaluate the segmentation accuracy of the continuous frame fiber patch segmentation network.

[0203] The evaluation results of this test are as follows:

[0204] Dice=0.9016; Hausdorff distance=0.1538mm.

[0205] The test results above show that the method of segmenting fiber patches using the continuous frame fiber patch segmentation network has high segmentation accuracy.

[0206] The above description of several specific embodiments further details the technical solution provided by the present invention in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, reasonable combinations of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for segmenting fiber patches in IVUS images, characterized in that, The method specifically includes the following steps: S1. Obtain the set of original IVUS images to be segmented; S2. Preprocess the original IVUS image set to obtain a dataset to be segmented, consisting of IVUS images; S3. Obtain the trained continuous frame fiber patch segmentation network; the trained continuous frame fiber patch segmentation network is obtained by training using transfer learning; the trained continuous frame fiber patch segmentation network includes a module for enhancing features in the IVUS image that are strongly correlated with fiber patch information, and a module for improving the IVUS image's attention to fiber patch information. S4. Using the trained continuous frame fiber patch segmentation network, process the dataset to be segmented to obtain a fiber patch segmentation result map; The trained continuous-frame fiber patch segmentation network is used to process the dataset to be segmented to obtain fiber patch segmentation result images, specifically: S4.1 The trained continuous frame fiber patch segmentation network sequentially extracts one IVUS image from the dataset to be segmented. S4.2 After each capture, the current frame IVUS image and the previous frame IVUS image are used together as the input of the trained continuous frame fiber patch segmentation network; if the current frame IVUS image is the first frame image, then the current frame IVUS image is used as its previous frame IVUS image, and together they are used as the input of the trained continuous frame fiber patch segmentation network. S4.

3. Encoding operations are performed on the current frame IVUS image and the previous frame IVUS image respectively to obtain multi-level current frame encoded feature maps and previous frame encoded feature maps; wherein, during the encoding operation, the trained continuous frame fiber patch segmentation network is used to obtain the correlation weight between each position point in the current frame IVUS image and the corresponding position point in the previous frame IVUS image, and according to the correlation weight, the position features with strong correlation in the multi-level current frame encoded feature maps are enhanced, and the position features with weak correlation in the multi-level current frame encoded feature maps are suppressed. S4.4 Perform multi-level decoding operation on the multi-level current frame encoded feature map to obtain a multi-level decoded feature map; wherein, during the decoding operation, the trained continuous frame fiber patch segmentation network is used to obtain the attention weight of each channel in the current frame IVUS image, and according to the attention weight, enhance the channels with strong attention in the multi-level current frame encoded feature map and suppress the channels with weak attention in the multi-level current frame encoded feature map. S4.

5. Based on the feature map obtained from the last decoding in the multi-level decoding feature map, obtain the fiber patch segmentation result map.

2. The method for segmenting fiber patches in IVUS images according to claim 1, characterized in that, In step S2, the original IVUS image set is preprocessed, specifically: S2.1, Perform image augmentation on the original IVUS image set; S2.

2. Perform center cropping on each IVUS original image in the set of original IVUS images after image amplification, and crop out the surrounding low signal and featureless regions to obtain the IVUS image; S2.

3. Combine all IVUS images into a set as the dataset to be segmented.

3. The method for segmenting fiber patches in IVUS images according to claim 1, characterized in that, In step S3, the trained continuous frame fiber patch segmentation network is obtained using a transfer learning training method, specifically: S3.1 Construct a single-frame fiber patch segmentation network; S3.

2. Using the training dataset, train the constructed single-frame fiber patch segmentation network to obtain pre-trained weights; wherein, the training dataset is obtained in the following manner: Obtain the original IVUS image set for training; preprocess the original IVUS image set for training to obtain a continuous sequence consisting of at least one IVUS image arranged in chronological order, which serves as the training dataset; S3.3 Construct a continuous frame fiber patch segmentation network; the continuous frame fiber patch segmentation network includes the network structure of the single frame fiber patch segmentation network; During the construction process, the pre-trained weights are used as the initial weights of the continuous frame fiber patch segmentation network; and the continuous frame fiber patch segmentation network includes a first current frame encoding module. With the previous frame encoding module The first current frame encoding module With the previous frame encoding module Shared parameters; S3.

4. Using the training dataset, train the constructed continuous frame fiber patch segmentation network to obtain the trained continuous frame fiber patch segmentation network.

4. The method for segmenting fiber patches in IVUS images according to claim 3, characterized in that, The continuous frame fiber patch segmentation network includes: n cascaded current frame coding modules and previous frame coding modules. End encoding module The system consists of a correlation weight acquisition module (NCC), n-1 cascaded downsampling modules, n cascaded decoding modules, n attention weight acquisition modules (ECA), and an output module (OP); where n=4. The i-th current frame coding module in the n cascaded current frame coding modules This is used to encode the result of multiplying the input (i-1)th level current frame encoded feature map with the (i-1)th level correlation weight map, and output the i-th level current frame encoded feature map; where i∈[1,n], and when i=1, the 0th level current frame encoded feature map is the current frame IVUS image; the (i-1)th level correlation weight map is obtained by the correlation weight acquisition module NCC and n-1 cascaded downsampling modules; The previous frame encoding module It is used to encode the previous frame of the input IVUS image and output the encoded feature map of the previous frame. The end encoding module This is used to encode the result of multiplying the input nth level current frame encoded feature map with the nth level correlation weight map, and output the (n+1)th level current frame encoded feature map; The input to the correlation weight acquisition module NCC is the first-level current frame coding feature map and the previous frame coding feature map, and its output is the first-level correlation weight map. The n-1 cascaded downsampling modules Each downsampling module , used for the input of the first The correlation weight map is downsampled and the first-order correlation weight map is output. Level correlation weight diagram, where ; The nth cascaded decoding module Decoding module , used for the input of the first Level 1 attention weight map and the 2nd level +1 level current frame decoding feature map is used for decoding operation, and the first level is output. The current frame decoding feature map of level n+1; wherein, when i=n, ​​the current frame decoding feature map of level n+1 is the current frame encoding feature map of level n+1; ... The level attention weight map is obtained by the n attention weight acquisition modules (ECA); The i-th attention weight acquisition module in the n attention weight acquisition modules ECA The input is the i-th level current frame encoded feature map, and the output is the i-th level attention weight map; The input to the output module OP is the first-level current frame decoding feature map, and the output is the fiber patch segmentation result map.

5. The method for segmenting fiber patches in IVUS images according to claim 4, characterized in that, The processing procedure of the correlation weight acquisition module NCC is as follows: The correlation weight acquisition module NCC will be processed by the first current frame encoding module. The obtained Level 1 current frame coding feature map The vector is then spatially flattened to obtain a two-dimensional vector corresponding to the current frame; similarly, the vector encoded by the previous frame module is... The obtained previous frame encoded feature map The vector is then spatially flattened to obtain a two-dimensional vector corresponding to the previous frame. Among them, the first-level current frame coding feature map and the encoded feature map of the previous frame The feature sizes are all × × ;in Represents the number of feature channels. Represents the height of the feature map. Represents the width of the feature map; The feature size of both of the two two-dimensional vectors is Each of the two-dimensional vectors is arranged sequentially after being flattened in space. Composed of column vectors; Using the NCC calculation formula, along the width direction of the two two-dimensional vectors, the two corresponding positions are obtained. The normalized cross-correlation coefficient of the column vectors yields a... A one-dimensional row vector; The NCC calculation formula is as follows: in, This indicates the first image being compared; This indicates the second image being compared; This represents the average value of the first image; This represents the average value of the first image; This represents the standard deviation of the first image; The first image represents the standard deviation of the first image; the first image refers to the image at the corresponding position in the two-dimensional vector corresponding to the current frame. The image represented by the ×1 column vector; the second image refers to the image at the corresponding position in the two-dimensional vector corresponding to the previous frame. The image represented by a ×1 column vector; Using the sigmoid function, the above... Data in a one-dimensional row vector is normalized to ; Normalized One-dimensional row vector reorganization yields a feature size of... The two-dimensional feature vector φ; the two-dimensional feature vector φ is the first-level correlation weight map.

6. The method for segmenting fiber patches in IVUS images according to claim 5, characterized in that, The i-th attention weight acquisition module The processing procedure is as follows: Obtain the i-th level current frame encoded feature map F, which has the following number of channels: ; For the i-th level current frame encoded feature map F, perform global average pooling along its channel dimension to obtain a feature map with... There are 1×1 feature vectors M with each channel having a feature size of 1×1. Based on the number of channels of the feature vector M To obtain the size of the adaptive convolution kernel k， The formula is as follows: in, It is 2. b =1; The size is k The adaptive convolution kernel operates on the channel dimension of the feature vector M to obtain the output N; The resulting output N is activated using the sigmoid function to obtain the i-th level attention weight map; the i-th level attention weight map contains the attention weight S for each channel.

7. The fiber patch segmentation method in IVUS images according to claim 6, characterized in that, The n cascaded current frame encoding modules and previous frame encoding modules and end encoding module Each encoding module consists of at least two stacked encoding sub-modules; where: The first current frame encoding module The second current frame encoding module and the previous frame encoding module Each is composed of two stacked encoding submodules; The third current frame encoding module and the 4th current frame encoding module Each of them is composed of three stacked encoding sub-modules; The end encoding module It consists of 4 stacked encoding sub-modules; The encoding submodule includes: two 3×3 convolutions, a BN layer, a ReLU activation layer, and an output; the output of the encoding submodule is further enhanced with a Dropout regularization.

8. A fiber patch segmentation device for IVUS images, characterized in that, The device specifically includes the following modules: The acquisition module is used to acquire the set of original IVUS images to be segmented; The preprocessing module is used to preprocess the original IVUS image set to obtain a dataset to be segmented, consisting of IVUS images. A loading module is used to acquire a trained continuous frame fiber patch segmentation network; the trained continuous frame fiber patch segmentation network is obtained by training using transfer learning; the trained continuous frame fiber patch segmentation network includes a module for enhancing features in the IVUS image that are strongly correlated with fiber patch information, and a module for improving the IVUS image's attention to fiber patch information. The segmentation module is used to process the dataset to be segmented using the trained continuous frame fiber patch segmentation network to obtain a fiber patch segmentation result map. In the segmentation module, the trained continuous-frame fiber patch segmentation network is used to process the dataset to be segmented to obtain the fiber patch segmentation result map, which is achieved using the following method: S4.1 The trained continuous frame fiber patch segmentation network sequentially extracts one IVUS image from the dataset to be segmented. S4.2 After each capture, the current frame IVUS image and the previous frame IVUS image are used together as the input of the trained continuous frame fiber patch segmentation network; if the current frame IVUS image is the first frame image, then the current frame IVUS image is used as its previous frame IVUS image, and together they are used as the input of the trained continuous frame fiber patch segmentation network. S4.

3. Encoding operations are performed on the current frame IVUS image and the previous frame IVUS image respectively to obtain multi-level current frame encoded feature maps and previous frame encoded feature maps; wherein, during the encoding operation, the trained continuous frame fiber patch segmentation network is used to obtain the correlation weight between each position point in the current frame IVUS image and the corresponding position point in the previous frame IVUS image, and according to the correlation weight, the position features with strong correlation in the multi-level current frame encoded feature maps are enhanced, and the position features with weak correlation in the multi-level current frame encoded feature maps are suppressed. S4.4 Perform multi-level decoding operation on the multi-level current frame encoded feature map to obtain a multi-level decoded feature map; wherein, during the decoding operation, the trained continuous frame fiber patch segmentation network is used to obtain the attention weight of each channel in the current frame IVUS image, and according to the attention weight, enhance the channels with strong attention in the multi-level current frame encoded feature map and suppress the channels with weak attention in the multi-level current frame encoded feature map. S4.

5. Based on the feature map obtained from the last decoding in the multi-level decoding feature map, obtain the fiber patch segmentation result map.

9. An electronic device, comprising: A processor and a memory, characterized in that the memory is used to store executable instructions of the processor, the processor being configured to perform the fiber patch segmentation method in an IVUS image according to any one of claims 1-7 by executing the executable instructions.

Images