IVUS image processing method based on image boundary segmentation

Abstract

The invention relates to the technical field of image processing, in particular to an IVUS image processing method based on image boundary segmentation, which can remarkably improve the accuracy of intravascular ultrasonic imaging IVUS image boundary segmentation, and a proposed image segmentation model comprises two branch paths: a self-adaptive position sensing space path and a guided position mapping space path, wherein the adaptive position sensing space path comprises a down-sampling layer and an up-sampling layer and outputs backbone network parameters for generating position sensing sensitivity enhancement, the guide type position mapping space path comprises six convolution layers and pooling layers of the down-sampling layer and six convolution layers and pooling layers of up-sampling, and the backbone network parameters are fused. On the basis of fusion processing, the mark is replaced with a manual marking result; mapping of original image pixel points to a segmentation feature map is achieved through an image segmentation model, segmentation boundaries of the segmentation feature map are processed, and an external elastic membrane (EEM) and a blood vessel lumen boundary Lumen are obtained.

Description

Technical field:

[0001] The invention relates to the technical field of image processing, and in particular to an IVUS image processing method based on image boundary segmentation, which can significantly improve the accuracy of intravascular ultrasound imaging (IVUS) image boundary segmentation. Background technology:

[0002] Atherosclerosis is the main cause of cardiovascular diseases such as coronary heart disease, myocardial infarction and stroke. According to the World Health Organization (WHO), cardiovascular disease is the number one cause of death worldwide, and intravascular ultrasound imaging (IVUS) has become the preferred technology for coronary artery assessment due to its unique advantages. Accurate segmentation of the lumen and external elastic membrane in IVUS images is a key step in assessing the lumen and vessel wall size and plaque burden, which is crucial for formulating treatment plans and risk stratification of patients with confirmed cardiovascular-related diseases. This precise segmentation helps identify stenosis and plaques in blood vessels, thereby accurately assessing the lumen and vessel wall size and plaque burden, providing key information for subsequent clinical decision-making.

[0003] At present, there are the following difficulties in processing image data of intravascular ultrasound imaging (IVUS): First, the inherent variability of blood vessels, artifacts, and extensive acoustic shadow areas require accurate inference of areas with unclear features. The diverse morphology of vascular walls and plaques leads to complex contours, especially the shadows caused by calcified plaques, which not only obscure the underlying vascular structure but also interfere with the clear identification of boundaries. The signal deflection and attenuation caused by calcified plaques lead to the loss of features in calcified areas, and the identification of these areas usually relies on the inference of visible boundaries, which is experience-based and uncertain. In addition, the presence of side branches of blood vessels increases the complexity of image segmentation. The lack of obvious boundary features at the intersection of side branches and main vessels makes accurate segmentation particularly difficult. The beating of the heart causes changes in the position of blood vessels and the intermittent appearance of side branches in the image frame, further increasing the complexity of analysis. The characteristics of IVUS imaging, including its circumferential characteristics, the penetration limit of sound waves, the resolution limitation, and the complexity of dynamic imaging, all pose challenges to accurate image segmentation. The motion of the probe in the vessel may introduce specific artifacts, which further blur the vessel boundaries and make it more difficult to distinguish plaque from the vessel wall. Finally, the development of automatic segmentation algorithms is hampered by the lack of accurately annotated datasets. All of these factors together constitute major obstacles in IVUS image analysis and processing. Summary of the invention:

[0004] In view of the shortcomings and deficiencies in the prior art, the present invention proposes an IVUS image processing method based on image boundary segmentation, which can significantly improve the accuracy of intravascular ultrasound imaging IVUS image boundary segmentation.

[0005] The present invention is achieved by the following measures:

[0006] An IVUS image processing method based on image boundary segmentation is characterized in that an image segmentation model is first established and trained, the image segmentation model includes a downsampling layer, an upsampling layer, a deep feature aggregation bridge DFCH module and an integral size calibration module IDC, wherein the downsampling layer and the upsampling layer each include six layers, the first three layers of the downsampling layer each include a wavelet convolution spectrum reshaper WCSR module and a Conv convolution layer, the last three layers of the downsampling layer each include a wavelet convolution spectrum reshaper WCSR module and a MaHS module, the processing process of the upsampling training layer is mirrored to the processing process of the downsampling training layer, after obtaining the trained image segmentation model, the image segmentation model is applied to complete the mapping of the original image pixel points to the segmentation feature map, the segmentation boundary of the segmentation feature map is processed, and the final external elastic membrane EEM and vascular lumen boundary Lumen boundary are obtained.

[0007] The training image segmentation model of the present invention specifically includes the following steps:

[0008] Step 1: Input the discrete IVUS image X to the image segmentation model i , where i = 1, 2, ... 6, i is the corresponding number of layers;

[0009] Step 2: Downsample the data. The wavelet convolution spectrum reshaper WCSR in the downsampling layer uses four filters with convolution: a low-pass filter f LL , high pass filter f LH , high pass filter f HL , high pass filter f HH , respectively defined as follows:

[0010]

[0011] The feature map F1 generated by the first layer is fed into the filter f of the wavelet transform LL 、f LH 、f HL and f HH , and obtain four feature maps X of different frequencies LL , X LH , X HL , X HH ,definition Here represents the convolution operation, ↓2 means the convolution step size in the filter is 2, X LL , X LH , X HL , X HH It is expressed as:

[0012] X LL (n,m)=X(2n-1,2m-1)+X(2n-1,2m)+X(2n,2m-1)+X(2n,2m),

[0013] X LH (n,m)=-X(2n-1,2m-1)-X(2n-1,2m)+X(2n,2m-1)+X(2n,2m,

[0014] X HL (n,m)=--X(2n-1,2m-1)+X(2n-1,2m)-X(2n,2m-1)+X(2n,2m),

[0015] X HH (n,m)=X(2n-1,2m-1)-X(2n-1,2m)-X(2n,2m-1)+X(2n,2m),,

[0016] Where n and m represent n rows and m columns in the image respectively. LL , X LH , X HL , X HH Through a 3*3 convolution module fusion, the F of the current downsampling layer is output d i , i=1, sent to the wavelet convolution spectrum reshaper WCSR of the second layer, and data reshaping and fusion are repeated. The data output from the second layer is sent to the wavelet convolution spectrum reshaper WCSR of the third layer, and data reshaping and fusion are repeated;

[0017] Step 3: The data output from the third layer is fed into the wavelet convolution spectrum reshaper WCSR of the fourth layer. After the data is reshaped, the multi-axis inhomogeneous synthesis MaHS module of the fourth layer processes the data, which specifically includes:

[0018] Step 3-1: Use the average division method to cut the input feature map F, and the results are: F1, F2, F3, F4, expressed as the following formula:

[0019] [F i1 , F i2 , F i3 , F i4 ]=Spilt(F i );

[0020] Step 3-2: Set the self-attention module SA, the convolutional block attention module CBA, the squeeze and excitation attention module SEA, and the alignment feature cutting module AS. After the cut feature map passes through the four channels respectively, the enhanced feature map of each channel is fused to extract the spatial information around the blank area in the image.

[0021] Step 3-3: Perform a 3*3 convolution operation on the feature map after the feature fusion operation to generate Fout with the same dimension as the output as the output;

[0022] Step 4: Repeat step 3 until downsampling is completed;

[0023] Step 5: Perform upsampling training processing. The first three layers of the upsampling layer each contain a wavelet convolution spectrum reshaper WCSR module and a multi-axis inhomogeneous synthesis MaHS module, and the last three layers each contain a wavelet convolution spectrum reshaper WCSR module and a Conv module. The processing of upsampling training and downsampling training is a mirror image. After completing the upsampling training processing, the trained image segmentation model is obtained.

[0024] In step 3 of the present invention, when the multi-axis heterogeneous synthesis MaHS module processes the data, the i-th layer feature Fi is cut into equal parts to obtain F i1 、F i2 、F i3 and F i4 Four features are then passed through the SA module, CBA module, SEA module and Conv layer to obtain four new feature maps F (i+1)1、 F (i+1)2、 F (i+1)3、 F (i+1)4 After concatenation, the final feature F is obtained through a Conv layer. MaHS(i+1) ;

[0025] The self-attention module SA processes the input data as follows: first, the feature map is flattened by FL, and then passed through three linear layers to obtain Q, K, and V, and then reshaped, and the reshaped features of Q and K are Hadamard-producted, and finally Hadamard-producted with V and passed through a linear layer and reshaped to obtain a new feature map;

[0026] The convolutional block attention module CBA processes the input data as follows: in the first step, the features are averaged and maximized along the two channels respectively, and the results are concatenated and Hadamard products are performed with the original features. In the second step, the newly generated features are averaged and maximized in the channel dimension again. Consistent with the above process, the two new features that have been averaged and maximized in the channel are concatenated and passed through a 7*7 Conv convolution layer, and finally Hadamard products are performed with the new features generated in the first step to obtain the final feature map;

[0027] The squeeze and excite attention module SEA processes the input data as follows: the features are divided into two routes for operation, one is to continuously perform averaging and maximization operations on the channel dimension, and the other is to directly send the features, and finally the feature maps obtained by the two routes are Hadamard products to obtain the final feature map;

[0028] The alignment feature cutting module AS processes the input data as follows: the process is divided into three routes, the first is the Conv convolution layer, the second is continuous averaging operations through the channel dimension and a shared Conv convolution layer, and the third is the shared convolution layer. After that, the features generated by the first route and the second route are concatenated and then Hadamard products are performed with the features generated by the third route. Finally, the new features are equally cut to obtain F1DFCH(i+1), F2DFCH(i+1), F3DFCH(i+1), and F4DFCH(i+1).

[0029] The present invention also introduces an integral dimension calibration module IDC to reduce the impact of temporal forgetting. Convolution 2d and group normalization are used to operate the input features to obtain a feature factor of 1*H*W, where H is the height of the image and W is the width of the image. ReLu is defined as group normalization, which is expressed as follows:

[0030] In order to effectively solve the problem of missing spatial information due to the small receptive field of convolution when the model is up-sampled, in step 5, the deep feature aggregation bridge DFCH module and the integral size calibration module IDC are used to first convert the feature map F generated by the DFCH module of the i+1th layer DFCH(i+1) And the feature F obtained by downsampling at the i-th layer d i The data is cut into equal parts, and then the equally cut data is concatenated with the feature factors generated by the corresponding i-th layer IDC module to obtain four different sets of new feature maps, which are then passed through different filter convolution layers G 1、 G 2、 G 3、 G4, splicing to obtain the feature map F generated by the DFCH module of the i-th layer DFCH(i), where the integral size is calibrated i+1 represents the feature factor generated by upsampling the i-th layer, F DFCH(i+1) Represents the fused features generated by the i+1 layer DFCH module.

[0031] The present invention also includes step 6, respectively performing projection operations on the feature maps after matching and fusion of each group, defining g0 as consisting of convolution 2d (dilation=1) and layernorm, g1 as consisting of convolution 2d (dilation=3) and layernorm, g2 as consisting of convolution 2d (dilation=5) and layernorm, g3 as consisting of convolution 2d (dilation=7) and layernorm, and finally, performing feature fusion and splicing on the feature maps after projection of each group to obtain the final output of DFCH of layer i.

[0032] The present invention also includes the step of using the fmask module to cover a part of the IVUS original image X′ in the training set with a black mask when pre-training the image segmentation model, wherein the fmask module is used to randomly generate a masked area, thereby obtaining a masked image, and removing the mask from the masked image by the image segmentation model to obtain a masked image X″. The method also simulates the situation that dark areas without image information generally exist in IVUS images, so as to improve the model's ability to understand images and repair the visual blank areas of the images, and at the same time improves the segmentation network's ability to understand IVUS images without wasting accurate annotation image data, thereby reducing the dependence on large data sets.

[0033] Compared with the prior art, the image segmentation model proposed in the present invention includes two branch paths: an adaptive position-aware spatial path and a guided position mapping spatial path, wherein the adaptive position-aware spatial path includes a downsampling layer and an upsampling layer, and both the downsampling layer and the upsampling layer include six convolutional layers and a pooling layer, and uses a discrete IVUS image Xi as input data, and outputs a backbone network parameter θ that generates enhanced position-aware sensitivity. i The guided position mapping spatial path includes six convolutional layers and pooling layers for downsampling and six convolutional layers and pooling layers for upsampling, and incorporates the backbone network parameters θ i ; Based on the fusion processing, the labels are replaced with the results of manual annotation; thereafter, the image segmentation model is used to map the original image pixels to the segmentation feature map, and the segmentation boundaries of the segmentation feature map are processed to obtain the final external elastic membrane EEM and vascular lumen boundary Lumen boundary.

[0034] The present invention conducts a detailed evaluation of the wavelet-integrated multidimensional tensor harmonic coronary artery reconstruction framework on the largest IVUS boundary segmentation dataset reported so far. The results show that the wavelet-integrated multidimensional tensor harmonic coronary artery reconstruction framework significantly outperforms multiple SoTA methods in the field of medical image segmentation on the challenging Lumen and EEM boundary segmentation tasks. Description of the drawings:

[0035] Attached Figure 1 Schematic diagram of the network model pre-training process in an embodiment of the present invention.

[0036] Attached Figure 2 This is a flow chart of the image segmentation model training in the present invention.

[0037] Attached Figure 3 This is a workflow diagram of the deep feature aggregation bridge DFCH in the present invention.

[0038] Attached Figure 4 It is a schematic diagram of the wavelet convolution spectrum reshaper WCSR in the present invention.

[0039] Attached Figure 5 This is a schematic diagram of the multi-axis heterogeneous synthesis MaHS module in the present invention.

[0040] Attached Figure 6 This is a principle block diagram of the self-attention module SA in the present invention.

[0041] Attached Figure 7 This is the principle block diagram of the convolutional attention module CBA in the present invention.

[0042] Attached Figure 8 This is a principle block diagram of the SEA module for squeezing and stimulating attention of the present invention.

[0043] Attached Fig. 9 This is a principle block diagram of the alignment feature cutting module AS in the present invention. Specific implementation method:

[0044] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0045] Example:

[0046] This example proposes an automatic segmentation method for determining the boundaries of lumens and EEMs in IVUS images, which aims to learn and understand the original IVUS input and map the original pixels to the corresponding categories. In order to enable the model to learn more complete and detailed position feature information and position feature relationships, this example proposes a Wavelet convolutional spectrum reshaper module and a MaHS module, which aims to inject the model with the simulation of IVUS images in the wavelet domain and use multi-head heterogeneous attention to emphasize the relationship between positions. Therefore, the input of the network consists of two modal frames: X, X 小波卷积频谱重塑器 ;

[0047] like Figure 2 As shown in the figure, the backbone network, i.e., the wavelet-integrated multidimensional tensor harmonic coronary artery reconstruction framework, is divided into two branches: an adaptive position-aware spatial path and a guided position mapping spatial path. The adaptive position-aware spatial path of the wavelet-integrated multidimensional tensor harmonic coronary artery reconstruction framework uses discrete IVUS images Xi as input and generates model parameters θi with enhanced position-aware sensitivity as output. To achieve this goal, a frequency-domain-based encoder is first introduced to mask the original IVUS image using the edge information in the IVUS image. Secondly, the backbone network is used to process the masked feature map F and generate the reconstructed IVUS image. The backbone network uses the input as a label to update the output model parameters θ i ;

[0048] In order to effectively solve the blurring problem caused by a large number of artifacts and angiography in IVUS images due to the influence of blood, plaque components, vascular curvature, and imaging devices, this example incorporates a model parameter θ with position awareness. i This fusion can improve the model's sensitivity to blank areas and can more accurately understand and restore the texture information and morphological structure of blank areas in IVUS images. On the basis of fusion, the labels are replaced with the results of manual annotation. Then, the backbone network is used to map the original image pixels to the segmentation feature map, and the segmentation boundary of the segmentation feature map is processed to obtain the final EEM and Lumen boundary. The overall model structure is as follows: Figure 1 The detailed implementation steps of the entire framework are as follows:

[0049] Step a: In order to better utilize the frequency signal and alleviate the problem of motion artifacts caused by heartbeats reducing spatial information in the image, a wavelet convolution spectrum reshaper module is introduced. Its structure is shown in the figure Figure 4 As shown in Figure 1, this module uses wavelet transform as the sampling basis and performs wavelet transform-like calculations on the spatial information of the channel dimension in the feature map. The architecture of the wavelet convolution spectrum reshaper is shown in Figure 1. Figure 1As shown on the right, four filters with convolution are used here, namely low-pass filter fLL, high-pass filters fLH, fHL, and fHH. The filters used in this example are defined as follows:

[0050]

[0051] Through these four filters, four self-contained feature maps xLL, xLH, xHL, and xHH are obtained, respectively, defining Here represents the convolution operation, ↓2 means the convolution step size in the filter is 2.

[0052] xLL, xLH, xHL, xHH can be expressed as:

[0053] X LL (n,m)=X(2n-1,2m-1)+X(2n-1,2m)+X(2n,2m-1)+X(2n,2m),

[0054] X LH (n,m)=-X(2n-1,2m-1)-X(2n-1,2m)+X(2n,2m-1)+X(2n,2m,

[0055] X HL (n,m)=--X(2n-1,2m-1)+X(2n-1,2m)-X(2n,2m-1)+X(2n,2m),

[0056] X HH (n,m)=X(2n-1,2m-1)-X(2n-1,2m)-X(2n,2m-1)+X(2n,2m),,

[0057] Here n and m represent n rows and m columns in the image respectively.

[0058] Finally, the four feature maps xLL, xLH, xHL, and xHH are fused using feature fusion and a 3*3 convolution is performed to obtain the representation of the feature map in the wavelet domain.

[0059] Step b: Taking the feature map F as input, the multi-attention mechanism is used to enhance the relationship between the target region and the global and non-local regions, thereby strengthening the characteristics of the target region; MaHHS Figure 1 As shown in the lower right, the input feature map is cut using the average division method, and the results are: F1, F2, F3, F4, which can be expressed as

[0060] [F i1 , F i2 , F i3 , Fi4 ]=Spilt(F i )

[0061] Through this operation, the entire feature map can be efficiently operated. After embedding, in order to reduce the impact of contrast, artifacts or blurred areas, the grouped feature maps are subjected to different operations, defining the self-attention module (SA), convolutional block attention module (CBA), squeeze and excitation attention module (SEA), alignment feature cutting module (AS) and convolution operation of 3*3. After the cut feature map passes through four channels respectively, the enhanced feature map of each channel is fused to extract the spatial information around the blank area in the image, and then the feature map after the feature fusion operation is subjected to a 3*3 convolution operation to generate Fout with the same dimension as the output as the output;

[0062] Step c: In order to alleviate the impact of temporal forgetting, we propose an integral size calibration module, which uses convolution 2d and group normalization to operate the input features to obtain the feature factor of 1*H*W, and defines ReLu as group normalization, which can be expressed as:

[0063]

[0064] In order to effectively solve the problem of missing spatial information due to the small receptive field of convolution during upsampling, this example combines the feature factor integral size calibration i+1 (indicating the feature factor generated by the i-th layer of upsampling) output by the integral size calibration module and the feature maps FDFCH(i+1) (indicating the fused features generated by the i+1 layer DFCH module) and Fdi (indicating the feature map output by the i-th layer of downsampling) of different layers. Before fusion, this example groups all feature maps and matches them with the feature factors respectively. This fusion can supplement the spatial feature information in the upper feature map during upsampling.

[0065] In addition, this example performs projection operations on each group of matched fused feature maps: define g0 as consisting of convolution 2d (dilation = 1) and layernorm, g1 as consisting of convolution 2d (dilation = 3) and layernorm, g2 as consisting of convolution 2d (dilation = 5) and layernorm, and g3 as consisting of convolution 2d (dilation = 7) and layernorm. Finally, perform feature fusion and splicing on the feature maps after each group of projection to obtain the final output of DFCH of layer i.

[0066] In this example, an unlabeled IVUS image is used to learn IVUS image features based on the MAE method. The original image in the training set covers a small area of ​​the IVUS original image with a black mask, and the groundtruth uses the original image. This method also simulates the common dark areas without image information in IVUS images to improve the model's ability to understand the image and repair the visual blank areas of the image. At the same time, it improves the segmentation network's ability to understand IVUS images without wasting accurately annotated image data, reducing dependence on large data sets.

[0067] In this example, a detailed evaluation of the wavelet-integrated multidimensional tensor harmonic coronary artery reconstruction framework was conducted on the largest IVUS boundary segmentation dataset reported so far. The results showed that the wavelet-integrated multidimensional tensor harmonic coronary artery reconstruction framework significantly outperformed multiple SoTA methods in the field of medical image segmentation in the challenging Lumen and EEM boundary segmentation tasks. The specific performance comparison is shown in Table 1.

[0068] Table 1 Performance comparison results of this example and multiple SoTA methods

[0069]

[0070] Compared with the prior art, the present invention improves the model's ability to understand images and repair visual blank areas of images, while improving the segmentation network's ability to understand IVUS images without wasting accurately annotated image data, thereby reducing dependence on large data sets.

Claims

1. An IVUS image processing method based on image boundary segmentation, characterized in that: Firstly, an image segmentation model is established and trained, which includes a downsampling layer, an upsampling layer, a deep feature aggregation bridge DFCH module and an integral size calibration module IDC, wherein the downsampling layer and the upsampling layer each contain six layers, the first three layers of the downsampling layer each contain a wavelet convolution spectrum reshaper WCSR module and a Conv convolution layer, the last three layers of the downsampling layer each contain a wavelet convolution spectrum reshaper WCSR module and a MaHS module, the processing process of the upsampling training layer is mirrored to that of the downsampling training layer, after obtaining the trained image segmentation model, the image segmentation model is applied to complete the mapping of the original image pixels to the segmentation feature map, the segmentation boundary of the segmentation feature map is processed, and the final external elastic membrane EEM and vascular lumen boundary Lumen boundary are obtained.

2. The IVUS image processing method based on image boundary segmentation according to claim 1, characterized in that: The training image segmentation model specifically comprises the following steps: Step 1: Input the discrete IVUS image X to the image segmentation model i， Where i = 1, 2, ... 6, i is the corresponding layer number; Step 2: Downsample the data. The wavelet convolution spectrum reshaper WCSR in the downsampling layer uses four filters with convolution: a low-pass filter f LL , high pass filter f LH , high pass filter f HL , high pass filter f HH , respectively defined as follows: The feature map F1 generated by the first layer is fed into the filter f of the wavelet transform LL 、f LH 、f HL and f HH， Get four feature maps X with different frequencies LL , X LH , X HL , X HH ,definition Here represents the convolution operation, ↓2 means the convolution step size in the filter is 2, X LL , X LH , X HL , X HH It is expressed as: X LL (n,m)=X(2n-1,2m-1)+X(2n-1,2m)+X(2n,2m-1)+X(2n,2m), X LH (n,m)=-X(2n-1,2m-1)-X(2n-1,2m)+X(2n,2m-1)+X(2n,2m, X HL (n,m)=-X(2n-1,2m-1)+X(2n-1,2m)-X(2n,2m-1)+X(2n,2m), X HH (n,m)=X(2n-1,2m-1)-X(2n-1,2m)-X(2n,2m-1)+X(2n,2m),, Where i and j represent the i row and j column in the image respectively. LL , X LH , X HL , X HH Through a 3*3 convolution module fusion, the output of the current downsampling layer F d i , i=1, sent to the wavelet convolution spectrum reshaper WCSR of the second layer, and data reshaping and fusion are repeated. The data output from the second layer is sent to the wavelet convolution spectrum reshaper WCSR of the third layer, and data reshaping and fusion are repeated. Step 3: The data output from the third layer is fed into the wavelet convolution spectrum reshaper WCSR of the fourth layer. After the data is reshaped, the multi-axis inhomogeneous synthesis MaHS module of the fourth layer processes the data, which specifically includes: Step 3-1: Use the average division method to cut the input feature map F, and the results are: F1, F2, F3, F4, expressed as the following formula: [F i1 ,F i2 ,F i3 ,F i4 ]=Played(F i ); Step 3-2: Set the self-attention module SA, the convolutional block attention module CBA, the squeeze and excitation attention module SEA, and the alignment feature cutting module AS. After the cut feature map passes through the four channels respectively, the enhanced feature map of each channel is fused to extract the spatial information around the blank area in the image. Step 3-3: Perform a 3*3 convolution operation on the feature map after the feature fusion operation to generate Fout with the same dimension as the output as the output; Step 4: Repeat step 3 until downsampling is completed; Step 5: Perform upsampling training processing. The first three layers of the upsampling layer each contain a wavelet convolution spectrum reshaper WCSR module and a multi-axis inhomogeneous synthesis MaHS module, and the last three layers each contain a wavelet convolution spectrum reshaper WCSR module and a Conv module. The processing of upsampling training and downsampling training is a mirror image. After completing the upsampling training processing, the trained image segmentation model is obtained.

3. The IVUS image processing method based on image boundary segmentation according to claim 2, characterized in that: In step 3, when the multi-axis heterogeneous synthesis MaHS module processes the data, the i-th layer feature Fi is cut equally to obtain four features Fi1, Fi2, Fi3 and Fi4, and then it is passed through the SA module, CBA module, SEA module and Conv layer to obtain four new feature maps F (i+1)1 、F (i+1)2 、F (i+1)3 、F (i+1)4 After concatenation, the final feature F is obtained through a Conv layer. MaHS(i+1) ; The self-attention module SA processes the input data as follows: first, the feature map is flattened by FL, and then passed through three linear layers to obtain Q, K, and V, and then reshaped, and the reshaped features of Q and K are Hadamard-producted, and finally Hadamard-producted with V and passed through a linear layer and reshaped to obtain a new feature map; The convolutional block attention module CBA processes the input data as follows: in the first step, the features are averaged and maximized along the two channels respectively, and the results are concatenated and Hadamard products are performed with the original features. In the second step, the newly generated features are averaged and maximized in the channel dimension again. Consistent with the above process, the two new features that have been averaged and maximized in the channel are concatenated and passed through a 7*7 Conv convolution layer, and finally Hadamard products are performed with the new features generated in the first step to obtain the final feature map; The squeeze and excite attention module SEA processes the input data as follows: the features are divided into two routes for operation, one is to continuously perform averaging and maximization operations on the channel dimension, and the other is to directly send the features, and finally the feature maps obtained by the two routes are Hadamard products to obtain the final feature map; The alignment feature cutting module AS processes the input data as follows: the process is divided into three routes, the first is the Conv convolution layer, the second is continuous averaging operations through the channel dimension and a shared Conv convolution layer, and the third is the shared convolution layer. After that, the features generated by the first route and the second route are concatenated and then Hadamard products are performed with the features generated by the third route. Finally, the new features are equally cut to obtain F1DFCH(i+1), F2DFCH(i+1), F3DFCH(i+1), and F4DFCH(i+1).

4. The IVUS image processing method based on image boundary segmentation according to claim 3, characterized in that: The integral dimension calibration module IDC is also introduced to reduce the impact of temporal forgetting. Convolution 2d and group normalization are used to operate the input features to obtain the feature factor of 1*H*W, where H is the height of the image and W is the width of the image. ReLu is defined as group normalization, which is expressed as follows: In order to effectively solve the problem of missing spatial information due to the small receptive field of convolution when the model is up-sampled, in step 5, the deep feature aggregation bridge DFCH module and the integral size calibration module IDC are used to first convert the feature map F generated by the DFCH module of the i+1th layer DFCH(i+1) And the feature F obtained by downsampling at the i-th layer d i The data is cut into equal parts, and then the equally cut data is spliced ​​with the feature factors generated by the corresponding IDC module of the i-th layer, so as to obtain four different sets of new feature maps, which are then spliced ​​through different filter convolution layers G1, G2, G3, and G4 to obtain the feature map F generated by the DFCH module of the i-th layer. DFCH(i) , where the integral size is calibrated i+1 represents the feature factor generated by upsampling the i-th layer, F DFCH(i+1) Represents the fused features generated by the i+1 layer DFCH module.

5. The IVUS image processing method based on image boundary segmentation according to claim 4, characterized in that: It also includes step 6, performing projection operations on each group of matched fused feature maps, defining g0 as consisting of convolution 2d (dilation = 1) and layernorm, g1 as consisting of convolution 2d (dilation = 3) and layernorm, g2 as consisting of convolution 2d (dilation = 5) and layernorm, and g3 as consisting of convolution 2d (dilation = 7) and layernorm. Finally, the feature maps after each group of projections are fused and spliced ​​to obtain the final output of DFCH of layer i.

6. The IVUS image processing method based on image boundary segmentation according to claim 1, characterized in that: The method also includes pre-training the image segmentation model, using the fmask module to cover a part of the IVUS original image X′ in the training set with a black mask, wherein the fmask module is used to randomly generate a masked area, thereby obtaining a masked image, and removing the mask from the masked image through the image segmentation model to obtain an image X" after the mask is removed.

Images