An intravascular fiber plaque OCT image segmentation method based on a U-shaped hybrid network

Abstract

The application provides an intravascular fiber plaque OCT image segmentation method based on a U-shaped hybrid network, and belongs to the technical field of image processing. An encoder extracts multi-scale features of an OCT image; a decoder gradually restores spatial resolution, fuses the multi-scale features, and obtains an OCT image segmentation result; the decoder fuses features through a channel space additive attention block; the channel space additive attention block: performs depth separable convolution and a linear layer on input to obtain keys, values and queries; sequentially inputs the queries into a space module and a channel module, sequentially inputs the keys into the channel module and the space module, splices outputs, and fuses the values to obtain channel space additive attention output; splices the channel space additive attention output and input, and inputs the spliced output into a multilayer perceptron to obtain output features of the channel space additive attention block. Tokens are gathered to a position of interest through gradual clustering and merging of the tokens, visual tokens of different positions and sizes are generated for each image, and the precision of segmentation is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for OCT image segmentation of intravascular fibrous plaques based on a U-shaped hybrid network. Background Technology

[0002] Cardiovascular disease is the leading cause of death worldwide, primarily affecting the elderly, but in recent years showing a significant trend towards affecting younger people. Coronary atherosclerosis, a crucial pathological basis of cardiovascular disease, can trigger life-threatening clinical emergencies such as acute coronary syndrome and often causes irreversible myocardial damage. In clinical diagnosis, medical imaging technology is the core tool for assessing the extent of coronary artery disease, mainly including non-invasive methods (such as computed tomography (CT) and magnetic resonance imaging (MRI)) and invasive methods (such as intravascular ultrasound (IVUS) and optical coherence tomography (OCT). Among these, optical coherence tomography, with its micron-level resolution and excellent tissue penetration, is widely recognized as the most accurate imaging method for characterizing the composition and morphological features of arterial plaques.

[0003] Optical coherence tomography (OCT) uses near-infrared light interference to achieve cross-sectional tomographic imaging of the blood vessel wall, clearly identifying three main subtypes of vulnerable plaques: fibrous plaques, calcified plaques, and thin-cap fibro-atherosclerotic plaques. While this technology offers image resolution more than 10 times higher than traditional IVUS, the reliance on physicians manually interpreting massive amounts of OCT image data in clinical practice presents a significant efficiency bottleneck, with image analysis for a single patient typically taking over 30 minutes. Existing automated segmentation methods mostly employ traditional image processing algorithms (such as thresholding and region growing), which generally suffer from insufficient segmentation accuracy (average Dice coefficient below 0.75) and high missegmentation rates in calcified regions when faced with the unique speckle noise interference and complex tissue boundaries inherent in OCT images. Furthermore, while deep learning schemes based on conventional U-Net can improve the recognition rate of fibrous plaques, their skip connection structure easily loses morphological information of microcalcifications during the feature fusion stage, resulting in an overall error rate exceeding 15% in the quantitative analysis of plaque components. Summary of the Invention

[0004] In view of this, the present invention provides a method for OCT image segmentation of intravascular fibrous plaques based on a U-shaped hybrid network. The method constructs a target U-shaped hybrid network by using region-aware clustering attention blocks and channel-space additive attention blocks, which improves segmentation accuracy while reducing complexity.

[0005] Therefore, the present invention provides the following technical solution: A method for OCT image segmentation of intravascular fibrous plaques based on a U-shaped hybrid network, comprising: The OCT image is input into the target U-shaped hybrid network to obtain the OCT image segmentation result; The target U-shaped hybrid network includes: an encoder extracting multi-scale features of the OCT image; and a decoder progressively restoring the spatial resolution and fusing the multi-scale features to obtain the OCT image segmentation result. The decoder performs feature fusion through channel-space additive attention blocks; the channel-space additive attention block includes: performing depthwise separable convolutions and linear layers on the input to obtain keys, values, and queries; Input the query into the spatial module and the channel module in sequence, input the key into the channel module and the spatial module in sequence, and then concatenate the output and merge it with the value to obtain the channel-spatial additive attention output; The channel-space additive attention output and the input are concatenated and then input into a multilayer perceptron to obtain the output features of the channel-space additive attention block.

[0006] Further, the encoder includes: Phase 1: Using image patch embedding and two region-aware clustering attention blocks, a size of [size missing] is [value missing]. The input is converted to a size of Features; Phase 2: Transform the output feature size of Phase 1 into a new size using token merging and four region-aware clustering attention blocks. ; Phase 3: Transform the output feature size of Phase 2 into a new size using token merging and four region-aware clustering attention blocks. ; Stage 4: Transform the output feature size of Stage 3 into a new size using token merging and two region-aware clustering attention blocks. .

[0007] Furthermore, the region-aware clustering attention block includes: Get the key, value, and query based on the input; The fused key, fused value, and query are obtained by density peak clustering based on k nearest neighbors; The fused key, fused value, and query are sequentially input into the channel space additive attention block and the multilayer perceptron to obtain the output features of the region-aware clustering attention block.

[0008] Furthermore, the step of obtaining the fused key and fused value, as well as the query, through density peak clustering based on k nearest neighbors includes: Perform average pooling on the key and value to obtain and and through and The affinity matrix A is generated by the following formula:

[0009] Based on the affinity matrix, the local density of the visual token is determined using the following formula:

[0010] in, This indicates that the index in A is , Visual tokens; Visual tokens Local density, KNN represents visual tokens The k nearest neighbors; Determining the cluster centers of visual tokens based on their local density: Define a distance metric for each visual token. Visual tokens representing measurements The minimum distance between each visual token and other tokens with higher density. The defined formula is expressed as:

[0011] in, Indicates distance index, This represents local density. The score of the visual token at the cluster center is the local product density. and minimum distance The product of, expressed as The top k visual tokens with the highest scores are used as cluster centers, and the remaining tokens are assigned to the nearest cluster centers with higher density. Clustering is performed on each visual token based on its cluster center to obtain a cluster set, expressed by the formula:

[0012] in, Let i represent the i-th cluster set. Indicates a visual token. This represents the importance score for each token; By concatenating the cluster set and the key, we obtain the merged key; by concatenating the cluster set and the value, we obtain the merged value.

[0013] Furthermore, the space module includes:

[0014] in, This indicates the output characteristics of the space module. This indicates max pooling. Indicates average pooling. Represents convolution. Input to the channel space additive attention block.

[0015] Furthermore, the channel module includes:

[0016] in, This indicates the output characteristics of the channel module. Input to the channel space additive attention block.

[0017] Further, the step of sequentially inputting queries into the spatial module and the channel module, sequentially inputting keys into the channel module and the spatial module, and then concatenating the outputs and fusing them with the values ​​to obtain the channel-space additive attention output includes:

[0018] in, This indicates the channel-space additive attention output. Indicates the channel module. Represents a spatial module.

[0019] Furthermore, the decoder performs feature fusion through channel-space additive attention blocks, including:

[0020]

[0021]

[0022]

[0023] in, For upsampling, Add an attention block to the channel space. for convolution; The output features for stage 5, For the output features of stage 6, The output features for stage 7 This is the output feature for stage 8.

[0024] Advantages and positive effects of the present invention: In this method, the region-aware clustering attention component forms the encoder. By progressively clustering and merging tokens, tokens can be gathered to the location of interest, generating visual tokens of different locations and sizes for each image, thereby improving the segmentation accuracy.

[0025] In this method, the channel-space additive attention block adopts channel attention and spatial attention as a new interaction form. Compared with the traditional attention mechanism, this module eliminates complex matrix multiplication, softmax and other operations, which improves the segmentation accuracy while reducing the complexity. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of the OCT image segmentation method for intravascular fibrous plaques based on a U-shaped hybrid network in the embodiment; Figure 2 This is a schematic diagram of the target U-shaped hybrid network structure in the embodiment; Figure 3 This is a structural diagram of the channel space additive attention block in the embodiment; Figure 4 This is a structural diagram of the channel block and space block in the embodiment; Figure 5 This is a schematic diagram of the region-aware clustering attention block in the embodiment; Figure 6 This is a graph comparing the errors of our method with those of other methods. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] This invention provides a method for OCT image segmentation of intravascular fibrous plaques based on a U-shaped hybrid network, which performs OCT image segmentation using a target U-shaped hybrid network. Figure 2 As shown, the target U-shaped hybrid network includes: an encoder and a decoder; the encoder includes: Multiple image blocks are obtained by dividing the image into blocks; Phase 1: Image patch embedding and two region-aware clustering attention blocks, with a size of... The input image is converted to a size of Features; Phase 2: Transform the feature size by token merging and four region-aware clustering attention blocks. ; Phase 3: Transform the feature size by token merging and four region-aware clustering attention blocks. ; Phase 4: Transform the feature size by token merging and two region-aware clustering attention blocks. .

[0031] Stages 1-4 are expressed by the following formula:

[0032]

[0033]

[0034]

[0035] in, For input, The output features for stage 1 This is the output feature of stage two. For the output features of stage 3, Output features for stage 4; Embedding for image patches; For region-aware clustering attention blocks; This indicates token merging, i.e., downsampling.

[0036] The decoder comprises stages 5 through 8, where features in each stage are processed by channel-space additive attention blocks and fused with features of the same resolution from the encoder. Finally, the fused features are upsampled to obtain a size of [missing value]. The output.

[0037] The formula for stage 5-8 is expressed as follows:

[0038]

[0039]

[0040]

[0041] Based on the output features of stage 8, the segmentation result is obtained through a linear layer, as expressed by the formula:

[0042] in, For linear layers, For upsampling, Add an attention block to the channel space. for convolution; The output features for stage 5, For the output features of stage 6, The output features for stage 7 Output features for stage 8; This is the segmentation result.

[0043] Example The target U-shaped hybrid network includes: an encoder and a decoder; The encoder includes: an input OCT image obtained through image patch segmentation and image patch embedding. Non-overlapping image patches. Linear projection of the image patches yields key, value, and query representations as: key, value, and query. The outputs of stages 1-4 are obtained through token merging and region-aware clustering attention blocks.

[0044] The decoder comprises stages 5 through 8, where features in each stage are processed by channel-space additive attention blocks and fused with features of the same resolution from the encoder. Finally, the fused features are upsampled to obtain a size of [missing value]. The output.

[0045] The output of stage 8 is input into a linear layer to obtain the OCT image segmentation result.

[0046] In this embodiment, as Figure 3 As shown, the data processing flow of the channel space additive attention block includes: The input features are first processed through a 3×3 depthwise separable convolution to encode relative positions, followed by layer normalization. Then, channel-space additive attention blocks are applied to compute attention, and the results are fed into a multilayer perceptron module. Three residual connections are used in this process to help mitigate gradient vanishing and enhance model stability.

[0047] 1. Obtain the key, value, and query by performing depthwise separable convolution and linear layers on the input. The formula is expressed as:

[0048] in, For key, For value and For querying; Input to the channel space additive attention block.

[0049] 2. Input the query into the spatial module and the channel module in sequence, and input the key into the channel module and the spatial module in sequence. Then, concatenate the outputs and fuse them with the values ​​to obtain the channel-spatial additive attention output. The formula is expressed as:

[0050] in, This indicates the channel-space additive attention output. Indicates the channel module. Represents a spatial module.

[0051] 1) such as Figure 4 As shown, the data processing procedure for the space module is as follows:

[0052] 2) such as Figure 4 As shown, the data processing procedure of the channel module is as follows:

[0053] in, This indicates the output characteristics of the space module. This indicates the output characteristics of the channel module. This indicates max pooling. Indicates average pooling. Represents convolution; Indicates the input of the space module. This indicates the input of the channel module.

[0054] 3. The channel-space additive attention output and input are concatenated and then input into a multilayer perceptron to obtain the output features of the channel-space additive attention block; the formula is expressed as:

[0055] in, Indicates a linear layer. This represents the output characteristics of the channel space additive attention block.

[0056] In this embodiment, as Figure 5 As shown, the data processing flow for region-aware clustering attention blocks includes: 1. Perform average pooling on the key and value to obtain... and and through and The affinity matrix A is generated by the following formula:

[0057] 2. Based on the affinity matrix, the local density of the visual token is determined using (k-Nearest Neighbors-based Density PeakClustering, DPC-kNN). The calculation formula is as follows:

[0058] in, This indicates that the index in A is , Visual tokens; Visual tokens Local density, KNN represents visual tokens The k nearest neighbors.

[0059] 3. Determining the cluster centers of visual tokens based on their local density: Define a distance metric for each visual token. Visual tokens representing measurements The minimum distance between visual tokens and other tokens with higher density. (The minimum distance between each visual token) The defined formula is expressed as:

[0060] in, Indicates distance index, This represents local density. The score of the visual token at the cluster center is the local product density. and minimum distance The product of, expressed as A higher score indicates a greater density and distance between visual tokens, making them more likely to be cluster centers. The top k visual tokens with the highest scores are selected as cluster centers, and the remaining tokens are assigned to the nearest cluster centers with higher density.

[0061] 4. Cluster each visual token according to its cluster center to obtain a cluster set, expressed by the formula:

[0062] in, Let i represent the i-th cluster set. Indicates a visual token. This represents the importance score for each token.

[0063] Concatenate the cluster set and the key to obtain the merged key; concatenate the cluster set and the value to obtain the merged value.

[0064] in, Indicates the key after fusion. This represents the value after fusion.

[0065] 5. Input the fused key, fused value, and query into the channel spatial additive attention block and multilayer perceptron in sequence to obtain the output features of the region-aware clustering attention block.

[0066] This method is compared with other existing methods, as follows: Figure 1 As shown in Table 1, the experimental data and results are as follows: Figure 6 As shown.

[0067] Table 1

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for OCT image segmentation of intravascular fibrous plaques based on a U-shaped hybrid network, characterized in that, include: The OCT image is input into the target U-shaped hybrid network to obtain the OCT image segmentation result; The target U-shaped hybrid network includes: an encoder extracting multi-scale features of the OCT image; and a decoder progressively restoring the spatial resolution and fusing the multi-scale features to obtain the OCT image segmentation result. The decoder performs feature fusion through channel-space additive attention blocks; the channel-space additive attention block includes: performing depthwise separable convolutions and linear layers on the input to obtain keys, values, and queries; Input the query into the spatial module and the channel module in sequence, input the key into the channel module and the spatial module in sequence, and then concatenate the output and merge it with the value to obtain the channel-spatial additive attention output; The channel-space additive attention output and the input are concatenated and then input into a multilayer perceptron to obtain the output features of the channel-space additive attention block.

2. The method according to claim 1, characterized in that, The encoder includes: Phase 1: Using image patch embedding and two region-aware clustering attention blocks, a size of [size missing] is [value missing]. The input is converted to a size of Features; Phase 2: Transform the output feature size of Phase 1 into a new size using token merging and four region-aware clustering attention blocks. ; Phase 3: Transform the output feature size of Phase 2 into a new size using token merging and four region-aware clustering attention blocks. ; Stage 4: Transform the output feature size of Stage 3 into a new size using token merging and two region-aware clustering attention blocks. .

3. The method according to claim 2, characterized in that, The region-aware clustering attention block includes: Get the key, value, and query based on the input; The fused key, fused value, and query are obtained by density peak clustering based on k nearest neighbors; The fused key, fused value, and query are sequentially input into the channel space additive attention block and the multilayer perceptron to obtain the output features of the region-aware clustering attention block.

4. The method according to claim 3, characterized in that, The process of obtaining the fused key, fused value, and query through density peak clustering based on k-nearest neighbors includes: Perform average pooling on the key and value to obtain and and through and The affinity matrix A is generated by the following formula: Based on the affinity matrix, the local density of the visual token is determined using the following formula: in, This indicates that the index in A is , Visual tokens; Visual tokens Local density, KNN represents visual tokens The k nearest neighbors; Determining the cluster centers of visual tokens based on their local density: Define a distance metric for each visual token. Visual tokens representing measurements The minimum distance between each visual token and other tokens with higher density. The defined formula is expressed as: in, Indicates distance index, This represents local density. The score of the visual token at the cluster center is the local product density. and minimum distance The product of, expressed as The top k visual tokens with the highest scores are used as cluster centers, and the remaining tokens are assigned to the nearest cluster centers with higher density. Clustering is performed on each visual token based on its cluster center to obtain a cluster set, expressed by the formula: in, Let i represent the i-th cluster set. Indicates a visual token. This represents the importance score for each token; By concatenating the cluster set and the key, we obtain the merged key; by concatenating the cluster set and the value, we obtain the merged value.

5. The method according to claim 1, characterized in that, The space module includes: in, This indicates the output characteristics of the space module. This indicates max pooling. Indicates average pooling. Represents convolution. Input to the channel space additive attention block.

6. The method according to claim 1, characterized in that, The channel module includes: in, This indicates the output characteristics of the channel module. Input to the channel space additive attention block.

7. The method according to claim 1, characterized in that, The process of sequentially inputting queries into the spatial module and the channel module, sequentially inputting keys into the channel module and the spatial module, and then concatenating the outputs and fusing them with the values ​​to obtain the channel-spatial additive attention output includes: in, This indicates the channel-space additive attention output. Indicates the channel module. Represents a spatial module.

8. The method according to claim 1, characterized in that, The decoder performs feature fusion through channel-space additive attention blocks, including: in, For upsampling, Add an attention block to the channel space. for convolution; The output features for stage 5, For the output features of stage 6, The output features for stage 7 This is the output feature for stage 8.

Images