Image data analysis method based on display shape prior injection

Abstract

The invention discloses an image data analysis method based on display shape prior injection, which is based on an improved Ret-UNet model architecture and comprises the following steps: preprocessing input image data, inputting the preprocessed image into an encoder of a Ret-UNet model, and adjusting a receptive field of each pixel in the image through a prior matrix based on Euclidean distance; performing feature extraction and fusion on the low-resolution feature map processed by the encoder in a bottleneck layer; the feature map enters a decoder to be up-sampled layer by layer, the resolution of the feature map is gradually restored to the size of an original image through deconvolution, and multi-scale features in an encoder are transmitted to a corresponding decoding layer through jump connection; the output of the decoder maps the feature map into final output prediction through a convolutional layer; and performing post-processing on the final prediction result to obtain a final image analysis result. According to the scheme, the performance of the model in a complex image segmentation task is remarkably improved, and higher accuracy and robustness are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data analysis, and in particular to an image data analysis method based on display shape prior injection. Background Art

[0002] With the advancement of deep learning technology, especially the widespread application of convolutional neural networks (CNNs) in image processing, the analysis and processing of image data has made significant progress. However, existing CNN models perform poorly in capturing global spatial relationships in images, especially when dealing with complex shapes or structures of different scales. Existing solutions, such as U-Net and its variants, mainly rely on local feature extraction and have difficulty in fully understanding the global spatial relationships in images. In recent years, although self-attention mechanisms and visual Transformer models have been introduced, these methods are still insufficient in capturing spatial continuity and shape consistency. Summary of the invention

[0003] In view of the above technical problems, the present invention provides an image data analysis method based on display shape prior injection.

[0004] The present invention is implemented by adopting the following technical scheme: an image data analysis method based on display shape prior injection, based on an improved Ret-UNet model architecture, comprising the following steps: Step S1: preprocess the input image data to adapt to the input requirements of the model, and input the data into the Ret-UNet model encoder; Step S2: The preprocessed image is input into the encoder of the Ret-UNet model, and the receptive field of each pixel in the image is adjusted by a prior matrix based on Euclidean distance; Step S3: The low-resolution feature map processed by the encoder is subjected to feature extraction and fusion in the bottleneck layer; Step S4: The feature map enters the decoder and is upsampled layer by layer. The resolution of the feature map is gradually restored to the original image size through deconvolution, and the multi-scale features in the encoder are transferred to the corresponding decoding layer using skip connections; Step S5: The output of the decoder is mapped to the final output prediction through a convolutional layer; Step S6: The final prediction result is subjected to a post-processing step to obtain the final image analysis result.

[0005] Specifically, the encoder of the Ret-UNet model includes a downsampling layer and a corresponding series of convolutional layers. Each downsampling block of the downsampling layer convolves the input features through a 3x3 convolutional layer to extract low-level features, and uses strided convolution to reduce the resolution of the image. The encoder generates feature maps of different resolutions in each layer and stores the feature maps for jump connections in the decoder.

[0006] Specifically, the last three layers of the encoder are configured with a Ret-Block module to capture the global spatial relationship in the image through a conservative self-attention mechanism.

[0007] Specifically, the bottleneck layer sets two Ret-Block modules, which fully combine local and global information through a reserved self-attention mechanism combined with deep convolution operations to provide feature input for the decoder.

[0008] Specifically, each decoding layer of the decoder is composed of an upsampling module, and a 3x3 deconvolution layer is used to improve the resolution of the feature map. The last three layers of the decoder are also provided with a Ret-Block module.

[0009] Specifically, the skip connection establishes a cross-layer connection between the encoder and the decoder, directly transferring the multi-scale feature map in the encoder to the corresponding layer of the decoder.

[0010] Specifically, the conservative self-attention mechanism introduces a priori matrix based on Euclidean distance on the basis of the traditional self-attention mechanism to emphasize the shape information in the image.

[0011] Specifically, the Ret-Block module includes deep convolution, LayerNorm layer normalization, conservative self-attention and feedforward network to effectively capture local and global information in the image.

[0012] The beneficial effect of the present invention is that the present invention enhances the model's ability to capture global spatial relationships when analyzing image data by introducing explicit shape priors. The method uses a prior matrix based on Euclidean distance and combines it with a self-attention mechanism to form a new architecture with Retentive Self-Attention (RetSA). The architecture can effectively fuse local features and global contextual information to improve the processing capabilities of complex image structures, and is particularly suitable for anatomical structure segmentation tasks in medical images. This scheme significantly improves the performance of the model in complex image segmentation tasks, especially when processing multi-scale structures in medical images, showing higher accuracy and robustness. Experimental results show that this scheme outperforms the existing state-of-the-art models on multiple public data sets (such as ACDC, CAMUS), specifically in terms of indicators such as Dice coefficient, Hausdorff distance, and average surface distance. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0014] Figure 1 This is a diagram of the Ret-UNet model architecture in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0016] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0017] The following is combined with Figure 1 , some embodiments of the present invention are described in detail. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0018] The present invention proposes an image data analysis method based on display shape prior injection, based on an improved Ret-UNet model architecture, comprising the following steps: Step S1: preprocess the input image data to adapt to the input requirements of the model, and input the data into the Ret-UNet model encoder; Step S2: The preprocessed image is input into the encoder of the Ret-UNet model, and the receptive field of each pixel in the image is adjusted by a prior matrix based on Euclidean distance; Step S3: The low-resolution feature map processed by the encoder is subjected to feature extraction and fusion in the bottleneck layer; Step S4: The feature map enters the decoder and is upsampled layer by layer. The resolution of the feature map is gradually restored to the original image size through deconvolution, and the multi-scale features in the encoder are transferred to the corresponding decoding layer using skip connections; Step S5: The output of the decoder is mapped to the final output prediction through a convolutional layer; Step S6: The final prediction result is subjected to a post-processing step to obtain the final image analysis result.

[0019] In this embodiment, the encoder of the Ret-UNet model includes a downsampling layer and a corresponding series of convolutional layers. Each downsampling block of the downsampling layer convolves the input features through a 3x3 convolutional layer to extract low-level features, and uses stride convolution to reduce the resolution of the image. The encoder generates feature maps of different resolutions in each layer and stores the feature maps for jump connections in the decoder. The last three layers of the encoder are set with Ret-Block modules to capture the global spatial relationship in the image through a reserved self-attention mechanism.

[0020] In this embodiment, two Ret-Block modules are set in the bottleneck layer, which fully combine local and global information through the reserved self-attention mechanism and deep convolution operation to provide feature input for the decoder.

[0021] In this embodiment, each decoding layer of the decoder is composed of an upsampling module, and a 3x3 deconvolution layer is used to improve the resolution of the feature map. The last three layers of the decoder are also provided with a Ret-Block module.

[0022] In this embodiment, a skip connection establishes a cross-layer connection between the encoder and the decoder, directly transferring the multi-scale feature map in the encoder to the corresponding layer of the decoder.

[0023] In this embodiment, the conservative self-attention mechanism introduces a prior matrix based on Euclidean distance on the basis of the traditional self-attention mechanism to emphasize the shape information in the image; the Ret-Block module includes deep convolution, LayerNorm layer normalization, conservative self-attention and feedforward network to effectively capture local and global information in the image.

[0024] In one embodiment, the image data analysis based on display shape prior injection includes the following detailed steps: 1. Data preprocessing: The input image data first undergoes preprocessing steps, including image size normalization, data augmentation and other operations, to improve the generalization ability of the model and ensure the size consistency of the input image so that it adapts to the input requirements of the model.

[0025] 2. Encoding process: The preprocessed image is input into the encoder of Ret-UNet. The encoder extracts low-level features through a series of convolution and downsampling layers, while reducing the resolution of the feature map layer by layer. The encoder introduces the Retention Self-Attention Mechanism (RetSA) in the last few layers, which enhances the ability to capture global spatial information through a prior matrix based on Euclidean distance, which is particularly suitable for the expression of complex structures in images.

[0026] 3. Bottleneck processing: The low-resolution feature map processed by the encoder is further extracted and fused in the bottleneck layer. The Ret-Block module in the bottleneck layer uses a multi-layer self-attention mechanism combined with deep convolution operations to fully combine local and global information and provide high-quality feature input for the decoder.

[0027] 4. Decoding process: The feature map enters the decoder after passing through the bottleneck layer. The decoder upsamples layer by layer, gradually restores the resolution of the feature map to the original image size through deconvolution, and uses jump connections to pass the multi-scale features in the encoder to the corresponding decoding layer, thereby achieving multi-scale fusion of feature information. The last few layers of the decoder also contain Ret-Block modules to further enhance the understanding of global spatial relationships.

[0028] 5. Output generation: The output of the decoder maps the feature map to the final output prediction (such as a segmented image) through a 1x1 convolutional layer. The output has the same size as the input image and marks the boundary of the target area or the classification result.

[0029] 6. Post-processing: The final prediction results go through post-processing steps, such as smoothing and boundary refinement, to improve the interpretability and accuracy of the predictions and obtain the final image analysis results.

[0030] In another embodiment, the present invention improves upon the U-Net architecture and proposes a Ret-UNet model. The key technical points include: Reservative Self-Attention Mechanism (RetSA): Based on the traditional self-attention mechanism, a prior matrix based on Euclidean distance is introduced to emphasize the shape information in the image and enhance the modeling ability of global spatial relationships.

[0031] Ret-Block module: This module consists of deep convolution, layer normalization (LayerNorm), conservative self-attention and feedforward network, which can effectively capture local and global information in the image.

[0032] Multi-scale feature fusion: Through skip connections, the multi-scale information in the encoder and decoder is fused to ensure that information at different levels in the segmentation task can be effectively utilized.

[0033] In this embodiment, the model design diagram is as follows Figure 1 As shown, the specific workflow is as follows: 1. Input image: The input 2D image (such as MRI or CT scan in medical images) first enters the encoder part of Ret-UNet.

[0034] 2. Encoder: The encoder consists of a series of convolutional layers and downsampling layers. Each downsampling block convolves the input features through a 3x3 convolutional layer to extract low-level features, while using stride convolution to reduce the resolution of the image.

[0035] In the last three layers of the encoder, Ret-UNet introduces the Ret-Block module, which captures the global spatial relationship in the image through a conservative self-attention mechanism.

[0036] Each layer of the encoder generates feature maps of different resolutions, which are stored for skip connections in the decoder.

[0037] 3. Bottleneck layer: The feature map after downsampling by the encoder enters the bottleneck layer at the bottom layer. This layer is the centralized processing area of ​​feature information, further enhancing the fusion of global and local features.

[0038] The bottleneck layer contains two Ret-Blocks. These modules introduce the Retained Self-Attention (RetSA) mechanism and use a prior matrix based on Euclidean distance to adjust the receptive field of each pixel in the image, thereby enhancing the ability to capture shapes.

[0039] 4. Decoder: The decoder is responsible for gradually restoring the compressed feature map in the encoder back to a high-resolution image.

[0040] Each decoding layer consists of an upsampling module, using a 3x3 deconvolution layer (Transposed Convolutions) to increase the resolution of the feature map. Each upsampling doubles the resolution of the feature map.

[0041] In the last three layers of the decoder, the Ret-Block module is also used to further enhance the model's capture of global spatial information.

[0042] 5. Skip Connections: The Ret-UNet model uses the classic skip connection structure of U-Net, establishes a cross-layer connection between the encoder and the decoder, and directly transfers the multi-scale feature map in the encoder to the corresponding layer of the decoder. This allows high-resolution local information to be retained and fused with low-resolution global information, thereby improving the segmentation accuracy of the model.

[0043] 6. Output layer: The last convolutional layer of the decoder uses 1x1 convolution to map the decoded feature map to the final output layer, generating a prediction map of the same size as the input image (such as a segmented image).

[0044] For the aforementioned embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily required by the present application.

[0045] The above embodiments describe the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the changes and modifications made by those skilled in the art shall be within the scope of protection of the appended claims of the present invention without departing from the spirit and scope of the present invention.

Claims

1. An image data analysis method based on display shape prior injection, characterized in that: Based on the improved Ret-UNet model architecture, the following steps are included: Step S1: preprocess the input image data to adapt to the input requirements of the model, and input the data into the Ret-UNet model encoder; Step S2: The preprocessed image is input into the encoder of the Ret-UNet model, and the receptive field of each pixel in the image is adjusted by a prior matrix based on Euclidean distance; Step S3: The low-resolution feature map processed by the encoder is subjected to feature extraction and fusion in the bottleneck layer; Step S4: The feature map enters the decoder and is upsampled layer by layer. The resolution of the feature map is gradually restored to the original image size through deconvolution, and the multi-scale features in the encoder are transferred to the corresponding decoding layer using skip connections; Step S5: The output of the decoder is mapped to the final output prediction through a convolutional layer; Step S6: The final prediction result is subjected to a post-processing step to obtain the final image analysis result.

2. The image data analysis method based on display shape prior injection as claimed in claim 1, characterized in that: The encoder of the Ret-UNet model includes a downsampling layer and a corresponding series of convolutional layers. Each downsampling block of the downsampling layer convolves the input features through a 3x3 convolutional layer to extract low-level features, and uses strided convolution to reduce the resolution of the image. The encoder generates feature maps of different resolutions in each layer and stores the feature maps for jump connections in the decoder.

3. The image data analysis method based on display shape prior injection as claimed in claim 2, characterized in that: The last three layers of the encoder are configured with a Ret-Block module to capture the global spatial relationship in the image through a conservative self-attention mechanism.

4. The image data analysis method based on display shape prior injection as claimed in claim 3, characterized in that: The bottleneck layer is provided with two Ret-Block modules, which fully combine local and global information through a reserved self-attention mechanism combined with a deep convolution operation to provide feature input for the decoder.

5. The image data analysis method based on display shape prior injection as claimed in claim 4, characterized in that: Each decoding layer of the decoder is composed of an upsampling module, and a 3x3 deconvolution layer is used to improve the resolution of the feature map. The last three layers of the decoder are also provided with a Ret-Block module.

6. The image data analysis method based on display shape prior injection as claimed in claim 5, characterized in that: The skip connection establishes a cross-layer connection between the encoder and the decoder, directly transferring the multi-scale feature map in the encoder to the corresponding layer of the decoder.

7. The image data analysis method based on display shape prior injection as claimed in claim 6, characterized in that: The conservative self-attention mechanism introduces a priori matrix based on Euclidean distance on the basis of the traditional self-attention mechanism, emphasizing the shape information in the image.

8. The image data analysis method based on display shape prior injection as claimed in claim 7, characterized in that: The Ret-Block module includes deep convolution, LayerNorm layer normalization, conservative self-attention and feedforward network, which effectively captures local and global information in the image.

Images