A photoacoustic image reconstruction method based on multi-scale coordinate gated convolution

Abstract

The application discloses a kind of based on multiscale coordinate gated convolution photoacoustic image reconstruction method, the method includes the following steps: step 1: multiscale coordinate gated convolution module design;Step 2: photoacoustic image reconstruction network model architecture design;Step 3: photoacoustic image reconstruction.This method can dynamically adjust the size and shape of convolution kernel by introducing multiscale coordinate gated convolution module adaptive to PSF spatial variation, to adapt to the change of PSF at different spatial positions, better capture image details, realize the adaptive processing of features in different positions of image, compensate the image blur and distortion caused by PSF variation, thereby significantly improve the clarity and resolution of reconstructed photoacoustic image, realize higher resolution image reconstruction, meet the high requirements of clinical imaging and diagnosis.Experimental results show that the application significantly improves the imaging quality in photoacoustic image reconstruction, provides a new possibility for clinical application of photoacoustic imaging.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and relates to a photoacoustic image reconstruction method, in particular to a photoacoustic image reconstruction method based on multi-scale coordinate gated convolution. BACKGROUND

[0002] Photoacoustic tomography (PAT) systems are widely used for imaging small animals or human tissues. As a non-invasive biomedical imaging technique, photoacoustic imaging (PAI) can reveal the light absorption characteristics of tissues and obtain functional information combined with molecular probes. However, the data volume of full-sampling photoacoustic signals is huge, leading to significant increase in equipment cost and image reconstruction time. In order to reduce cost and improve reconstruction speed, some ring array systems reduce the number of transducer elements and control the sampling viewing angle, but this will reduce the quality of the reconstructed image. Traditional PAT reconstruction algorithms, such as filtered back projection (FBP) and time reversal (TR), are prone to image distortion and artifacts under limited viewing angle conditions. Although deep learning (DL) methods have made progress in improving PAT image quality in recent years, especially in handling complex nonlinear signals and improving image resolution, they still face the challenge of spatial variation of point spread function (PSF). The variation of PSF often leads to image blurring and distortion, especially under limited viewing angle conditions. SUMMARY

[0003] In order to solve the problem of image quality degradation caused by sparse sampling and limited viewing angle in traditional PAT, the application provides a photoacoustic image reconstruction method based on multi-scale coordinate gated convolution. This method introduces a multi-scale coordinate gated convolution module that adapts to the spatial variation of PSF, which can dynamically adjust the size and shape of the convolution kernel to adapt to the variation of PSF at different spatial positions, better capture image details, and realize adaptive processing of features at different positions in the image, compensating for image blurring and distortion caused by PSF variation, thereby significantly improving the clarity and resolution of reconstructed photoacoustic images, achieving higher resolution image reconstruction, and meeting the high requirements of clinical imaging and diagnosis.

[0004] The purpose of the application is achieved by the following technical solutions:

[0005] A photoacoustic image reconstruction method based on multi-scale coordinate gated convolution, comprising the following steps:

[0006] Step 1: Multi-scale coordinate gated convolution module design

[0007] Step 1.1: Input feature map After standard Conv 3x3 module h processing, preliminary features h(x) of C out channels are extracted, where H is the height, W is the width, and Cin denotes the number of input channels, h(x) is the feature map extracted by the standard Conv3x3 module; C out is the number of output feature channels;

[0008] Step 1.2: Static coordinate map Input into the multi-scale convolutional encoding module g composed of Conv n x n and Conv 1 x 1 layers, where n ≥ 1, to generate a coordinate-based gating mask

[0009]

[0010] Step 1.3: To achieve selective filtering, h(x) and g(c) are multiplied element- wise in the channel dimension, which is represented by the operation:

[0011] y = h(x) ȏ g(c);

[0012] where y is the final generated feature map;

[0013] Step 2: Photoacoustic image reconstruction network model architecture design

[0014] The photoacoustic image reconstruction network model is composed of an encoder, a decoder, and a multi-scale gating convolution module, wherein:

[0015] In the encoder part, the input image is subjected to layer-by-layer feature extraction through a series of down-sampling modules, which are composed of standard convolution modules, multi-scale gating convolution modules, and max-pooling layers. Each standard convolution module is connected to a multi-scale coordinate gating convolution module.

[0016] In the decoder part, the spatial resolution of the image is restored layer by layer through up-sampling modules, which are composed of standard convolution modules, multi-scale gating convolution modules, and up-sampling layers. Each standard convolution module is connected to a multi-scale coordinate gating convolution module.

[0017] In the photoacoustic image reconstruction network model, a skip connection is introduced to directly pass the output feature map of each layer of the encoder to the corresponding layer of the decoder.

[0018] Step 3: Photoacoustic image reconstruction

[0019] Step 3.1: Create a photoacoustic image dataset, which is divided into training, validation, and test sets in proportion;

[0020] Step 3.2: Load the images in the training set into the photoacoustic image reconstruction network model for training. The network parameters are iteratively updated by the optimizer. The performance of the photoacoustic image reconstruction network model on the training set and the validation set is monitored by the loss function to ensure that the performance of the photoacoustic image reconstruction network model on the validation set is optimal and to prevent overfitting.

[0021] Step 3.3: Reconstruct the images in the test set using the photoacoustic image reconstruction network model.

[0022] Compared with the prior art, the present application has the following advantages:

[0023] 1. The multi-scale coordinate gated convolution module designed in the present application can selectivity filter and feature extraction at spatial positions through coordinate attention according to the spatial variation characteristics of PSF, not only enhancing the feature extraction capability of the network when processing different spatial positions and complex tissue structures, but also better restoring image details, while still maintaining high-quality image reconstruction effect in the case of limited viewing angle and insufficient sampling.

[0024] 2. The present application effectively reduces the parameter amount in the calculation process by introducing n x n and 1 x 1 point convolution operations for coordinate coding, reducing the computational burden, which enables the model to complete the image reconstruction task in a shorter time while ensuring high precision, significantly improving the overall computing efficiency.

[0025] 3. The multi-scale convolution gating mechanism can adaptively adjust the size of the convolution kernel according to the dimension and spatial feature distribution of the input image, enhancing the adaptability of the model in different photoacoustic imaging scenarios, and providing a more flexible and efficient solution for diversified applications.

[0026] 4. The experimental results show that the present application significantly improves the imaging quality in photoacoustic image reconstruction, providing new possibilities for the clinical application of photoacoustic imaging. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The multi-scale coordinate gated convolution module proposed in the present application is shown in the figure.

[0028] Figure 2 The deep learning model structure integrating the multi-scale coordinate gated convolution proposed in the present application is shown in the figure.

[0029] Figure 3 The application effect example of the method proposed in the present application in sparse sampling photoacoustic image reconstruction is shown in the figure. DETAILED DESCRIPTION

[0030] The technical solutions of the present application are further described below in conjunction with the drawings, but are not limited thereto, and any modifications or equivalent replacements to the technical solutions of the present application without departing from the spirit and scope of the present application shall be covered in the protection scope of the present application.

[0031] The present application provides a multi-scale coordinate gated convolution based photoacoustic image reconstruction method, which comprises the following steps:

[0032] Step 1: Design of multiscale coordinate gate convolution module

[0033] To adapt to the spatial variation characteristics of PSF, while reducing the number of parameters required for calculation and reducing the computational burden, the present application proposes a multiscale coordinate gate convolution module for coordinate coding through n×n and 1×1 convolution (where n≥1). As shown in Figure 1 , the input feature map is first processed through a standard Conv 3×3 module h to extract the generated C out channel preliminary features h(x), where H is the height, W is the width, C in represents the number of input channels, and h(x) is the feature map extracted by the standard Conv 3×3 module; C out is the number of output feature channels. The static coordinate map is input into the multiscale convolution coding module g, which is composed of Conv n×n and Conv 1×1 layers to generate a coordinate-based gate mask The mask is completely dependent on the coordinates. To achieve selective filtering, the preliminary extracted feature map h(x) is element-wise multiplied (Hadamard product) with g(c) in the channel dimension, which can be represented as:

[0034] y=h(x)⊙g(c)。

[0035] where y is the final generated feature map.

[0036] Step 2: Design of photoacoustic image reconstruction network model architecture

[0037] The photoacoustic image reconstruction network model proposed by the present application is based on the classic encoding-decoding structure U-Net network, and integrates the multiscale coordinate gate convolution (Multiscale-Gate) module in the convolution and deconvolution process to enhance the network's spatial feature extraction ability at different scales. As shown in Figure 2 , the entire network structure is composed of an encoder, a decoder, and a multiscale gate convolution module, forming a powerful image reconstruction framework.

[0038] In the encoder part, the input image is subjected to a series of down-sampling modules for layer-by-layer feature extraction, as shown in Figure 2 . Each down-sampling module is composed of a standard convolution module (Conv), a multiscale gate convolution module (MGate), and a maximum pooling layer (Down). The standard convolution module (Conv) is composed of a standard Conv 3×3 convolution layer, a batch normalization (Batch Normalization, BN) layer, and a ReLU activation function, and is connected with the multiscale coordinate gate convolution module after it to enhance the feature extraction ability.

[0039] At the decoder part, the spatial resolution of the image is recovered layer by layer by upsampling modules, as shown in Figure 2 Each upsampling module is composed of a standard convolution module (Conv), a multi-scale gated convolution module (MGate), and an upsampling layer (Up). To further enhance feature extraction, a multi-scale coordinate gated convolution module is connected after each standard convolution module. This design not only improves the overall quality of image reconstruction but also enhances the network's ability to handle complex photoacoustic images.

[0040] In addition, a skip connection is introduced in the network, which directly transmits the output feature maps of each layer of the encoder to the corresponding layer of the decoder. This connection preserves more detailed information, enabling the decoder to combine features of different scales for reconstruction and generating more accurate photoacoustic images.

[0041] In the entire network architecture, the multi-scale gated convolution module serves as the core component, adaptively handling feature variations at different spatial positions. This enables the network to better capture and restore the detailed structures in photoacoustic images, making it particularly suitable for handling complex tissue structures and nonlinear signals.

[0042] Step 3: Photoacoustic image reconstruction

[0043] Step 3.1: Create a photoacoustic image dataset and divide it into training, validation, and test sets in a certain proportion. During dataset construction, perform necessary preprocessing operations on the image data, including normalization, noise removal, and image enhancement, to ensure the consistency and quality of the input images. At the same time, generate more diverse image samples through data augmentation (such as random rotation, translation, and scaling) to improve the model's generalization ability.

[0044] Step 3.2: Load the training set images into the model for training, and iteratively update the network parameters through the optimizer. Monitor the model's performance on the training set and validation set through the loss function to ensure that the model's performance on the validation set is optimal and prevent overfitting.

[0045] Step 3.3: The model reconstructs the images in the test set and conducts a comprehensive analysis of the reconstruction results using quantitative evaluation indicators. Specifically, use PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index), and other evaluation indicators to quantitatively evaluate the image quality, measuring the clarity and structural fidelity of the reconstructed images. In addition, combined with qualitative visual analysis, verify the actual application effect of the reconstructed structure.

[0046] Step 4: Experimental verification and application

[0047] In order to verify the effectiveness of the photoacoustic image reconstruction network model, a photoacoustic image reconstruction experiment is carried out. Based on 756 public mouse brain photoacoustic images, a simulation photoacoustic image dataset with a size of 128x128 is generated using the MATLAB toolbox k-Wave. In the simulation, a 360° full-view transducer ring array is used, with a radius of 55mm, containing 128 detection elements, a center frequency of 5.5MHz, a bandwidth of 80%, and a sound speed of 1540m / s. The full-ring 64-element subsampled photoacoustic signal reconstruction image is used as the model input, and the original 128-element full-sampled photoacoustic signal reconstruction image is used as the target image. The dataset is divided into 70% training set, 10% validation set and 20% test set. Due to the small image size, the n in the multi-scale coordinate gated convolution module is set to 1. During the model training process, the mean square error (MSE) loss function is used to measure the difference between the reconstructed image and the true image, and the Adam optimizer is used to adjust the network parameters to ensure that the performance of the model on the validation set is optimal. In the test stage, the model reconstructs the images in the test set, and the PSNR (peak signal-to-noise ratio), SSIM (structural similarity) and other evaluation indicators are used to quantitatively analyze the reconstruction results, so as to comprehensively evaluate the performance of the method proposed in the application. In the simulation of human brain photoacoustic image reconstruction, the deep learning model integrating multi-scale coordinate gated convolution proposed in the application achieves a PSNR of 32.83dB and an SSIM of 0.89.

[0048] Figure 3 The visualization results of the model in the sparse sampling photoacoustic image reconstruction are shown. The experimental results show that the model of the application performs excellently in the photoacoustic image reconstruction task, successfully captures fine structural details, and can effectively improve the imaging quality. Under the condition of limited view angle and sparse sampling, the model can still generate high-quality photoacoustic images, which provides strong support for the further development of photoacoustic imaging in clinical applications.

Claims

1. A photoacoustic image reconstruction method based on multi-scale coordinate gated convolution, characterized in that The method comprises the following steps: Step 1: Multi-scale coordinate gated convolution module design Step 1.1: Input feature map After the standard Conv 3×3 module Processing, extraction and generation Preliminary characteristics of the channel , where H is the height and W is the width, Indicates the number of input channels, Feature maps extracted for the standard Conv 3×3 module; is the number of channels of the output feature; Step 1.2: Static Coordinate Graph Input to the multi-scale convolutional encoding module consisting of Conv n×n and Conv 1×1 layers Where n≥1, generate a coordinate-based gated mask ; Step 1.3: To achieve selective filtering, and Perform element-by-element multiplication on the channel dimension, and its operation is expressed as: ； in, To finally generate the feature map; Step 2: Photoacoustic image reconstruction network model architecture design The photoacoustic image reconstruction network model consists of an encoder, a decoder, and a multi-scale coordinate gated convolution module, where: In the encoder part, the input image is passed through a series of downsampling modules for layer-by-layer feature extraction. The downsampling module consists of a standard convolution module, a multi-scale coordinate gated convolution module, and a maximum pooling layer. Each standard convolution module is connected to a multi-scale coordinate gated convolution module. In the decoder part, the spatial resolution of the image is restored layer by layer through the upsampling module. The upsampling module consists of a standard convolution module, a multi-scale coordinate gated convolution module and an upsampling layer. Each standard convolution module is connected to a multi-scale coordinate gated convolution module. Skip connections are introduced into the photoacoustic image reconstruction network model to directly transfer the output feature maps of each layer of the encoder to the corresponding layer of the decoder; Step 3: Photoacoustic image reconstruction Step 3.1: Create a photoacoustic image dataset and divide it into training set, validation set, and test set in proportion; Step 3.2: Load the images in the training set into the photoacoustic image reconstruction network model for training. Use the optimizer to iteratively update the network parameters. Use the loss function to monitor the performance of the photoacoustic image reconstruction network model on the training set and validation set to ensure that the performance of the photoacoustic image reconstruction network model on the validation set is optimal and to prevent overfitting. Step 3.3: Use the photoacoustic image reconstruction network model to reconstruct the images in the test set.

2. The photoacoustic image reconstruction method based on multi-scale coordinate gated convolution according to claim 1, characterized in that The standard convolution module consists of a standard Conv 3×3 convolution layer, a batch normalization layer, and a ReLU activation function.

3. The photoacoustic image reconstruction method based on multi-scale coordinate gated convolution according to claim 1, characterized in that During the construction of the photoacoustic image dataset, the image data was normalized, denoised, and preprocessed with image enhancement to ensure the consistency and quality of the input images. At the same time, more diverse image samples were generated through data augmentation to improve the generalization ability of the model.

Images