Photoacoustic image reconstruction method based on multi-scale coordinate gating convolution

Abstract

The invention discloses a photoacoustic image reconstruction method based on multi-scale coordinate gating convolution. The method comprises the following steps: step 1, designing a multi-scale coordinate gating convolution module; 2, designing a photoacoustic image reconstruction network model architecture; and 3, reconstructing a photoacoustic image. According to the method, by introducing a multi-scale coordinate gating convolution module adaptive to PSF spatial change, the size and shape of a convolution kernel can be dynamically adjusted to adapt to the change of PSF at different spatial positions, image details are better captured, adaptive processing of features at different positions in an image is realized, image blurring and distortion caused by the PSF change are compensated, and the image quality is improved. Therefore, the definition and the resolution of the reconstructed photoacoustic image are remarkably improved, image reconstruction with higher resolution is realized, and high requirements of clinical imaging and diagnosis are met. Experimental results show that the imaging quality is remarkably improved in photoacoustic image reconstruction, and new possibility is provided for clinical application of photoacoustic imaging.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and relates to a photoacoustic image reconstruction method, and specifically relates to a photoacoustic image reconstruction method based on multi-scale coordinate gated convolution. Background Art

[0002] Photoacoustic tomography (PAT) systems are widely used for imaging of the whole body of small animals or human tissues. As a non-invasive biomedical imaging technique, photoacoustic imaging (PAI) can reveal the light absorption properties of tissues and obtain functional information in combination with molecular probes. However, the data volume of fully sampled photoacoustic signals is huge, resulting in a significant increase in equipment cost and image reconstruction time. In order to reduce costs and increase reconstruction speed, some annular array systems reduce the number of transducer elements and control the sampling angle, but this will reduce the quality of the reconstructed image. Traditional PAT reconstruction algorithms, such as filtered back projection (FBP) and time reversal (TR), are prone to image distortion and artifacts when the viewing angle is limited. Although deep learning (DL) methods have made progress in improving PAT image quality in recent years, especially in processing complex nonlinear signals and improving image resolution, they still face challenges caused by spatial variations of the point spread function (PSF). Changes in PSF often lead to image blur and distortion, which are more obvious when the viewing angle is limited. Summary of the invention

[0003] In order to solve the problem of image quality degradation caused by sparse sampling and limited viewing angle in traditional PAT, the present invention provides a photoacoustic image reconstruction method based on multi-scale coordinate gated convolution. By introducing a multi-scale coordinate gated convolution module adapted to the spatial variation of PSF, the method can dynamically adjust the size and shape of the convolution kernel to adapt to the variation of PSF at different spatial positions, better capture image details, realize adaptive processing of features at different positions in the image, and compensate for image blur and distortion caused by PSF variation, thereby significantly improving the clarity and resolution of the reconstructed photoacoustic image, realizing higher resolution image reconstruction, and meeting the high requirements of clinical imaging and diagnosis.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] A photoacoustic image reconstruction method based on multi-scale coordinate gated convolution comprises the following steps:

[0006] Step 1: Multi-scale coordinate gated convolution module design

[0007] Step 1.1: Input feature map After processing by the standard Conv 3×3 module h, the generated C out The preliminary features of the channel are h(x), where H is the height, W is the width, and Cin represents the number of input channels, h(x) is the feature map extracted by the standard Conv3×3 module; C out is the number of channels of the output feature;

[0008] Step 1.2: Static Coordinate Graph Input to a multi-scale convolutional encoding module g consisting of Conv n×n and Conv 1×1 layers, where n ≥ 1, to generate a coordinate-based gated mask

[0009]

[0010] Step 1.3: To achieve selective filtering, h(x) and g(c) are element-wise multiplied in the channel dimension. The operation is expressed as:

[0011] y=h(x)⊙g(c);

[0012] Among them, y is the final generated feature map;

[0013] Step 2: Photoacoustic image reconstruction network model architecture design

[0014] The photoacoustic image reconstruction network model consists of an encoder, a decoder, and a multi-scale gated convolution module, where:

[0015] In the encoder part, the input image is subjected to a series of downsampling modules for layer-by-layer feature extraction. The downsampling modules consist of a standard convolution module, a multi-scale gated convolution module, and a maximum pooling layer. Each standard convolution module is connected to a multi-scale coordinate gated convolution module.

[0016] In the decoder part, the spatial resolution of the image is restored layer by layer through an upsampling module, which consists of a standard convolution module, a multi-scale gated convolution module and an upsampling layer, and each standard convolution module is connected to a multi-scale coordinate gated convolution module;

[0017] The skip connection is introduced into the photoacoustic image reconstruction network model to directly transfer the output feature maps of each layer of the encoder to the corresponding layer of the decoder;

[0018] Step 3: Photoacoustic image reconstruction

[0019] Step 3.1: Prepare a photoacoustic image dataset and divide it into training set, validation set and test set in proportion;

[0020] Step 3.2: Load the images in the training set into the photoacoustic image reconstruction network model for training, iteratively update the network parameters through the optimizer, and monitor the performance of the photoacoustic image reconstruction network model on the training set and the validation set through the loss function to ensure that the performance of the photoacoustic image reconstruction network model on the validation set is optimal and prevent overfitting;

[0021] Step 3.3: Use the photoacoustic image reconstruction network model to reconstruct the images in the test set.

[0022] Compared with the prior art, the present invention has the following advantages:

[0023] 1. The multi-scale coordinate gated convolution module designed in the present invention can selectively filter and extract features in spatial positions through coordinate attention according to the spatial variation characteristics of PSF, which not only enhances the feature extraction ability of the network when processing different spatial positions and complex tissue structures, but also better restores image details. At the same time, it can still maintain high-quality image reconstruction effects under conditions of limited viewing angle and insufficient sampling.

[0024] 2. The present invention introduces n×n and 1×1 point convolution operations for coordinate encoding, which effectively reduces the number of parameters in the calculation process and reduces the computational burden. This enables the model to complete the image reconstruction task in a shorter time while ensuring high accuracy, significantly improving the overall computational efficiency.

[0025] 3. The multi-scale convolution gating mechanism can adaptively adjust the convolution kernel size according to the dimension and spatial feature distribution of the input image, which enhances the adaptability of the model in different photoacoustic imaging scenarios and provides a more flexible and efficient solution for diverse applications.

[0026] 4. Experimental results show that the present invention significantly improves the imaging quality in photoacoustic image reconstruction and provides new possibilities for the clinical application of photoacoustic imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of the multi-scale coordinate gated convolution module proposed in the present invention;

[0028] Figure 2 This is a schematic diagram of the structure of the deep learning model integrating multi-scale coordinate gated convolution proposed in the present invention;

[0029] Figure 3 This is an example of the application effect of the method proposed in the present invention in sparse sampling photoacoustic image reconstruction. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0031] The present invention provides a photoacoustic image reconstruction method based on multi-scale coordinate gated convolution, the method comprising the following steps:

[0032] Step 1: Multi-scale coordinate gated convolution module design

[0033] In order to adapt to the spatial variation characteristics of PSF and reduce the number of parameters required for calculation and reduce the computational burden, the present invention proposes a multi-scale coordinate gated convolution module that performs coordinate encoding through n×n and 1×1 convolutions (where n≥1). Figure 1 As shown, first the input feature map After processing by the standard Conv 3×3 module h, the generated C out The preliminary features of the channel are h(x), where H is the height, W is the width, and C in represents the number of input channels, h(x) is the feature map extracted by the standard Conv 3×3 module; C out is the number of channels of the output feature. Static coordinate diagram The input is fed into a multi-scale convolutional encoding module g, which consists of Conv n×n and Conv 1×1 layers to generate a coordinate-based gated mask This mask is completely dependent on the coordinates. To achieve selective filtering, the initially extracted feature map h(x) is multiplied element-wise (Hadamard product) with g(c) generated by the coordinates in the channel dimension. The operation can be expressed as:

[0034] y=h(x)⊙g(c).

[0035] Among them, y is the final generated feature map.

[0036] Step 2: Photoacoustic image reconstruction network model architecture design

[0037] The photoacoustic image reconstruction network model proposed in the present invention is based on the classic encoding-decoding structure U-Net network, and integrates a multiscale coordinate gated convolution (Multiscale-Gate) module in the convolution and deconvolution process to enhance the network's ability to extract spatial features at different scales. Figure 2 As shown in the figure, the entire network structure consists of an encoder, a decoder, and a multi-scale gated convolution module, forming a powerful image reconstruction framework.

[0038] In the encoder part, the input image is passed through a series of downsampling modules for layer-by-layer feature extraction, such as Figure 2 As shown in the figure, each downsampling module consists of a standard convolution module (Conv), a multi-scale gated convolution module (MGate) and a maximum pooling layer (Down). The standard convolution module (Conv) consists of a standard Conv 3×3 convolution layer, a batch normalization (BatchNormalization, BN) layer and a ReLU activation function, and is followed by a multi-scale coordinate gated convolution module to enhance feature extraction capabilities.

[0039] In the decoder part, the spatial resolution of the image is restored layer by layer through the upsampling module, such as Figure 2 As shown in the figure, each upsampling module consists of a standard convolution module (Conv), a multi-scale gated convolution module (MGate), and an upsampling layer (Up). To further improve the feature extraction effect, each standard convolution module is connected to a multi-scale coordinate gated convolution module. This design not only improves the overall quality of image reconstruction, but also enhances the network's ability to process complex photoacoustic images.

[0040] In addition, skip connections are introduced into the network to directly pass the output feature maps of each layer of the encoder to the corresponding layer of the decoder. This connection method retains more detailed information, allowing the decoder to combine features of different scales for reconstruction, thereby generating more accurate photoacoustic images.

[0041] In the entire network architecture, the multi-scale gated convolution module is the core component that adaptively processes feature changes at different spatial locations. This enables the network to better capture and restore detailed structures in photoacoustic images, and is particularly suitable for processing complex tissue structures and nonlinear signals.

[0042] Step 3: Photoacoustic image reconstruction

[0043] Step 3.1: Create a photoacoustic image dataset and divide it into a training set, a validation set, and a test set in a certain ratio. During the dataset construction process, the image data is preprocessed, including normalization, noise removal, and image enhancement, to ensure the consistency and quality of the input image. At the same time, more diverse image samples are generated through data augmentation (such as random rotation, translation, and scaling) to improve the generalization ability of the model.

[0044] Step 3.2: Load the training set images into the model for training, iteratively update the network parameters through the optimizer, and monitor the performance of the model on the training set and validation set through the loss function to ensure that the performance of the model on the validation set is optimal and prevent overfitting.

[0045] Step 3.3: The model reconstructs the images in the test set and uses quantitative evaluation indicators to comprehensively analyze the reconstruction results. Specifically, the image quality is quantitatively evaluated using evaluation indicators such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity) to measure the clarity and structural fidelity of the reconstructed image. In addition, combined with qualitative visual analysis, the actual application effect of the reconstructed structure is verified.

[0046] Step 4: Experimental verification and application

[0047] In order to verify the effectiveness of the photoacoustic image reconstruction network model, a photoacoustic image reconstruction experiment was conducted. Based on 756 publicly available mouse brain photoacoustic images, a simulated photoacoustic image dataset with a size of 128×128 was generated using the MATLAB toolbox k-Wave. In the simulation, a 360° full-view transducer annular array with a radius of 55 mm, 128 detection elements, a center frequency of 5.5 MHz, a bandwidth of 80%, and a sound speed of 1540 m / s was used. The image reconstructed by the sub-sampling photoacoustic signal of the full ring 64 array elements was used as the model input, and the image reconstructed by the original 128 array element full sampling photoacoustic signal was used as the target image. The dataset was divided into 70% training set, 10% validation set, and 20% test set. Due to the small image size, n was set to 1 in the multi-scale coordinate gated convolution module. During the model training process, the mean square error (MSE) loss function was used to measure the difference between the reconstructed image and the real image, and the network parameters were adjusted by the Adam optimizer to ensure that the performance of the model on the validation set was optimal. During the testing phase, the model reconstructed the images in the test set and used evaluation indicators such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity) to quantitatively analyze the reconstruction results to comprehensively evaluate the performance of the method proposed in the present invention. In the simulated human brain photoacoustic image reconstruction, the deep learning model integrating multi-scale coordinate gated convolution proposed in the present invention achieved a PSNR of 32.83dB and an SSIM of 0.89.

[0048] Figure 3 The model is visualized in sparse sampling photoacoustic image reconstruction. The experimental results show that the model of the present invention performs well in the photoacoustic image reconstruction task, successfully captures fine structural details, can effectively improve the imaging quality, and can still generate high-quality photoacoustic images under the conditions of limited viewing angle and sparse sampling, which provides strong support for the further development of photoacoustic imaging in clinical applications.

Claims

1. A photoacoustic image reconstruction method based on multi-scale coordinate gated convolution, characterized in that The method comprises the following steps: Step 1: Multi-scale coordinate gated convolution module design Step 1.1: Input feature map After processing by the standard Conv 3×3 module h, the generated C out The preliminary features of the channel are h(x), where H is the height, W is the width, and C in represents the number of input channels, h(x) is the feature map extracted by the standard Conv 3×3 module; C out is the number of channels of the output feature; Step 1.2: Static Coordinate Graph Input to a multi-scale convolutional encoding module g consisting of Conv n×n and Conv 1×1 layers, where n ≥ 1, to generate a coordinate-based gated mask Step 1.3: To achieve selective filtering, h(x) and g(c) are element-wise multiplied in the channel dimension. The operation is expressed as: y=h(x)⊙g(c); Among them, y is the final generated feature map; Step 2: Photoacoustic image reconstruction network model architecture design The photoacoustic image reconstruction network model consists of an encoder, a decoder, and a multi-scale gated convolution module, where: In the encoder part, the input image is subjected to a series of downsampling modules for layer-by-layer feature extraction. The downsampling modules consist of a standard convolution module, a multi-scale gated convolution module, and a maximum pooling layer. Each standard convolution module is connected to a multi-scale coordinate gated convolution module. In the decoder part, the spatial resolution of the image is restored layer by layer through an upsampling module, which consists of a standard convolution module, a multi-scale gated convolution module and an upsampling layer, and each standard convolution module is connected to a multi-scale coordinate gated convolution module; The skip connection is introduced into the photoacoustic image reconstruction network model to directly transfer the output feature maps of each layer of the encoder to the corresponding layer of the decoder; Step 3: Photoacoustic image reconstruction Step 3.1: Prepare a photoacoustic image dataset and divide it into training set, validation set and test set in proportion; Step 3.2: Load the images in the training set into the photoacoustic image reconstruction network model for training, iteratively update the network parameters through the optimizer, and monitor the performance of the photoacoustic image reconstruction network model on the training set and the validation set through the loss function to ensure that the performance of the photoacoustic image reconstruction network model on the validation set is optimal and prevent overfitting; Step 3.3: Use the photoacoustic image reconstruction network model to reconstruct the images in the test set.

2. The photoacoustic image reconstruction method based on multi-scale coordinate gated convolution according to claim 1, characterized in that The standard convolution module consists of a standard Conv 3×3 convolution layer, a batch normalization layer, and a ReLU activation function.

3. The photoacoustic image reconstruction method based on multi-scale coordinate gated convolution according to claim 1, characterized in that During the construction of the photoacoustic image dataset, the image data is normalized, denoised, and preprocessed with image enhancement to ensure the consistency and quality of the input image. At the same time, more diverse image samples are generated through data augmentation to improve the generalization ability of the model.

Images