A photoacoustic image reconstruction method and device based on deep learning

Abstract

The embodiment of the application provides a kind of based on deep learning's photoacoustic image reconstruction method and device, wherein the method comprises: in response to PACT imaging equipment photoacoustic detection is carried out to the target to be imaged, original photoacoustic signal data is obtained by sampling;The original photoacoustic signal data is input into the preset photoacoustic image prediction model to obtain the reconstructed photoacoustic image output by the photoacoustic image prediction model, wherein the photoacoustic image prediction model is the deep learning network model trained by pre-obtained training data.In the embodiment of the application, the reconstruction of photoacoustic image is realized based on the pre-trained deep learning network model, compared with prior art, the photoacoustic image reconstruction method provided in the embodiment of the application considers the robustness of traditional algorithm and the adaptability of deep learning algorithm, and the performance is more optimal, and the imaging quality of PACT system photoacoustic image is improved.

Description

Technical Field

[0001] This application relates to the field of medical image reconstruction, and in particular to a photoacoustic image reconstruction method and apparatus based on deep learning. Background Technology

[0002] Photoacoustic Computed Tomography (PACT) is a non-destructive biomedical imaging technique that combines the advantages of high contrast in optical imaging and deep penetration in acoustic imaging, leading to its widespread application in the biomedical field. The basic principle of PACT imaging is as follows: When an object is irradiated by a pulsed laser, the absorbers inside the object heat up and expand, emitting ultrasonic signals. These signals propagate outward and are received by an ultrasonic detector. Through appropriate image reconstruction techniques, a visualized image of the object's internal structure can be obtained.

[0003] Image reconstruction has a crucial impact on the imaging quality of PACT (Photoacoustic Acoustic Reconstruction). Currently, PACT image reconstruction algorithms mainly fall into three categories: analytical reconstruction algorithms, iterative reconstruction algorithms, and deep learning reconstruction algorithms. Among these, existing deep learning-based PACT image reconstruction methods primarily learn the mapping relationship between low-quality and high-quality images through neural networks to achieve image artifact suppression and accuracy improvement. However, these methods can only make simple corrections to the reconstruction results in the image domain, and their improvement effect is limited when the input image quality is poor. Therefore, there is an urgent need for a photoacoustic image reconstruction method that can provide superior performance to improve the imaging quality of photoacoustic images in the PACT system. Summary of the Invention

[0004] In view of this, this application provides a photoacoustic image reconstruction method and apparatus based on deep learning, which can provide a photoacoustic image reconstruction method with better performance, so as to improve the imaging quality of photoacoustic images in the PACT system.

[0005] The technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a photoacoustic image reconstruction method based on deep learning, the method comprising:

[0007] In response to the PACT imaging device performing photoacoustic detection on the target to be imaged, the raw photoacoustic signal data is sampled and obtained.

[0008] The original photoacoustic signal data is input into a preset photoacoustic image prediction model to obtain a reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained with pre-obtained training data. The training data includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images. The photoacoustic training signals and photoacoustic training images correspond one-to-one.

[0009] Optionally, the raw photoacoustic signal data includes: sparse view sampling data or finite view sampling data.

[0010] Optionally, the photoacoustic image prediction model includes: a filtering module, a back projection module, and a fusion module; wherein, the filtering module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a skip connection layer connected in sequence; the back projection module includes a sparse transformation matrix and a dense decomposition matrix; and the fusion module includes an encoder, a decoder, and a size adjustment layer.

[0011] Optionally, the filtering module is used to convert the input raw photoacoustic signal into a back projection signal, and input the back projection signal into the back projection module;

[0012] The first convolutional layer includes 16 convolutional kernels with a size of 1×L×1 and one activation function; the second convolutional layer includes one convolutional kernel with a size of 1×L×16 and one activation function; the third convolutional layer includes N... s Each dimension is N d A 1×1 convolutional kernel and a single dimension of 1×1×N d N s The convolutional kernel, the skip connection layer is used to obtain the back-projection signal based on the original photoacoustic signal and the output signal of the third convolutional layer, the back-projection signal having a dimension of N. d ×N s ×1, where L is 1 / 4 of the original photoacoustic signal sampling length, and N s The original photoacoustic signal sampling length is N. d The number of detectors in the PACT imaging device.

[0013] Optionally, the back-projection module is used to convert the input back-projection signal into a projected image and input the projected image into the fusion module;

[0014] The sparse transformation matrix has a dimension of N. x N y ×N d N s ×1, the size dimension of the dense decomposition matrix is ​​N.x ×N y ×N d The dimensions of the projected image are N. x ×N y ×N d , wherein, the N x With the N y N represents the output image size dimension of the photoacoustic image prediction model. d The number of detectors in the PACT imaging device.

[0015] Optionally, the fusion module is used to convert the input projection image into the reconstructed photoacoustic image;

[0016] The encoder includes one feature extraction submodule and four shrinking submodules connected in sequence. The feature extraction submodule consists of a convolutional layer, a normalization layer, and an activation function layer. The shrinking submodule consists of a convolutional layer, a normalization layer, an activation function layer, and a max pooling layer.

[0017] The decoder includes four extended sub-modules connected in sequence. Each extended sub-module consists of a convolutional layer, a normalization layer, an activation function layer, and a size adjustment layer. The size adjustment layer consists of a convolutional layer and an upsampling layer, or a convolutional layer and a downsampling layer.

[0018] Optionally, the photoacoustic image prediction model includes:

[0019] p0(r s )=f Fusion (f Back-projection (f Filtering (p(r d ,t)))),

[0020] Wherein, p0(r) s To reconstruct the photoacoustic image, the f Fusion As a fusion module, the f Back-projection For the back projection module, the f Filtering For the filtering module, the p(r) d ,t) represents the original photoacoustic signal data, and r s The position of the photoacoustic source of the target to be imaged, r d This refers to the position of the detector in the PACT imaging device.

[0021] Secondly, embodiments of this application provide a photoacoustic image reconstruction apparatus based on deep learning, the apparatus comprising:

[0022] The data acquisition module is used to respond to the photoacoustic detection of the target to be imaged by the PACT imaging device and sample and obtain the raw photoacoustic signal data.

[0023] The image reconstruction module is used to input the original photoacoustic signal data into a preset photoacoustic image prediction model to obtain the reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained with pre-obtained training data. The training data includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images. The photoacoustic training signals and the photoacoustic training images correspond one-to-one.

[0024] Thirdly, embodiments of this application provide a photoacoustic image reconstruction device based on deep learning, comprising:

[0025] Memory, used to store computer programs;

[0026] A processor for executing the computer program to implement the steps of the deep learning-based photoacoustic image reconstruction method as described above.

[0027] Fourthly, embodiments of this application provide a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the aforementioned deep learning-based photoacoustic image reconstruction methods.

[0028] The above technical solution has the following beneficial effects:

[0029] This application provides a deep learning-based photoacoustic image reconstruction method. When executing the method, in response to the PACT imaging device performing photoacoustic detection on the target to be imaged, raw photoacoustic signal data is sampled and obtained. The raw photoacoustic signal data is input into a preset photoacoustic image prediction model to obtain a reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained using pre-obtained training data. The training data includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images, with a one-to-one correspondence between the photoacoustic training signals and the photoacoustic training images. This application embodiment reconstructs photoacoustic images based on a pre-trained deep learning network model. Compared with existing technologies, the photoacoustic image reconstruction method provided in this application embodiment combines the robustness of traditional algorithms with the adaptability of deep learning algorithms, resulting in superior performance and improved imaging quality of photoacoustic images in the PACT system.

[0030] This application also provides apparatus, devices, and storage media corresponding to the above methods, which have the same beneficial effects as the above methods. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A flowchart illustrating a deep learning-based photoacoustic image reconstruction method provided in this application embodiment;

[0033] Figure 2 A schematic diagram illustrating the workflow of a photoacoustic image prediction model involved in a deep learning-based photoacoustic image reconstruction method provided in this application embodiment;

[0034] Figure 3 A schematic diagram of the photoacoustic image prediction model network involved in a deep learning-based photoacoustic image reconstruction method provided in this application embodiment;

[0035] Figure 4 This is a comparison image of the reconstructed image of the photoacoustic image prediction model in this application under input sparse viewpoint photoacoustic data and the reconstructed image of the FBP algorithm.

[0036] Figure 5 This is a comparison chart of the quantitative evaluation results of the reconstructed image of the photoacoustic image prediction model in this application embodiment under input sparse viewpoint photoacoustic data and the reconstructed image of the FBP algorithm.

[0037] Figure 6 This is a comparison image of the reconstructed image of the photoacoustic image prediction model in this application under input limited-view photoacoustic data and the reconstructed image of the FBP algorithm;

[0038] Figure 7 This is a comparison chart of the quantitative evaluation results of the reconstructed image of the photoacoustic image prediction model in this application embodiment under input limited viewpoint photoacoustic data and the reconstructed image of the FBP algorithm;

[0039] Figure 8 This is a schematic diagram of a photoacoustic image reconstruction device based on deep learning, provided in an embodiment of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] To improve the imaging quality of photoacoustic images in the PACT system, this application provides a deep learning-based photoacoustic image reconstruction method. Please refer to [link to relevant documentation]. Figure 1 This method can be applied to PACT imaging devices, and the method may include:

[0042] Step S100: In response to the PACT imaging device performing photoacoustic detection on the target to be imaged, sample and obtain raw photoacoustic signal data.

[0043] Specifically, the PACT imaging device is used to perform photoacoustic detection on the target to be imaged, and raw photoacoustic signal data is acquired. The raw photoacoustic signal data is used for subsequent reconstruction of photoacoustic images.

[0044] It should be noted that the raw photoacoustic signal data can include either sparse-view sampling data or limited-view sampling data. Sparse-view sampling data refers to the raw photoacoustic signal data acquired by the PACT imaging device with its detectors sparsely arranged in a 360-degree field of view, resulting in a spatially undersampled photoacoustic signal. Limited-view sampling data refers to the raw photoacoustic signal data acquired by the PACT imaging device with its scanning angle limited to a specific angle, resulting in limited-view sampling data.

[0045] Understandably, the photoacoustic image reconstructed from sparse-view sampling data covers the same area. The denser the detector arrangement, the richer the original photoacoustic signal data sampling, resulting in a better reconstructed photoacoustic image. However, the increased sampling volume also increases both sampling and computational costs. Furthermore, limited-view sampling, due to the restriction of the scanning angle, limits its application in practical applications such as breast imaging and skin imaging, as it cannot acquire photoacoustic signal data from the entire viewing angle.

[0046] In practical applications, the training data used to train the photoacoustic image prediction model includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images, with a one-to-one correspondence between the photoacoustic training signals and the photoacoustic training images.

[0047] In practice, to achieve better model training results, the training data used to train the model can be obtained by sampling photoacoustic signals in a higher configuration mode using the PACT imaging device, and then reconstructing photoacoustic training images from the photoacoustic signals. For example, if the photoacoustic training signals used to train the model are sparse view sampling data or limited view sampling data, then the PACT imaging device detector can be selected to sample them in a full sampling mode to ensure the comprehensiveness and richness of the sampled data as much as possible, so as to ensure that the reconstructed photoacoustic training images used for model training are of better quality, thereby ensuring the photoacoustic image reconstruction effect of the final trained photoacoustic image prediction model.

[0048] In specific applications, the reconstruction method for obtaining photoacoustic training images can employ the Filtered Back Projection (FBP) algorithm, and its reconstruction process can be expressed as follows:

[0049]

[0050] In the formula,

[0051]

[0052] b(r d The term ,t) is called the back projection term, where v is the speed of sound and r is the velocity of sound. s and r d Let represent the positions of the photoacoustic source and the detector, respectively. Ω is the solid angle enclosed by the detector surface, and dΩ is the solid angle corresponding to the detector element dσ, which can be expressed as:

[0053]

[0054] In the formula, n d It is the unit normal vector of the detector surface pointing to the region of interest.

[0055] Step S200: Input the original photoacoustic signal data into the preset photoacoustic image prediction model to obtain the reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained with pre-obtained training data.

[0056] Specifically, the photoacoustic image prediction model is a deep learning network model pre-trained using training data. In practical applications, the photoacoustic image prediction model can include: a filtering module, a back-projection module, and a fusion module, corresponding to the three processing steps of filtering, back-projection, and fusion, respectively. Figure 2 The diagram shows the workflow of the photoacoustic image prediction model. The photoacoustic signal is filtered by the filtering module to obtain the back-projection signal. The back-projection signal is back-projected by the back-projection module to obtain multiple projection images. The fusion module fuses the multiple projection images to obtain the reconstructed photoacoustic image.

[0057] Further, see Figure 3 The diagram shown is a schematic of the photoacoustic image prediction model network:

[0058] The filtering module can be represented as Y1 = F1(X), where X is the photoacoustic signal, F1 is the filtering module network model, and Y1 is the back-projection signal output by the filtering module. The filtering module network model includes a first convolutional layer L1, a second convolutional layer L2, a third convolutional layer L3, and skip connection layers connected sequentially. The first convolutional layer includes 16 convolutional kernels with a size of 1×L×1 and one activation function; the second convolutional layer includes one convolutional kernel with a size of 1×L×16 and one activation function; the third convolutional layer includes N... s Each dimension is N d A 1×1 convolutional kernel and a single dimension of 1×1×N d N s The convolutional kernel and skip connection layer transmit the original photoacoustic signal to the output of the third convolutional layer, where it is combined with the output signal of the third convolutional layer to obtain the backprojection signal. The size dimension of the backprojection signal is N. d ×N s ×1, L is 1 / 4 of the original photoacoustic signal sampling length, N s N represents the original photoacoustic signal sampling length. d This represents the number of detectors in the PACT imaging device. It should be noted that the Tanh activation function can be used.

[0059] The backprojection module can be represented as Y2 = F2(Y1), where Y1 is the backprojection signal mentioned above, F2 is the backprojection module network model, and Y2 is the projected image output by the backprojection module. The backprojection module network model includes a sparse transformation matrix L4 and a dense decomposition matrix L5; wherein the size dimension of the sparse transformation matrix is ​​N. x N y ×N d N s ×1, the size dimension of the dense decomposition matrix is ​​N. x ×N y ×N d The size dimension is N d ×N s The back-projection signal of size ×1 is multiplied and rearranged by a sparse transformation matrix to output a signal of size N. x ×N y The image is then decomposed into three-dimensional projected images (i.e., multiple two-dimensional images) by a decomposition matrix, with a size of N. x ×N y ×N d N x With N y N represents the output image size dimension of the photoacoustic image prediction model. This output image size dimension can be preset by the technician according to their needs. d This represents the number of detectors in the PACT imaging device.

[0060] The fusion module can be represented as Y3 = F3(Y2), where Y2 is the projected image mentioned above, F3 is the fusion module network model, and Y3 is the reconstructed photoacoustic image output by the fusion module. The fusion module network model mainly includes an encoder, a decoder, and a scaling layer.

[0061] The encoder comprises a feature extraction submodule L6 and four shrinking submodules L7 connected sequentially. The feature extraction submodule consists of a convolutional layer with 64 kernels of size 3×3, a normalization layer, and an activation function layer. The shrinking submodule consists of a convolutional layer with kernels of size 3×3 and a stride of 1, a normalization layer, an activation function layer, and a max pooling layer with kernel size 2×2 and a stride of 2. It should be noted that, in order to extract more features, the number of kernels in the four shrinking submodules gradually doubles along the encoder processing progress. That is, the first shrinking submodule includes 128 kernels of size 3×3, the second shrinking submodule includes 256 kernels of size 3×3, the third shrinking submodule includes 512 kernels of size 3×3, and the fourth shrinking submodule includes 1024 kernels of size 3×3.

[0062] The decoder consists of four sequentially connected expansion sub-modules. Each expansion sub-module is composed of a convolutional layer consisting of 64 convolutional kernels with a size dimension of 3×3 and a stride of 1, a normalization layer, an activation function layer, and a corresponding size adjustment layer.

[0063] The size adjustment layer consists of a convolutional layer with 64 kernels of size dimension 3×3 and stride 1, and a downsampling layer or an upsampling layer. By setting an appropriate sampling factor, features of different sizes can be normalized to the same size.

[0064] It should be noted that the model can finally add the 3D projection image output by the backprojection module to the last layer through a convolutional layer with a kernel size of 1×1, in order to reduce the learning difficulty of the network.

[0065] In practical applications, to accelerate convergence and improve network performance, group normalization and activation functions are added after the convolutional layers. The normalization layer can use the Group Normalization (GN) algorithm, and the activation function layer can use the ReLU activation function.

[0066] In one optional implementation, the photoacoustic image prediction model in this application embodiment may include:

[0067] p0(r s )=f Fusion (f Back-projection(f Filtering (p(r d ,t)))),

[0068] Where p0(r) s To reconstruct the photoacoustic image, f Fusion For the fusion module, f Back-projection For the back projection module, f Filtering For the filtering module, p(r) d ,t) represents the original photoacoustic signal data, r s Let r be the position of the target to be imaged. d This indicates the position of the detector in the PACT imaging device.

[0069] As can be seen from the above, the photoacoustic image prediction model provided in this application embodiment is a physical model-driven deep neural network. This model network can simultaneously take into account the robustness of traditional algorithms and the adaptability of deep learning algorithms, thus having better image reconstruction performance in practical applications.

[0070] In practice, training a photoacoustic image prediction model can involve the following steps: first, training the filtering module separately; then, training the filtering module and the backprojection module together; and finally, performing end-to-end training on the entire network. The batch sizes for the three training stages are set to 1, 16, and 3, respectively. The Adam optimizer and mean squared error (MSE) loss function can be used during training, with the learning rate set to 1.0 × 10⁻⁶ in the first and third stages. -4 The second stage is set to 2.5 × 10 -5 All training can be implemented using the TensorFlow 2.0 framework, with the code deployed on a single NVIDIA RTX TITAN GPU to accelerate network training.

[0071] Figure 4 The diagram illustrates a comparison between the reconstructed image from the photoacoustic image prediction model of this application, based on input sparse viewpoint photoacoustic data, and the reconstructed image from the FBP algorithm. The number of channels for the four sets of sparse viewpoint data are n = 32, 64, 128, and 256, respectively. The sparse viewpoint photoacoustic data serves as the network input, and the image reconstructed from the 2π viewpoint 512-channel photoacoustic data serves as the reference for the network output. The diagram shows that, due to spatial sparse sampling, the image reconstructed by the FBP algorithm exhibits numerous stripe artifacts and loses much valuable structural information. In contrast, the photoacoustic image prediction model provided in this application effectively suppresses these artifacts, recovers more structural information, and obtains a higher-quality reconstructed image.

[0072] Figure 5The diagram shows a comparison of the quantitative evaluation results of the reconstructed image of the photoacoustic image prediction model of this application under input sparse viewpoint photoacoustic data and the FBP algorithm. It can be seen that the reconstruction effect of the photoacoustic image prediction model provided by this application under input sparse viewpoint photoacoustic data is better than that of the FBP algorithm in three evaluation indicators: root mean square error (RMSE), peak signal to noise ratio (PSNR), and structural similarity index measure (SSIM).

[0073] Figure 6 The diagram shows a comparison between the reconstructed image from the photoacoustic image prediction model of this application, based on input finite-view photoacoustic data, and the reconstructed image from the FBP algorithm. The detection angles for the four finite-viewpoints are Ω = π / 4, π / 2, 3π / 4, and π. The finite-viewpoint photoacoustic data serves as the network input, and the image reconstructed from 512 channels of photoacoustic data at a 2π-viewpoint serves as the network output reference. As can be seen from the reconstruction results of the finite-viewpoint photoacoustic data, due to the finite-angle sampling, the image reconstructed by the FBP algorithm is severely distorted, blurry, and loses many details. The photoacoustic image prediction model provided in this application effectively mitigates these problems, resulting in a clearer reconstructed image with fewer artifacts and enhanced detail visibility.

[0074] Figure 7 The diagram shows a comparison of the quantitative evaluation results of the reconstructed image of the photoacoustic image prediction model of this application under input limited-view photoacoustic data and the FBP algorithm. It can be seen that the reconstruction effect of the photoacoustic image prediction model provided by this application under input limited-view photoacoustic data is better than that of the FBP algorithm in terms of the three evaluation indicators: root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM).

[0075] In summary, this application provides a photoacoustic image reconstruction method based on deep learning. When executing this method, in response to the PACT imaging device performing photoacoustic detection on the target to be imaged, raw photoacoustic signal data is sampled and obtained. The raw photoacoustic signal data is input into a preset photoacoustic image prediction model to obtain a reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained using pre-obtained training data. The training data includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images, with a one-to-one correspondence between the photoacoustic training signals and the photoacoustic training images. This application embodiment achieves photoacoustic image reconstruction based on a pre-trained deep learning network model. Compared with existing technologies, the photoacoustic image reconstruction method provided by this application embodiment combines the robustness of traditional algorithms with the adaptability of deep learning algorithms, resulting in superior performance and improved imaging quality of photoacoustic images in the PACT system.

[0076] Corresponding to the above methods, this application also provides a photoacoustic image reconstruction apparatus based on deep learning. Please refer to [link to relevant documentation]. Figure 8 The diagram shows the structure of the device, which may include:

[0077] The data acquisition module 801 is used to respond to the photoacoustic detection of the target to be imaged by the PACT imaging device and sample and obtain the original photoacoustic signal data.

[0078] The image reconstruction module 802 is used to input the original photoacoustic signal data into a preset photoacoustic image prediction model to obtain the reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained with pre-obtained training data. The training data includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images. The photoacoustic training signals and photoacoustic training images correspond one-to-one.

[0079] In one alternative implementation, the raw photoacoustic signal data includes: sparse viewpoint sampling data or finite viewpoint sampling data.

[0080] In one optional implementation, the photoacoustic image prediction model includes: a filtering module, a back-projection module, and a fusion module; wherein, the filtering module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a skip connection layer connected in sequence; the back-projection module includes a sparse transformation matrix and a dense decomposition matrix; and the fusion module includes an encoder, a decoder, and a size adjustment layer.

[0081] In one optional implementation, the filtering module is specifically used to convert the input raw photoacoustic signal into a back projection signal, and input the back projection signal into the back projection module;

[0082] The first convolutional layer includes 16 convolutional kernels with a size of 1×L×1 and one activation function; the second convolutional layer includes one convolutional kernel with a size of 1×L×16 and one activation function; the third convolutional layer includes N... s Each dimension is N d A 1×1 convolutional kernel and a single dimension of 1×1×N d N s The convolutional kernel, the skip connection layer is used to obtain the back-projection signal based on the original photoacoustic signal and the output signal of the third convolutional layer, the back-projection signal having a dimension of N. d ×N s ×1, where L is 1 / 4 of the original photoacoustic signal sampling length, and N s The original photoacoustic signal sampling length is N. d The number of detectors in the PACT imaging device.

[0083] In one optional implementation, the back-projection module is specifically used to convert the input back-projection signal into a projected image and input the projected image into the fusion module;

[0084] The sparse transformation matrix has a dimension of N. x N y ×N d N s ×1, the size dimension of the dense decomposition matrix is ​​N. x ×N y ×N d The dimensions of the projected image are N. x ×N y ×N d , wherein, the N x With the N y N represents the output image size dimension of the photoacoustic image prediction model. d The number of detectors in the PACT imaging device.

[0085] In one alternative implementation, the fusion module is specifically used to convert the input projected image into the reconstructed photoacoustic image;

[0086] The encoder includes one feature extraction submodule and four shrinking submodules connected in sequence. The feature extraction submodule consists of a convolutional layer, a normalization layer, and an activation function layer. The shrinking submodule consists of a convolutional layer, a normalization layer, an activation function layer, and a max pooling layer.

[0087] The decoder includes four extended sub-modules connected in sequence. Each extended sub-module consists of a convolutional layer, a normalization layer, an activation function layer, and a size adjustment layer. The size adjustment layer consists of a convolutional layer and an upsampling layer, or a convolutional layer and a downsampling layer.

[0088] In one optional implementation, the photoacoustic image prediction model specifically includes:

[0089] p0(r s )=f Fusion (f Back-projection (f Filtering (p(r d ,t)))),

[0090] Wherein, p0(r) s To reconstruct the photoacoustic image, f Fusion For the fusion module, f Back-projection For the back projection module, f Filtering For the filtering module, p(r) d ,t) represents the original photoacoustic signal data, r s r represents the position of the photoacoustic source of the target to be imaged. d This refers to the position of the detector in the PACT imaging device.

[0091] It should be noted that the steps and related technical features of each module in the photoacoustic image reconstruction device based on deep learning provided in this application correspond to the method provided in the application embodiment. The description of the device part can be found in the embodiments of the aforementioned method part, and will not be repeated here.

[0092] In summary, this application provides a photoacoustic image reconstruction device based on deep learning. The device includes a data acquisition module and an image reconstruction module. The data acquisition module is used to sample and obtain raw photoacoustic signal data in response to photoacoustic detection of the target by the PACT imaging device. The image reconstruction module is used to input the raw photoacoustic signal data into a preset photoacoustic image prediction model to obtain a reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained using pre-obtained training data. The training data includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images, with a one-to-one correspondence between the photoacoustic training signals and the photoacoustic training images. This application embodiment achieves photoacoustic image reconstruction based on a pre-trained deep learning network model. Compared with existing technologies, the photoacoustic image reconstruction method provided by this application embodiment combines the robustness of traditional algorithms with the adaptability of deep learning algorithms, resulting in superior performance and improved imaging quality of photoacoustic images in the PACT system.

[0093] In another embodiment, this application also provides a photoacoustic image reconstruction device based on deep learning, comprising:

[0094] Memory, used to store computer programs;

[0095] A processor is configured to execute the computer program to implement the steps of any of the deep learning-based photoacoustic image reconstruction methods described in the foregoing embodiments.

[0096] In another embodiment, this application also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the deep learning-based photoacoustic image reconstruction methods in the foregoing embodiments.

[0097] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0098] Those skilled in the art will understand that the flowchart shown is merely an example in which the embodiments of this application can be implemented, and the scope of application of the embodiments of this application is not limited by any aspect of the flowchart.

[0099] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and devices can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, the functional units in the various embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0101] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A photoacoustic image reconstruction method based on deep learning, characterized in that, The method includes: In response to the PACT imaging device performing photoacoustic detection on the target to be imaged, the raw photoacoustic signal data is sampled and obtained. The original photoacoustic signal data is input into a preset photoacoustic image prediction model to obtain a reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained with pre-obtained training data. The training data includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images. The photoacoustic training signals and photoacoustic training images correspond one-to-one. The photoacoustic image prediction model includes a filtering module, a back-projection module, and a fusion module; wherein, the filtering module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a skip connection layer connected in sequence; the back-projection module includes a sparse transformation matrix and a dense decomposition matrix; the fusion module includes an encoder, a decoder, and a size adjustment layer. The filtering module is used to convert the input raw photoacoustic signal into a back projection signal, and input the back projection signal into the back projection module; The first convolutional layer includes 16 layers with a size dimension of 1× L The second convolutional layer consists of a 1×1 convolutional kernel and an activation function. L The third convolutional layer includes a ×16 convolutional kernel and an activation function. N s Each dimension is N d A 1×1 convolutional kernel and a dimension of 1×1× N d N s The convolutional kernel, the skip connection layer is used to obtain the back-projection signal based on the original photoacoustic signal and the output signal of the third convolutional layer, the size dimension of the back-projection signal is... N d × N s ×1, where the L The original photoacoustic signal sampling length is 1 / 4 of the original sample length. N s The original photoacoustic signal sampling length, the N d The number of detectors in the PACT imaging device.

2. The method according to claim 1, characterized in that, The original photoacoustic signal data includes: sparse view sampling data or finite view sampling data.

3. The method according to claim 1, characterized in that, The back-projection module is used to convert the input back-projection signal into a projected image and input the projected image into the fusion module; The size dimension of the sparse transformation matrix is N x N y × N d N s ×1, the size dimension of the dense decomposition matrix is N x × N y × N d The dimensions of the projected image are N x × N y × N d , wherein N x With the N y The output image size dimension of the photoacoustic image prediction model is... N d The number of detectors in the PACT imaging device.

4. The method according to claim 3, characterized in that, The fusion module is used to convert the input projection image into the reconstructed photoacoustic image; The encoder includes one feature extraction submodule and four shrinking submodules connected in sequence. The feature extraction submodule consists of a convolutional layer, a normalization layer, and an activation function layer. The shrinking submodule consists of a convolutional layer, a normalization layer, an activation function layer, and a max pooling layer. The decoder includes four extended sub-modules connected in sequence. Each extended sub-module consists of a convolutional layer, a normalization layer, an activation function layer, and a size adjustment layer. The size adjustment layer consists of a convolutional layer and an upsampling layer, or a convolutional layer and a downsampling layer.

5. The method according to claim 1, characterized in that, The photoacoustic image prediction model includes: ， Among them, the To reconstruct the photoacoustic image, the As a fusion module, the For the back projection module, the For filtering modules, the The original photoacoustic signal data, the The location of the target to be imaged, the This refers to the position of the detector in the PACT imaging device.

6. A photoacoustic image reconstruction device based on deep learning, characterized in that, The device includes: The data acquisition module is used to respond to the photoacoustic detection of the target to be imaged by the PACT imaging device and sample and obtain the raw photoacoustic signal data. The image reconstruction module is used to input the original photoacoustic signal data into a preset photoacoustic image prediction model to obtain the reconstructed photoacoustic image output by the photoacoustic image prediction model. The photoacoustic image prediction model is a deep learning network model trained with pre-obtained training data. The training data includes photoacoustic training signals sampled by the PACT imaging device and reconstructed photoacoustic training images. The photoacoustic training signals and the photoacoustic training images correspond one-to-one. The photoacoustic image prediction model includes a filtering module, a back-projection module, and a fusion module; wherein, the filtering module includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a skip connection layer connected in sequence; the back-projection module includes a sparse transformation matrix and a dense decomposition matrix; the fusion module includes an encoder, a decoder, and a size adjustment layer. The filtering module is specifically used to convert the input original photoacoustic signal into a back projection signal, and input the back projection signal into the back projection module; The first convolutional layer includes 16 layers with a size dimension of 1× L The second convolutional layer consists of a 1×1 convolutional kernel and an activation function. L The third convolutional layer includes a ×16 convolutional kernel and an activation function. N s Each dimension is N d A 1×1 convolutional kernel and a dimension of 1×1× N d N s The convolutional kernel, the skip connection layer is used to obtain the back-projection signal based on the original photoacoustic signal and the output signal of the third convolutional layer, the size dimension of the back-projection signal is... N d × N s ×1, where the L The original photoacoustic signal sampling length is 1 / 4 of the original sample length. N s The original photoacoustic signal sampling length, the N d The number of detectors in the PACT imaging device.

7. A photoacoustic image reconstruction device based on deep learning, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the deep learning-based photoacoustic image reconstruction method as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the deep learning-based photoacoustic image reconstruction method as described in any one of claims 1 to 5.

Images