Multi-bandwidth photoacoustic imaging system and method based on deep learning cascaded enhancement

Abstract

The application discloses a kind of based on deep learning cascade enhancement multi-bandwidth photoacoustic imaging system and method, the system is by pulsed laser, two half annular array probes, multichannel data acquisition card and central computer composition, pulsed laser is used to excite the light-absorbing body in target area, generates photoacoustic signal;Two half annular array probes respectively have high-frequency probe with center frequency 5.5MHz and 2.5MHz low-frequency probe, high-frequency probe is used to obtain high-resolution image, low-frequency probe is used to detect larger light-absorbing body and low signal-to-noise ratio structural features;Multichannel data acquisition card is used to carry out digitization processing to photoacoustic signal received by two half annular array probes, and the data after processing is transmitted to central computer, reconstructs photoacoustic image based on filter back-projection reconstruction algorithm.The multi-bandwidth photoacoustic imaging system of the application shows very high image reconstruction capability, provides more accurate solution for biomedical imaging.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and relates to a multi-bandwidth photoacoustic imaging system and method, in particular to a multi-bandwidth photoacoustic imaging system and method based on deep learning cascade enhancement. BACKGROUND

[0002] In recent years, photoacoustic imaging technology (PAI) has made significant progress in the field of biomedicine, especially in tumor detection and functional imaging, showing great potential. However, the traditional photoacoustic imaging method is limited by the bandwidth and response characteristics of the detector, resulting in the loss of part of the spectral information, thereby limiting the resolution and information integrity of the imaging. Although multi-bandwidth imaging technology alleviates this problem by using detectors with different center frequencies, there are still challenges such as low information integration efficiency and low imaging quality. With the rapid development of deep learning technology, especially in the field of image reconstruction and signal processing, new solutions are provided. SUMMARY

[0003] In order to overcome the shortcomings of limited signal bandwidth, insufficient image resolution and quality in existing photoacoustic imaging technology, the present application provides a multi-bandwidth photoacoustic imaging system and method based on deep learning cascade enhancement. By introducing a cascade deep learning network architecture, the present application effectively integrates multi-frequency photoacoustic signals, realizes image reconstruction from coarse to fine, significantly improves imaging accuracy and quality, reduces the complexity of the system hardware, and realizes higher quality photoacoustic image reconstruction, suitable for high-resolution, multi-bandwidth imaging requirements in biomedical imaging.

[0004] The purpose of the present application is realized by the following technical solutions:

[0005] A multi-bandwidth photoacoustic imaging system based on deep learning cascade enhancement, comprising a pulsed laser, two semi-ring array probes with different center frequencies, a multi-channel data acquisition card and a central computer, wherein:

[0006] The pulsed laser is used to excite light absorbers in the target area to generate photoacoustic signals;

[0007] The two semi-ring array probes have a high-frequency probe with a center frequency of 5.5 MHz and a low-frequency probe with a center frequency of 2.5 MHz, the high-frequency probe is used to obtain high-resolution images, and the low-frequency probe is used to detect larger light absorbers and low signal-to-noise ratio structural features;

[0008] The multi-channel data acquisition card is used to digitize the photoacoustic signals received by the two semi-ring array probes, and transmit the processed data to the central computer, and reconstruct the photoacoustic image based on the filtered back-projection reconstruction algorithm.

[0009] A multi-bandwidth photoacoustic imaging method based on deep learning cascade enhancement includes the following steps:

[0010] Step 1: Multi-bandwidth photoacoustic imaging system hardware composition and configuration

[0011] The multi-bandwidth photoacoustic imaging system consists of a pulsed laser, two semi-circular array probes with different center frequencies, a multi-channel data acquisition card, and a central computer.

[0012] The pulse laser is used to excite the light absorber in the target area to generate a photoacoustic signal;

[0013] The two semi-annular array probes respectively have a high-frequency probe with a center frequency of 5.5 MHz and a low-frequency probe with a center frequency of 2.5 MHz. The high-frequency probe is used to obtain high-resolution images, and the low-frequency probe is used to detect large light absorbers and structural features with low signal-to-noise ratio.

[0014] The multi-channel data acquisition card is used to digitally process the photoacoustic signals received by the two semi-circular array probes, and transmit the processed data to the central computer to reconstruct the photoacoustic image based on the filtered back projection reconstruction algorithm;

[0015] Step 2: Signal acquisition and preprocessing

[0016] Step 2.1: The pulsed laser emits short pulses of light, which irradiate the tissue within the target imaging area, stimulating photoacoustic signals;

[0017] Step 2.2: The two semi-circular array probes receive photoacoustic signals of different frequencies respectively. These photoacoustic signals are collected by a multi-channel data acquisition card and stored in a central computer.

[0018] Step 2.3: The central computer pre-processes the collected photoacoustic signals;

[0019] Step 2.4: Use the reconstruction algorithm to generate high-frequency photoacoustic image x after preprocessing the photoacoustic signal h and low-frequency photoacoustic images x l , and divided into training set, validation set and test set for subsequent deep learning cascade network training and performance verification;

[0020] Step 3: Construction and training of deep learning cascade network

[0021] Step 3.1: The cascaded deep learning network consists of two serially connected U-Net modules. First, the high-frequency photoacoustic image x generated by preprocessing is h and low-frequency photoacoustic image x lThe shallow feature extraction is respectively carried out through a convolution kernel with a size of 3*3, then the extracted features are fused along the channel direction, and the fused features are input into a U-Net module of a first encoding-decoding structure, in the encoding process of the module, feature extraction is carried out through a double Double Conv3*3 convolution layer, and down-sampling is realized through a four-level maximum pooling layer, in the decoding process, up-sampling is realized through a Double Deconv3*3 deconvolution layer, finally, rough reconstruction is completed through an output layer with a convolution kernel size of 1*1, and a preliminary full annular image y1 is generated; then, the generated full annular image y1 and the original input high-frequency photoacoustic image x h and the low-frequency photoacoustic image x l are fused along the channel direction, the fused features are input into a U-Net module of a second encoding-decoding structure, in the encoding process of the module, feature extraction is carried out through a double Double Conv3*3 convolution layer, and down-sampling is realized through a four-level maximum pooling layer, in the decoding process, up-sampling is realized through a Double Deconv3*3 deconvolution layer, finally, fine reconstruction is completed through an output layer with a convolution kernel size of 1*1, and a final full annular image y2 is generated;

[0022] Step 3.2: In the training process, a joint loss function is used to measure the difference between the reconstructed image and the real image, and the loss function is composed of two parts:

[0023] L = alpha * L1 + beta * L2

[0024]

[0025]

[0026] Wherein, L1 is the MSE loss between y1 output by the first-level rough reconstruction model and the real image y, L2 is the MSE loss between y2 output by the second-level fine reconstruction model and the real image y, alpha is the L1 loss weight, and beta is the L2 loss weight;

[0027] Step 3.3: the Adam optimizer is used to optimize and adjust the network parameters;

[0028] Step 4: multi-bandwidth photoacoustic reconstruction:

[0029] The photoacoustic images collected by the 2.5MHz half-ring array and the photoacoustic images collected by the 5.5MHz half-ring array are used as the input of the deep learning cascade network, the trained deep learning cascade network is used for multi-bandwidth photoacoustic reconstruction, and the 2.5MHz and 5.5MHz full-ring photoacoustic images are obtained.

[0030] Compared with the prior art, the present application has the following advantages:

[0031] 1、The present application realizes a two-stage image reconstruction process from coarse to fine through a cascaded deep learning network (CU-Net), significantly improving the resolution and information integrity of photoacoustic images, especially in the fusion and processing of low-frequency and high-frequency signals.

[0032] 2、The present application uses a semi-ring array probe with different center frequencies, which can capture a wider range of photoacoustic signals, realize multi-scale and multi-bandwidth imaging, and meet the detection needs of large light absorbers and fine structures.

[0033] 3、The present application uses deep learning technology to realize automatic image reconstruction, reduces the dependence on manual adjustment, greatly improves the processing efficiency, and is suitable for real-time imaging and large-scale application scenarios

[0034] 4、The present application uses deep learning technology to realize automatic image reconstruction, reduces the dependence on manual adjustment, greatly improves the processing efficiency, and is suitable for real-time imaging and large-scale application scenarios

[0035] 5、In in vivo experiments, the multi-bandwidth photoacoustic imaging system is successfully applied to photoacoustic imaging of mouse abdomen, and CU-Net performs well in image reconstruction PSNR (peak signal-to-noise ratio), MAE (mean absolute error) and SSIM (structural similarity index) and other indicators, verifying its reliability and wide applicability in practical application.

[0036] 6、The multi-bandwidth photoacoustic imaging system of the present application shows high image reconstruction capability, providing a more accurate solution for biomedical imaging. The system integrates high and low frequency probes, and through the deep learning cascaded enhancement model CU-Net, it combines the wide coverage capability of low frequency probes in overall structure imaging and the high resolution characteristics of high frequency probes in detail capture, thus showing significant advantages and great application potential in multi-bandwidth photoacoustic imaging tasks. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 Figure 1 is a schematic diagram of the workflow of the multi-bandwidth photoacoustic tomography system of the present application;

[0038] Figure 2 Figure 2 is a schematic diagram of the deep learning cascaded enhancement model CU-Net structure proposed by the present application;

[0039] Figure 3 Figure 3 is an example of the application effect of the method proposed by the present application in multi-bandwidth photoacoustic reconstruction. DETAILED DESCRIPTION

[0040] The technical solutions of the present application are further described below with reference to the drawings, but are not limited thereto, and any modification or equivalent replacement within the spirit and scope of the technical solutions of the present application shall be covered in the protection scope of the present application.

[0041] The present application provides a multi-bandwidth photoacoustic imaging method based on deep learning cascade enhancement, which comprises the following steps:

[0042] Step 1: Hardware composition and configuration of multi-bandwidth photoacoustic imaging system

[0043] As shown in Figure 1 The multi-bandwidth photoacoustic imaging system proposed by the present application is composed of the following main components: a pulsed laser, two semi-ring array probes with different center frequencies, a multi-channel data acquisition card and a central computer. The pulsed laser is used to excite the light absorber in the target region to generate photoacoustic signals. The two semi-ring array probes have a high-frequency probe with a center frequency of 5.5 MHz and a low-frequency probe with a center frequency of 2.5 MHz. The high-frequency probe is mainly used to obtain high-resolution images, while the low-frequency probe is used to detect larger light absorbers and low signal-to-noise ratio structural features. The multi-channel data acquisition card is used to digitize the photoacoustic signals received by the probe and transmit the processed data to the central computer, and the photoacoustic image is reconstructed based on the filtered back-projection reconstruction algorithm.

[0044] Step 2: Signal acquisition and preprocessing

[0045] During the experimental operation, the pulsed laser emits short pulse light, which is irradiated on the tissue in the target imaging area to excite photoacoustic signals. The generation of photoacoustic signals can be represented by the following formula:

[0046] p0(r)=Γ·μ a (r)·F(r)

[0047] Where p0(r) represents the initial sound pressure distribution at position r, Γ is the Grüneisen coefficient, μ a (r) is the light absorption coefficient of the tissue, and F(r) represents the local light energy density.

[0048] The propagation of photoacoustic signals in the tissue can be described by the following wave equation:

[0049]

[0050] Where p(r,t) is the sound pressure at position r at time t, c represents the sound speed in the tissue, and δ(t) represents the instantaneous excitation of the laser pulse.

[0051] Two half-ring array probes receive photoacoustic signals of different frequencies, which are collected by a multi-channel data acquisition card and stored in a central computer. In order to ensure the accuracy and efficiency of subsequent image reconstruction, the collected photoacoustic signals need to be preprocessed first. The preprocessing steps include removing noise, enhancing the effective part of the signal, and normalizing the data. The preprocessed photoacoustic signals are used to generate photoacoustic image data using traditional reconstruction algorithms such as delay-and-sum (DAS), and are divided into training set, validation set and test set according to a certain proportion, which are used for subsequent training and performance verification of deep learning cascade network.

[0052] Step 3: Construction and training of deep learning cascade network

[0053] In order to realize the reconstruction of high-quality images from coarse to fine, the present application constructs a cascade deep learning network (Cascade U-Net, CU-Net). The network consists of two U-Net modules connected in series. First, the high-frequency photoacoustic image x h and the low-frequency photoacoustic image x l generated by preprocessing are respectively extracted by a shallow feature extraction layer (Conv3x3) with a kernel size of 3x3, and then the extracted features are fused along the channel direction and input into the first U-Net module with an encoding-decoding structure. In the encoding process, the features are extracted by a double Conv3x3 convolution layer, and down-sampling is achieved by a four-level max-pooling layer. In the decoding process, up-sampling is achieved by a Double Deconv3x3 deconvolution layer, and finally a rough reconstruction is completed by an output layer with a kernel size of 1x1, generating a preliminary full-ring image y1. Then, the generated intermediate image y1 is fused with the original input high-frequency image x h and the low-frequency image x l along the channel direction, and the fused features are input into the second U-Net module with an encoding-decoding structure. This module also performs fine reconstruction through the encoding-decoding structure to generate the final full-ring image y2, thereby improving the quality and resolution of the image.

[0054] In the training process, the mean square error (MSE) loss function is used to measure the difference between the reconstructed image and the true image. To ensure that both stages of the cascade U-Net play a key role in improving the overall quality and accuracy of the reconstructed image, the present application designs a joint loss function, which consists of two parts:

[0055]

[0056] where L1 is the MSE loss between y1 output by the first-stage coarse reconstruction model and the true image y.

[0057]

[0058] wherein L2 is the MSE loss between the second level fine reconstruction model output y2 and the real image y. The final joint loss function L is represented as:

[0059] L = aL1 + bL2

[0060] wherein a is the coarse reconstruction L1 loss weight, b is the fine reconstruction L2 loss weight, and in the embodiments, both a and b are set to 1.

[0061] The network parameters are optimized and adjusted using the Adam optimizer. The learning rate in the training process is set to an initial value of 0.0001, and a step-by-step decreasing method is used for optimization to ensure the stability and convergence of the training.

[0062] Step 4: Multi-bandwidth photoacoustic reconstruction:

[0063] The photoacoustic images collected by the 2.5MHz half-ring array and the photoacoustic images collected by the 5.5MHz half-ring array are used as the input of the deep learning cascade network, and the trained deep learning cascade network is used for multi-bandwidth photoacoustic reconstruction to obtain 2.5MHz and 5.5MHz full-ring photoacoustic images.

[0064] Step 5: System verification and application

[0065] In order to verify the effectiveness of the present application, in-vivo experiments are carried out to verify the reliability and applicability in practical application.

[0066] The following are the reconstruction indicators and visual effect examples of the model of the present application, which demonstrate its excellent performance in practical application. The system is applied to photoacoustic imaging of the abdomen of a mouse, low-frequency 2.5MHz photoacoustic signals and high-frequency 5.5MHz photoacoustic signals are collected, 240 photoacoustic image data are generated using the delay-and-sum algorithm, and are divided into training set, validation set and test set according to the ratio of 7:1:2. The low-frequency 2.5MHz half-ring (180°) array photoacoustic image x l and the high-frequency half-ring (180°) array photoacoustic image x h are used as the input of the deep learning cascade enhancement model CU-Net, the low-frequency 2.5MHz full-view (360°) photoacoustic image and the high-frequency 2.5MHz full-view (360°) photoacoustic image are used as the imaging target y respectively, and the model reconstruction performance test is carried out. The experimental results show that the cascade deep learning model CU-Net proposed in the present application has good reconstruction performance in the application of multi-bandwidth photoacoustic imaging, and performs excellently in image quality indicators such as PSNR, MAE and SSIM, as shown in Table 1.

[0067] Table 1: Mouse abdomen reconstruction indicators

[0068]

[0069] Figure 3 The visualization results of CU-Net in multi-bandwidth photoacoustic imaging tasks are demonstrated, which effectively capture fine structural details and significantly improve imaging quality, demonstrating its outstanding performance in practical applications. The system integrates high and low frequency probes, and through the deep learning cascade enhancement model CU-Net, it combines the wide coverage ability of the low frequency probe in the overall structure imaging and the high resolution characteristics of the high frequency probe in the detail capture, thus showing significant advantages and great application potential in multi-bandwidth photoacoustic imaging tasks.

Claims

1. A multi-bandwidth photoacoustic imaging system based on deep learning cascade enhancement, characterized in that The multi-bandwidth photoacoustic imaging system is composed of a pulsed laser, two semi-annular array probes with different center frequencies, a multi-channel data acquisition card, and a central computer, wherein: The pulse laser is used to excite the light absorber in the target area to generate a photoacoustic signal; The two semi-annular array probes respectively have a high-frequency probe with a center frequency of 5.5 MHz and a low-frequency probe with a center frequency of 2.5 MHz. The high-frequency probe is used to obtain high-resolution images, and the low-frequency probe is used to detect large light absorbers and structural features with low signal-to-noise ratio. The multi-channel data acquisition card is used to digitally process the photoacoustic signals received by the two semi-circular array probes, and transmit the processed data to a central computer to reconstruct the photoacoustic image based on the filtered back projection reconstruction algorithm.

2. A multi-bandwidth photoacoustic imaging method based on deep learning cascade enhancement, characterized in that The method comprises the following steps: Step 1: Multi-bandwidth photoacoustic imaging system hardware composition and configuration The multi-bandwidth photoacoustic imaging system consists of a pulsed laser, two semi-circular array probes with different center frequencies, a multi-channel data acquisition card, and a central computer. The pulse laser is used to excite the light absorber in the target area to generate a photoacoustic signal; The two semi-annular array probes respectively have a high-frequency probe with a center frequency of 5.5 MHz and a low-frequency probe with a center frequency of 2.5 MHz. The high-frequency probe is used to obtain high-resolution images, and the low-frequency probe is used to detect large light absorbers and structural features with low signal-to-noise ratio. The multi-channel data acquisition card is used to digitally process the photoacoustic signals received by the two semi-circular array probes, and transmit the processed data to the central computer to reconstruct the photoacoustic image based on the filtered back projection reconstruction algorithm; Step 2: Signal acquisition and preprocessing Step 2.1: The pulsed laser emits short pulses of light, which irradiate the tissue within the target imaging area, stimulating photoacoustic signals; Step 2.2: The two semi-circular array probes receive photoacoustic signals of different frequencies respectively. These photoacoustic signals are collected by a multi-channel data acquisition card and stored in a central computer. Step 2.3: The central computer pre-processes the collected photoacoustic signals; Step 2.4: Use the reconstruction algorithm to generate high-frequency photoacoustic image x after preprocessing the photoacoustic signal h and low-frequency photoacoustic images x l , and divided into training set, validation set and test set for subsequent deep learning cascade network training and performance verification; Step 3: Construction and training of deep learning cascade network Step 3.1: The cascaded deep learning network consists of two serially connected U-Net modules. First, the high-frequency photoacoustic image x generated by preprocessing is h and low-frequency photoacoustic image x l Shallow feature extraction is performed respectively through a convolution kernel of size 3×3, and then the extracted features are fused along the channel direction, and the fused features are input into the first U-Net module of the encoding-decoding structure. During the encoding process, the module extracts features through a double-layer Double Conv3×3 convolution layer and implements downsampling through a four-level maximum pooling layer. During the decoding process, upsampling is performed through a Double Deconv3×3 deconvolution layer, and finally a rough reconstruction is completed through an output layer with a convolution kernel size of 1×1 to generate a preliminary full-ring image y1; then, the generated full-ring image y1 is compared with the original input high-frequency photoacoustic image x h and low-frequency photoacoustic images x l The features are fused along the channel direction and input into the second U-Net module of the encoding-decoding structure. During the encoding process, this module extracts features through a double-layer Double Conv3×3 convolution layer and implements downsampling through a four-level maximum pooling layer. During the decoding process, upsampling is performed through a Double Deconv3×3 deconvolution layer. Finally, fine reconstruction is completed through an output layer with a convolution kernel size of 1×1 to generate the final full ring image y2; Step 3.2: During training, a joint loss function is used to measure the difference between the reconstructed image and the real image. Step 3.3: Use Adam optimizer to optimize the network parameters; Step 4: Multi-bandwidth photoacoustic reconstruction: The photoacoustic images collected by the 2.5MHz half-ring array and the photoacoustic images collected by the 5.5MHz half-ring array were used as the input of the deep learning cascade network. The trained deep learning cascade network was used to perform multi-bandwidth photoacoustic reconstruction to obtain 2.5MHz and 5.5MHz full-ring photoacoustic images.

3. The multi-bandwidth photoacoustic imaging method based on deep learning cascade enhancement according to claim 2 is characterized in that The joint loss function consists of two parts: L=αL1+βL2 Among them, L1 is the MSE loss between y1 output by the first-level coarse reconstruction model and the real image y, L2 is the MSE loss between y2 output by the second-level fine reconstruction model and the real image y, α is the L1 loss weight, and β is the L2 loss weight.

Images