Photoacoustic imaging reflection artifact elimination method based on neural network and spatial filtering

Abstract

The invention discloses a photoacoustic imaging reflection artifact elimination method based on a neural network and spatial filtering, which comprises the following steps of: firstly, eliminating internal artifacts of optical signals by adopting singular filtering, and then eliminating boundary artifacts of the optical signals by adopting the neural network. The imaging definition and accuracy are improved, the contrast ratio and the resolution ratio of the image are enhanced by reducing artifacts and noise, so that the imaging result is closer to the real situation, the clearer image is beneficial to more accurate diagnosis of doctors, and the reliability of medical diagnosis is improved. The method is compatible with an existing photoacoustic imaging system and can be applied to various imaging platforms, the deep learning technology is utilized, the data processing efficiency is improved, the requirement for computing resources is reduced, the deep learning model can stably work under different conditions after being fully trained, and dependence on prior information is reduced. Compared with some traditional photoacoustic imaging algorithms, the deep learning method may reduce the demand for computing resources, so that the imaging process is more efficient.

Description

Technical Field

[0001] The invention belongs to the technical field of acousto-optic signal processing, and in particular relates to a method for eliminating reflection artifacts in photoacoustic imaging based on neural network and spatial filtering. Background Art

[0002] Photoacoustic imaging technology is a new imaging modality that has developed rapidly in recent years. It has the advantages of high resolution, non-invasiveness, specific imaging and high imaging depth. However, traditional photoacoustic imaging technology has artifacts, which seriously affects the judgment of imaging. For example, in photoacoustic imaging, boundary reflection signals will cause errors in the imaging results, affecting the clarity of the image and the accuracy of the diagnosis. Internal reflection signals will produce multiple reflections during the imaging process, forming complex signals, which increases the difficulty of imaging processing. Moreover, photoacoustic imaging technology has certain limitations in imaging depth, especially under high resolution requirements. Technologies such as quantitative photoacoustic tomography (QPAT) require a lot of computing resources and time. Based on the above problems, the existing photoacoustic imaging technology cannot be popularized in the medical field. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a method for eliminating reflection artifacts in photoacoustic imaging based on neural network and spatial filtering, which has high imaging quality, high accuracy and low demand for computing resources.

[0004] The technical solution adopted to solve the above technical problems is: a method for eliminating reflection artifacts in photoacoustic imaging based on neural network and spatial filtering, comprising the following steps:

[0005] Step 1. Use singular filtering to eliminate internal artifacts of the optical signal

[0006] Step 1.1. Reorganize the received RF signal according to the number of channels, perform singular value decomposition on the reorganized signal, and obtain a spatial matrix, a singular value matrix and a time matrix. The singular value matrix contains the energy distribution of the signal;

[0007] Step 1.2. Using the space matrix, calculate the covariance between the space vectors according to the following formula to obtain the covariance matrix, which is used to obtain the spatial correlation of the signal;

[0008]

[0009] In the formula, C u (n,m) is the covariance of signal U between the nth and mth sampling points, n z is the number of sampling points in the depth direction, n x is the number of sampling points in the horizontal direction, <|U m(z,x)|> is the statistical average of the signal amplitudes of all (z,x) sampling points at time m, <|U n (z,x)|>The statistical average of the signal amplitudes of all (z,x) sampling points at time n;

[0010] Step 1.3. Use the full search method along the diagonal to divide the covariance matrix into blocks of the same size, and define the block with the peak value in the diagonal as the central block. Determine the minimum mean square error function SSD between the central block and the block to be matched, which is used to evaluate the similarity between the central block and the block to be matched, so as to find the most similar block;

[0011] Step 1.4. Use the block matching method to locate the direct information of the light source, identify and distinguish the direct signal and the reflected signal, use a filter to filter out the reflected signal, eliminate the artifacts generated by the internal reflection, generate and output a light signal containing only the boundary artifacts;

[0012] Step 2. Use a neural network to eliminate the boundary artifacts of the output light signal in step 1

[0013] Step 2.1. Prepare the dataset

[0014] The optical signal is collected and processed to obtain the original time-frequency diagram of the optical signal. The position of the photoacoustic source is determined according to the time-frequency diagram and the DAS algorithm. The ideal binary mask IBM of the optical signal with the same size as the time-frequency diagram is obtained according to the following formula, which is used to mark the valid signal and artifacts. Among them, 1 is the valid signal and 0 is the artifact. IBM is used as the annotation information and the time-frequency diagram. Figure 1 The data set is constructed, the data set is normalized, and divided into a training set, a validation set, and a test set according to the proportion. The samples of the training set and the validation set are composed of a time-frequency diagram and its corresponding IBM, and the samples of the test set are composed of a time-frequency diagram;

[0015]

[0016] Where IBM(·) is the ideal binary mask, t is time, f is frequency, M is the modulus of the element in the time-frequency spectrum, and LC is the threshold of the modulus;

[0017] Step 2.2. Optimize the deep neural network model DNN to obtain a DNN model with IBM function

[0018] The deep neural network model DNN includes an input layer, multiple hidden layers, and an output layer. The Sigmoid function is used as the activation function of the hidden layer, and the biased Sigmoid function is used as the activation function of the output layer.

[0019] Use the training set to train the deep neural network model DNN, use the mean square error as the loss function, and continuously iterate and optimize until the number of iterations reaches the upper limit of the number of iterations, and obtain a DNN model with IBM function;

[0020] Use the validation set to evaluate the performance of the DNN model with IBM function, and adjust the model's hyperparameters based on the results of the performance evaluation to ensure that the network can accurately predict IBM, thereby obtaining the final DNN model with IBM function;

[0021] The test set is input into the final DNN model with IBM function, and the final DNN model with IBM function outputs the estimated IBM;

[0022] Step 2.3. After the original time-frequency map is multiplied by the estimated IBM, it is converted back to the original time or space domain through inverse continuous wavelet transform to reconstruct the photoacoustic signal with artifacts removed.

[0023] As a preferred technical solution, in step 1.3, the minimum mean square error function SSD of the central block and the block to be matched is determined as:

[0024]

[0025] In the formula, ρ m,n is the similarity, X i,j is the pixel value of the center block X at coordinate (i, j), Y i+m,j+n is the gray value of the matching block Y at the coordinate (i+m,j+n), l is the width of the center block, and k is the length of the center block.

[0026] As a preferred technical solution, in step 2.1, the ratio of the training set, the validation set, and the test set is 8:1:1.

[0027] The beneficial effects of the present invention are as follows:

[0028] The present invention effectively eliminates boundary reflection and internal reflection artifacts in photoacoustic imaging, improves the clarity and accuracy of imaging, and enhances the contrast and resolution of images by reducing artifacts and noise, so that the imaging results are closer to the real situation. Clearer images help doctors to diagnose more accurately and improve the reliability of medical diagnosis. The method of the present invention is compatible with existing photoacoustic imaging systems and can be applied to a variety of imaging platforms. It uses deep learning technology to improve the efficiency of data processing and reduce the demand for computing resources. After being fully trained, the deep learning model can work stably under different conditions and reduce the dependence on prior information. Compared with some traditional photoacoustic imaging algorithms, the deep learning method may reduce the demand for computing resources and make the imaging process more efficient. The present invention increases the effective depth of imaging by eliminating artifacts, making deep tissue imaging clearer. The improved imaging quality makes the technology have greater potential in clinical applications, especially in fields requiring high-resolution imaging, providing a new tool for biomedical research and helping to promote scientific research in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the process of the present invention.

[0030] Figure 2 Imaging results of the phantom used in the experiment.

[0031] Figure 3 This is a comparison chart of the RF signal before and after filtering.

[0032] Figure 4 This is the imaging result of the phantom I experiment.

[0033] Figure 5 This is the imaging result of the phantom II experiment.

[0034] Figure 6 This is the imaging result of the phantom III experiment.

[0035] Figure 7 is the estimated mask error in the phantom and ex vivo sample experiments. DETAILED DESCRIPTION

[0036] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0037] exist Figure 1 In this embodiment, the method for eliminating reflection artifacts in photoacoustic imaging based on neural network and spatial filtering includes the following steps:

[0038] Step 1. Use singular filtering to eliminate internal artifacts of the optical signal

[0039] Step 1.1. Reorganize the received RF signal according to the number of channels, perform singular value decomposition on the reorganized signal, and obtain a spatial matrix, a singular value matrix and a time matrix. The singular value matrix contains the energy distribution of the signal;

[0040] Step 1.2. Using the space matrix, calculate the covariance between the space vectors according to the following formula to obtain the covariance matrix, which is used to obtain the spatial correlation of the signal;

[0041]

[0042] In the formula, C u (n,m) is the covariance of signal U between the nth and mth sampling points, n z is the number of sampling points in the depth direction, n x is the number of sampling points in the horizontal direction, <|U m (z,x)|> is the statistical average of the signal amplitudes of all (z,x) sampling points at time m, <|U n(z,x)|>The statistical average of the signal amplitudes of all (z,x) sampling points at time n;

[0043] Step 1.3. Use the full search method along the diagonal to divide the covariance matrix into blocks of the same size, and define the block with the peak value in the diagonal as the central block. According to the following formula, the minimum mean square error function SSD between the central block and the block to be matched is determined to evaluate the similarity between the central block and the block to be matched, so as to find the most similar block;

[0044]

[0045] In the formula, ρ a,b is the similarity, X i,j is the pixel value of the center block X at coordinate (i, j), Y i+a,j+b is the gray value of the matching block Y at the coordinate (i+a,j+b), l is the length of the center block X, and k is the width of the center block X.

[0046] Step 1.4. Use the block matching method to locate the direct information of the light source, identify and distinguish the direct signal and the reflected signal, use a filter to filter out the reflected signal, eliminate the artifacts generated by the internal reflection, generate and output a light signal containing only the boundary artifacts;

[0047] Step 2. Use a neural network to eliminate the boundary artifacts of the output light signal in step 1

[0048] Step 2.1. Prepare the dataset

[0049] The collected optical signal is transformed into the original time-frequency diagram of the optical signal through continuous wavelet transform. The position of the photoacoustic source is determined according to the time-frequency diagram and DAS algorithm. The ideal binary mask IBM of the optical signal with the same size as the time-frequency diagram is obtained according to the following formula, which is used to mark the valid signal and artifacts. Among them, 1 is the valid signal and 0 is the artifact. IBM is used as the labeling information and the time-frequency diagram. Figure 1 The data set is constructed and normalized, and divided into a training set, a validation set, and a test set according to a ratio of 8:1:1. The samples of the training set and the validation set consist of a time-frequency graph and its corresponding IBM, and the samples of the test set consist of a time-frequency graph.

[0050]

[0051] Where IBM(·) is the ideal binary mask, t is time, f is frequency, M is the modulus of the element in the time-frequency spectrum, and LC is the threshold of the modulus;

[0052] Step 2.2. Optimize the deep neural network model DNN to obtain a DNN model with IBM function

[0053] The deep neural network model DNN includes an input layer, multiple hidden layers, and an output layer. The Sigmoid function is used as the activation function of the hidden layer, and the biased Sigmoid function is used as the activation function of the output layer.

[0054] Use the training set to train the deep neural network model DNN, use the mean square error as the loss function, and continuously iterate and optimize until the number of iterations reaches the upper limit of the number of iterations, and obtain a DNN model with IBM function;

[0055] Use the validation set to evaluate the performance of the DNN model with IBM function, and adjust the model's hyperparameters based on the results of the performance evaluation to ensure that the network can accurately predict IBM, thereby obtaining the final DNN model with IBM function;

[0056] The test set is input into the final DNN model with IBM function, and the final DNN model with IBM function outputs the estimated IBM;

[0057] Step 2.3. After the original time-frequency map is multiplied by the estimated IBM, it is converted back to the original time or space domain through inverse continuous wavelet transform to reconstruct the photoacoustic signal with artifacts removed.

[0058] experiment

[0059] In order to verify the beneficial effects of the present invention, the inventors applied the technical solutions of the embodiments and the prior art to conduct the following experiments:

[0060] 1. Experimental Preparation

[0061] 1) Using a pulsed laser to emit laser light of a specific wavelength (e.g., 532 nm), and configuring a multi-channel ultrasound platform and an ultrasound array (e.g., L11-5, Verasonics) to receive the signal;

[0062] 2) Make a phantom model with different light and sound sources, such as embedding thin steel wire, pencil lead, copper core, etc. into an agar model (4×5×6cm 3 )middle;

[0063] 3) Bandpass filter the collected signal to remove noise and retain useful signal components.

[0064] 2. Remove internal artifacts

[0065] 1) Experiments using different photoacoustic sources and reflector phantoms

[0066] A pencil lead was embedded in the phantom as a photoacoustic source, and three rubber bands were added as acoustic reflectors to simulate inhomogeneous media.

[0067] The experimental results are as follows Figure 2 As shown, Figure 2(a) is the reconstructed image of the original signal, and (b) is the reconstructed image of the filtered signal. The image quality is evaluated by enhancing the contrast (Contrast to Noise Ratio, CNR). The CNRs of the original image and the processed image are 3.53dB and 6.37dB respectively, indicating that the present invention can effectively filter the reflected signal.

[0068] The RF signal is Figure 3 exhibit. Figure 3 (a) is the original signal before filtering; Figure 3 (b) is the signal after filtering. There are multiple clutters in the original signal, among which the reflected signal below the direct signal is generated by the gap between the pencil lead and the phantom, and the reflected signal in the distance is generated by the reflector and the boundary. We evaluate the filtering effect by the signal-to-noise ratio (SNR) of the RF signal, and the SNR of the original signal is 14.38dB, and the SNR of the filtered signal is 43.86dB. The images of the signal-to-noise ratio and the RF signal show that the present invention can effectively filter the reflected signal and improve the signal-to-noise ratio.

[0069] 3. Remove boundary artifacts

[0070] 1) Experiments with different photoacoustic sources

[0071] A thin steel wire and a pencil lead were used as two highly optically absorbing photoacoustic sources embedded in an agar model (4×5×6cm 3 ), the experimental results are as follows Figure 4 As shown, Figure 4 (a) Results of the unoptimized reconstruction process; Figure 4 (b) is the ROI in (a); Figure 4 (c) Reconstruction results of the optimization process; Figure 4 (d) is the ROI value in (c). And the contrast images of three photoacoustic sources (copper core, steel wire, pencil core) as phantoms, such as Figure 5 , Figure 5 (a) Results of the unoptimized reconstruction process; Figure 5 (b) is the ROI in (a); Figure 5 (c) Reconstruction results of the optimization process; Figure 5 (d) is the ROI value in (c).

[0072] 2) Experiments with different photoacoustic sources and reflector phantoms

[0073] Two plastic tubes filled with black ink were embedded as low optical absorption photoacoustic sources, and five rubber bands were added to simulate inhomogeneous media as model phantom imaging. In addition, we embedded a photoacoustic source (pencil lead) in the inhomogeneous medium as a model phantom imaging. The experimental results are shown in Figure 6(a) Results of non-optimized reconstruction process, Figure 6 (b) Reconstruction results of the optimization process; Figure 6 (c) Fusion image of photoacoustic and B-ultrasound images.

[0074] The quantitative evaluation results of the distribution relative error (DRE) between the calculated reconstructed photoacoustic image and its standard image are shown in Table 1.

[0075] Table 1. Distribution relative error of images without optimization and traditional reconstruction process

[0076]

[0077] Experimental signal mask error, such as Figure 7 , Figure 7 (a) shows the error of phantom I. The mask error between -0.01 and 0 is concentrated at 0.90 probability. The mask error of phantom II is shown in Figure 7 (b) shows that the mask error between -0.02 and -0.01 has been concentrated at a probability of 0.43. Figure 7 For phantom III in (c), the mask error between -0.01 and -0.00 has been concentrated at a probability of 0.45. Figure 7 In the in vitro tissue in (d), the probability of mask error between -0.02 and -0 is concentrated at 0.37. The mean absolute error (MAE) between the verified signal mask and the estimated mask in the phantom I, phantom II, phantom III, and in vitro sample experiments is 0.0163, 0.0685, 0.0146, and 0.0259, respectively.

[0078] The experimental results show that the reconstruction process optimized by neural network can effectively suppress the boundary reflection signal in photoacoustic imaging, significantly reduce the boundary artifacts, and further eliminate the artifacts caused by internal reflections by using spatial singular value filters, thereby improving the clarity and accuracy of photoacoustic imaging. Through the evaluation of quantitative indicators such as distributed relative error (DRE), the optimized reconstruction process and filter performed well in reducing errors. The improvement of indicators such as contrast-to-noise ratio (CNR) and signal-to-noise ratio (SNR) of experimental images proves that the image quality has been significantly improved. The effectiveness of the proposed method was verified by agar phantom and in vitro tissue experiments, proving the applicability and robustness of the method of the present invention under different conditions. Compared with traditional photoacoustic imaging technology, the present invention has obvious advantages in eliminating artifacts and improving imaging quality.

Claims

1. A method for eliminating reflection artifacts in photoacoustic imaging based on neural network and spatial filtering, characterized in that: The following steps are involved: Step 1. Use singular filtering to eliminate internal artifacts of the optical signal Step 1.

1. Reorganize the received RF signal according to the number of channels, perform singular value decomposition on the reorganized signal, and obtain a spatial matrix, a singular value matrix and a time matrix. The singular value matrix contains the energy distribution of the signal; Step 1.

2. Using the space matrix, calculate the covariance between the space vectors according to the following formula to obtain the covariance matrix, which is used to obtain the spatial correlation of the signal; In the formula, C u (n,m) is the covariance of signal U between the nth and mth sampling points, n z is the number of sampling points in the depth direction, n x is the number of sampling points in the horizontal direction, <|U m (z,x)|> is the statistical average of the signal amplitudes of all (z,x) sampling points at time m, <|U n (z,x)|>The statistical average of the signal amplitudes of all (z,x) sampling points at time n; Step 1.

3. Use the full search method along the diagonal to divide the covariance matrix into blocks of the same size, and define the block with the peak value in the diagonal as the central block. Determine the minimum mean square error function SSD between the central block and the block to be matched, which is used to evaluate the similarity between the central block and the block to be matched, so as to find the most similar block; Step 1.

4. Use the block matching method to locate the direct information of the light source, identify and distinguish the direct signal and the reflected signal, use a filter to filter out the reflected signal, eliminate the artifacts generated by the internal reflection, generate and output a light signal containing only the boundary artifacts; Step 2. Use a neural network to eliminate the boundary artifacts of the output light signal in step 1 Step 2.

1. Prepare the dataset The optical signal is collected and processed to obtain the original time-frequency graph of the optical signal. The position of the photoacoustic source is determined according to the time-frequency graph and the DAS algorithm. The ideal binary mask IBM of the optical signal with the same size as the time-frequency graph is obtained according to the following formula, which is used to mark the valid signal and the artifact, where 1 is the valid signal and 0 is the artifact. IBM, as the annotation information, constitutes a data set together with the time-frequency graph. The data set is normalized and divided into a training set, a validation set, and a test set according to the proportion. The samples of the training set and the validation set consist of a time-frequency graph and its corresponding IBM, and the samples of the test set consist of a time-frequency graph; Where IBM(·) is the ideal binary mask, t is time, f is frequency, M is the modulus of the element in the time-frequency spectrum, and LC is the threshold of the modulus; Step 2.

2. Optimize the deep neural network model DNN to obtain a DNN model with IBM function The deep neural network model DNN includes an input layer, multiple hidden layers, and an output layer. The Sigmoid function is used as the activation function of the hidden layer, and the biased Sigmoid function is used as the activation function of the output layer. Use the training set to train the deep neural network model DNN, use the mean square error as the loss function, and continuously iterate and optimize until the number of iterations reaches the upper limit of the number of iterations, and obtain a DNN model with IBM function; Use the validation set to evaluate the performance of the DNN model with IBM function, and adjust the model's hyperparameters based on the results of the performance evaluation to ensure that the network can accurately predict IBM, thereby obtaining the final DNN model with IBM function; The test set is input into the final DNN model with IBM function, and the final DNN model with IBM function outputs the estimated IBM; Step 2.

3. After the original time-frequency map is multiplied by the estimated IBM, it is converted back to the original time or space domain through inverse continuous wavelet transform to reconstruct the photoacoustic signal with artifacts removed.

2. The method for eliminating reflection artifacts in photoacoustic imaging based on neural network and spatial filtering according to claim 1, characterized in that: In step 1.3, the minimum mean square error function SSD of the central block and the block to be matched is determined as: In the formula, ρ m,n is the similarity, X i,j is the pixel value of the center block X at coordinate (i, j), Y i+m,j+n is the gray value of the matching block Y at the coordinate (i+m,j+n), l is the width of the center block, and k is the length of the center block.

3. The method for eliminating reflection artifacts in photoacoustic imaging based on neural network and spatial filtering according to claim 1, characterized in that: The ratio of the training set, validation set, and test set in step 2.1 is 8:1:1.

Images