Photoacoustic microscopic depth-of-field enhanced imaging method based on three-dimensional continuity and sparsity deconvolution

Abstract

The invention discloses a photoacoustic microscopic depth-of-field enhanced imaging method based on three-dimensional continuity and sparsity deconvolution. The method comprises the steps of data acquisition, construction of a point spread function based on a photoacoustic microscopic imaging system, construction of continuity and sparsity constraints, solving of a constraint optimization problem, three-dimensional deconvolution and the like. According to the invention, the deconvolution framework is designed, and continuity and sparsity constraint conditions are combined, so that depth-of-field expansion of the photoacoustic microscopic imaging system is realized; by introducing the AN-ADMM algorithm of momentum acceleration, the convergence speed is accelerated, and the stability of image restoration is enhanced. According to the method, the imaging quality of the photoacoustic microscopic imaging system at different depths is remarkably improved, the depth of field is expanded, the limitation of a traditional method in the depth direction is overcome, and a depth-of-field expansion scheme which is more efficient and more accurate than that of the traditional method is provided in the field of photoacoustic microscopic imaging.

Description

Technical Field

[0001] The present invention belongs to the field of photoacoustic microscopy, and relates to a method for expanding the depth of field of a photoacoustic microscope by using a three-dimensional deconvolution algorithm, specifically to a photoacoustic microscopy depth-of-field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution. Background Art

[0002] An optical-resolution photoacoustic microscopy (OR-PAM) system is a new non-invasive medical imaging technology that combines the advantages of optical imaging and ultrasonic imaging. Compared with traditional single optical imaging or ultrasonic imaging, it can achieve a lateral resolution from micrometers to sub-micrometers, and can perform non-radiative and high-resolution imaging on microscopic biological tissues, and is widely used in biomedical fields such as microvascular imaging, brain imaging, oral imaging, and bone tissue imaging. In OR-PAM, a Gaussian beam that can be focused to a micrometer-sized spot is usually used as the excitation of the photoacoustic signal. After the light absorber in the biological tissue absorbs the laser energy, under the conditions of thermal limitation and pressure limitation, ultrasonic waves are emitted based on the photo-thermal elastic effect.

[0003] In an actual system, OR-PAM receives the time data in each ultrasonic wave through A-line scanning, and then through two-dimensional raster scanning, three-dimensional imaging with depth information can be achieved. However, due to the diffraction phenomenon of light, the Gaussian beam diverges rapidly along its propagation path, resulting in a rapid expansion of the spot outside the focal plane, and the signal-to-noise ratio of the photoacoustic image in the defocused area decreases accordingly. This will lead to a very limited depth of field of OR-PAM, and it is impossible to obtain a three-dimensional image with a consistent lateral resolution in the depth direction, and the obtained structural information is less and inaccurate. Therefore, in the practical application of traditional OR-PAM, there are challenges such as limited depth of field and poor signal-to-noise ratio at defocus, and new technologies are urgently needed to increase the depth of field to meet the precise photoacoustic structure imaging and analysis requirements at different depths.

[0004] With the development of deconvolution technology, deconvolution-based algorithms have been widely applied to fluorescence microscopy, optical coherence tomography, ultrasonic imaging, and photoacoustic imaging. In fluorescence microscopy and photoacoustic imaging, deconvolution shows promise in improving spatial resolution, suppressing artifacts, and expanding the depth of field, and can be applied to a photoacoustic microscopy system. The degradation model of a photoacoustic image can be described by the convolution of the original image and the point spread function of the imaging system. Deconvolving the photoacoustic image based on the point spread function can restore the original images at different depths, and has a strong ability to expand the depth of field. Summary of the Invention

[0005] In order to overcome the problem of subsequent imaging effects caused by limited depth of field during the three-dimensional imaging process of a photoacoustic microscopy system, the present invention provides a photoacoustic microscopy depth-of-field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution. By designing a deconvolution framework, this method effectively extends the depth of field, providing a more efficient and stable method for expanding the depth of field of a photoacoustic microscopy system without increasing the system cost.

[0006] The object of the present invention is achieved through the following technical solutions:

[0007] A photoacoustic microscopy depth-of-field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution includes the following steps:

[0008] Step 1) Collect photoacoustic data using a photoacoustic microscopy system:

[0009] Collect photoacoustic data using a photoacoustic microscopy system based on a single-element ultrasonic transducer, including photoacoustic image data of phantoms and in vivo tissues. The specific steps are as follows:

[0010] Step 11) The photoacoustic microscopy imaging system uses a pulsed laser to excite a biological tissue sample. The laser pulse is absorbed by the light absorber in the biological tissue, triggering a thermoelastic effect and generating an ultrasonic signal;

[0011] Step 12) The photoacoustic microscopy imaging system uses a single-element ultrasonic detector array to receive the propagation of ultrasonic signals in the tissue, and then collects information at different depths. Through two-dimensional raster scanning, three-dimensional imaging with depth information is achieved. The imaging process is expressed as:

[0012] f(x,y,z) = p(x,y,z) * h(x,y,z) + n(x,y,z)

[0013] where f(x,y,z) represents the photoacoustic microscopy image, p(x,y,z) represents the point spread function at different depths, h(x,y,z) represents the depth-of-field extended image, and n(x,y,z) represents the noise term of the photoacoustic microscopy imaging system;

[0014] Step 2) Construct the point spread function based on the photoacoustic microscopy imaging system:

[0015] Step 21) Calculate the point spread function of the focal region of the photoacoustic microscopy imaging system;

[0016] Step 22) Estimate the point spread function of the defocused region;

[0017] Step 23) Initially estimate using the point spread function of the focal region as the initial value, and then perform blind deconvolution separately at each defocused layer to obtain the point spread function at that depth. The PSF construction model is expressed as:

[0018] PSF(z) = Gaussian(z, σ(z))

[0019] Wherein, PSF(z) is the point spread function of the photoacoustic microscopy system, z is the depth of the photoacoustic microscopy system, and σ(z) is the standard deviation at this depth;

[0020] Step 3) Construct continuity and sparsity constraints:

[0021] The continuity constraint is expressed as:

[0022]

[0023] Wherein, R Hessian (h) represents the continuity constraint, h xx , h xy , h xz , ··· represent the gradients of the image h in different directions, and λ1 is used to adjust the weight of the Hessian regularization dimension;

[0024] The sparsity constraint is expressed as:

[0025] R L = λ L1 ‖h‖1

[0026] Wherein, R L represents the sparsity constraint, and λ1 is used to adjust the weight of the sparse regularization dimension;

[0027] Step 4) Solve the constrained optimization problem:

[0028] Use the accelerated Nesterov's alternating direction multiplier method to solve the constrained optimization problem. The specific steps are as follows:

[0029] Step 41) Construct a new optimization problem based on the augmented Lagrangian function, and realize the optimization process by alternately optimizing the main variable and the auxiliary variable:

[0030]

[0031] Wherein, is to construct a new optimization problem based on the augmented Lagrangian function. U and V are auxiliary variables, representing the continuity and sparsity constraints on the photoacoustic microscopy image. λ1 and λ2 are regularization parameters. By alternately optimizing the photoacoustic image H, the auxiliary variables U and V, the image restoration at different depths is realized;

[0032] Step 42) At each gradient update, use a momentum term to accelerate the convergence process. The specific update formula is:

[0033]

[0034] Wherein, Hk is the updated and iterated photoacoustic image, β is the Nesterov momentum factor, η is the learning rate, is the gradient of the objective function;

[0035] Step 5) 3D deconvolution:

[0036] After the image obtained by accelerating the Nesterov's alternating direction multiplier method approaches the ideal state, the Richardson-Lucy deconvolution method is applied to further optimize the image to obtain a 3D photoacoustic microscopy image with a larger depth of field. The Richardson-Lucy deconvolution method is expressed as:

[0037]

[0038] where, o (k) (x, y, z) represents the depth-of-field extended image after deconvolution, h(x, y, z) represents the photoacoustic microscopy image, and p(x, y, z) represents the point spread function at different depths.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] 1. By designing a deconvolution framework and combining continuity and sparsity constraints, the present invention realizes the depth-of-field extension of the photoacoustic microscopy imaging system.

[0041] 2. By introducing the momentum-accelerated AN-ADMM algorithm, the present invention speeds up the convergence rate and enhances the stability of image restoration.

[0042] 3. The present invention significantly improves the imaging quality of the photoacoustic microscopy imaging system at different depths, extends the depth of field, overcomes the limitations of traditional methods in the depth direction, and provides a more efficient and accurate depth-of-field extension scheme than traditional methods in the field of photoacoustic microscopy imaging.

[0043] 4. Compared with traditional depth-of-field extension methods, the present invention can effectively restore the structural information of the defocused area and improve the spatial resolution of the 3D image without increasing the system cost, breaking through the diffraction limit of the traditional OR-PAM system and providing higher-quality images for biological tissue imaging. Description of the Drawings

[0044] Figure 1 is a flowchart of the photoacoustic microscopy depth-of-field enhancement imaging method based on 3D continuity and sparsity deconvolution.

[0045] Figure 2This is an example of the application effect of the method proposed in the present invention for a tungsten wire in a photoacoustic microscopy imaging system, which shows the comparison of the original standard photoacoustic image, the photoacoustic image obtained by the traditional deconvolution method, and the photoacoustic image obtained by the method of the present invention, as well as the corresponding depth cross-sectional images.

[0046] Figure 3 This is an example of the application effect of the method proposed in the present invention for a mouse ear in a photoacoustic microscopy imaging system, which shows the comparison of the original standard photoacoustic image and the photoacoustic image obtained by the method of the present invention. Specific embodiments

[0047] The technical solution of the present invention will be further described below in conjunction with the accompanying drawings, but it is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention shall be covered by the protection scope of the present invention.

[0048] The present invention provides a photoacoustic microscopy depth-of-field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution. The method includes processes such as data acquisition, construction of a point spread function based on a photoacoustic microscopy imaging system, construction of continuity and sparsity constraints, solution of a constrained optimization problem, and three-dimensional deconvolution. As Figure 1 shown, the specific steps are as follows:

[0049] Step 1) Use a photoacoustic microscopy imaging system to collect photoacoustic data:

[0050] The photoacoustic microscopy imaging system uses a pulsed laser to excite a biological tissue sample. The laser pulse is absorbed by the light absorber in the biological tissue, triggering a thermoelastic effect and generating an ultrasonic signal. The photoacoustic microscopy imaging system uses a single-element ultrasonic detector array to receive the propagation of the ultrasonic signal in the tissue, and then collects information at different depths. Through two-dimensional raster scanning, the system can achieve three-dimensional imaging with depth information.

[0051] In a photoacoustic microscopy imaging system, the collected image can usually be described by convolving the original image with a point spread function. Assuming that the imaging system is a linear time-invariant system, the imaging process can be expressed as:

[0052] f(x,y,z) = p(x,y,z) * h(x,y,z) + n(x,y,z)

[0053] where f(x,y,z) represents the photoacoustic microscopy image, p(x,y,z) represents the point spread function at different depths, h(x,y,z) represents the depth-of-field extended image, and n(x,y,z) represents the noise term of the photoacoustic microscopy imaging system.

[0054] Step 2) Construct a point spread function based on a photoacoustic microscopy imaging system:

[0055] First, calculate the point spread function of the focus region of the photoacoustic microscopy system. In the focus region, the resolution of the system is known and is represented as a standard Gaussian function based on the system resolution. Secondly, estimate the point spread function of the defocus region. In the defocus region, the point spread function will spread due to the diffraction effect. Considering that the point spread function of the photoacoustic microscopy image varies at different depths, the present invention constructs a blind deconvolution method to dynamically estimate the point spread function. Traditional methods usually assume that the point spread function is fixed, but due to the diffusion effect of the light spot, the point spread function will change with the imaging depth. To improve the recovery accuracy, the initial estimate uses the point spread function of the focus region as the initial value, and then performs blind deconvolution separately at each defocus layer to obtain the point spread function at that depth. The PSF construction model can be expressed as:

[0056] PSF(z) = Gaussian(z, σ(z))

[0057] Where PSF(z) is the point spread function of the photoacoustic microscopy system, z is the depth of the photoacoustic microscopy system, and σ(z) is the standard deviation at this depth. As the depth increases, the expansion of the light spot will increase.

[0058] Step 3) Construct continuity and sparsity constraints:

[0059] Due to the noise sensitivity problem of the photoacoustic microscopy system, traditional deconvolution methods are often affected by noise and converge to solutions misled by noise. Therefore, in order to recover the lost structural information in the defocus region from noise, the present invention introduces continuity constraints and sparsity constraints in the optimization process to improve the stability of image recovery, ensuring that the recovered image has a high signal-to-noise ratio and can effectively extract high-frequency information. At the same time, by combining the Hessian continuity constraint and the norm sparsity constraint, it is possible to avoid the problem of blurred detail information during the increase of the signal-to-noise ratio, making the acquired photoacoustic microscopy image approach the ideal state.

[0060] Continuity constraint: Assume that the structure of biological tissue is continuous in space and time, while noise is discontinuous. To reduce the impact of noise on image recovery, a continuity constraint is introduced. The Hessian matrix is constructed to constrain the image. The Hessian matrix can describe the local changes of the image, thereby suppressing the irregular changes in the image and maintaining the smoothness of the structure. The continuity constraint can be expressed as:

[0061]

[0062] Where R Hessian (h) represents the continuity constraint, h xx , h xy , h xz, ··· represents the gradients of the image h in different directions, and λ1 is used to adjust the weight of the Hessian regularization dimension.

[0063] Sparsity constraint: The high-frequency part of an image usually represents details and edge information, which is very important in a photoacoustic microscopy system. By introducing a sparsity constraint, it is ensured that more high-frequency information can be retained during the image restoration process. Applying the norm regularization to the high-frequency components of the image can effectively maintain the sparse structure. The sparsity constraint can be expressed as:

[0064] R L = λ L1 ‖h‖1

[0065] where R L represents the sparsity constraint, and λ1 is used to adjust the weight of the sparse regularization dimension.

[0066] The constraint function can be expressed as:

[0067]

[0068] where the first term is the data fidelity term, λ represents the regularization parameter, which can be set according to the noise level to adjust the weight of the data fidelity term; the second term is the continuity constraint. Based on the prior knowledge that the sample is smooth in space-time and space while the noise is discontinuous, through the Hessian matrix regularization, it can effectively suppress the noise in the photoacoustic microscopy system and at the same time increase the smoothness of the structural information; the third term is the sparsity constraint, λ L1 is used to adjust the sparsity. Based on the prior knowledge that the convolution of the sample with a smaller point spread function always gives relative sparsity, by processing the absolute sparse structure and the relative sparse structure with the norm, it can effectively extract the high-frequency information of the photoacoustic microscopy image.

[0069] Step 4) Solve the constrained optimization problem:

[0070] To stably and quickly solve the above optimization problem with multiple regularization terms, the present invention adopts the accelerated Nesterov's alternating direction multiplier method.

[0071] First, construct a new optimization problem based on the augmented Lagrangian function, and realize the optimization process by alternately optimizing the main variable and the auxiliary variable:

[0072]

[0073] where To construct a new optimization problem based on the augmented Lagrangian function, U and V are auxiliary variables representing the continuity and sparsity constraints on the photoacoustic microscopy image, and λ1 and λ2 are regularization parameters. By alternately optimizing the photoacoustic image H, the auxiliary variables U and V, the image restoration at different depths is achieved.

[0074] At each gradient update, a momentum term is adopted to accelerate the convergence process. The specific update formula is:

[0075]

[0076] where H k is the updated photoacoustic image, β is the Nesterov momentum factor, η is the learning rate, is the gradient of the objective function. The optimized photoacoustic microscopy image will approach the ideal state, providing high-quality images for the subsequent depth of field extension of the photoacoustic microscopy imaging system.

[0077] By introducing the momentum term, this method can stably handle the constrained optimization problem, accelerate convergence, and effectively suppress the interference of noise.

[0078] Step 5) 3D deconvolution:

[0079] After obtaining the photoacoustic images of all depth layers, the images at each depth are optimized using the above process. On this basis, the images are further optimized by the Richardson-Lucy deconvolution method to further solve the image blurring problem in the defocused area caused by spot diffusion, thereby extending the depth of field of the photoacoustic microscopy imaging system, breaking through the diffraction limit of the photoacoustic microscopy imaging system, and obtaining a photoacoustic microscopy image with a larger depth of field. The Richardson-Lucy deconvolution method can be expressed as:

[0080]

[0081] where o (k) (x, y, z) represents the depth of field extended image after deconvolution, h(x, y, z) represents the photoacoustic microscopy image, and p(x, y, z) represents the point spread function at different depths.

[0082] Step 6) System verification and application:

[0083] To verify the effectiveness of the method for extending the depth of field of the photoacoustic microscope based on 3D continuity and sparsity deconvolution proposed in the present invention, phantom and in vivo experiments were carried out to verify the reliability and applicability of the method in an actual photoacoustic microscopy imaging system.

[0084] The tungsten wire phantom experiment is used to verify the performance of the method of the present invention on standard physical samples, mainly evaluating the image spatial resolution, noise suppression ability and depth of field extension effect of the algorithm at different depths. The tungsten wire is placed in the focal region of the photoacoustic microscopy system and scanned to obtain the original photoacoustic images of the tungsten wire phantom at different depths. By comparing the original standard photoacoustic images, the photoacoustic images obtained by the traditional deconvolution method and the photoacoustic images obtained by the method of the present invention, as well as the corresponding depth cross-sectional views, the effectiveness of the deconvolution method is verified. Figure 2 It shows the visualization results of the tungsten wire in the depth of field extension task of the photoacoustic microscopy system based on the three-dimensional continuity and sparsity deconvolution method. It can be seen that in the focal region, the image after deconvolution of the present invention shows a high spatial resolution and clear details. In the defocus region, the image restored by the method of the present invention significantly improves the signal-to-noise ratio, effectively supplements the structural information, extends the depth of field in the depth direction, and successfully restores the photoacoustic image missing due to spot diffusion. Compared with the traditional deconvolution method, the image quality of each depth layer using the method of the present invention is generally higher, and consistent resolution can be restored within different depth ranges, effectively extending the depth of field.

[0085] The mouse ear experiment is used to verify the applicability of the method of the present invention in biological tissue imaging, especially in microvascular imaging and structural imaging. The mouse ear is imaged using a photoacoustic microscopy system, and the photoacoustic images of the mouse ear tissue at different depths are collected. The deconvolution method of the present invention is applied to process the obtained images to restore the structural information at different depths. Figure 3 It shows the visualization results of the mouse ear in the depth of field extension task of the photoacoustic microscopy system based on the three-dimensional continuity and sparsity deconvolution method. It can be seen from the results that in the shallow tissue of the mouse ear, the deconvolved image clearly shows the tiny structure of the blood vessels. In the deep tissue, the method of the present invention restores the detailed information lost due to out-of-focus spot diffusion. Through the depth of field extension, the image can accurately reflect the structural changes at different depths, especially showing significant details in the blood vessel morphology and distribution. Using the method of the present invention can effectively suppress the noise of the photoacoustic microscopy image, improve the signal-to-noise ratio of the image, and extend the depth of field, enabling the tiny structure of the deep tissue to be shown more clearly, and can be widely applied to photoacoustic microscopy imaging and other high-precision image processing applications.

Claims

1. A photoacoustic microscopy depth of field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution, characterized in that The method includes the following steps: Step 1) Collect photoacoustic data using a photoacoustic microscopy system: Collect photoacoustic data using a photoacoustic microscopy system based on a single-element ultrasonic transducer; Step 2) Construct a point spread function based on the photoacoustic microscopy imaging system: Step 21) Calculate the point spread function in the focal region of the photoacoustic microscopy imaging system; Step 22) Estimate the point spread function in the defocused region; In step 23), initially estimate the point spread function in the focal region as the initial value, and then perform blind deconvolution separately at each defocused layer to obtain the point spread function at that depth; Step 3) Construct continuity and sparsity constraints: The continuity constraint is expressed as: Among them, R Hessian (h) represents a continuity constraint, h xx , h xy , h xz , ··· represent the gradients of the image h in different directions, and λ1 is used to adjust the weight of the Hessian regularization dimension; The sparsity constraint is expressed as: R L = λ L1 ‖h‖1 Among them, R L represents the sparsity constraint, and λ1 is used to adjust the weight of the sparse regularization dimension; Step 4) Solve the constrained optimization problem: Use the accelerated Nesterov's alternating direction multiplier method to solve the constrained optimization problem; Step 5) 3D deconvolution: After the image obtained by the accelerated Nesterov's alternating direction multiplier method approaches the ideal state, apply the Richardson-Lucy deconvolution method to further optimize the image and obtain a 3D photoacoustic microscopy image with a larger depth of field.

2. The photoacoustic microscopy depth of field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution according to claim 1, characterized in that The specific steps of step 1) are as follows: Step 11) The photoacoustic microscopy imaging system uses a pulsed laser to excite a biological tissue sample. The laser pulse is absorbed by the light absorber in the biological tissue, triggering a thermoelastic effect and generating an ultrasonic signal; Step 12) The photoacoustic microscopy imaging system uses a single-element ultrasonic detector array to receive the propagation of ultrasonic signals in the tissue, and then collects information at different depths. Through two-dimensional raster scanning, 3D imaging with depth information is achieved. The imaging process is expressed as: f(x,y,z)=p(x,y,z)*h(x,y,z)+n(x,y,z) f(x,y,z) represents the photoacoustic microscopy image, p(x,y,z) represents the point spread function at different depths, h(x,y,z) represents the depth-of-field extended image, and n(x,y,z) represents the noise term of the photoacoustic microscopy imaging system.

3. The photoacoustic microscopy depth of field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution according to claim 1, characterized in that In step 1), the photoacoustic data includes photoacoustic image data of phantoms and in vivo samples.

4. The photoacoustic microscopy depth of field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution according to claim 1, characterized in that In step 23), the PSF construction model is expressed as: PSF(z)=Gaussian(z,σ(z)) where PSF(z) is the point spread function of the photoacoustic microscopy imaging system, z is the depth of the photoacoustic microscopy imaging system, and σ(z) is the standard deviation at this depth.

5. The photoacoustic microscopy depth of field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution according to claim 1, wherein The specific steps of step 4) are as follows: Step 41) Construct a new optimization problem based on the augmented Lagrangian function and achieve the optimization process by alternately optimizing the main variable and the auxiliary variable: Among them, a new optimization problem is constructed based on the augmented Lagrangian function. U and V are auxiliary variables, representing the continuity and sparsity constraints on the photoacoustic microscopy image. λ1 and λ2 are regularization parameters. By alternately optimizing the photoacoustic image H, the auxiliary variables U and V, image restoration at different depths is achieved. Step 42) At each gradient update, use a momentum term to accelerate the convergence process. The specific update formula is: Among them, H k is the updated photoacoustic image, β is the Nesterov momentum factor, η is the learning rate, is the gradient of the objective function.

6. The photoacoustic microscopy depth-of-field enhancement imaging method based on three-dimensional continuity and sparsity deconvolution according to claim 1, wherein In step 5), the Richardson-Lucy deconvolution method is expressed as: Among them, o (k) (x, y, z) represents the depth-of-field extended image after deconvolution, h(x, y, z) represents the photoacoustic microscopy image, and p(x, y, z) represents the point spread function at different depths.

Images