Multi-focus photoacoustic imaging registration fusion method based on blind deconvolution

Abstract

The invention relates to a multi-focus photoacoustic imaging registration fusion method based on blind deconvolution. The method comprises the following steps: obtaining photoacoustic images with different laser wavelengths and different depths of field by using a rapid photoacoustic imaging device; establishing a blind deconvolution model and initializing a blind deconvolution fuzzy kernel; based on the blind deconvolution model, deconvolution and fuzzy kernel updating are carried out on the photoacoustic image, the photoacoustic image after blind deconvolution optimization is obtained, and optimization is carried out continuously through an image quality judgment function; determining an optimal transformation parameter based on intensity information by using an intensity-based photoacoustic image registration algorithm, and performing registration on the photoacoustic images with different depths of field after blind deconvolution optimization; and fusing the registered photoacoustic images by using an image fusion algorithm to generate a focused and fused high-quality super-resolution photoacoustic image. Compared with the prior art, the method has the advantages of being non-invasive, non-destructive, convenient to operate and capable of rapidly achieving feature registration and fusion of photoacoustic images with different depths of field and recovering high-resolution imaging.

Description

Technical Field

[0001] The invention relates to the field of photoacoustic imaging, and in particular to a multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution. Background Art

[0002] Biomedical imaging emerged in the late 19th century. Since the 21st century, although a variety of biomedical imaging methods have been proposed, some of them are expensive to produce, and some will affect human health, and cannot meet the requirements of low-cost, high-quality, and harmless medical imaging methods. In recent years, photoacoustic imaging technology has been greatly developed, with the advantages of high image contrast and high penetration depth, but for imaging of multi-layered structures or irregular biological tissues, for non-focusing photoacoustic imaging, there is a problem of poor effect in the out-of-focus area, that is, there will be problems of reduced resolution and reduced sensitivity in areas outside the acoustic and optical focus. Some equipment can solve this problem. For example, the optical resolution photoacoustic microscope with dynamic focusing can achieve focused imaging of different depths of field through free focusing, but dynamic focusing involves liquid lenses or Z-axis electric adjustment. Liquid lenses often change the optical aperture and make it difficult to achieve large-scale super-resolution imaging. The mechanical inertia of Z-axis adjustment will slow down the dynamic focusing speed.

[0003] However, traditional photoacoustic imaging equipment can only obtain longitudinal or transverse profile imaging results within a certain range. For example, photoacoustic microscopes can only obtain horizontal imaging at a certain depth and cannot fully display imaging information of biological tissues with multilayer structures or irregular structures. At present, in the study of extending the depth of field of traditional photoacoustic microscopes, dynamic remote focusing, spatial and spectral multiplexing, and decoupled illumination and detection in microscopes can be used, but they are not only very complex in design, but also very expensive. The optical resolution of the synchronous zoom photoacoustic objective lens is rapidly controllable, and the confocal photoacoustic microscope solves this problem and can obtain horizontal imaging of different depths of field of organisms. Therefore, it is necessary to consider how to quickly obtain multi-focus super-resolution registered fusion photoacoustic images of organisms. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution. The blind deconvolution method can obtain an approximate system response function when the system response function is unknown, thereby improving the image resolution. The intensity-based image registration and fusion algorithm can effectively fuse the features of photoacoustic images with different depths of field. The method is not only applicable to photoacoustic microscopy imaging, but also to photoacoustic tomography imaging, and can quickly obtain multi-focus super-resolution registration and fusion images of organisms.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution comprises the following steps:

[0007] Step 1, using a fast photoacoustic imaging device to obtain photoacoustic images with different laser wavelengths and different depths of field;

[0008] Step 2: Based on the photoacoustic image, a blind deconvolution model is established and the blind deconvolution blur kernel is initialized by using a blind deconvolution algorithm;

[0009] Step 3: Based on the blind deconvolution model established in step 2, the photoacoustic image is deconvolved and the blur kernel is updated to obtain the photoacoustic image after blind deconvolution optimization, and the image quality judgment function is continuously optimized;

[0010] Step 4: using an intensity-based photoacoustic image registration algorithm, determining optimal transformation parameters based on the intensity information of the image, and registering the photoacoustic images of different depths of field after blind deconvolution optimization based on the optimal transformation parameters;

[0011] Step 5: Use an image fusion algorithm to fuse the photoacoustic images registered in step 4 to generate a focused fused high-quality super-resolution photoacoustic image.

[0012] The blind deconvolution model is a linear fuzzy mathematical model used to describe the image blurring process, which is expressed as: g(x,y)=o(x,y)*h(x,y)+n(x,y), wherein g(x,y) is a photoacoustic image acquired by a photoacoustic imaging device, o(x,y) is an original clear photoacoustic image, h(x,y) is a system function of the photoacoustic imaging device, i.e., a blur kernel, and n(x,y) refers to additive noise in the imaging process.

[0013] In step 2, the blind deconvolution blur kernel is initialized to a known uniform blur kernel.

[0014] The step 3 is specifically as follows: performing deconvolution processing on the photoacoustic image using the initially estimated blur kernel by means of frequency domain transformation, and updating the estimated blur kernel by means of iterative optimization according to the difference between the deconvolved image and the original image.

[0015] In step 3, the blur kernel is best estimated by the maximum likelihood function, and the estimate of the blur kernel is gradually improved through repeated deconvolution and blur kernel update iterative processes. Whether the iteration is terminated is determined by setting a maximum number of iterations or setting the transformation of the blur kernel to be less than a certain threshold.

[0016] The step 4 comprises the following steps:

[0017] Respectively projecting the amplitudes or parameters of the fixed photoacoustic image after blind deconvolution optimization and the photoacoustic image to be registered to obtain intensity images with relevant intensity information;

[0018] Extract feature points and feature descriptors of fixed intensity image and intensity image to be registered based on robust feature algorithm;

[0019] Using a K-nearest matching algorithm to determine the best match between features of the fixed intensity image and the intensity image to be registered;

[0020] The pixel coordinates of the intensity image are converted into homogeneous coordinates by estimating the affine transformation matrix through a random sampling consensus algorithm, and the secondary coordinates are transformed by aligning the affine transformation matrix;

[0021] Bilinear interpolation is performed to obtain the transformed pixel values ​​to achieve the transformation alignment of the images to be registered.

[0022] The step 5 is specifically as follows:

[0023] When there are two registered photoacoustic images, a new composite image is created directly based on the two photoacoustic images, and the image features of the two images are complementary and fused;

[0024] When there are three or more registered photoacoustic images, they are first fused in pairs to initially fuse the photoacoustic image features of different depths of field into the same image, and the differences between different input images are displayed through pseudo-color, that is, the color of each pixel of the processed images at different depths of field is adjusted to different proportions of the three primary colors of R, G, and B, and then the multiple images after preliminary fusion are superimposed to obtain the final fusion result.

[0025] The photoacoustic image is a photoacoustic microscopic image or a photoacoustic tomographic image.

[0026] The photoacoustic images with different depths of field are obtained by adjusting the optical focus or the focusing plane of the acoustic transducer, or by directly obtaining the photoacoustic imaging pictures of different layers.

[0027] The different laser wavelengths include 266 nm, 532 nm, near infrared I region and II region.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] For traditional photoacoustic imaging systems, if high-definition focused images of different depths are to be obtained, it is necessary to image biological tissues of different depths by adjusting the focusing plane or reconstruction principle. Therefore, it is impossible to obtain the full picture of biological tissues from a single imaging result. The method of the present invention can achieve the focused imaging results of biological tissues with multiple depths of field from a single imaging result for multi-focused photoacoustic imaging results. First, the photoacoustic images of different depths of field output by the photoacoustic imaging system are processed using a blind deconvolution algorithm to reduce the influence of the response function of the photoacoustic imaging system on the image clarity, and to a certain extent reduce the influence of light penetration distribution attenuation on the clarity of photoacoustic imaging; then, the photoacoustic images of different depths of field are initially aligned using an affine transformation model through an intensity-based photoacoustic image alignment algorithm. Finally, the features of different focused depths of field are quickly extracted through an image fusion method to generate focused fused high-quality photoacoustic images. The present invention has the advantages of being non-invasive, non-destructive, easy to operate, and can quickly achieve feature alignment and fusion of photoacoustic images of different depths of field, and restore high-resolution imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the method of the present invention;

[0031] Figure 2 FIG. 4 is a diagram showing the photoacoustic image registration and fusion result of the present invention in an embodiment. DETAILED DESCRIPTION

[0032] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0033] This embodiment provides a multi-focus photoacoustic imaging registration fusion method based on blind deconvolution (Multi-focus image fusion via blind deconvolution, MIFBD), such as Figure 1 As shown, the following steps are included:

[0034] Step 1: Use fast photoacoustic imaging equipment to obtain photoacoustic images with different laser wavelengths and different depths of field.

[0035] In this embodiment, the photoacoustic image is a photoacoustic microscopic image or a photoacoustic tomographic image or other photoacoustic imaging images. Photoacoustic images with different depths of field are obtained by adjusting the optical focus or the focusing plane of the acoustic transducer, or by directly obtaining photoacoustic imaging images of different layers. The different laser wavelengths that can be processed include 266nm wavelength, 532nm wavelength, near infrared I zone and II zone.

[0036] Step 2: Based on the photoacoustic image, a blind deconvolution model is established and the blind deconvolution blur kernel is initialized through a blind deconvolution algorithm.

[0037] The blind deconvolution model is a linear fuzzy mathematical model used to describe the image blurring process, which is expressed as: g(x,y)=o(x,y)*h(x,y)+n(x,y), where g(x,y) is the photoacoustic image acquired by the photoacoustic imaging device, o(x,y) is the original clear photoacoustic image, h(x,y) is the system function of the photoacoustic imaging device, i.e., the blur kernel, and n(x,y) refers to the additive noise in the imaging process.

[0038] In this embodiment, the blind deconvolution blur kernel is initialized to a known uniform blur kernel in order to provide a starting point for iterative optimization estimation.

[0039] Step 3: Based on the blind deconvolution model established in step 2, the photoacoustic image is deconvolved and the blur kernel is updated to obtain the photoacoustic image after blind deconvolution optimization, and the image quality judgment function is continuously optimized.

[0040] Specifically: the photoacoustic image is deconvolved using an initially estimated blur kernel by frequency domain transformation, and the estimated blur kernel is updated by iterative optimization according to the difference between the deconvolved image and the original image.

[0041] This embodiment makes the best estimate of the blur kernel through the maximum likelihood function, and gradually improves the estimate of the blur kernel through repeated deconvolution and blur kernel update iterative processes. As the number of iterations increases, a ringing effect may appear. Whether the iteration is terminated is determined by setting the maximum number of iterations or setting the transformation of the blur kernel to be less than a certain threshold.

[0042] Step 4: Using the intensity-based photoacoustic image registration algorithm, the optimal transformation parameters are determined based on the intensity information of the image, and the photoacoustic images of different depths of field after blind deconvolution optimization are registered based on the optimal transformation parameters.

[0043] Step 4 includes the following steps:

[0044] Step 41, respectively projecting the amplitudes or parameters of the fixed photoacoustic image after blind deconvolution optimization and the photoacoustic image to be registered to obtain intensity images of relevant intensity information, where the intensity information is represented by grayscale values ​​of different intensities;

[0045] Step 42: extracting feature points and feature descriptors of the fixed intensity image and the intensity image to be registered based on a robust feature algorithm;

[0046] Step 43, using a K nearest matching algorithm to determine the best match between the features of the fixed intensity image and the intensity image to be registered;

[0047] Step 44, estimating an affine transformation matrix by a random sampling consistency algorithm, converting the pixel coordinates of the intensity image into homogeneous coordinates, and aligning the secondary coordinates for transformation using the affine transformation matrix;

[0048] Step 45: perform bilinear interpolation to obtain transformed pixel values, and realize transformation alignment of the image to be registered, so that the feature points on the photoacoustic image achieve one-to-one mapping.

[0049] Step 5: Use an image fusion algorithm to fuse the photoacoustic images registered in step 4 to generate a focused fused high-quality super-resolution photoacoustic image.

[0050] When there are two registered photoacoustic images, a new composite image is created directly based on the two photoacoustic images, and the image features of the two images are complementary and fused.

[0051] When the number of registered photoacoustic images is greater than or equal to three, pairwise fusion is first performed to initially fuse the photoacoustic image features of different depths of field into the same image, and the differences between different input images are displayed through pseudo-color, such as Figure 2 As shown, that is, the color of each pixel of the processed images at different depths of field is adjusted to different proportions of the three primary colors of R, G, and B, and then the multiple images after preliminary fusion are superimposed to obtain the final fusion result, highlighting the feature differences therein.

[0052] In this embodiment, half of the peak intensity corresponding to the marked position in the photoacoustic image is normalized and compared, and is used as one of the clarity indicators of the registered fusion image to quantitatively evaluate the image.

[0053] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution, characterized in that: The following steps are involved: Step 1, using a fast photoacoustic imaging device to obtain photoacoustic images with different laser wavelengths and different depths of field; Step 2: Based on the photoacoustic image, a blind deconvolution model is established and the blind deconvolution blur kernel is initialized by using a blind deconvolution algorithm; Step 3: Based on the blind deconvolution model established in step 2, the photoacoustic image is deconvolved and the blur kernel is updated to obtain the photoacoustic image after blind deconvolution optimization, and the image quality judgment function is continuously optimized; Step 4: using an intensity-based photoacoustic image registration algorithm, determining optimal transformation parameters based on the intensity information of the image, and registering the photoacoustic images of different depths of field after blind deconvolution optimization based on the optimal transformation parameters; Step 5: Use an image fusion algorithm to fuse the photoacoustic images registered in step 4 to generate a focused fused high-quality super-resolution photoacoustic image.

2. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: The blind deconvolution model is a linear fuzzy mathematical model used to describe the image blurring process, which is expressed as: g(x,y)=o(x,y)*h(x,y)+n(x,y), wherein g(x,y) is a photoacoustic image acquired by a photoacoustic imaging device, o(x,y) is an original clear photoacoustic image, h(x,y) is a system function of the photoacoustic imaging device, i.e., a blur kernel, and n(x,y) refers to additive noise in the imaging process.

3. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: In step 2, the blind deconvolution blur kernel is initialized to a known uniform blur kernel.

4. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: The step 3 is specifically as follows: performing deconvolution processing on the photoacoustic image using the initially estimated blur kernel by means of frequency domain transformation, and updating the estimated blur kernel by means of iterative optimization according to the difference between the deconvolved image and the original image.

5. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: In step 3, the blur kernel is best estimated by the maximum likelihood function, and the estimate of the blur kernel is gradually improved through repeated deconvolution and blur kernel update iterative processes. Whether the iteration is terminated is determined by setting a maximum number of iterations or setting the transformation of the blur kernel to be less than a certain threshold.

6. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: The step 4 comprises the following steps: Respectively projecting the amplitudes or parameters of the fixed photoacoustic image after blind deconvolution optimization and the photoacoustic image to be registered to obtain intensity images with relevant intensity information; Extract feature points and feature descriptors of fixed intensity image and intensity image to be registered based on robust feature algorithm; Using a K-nearest matching algorithm to determine the best match between features of the fixed intensity image and the intensity image to be registered; The pixel coordinates of the intensity image are converted into homogeneous coordinates by estimating the affine transformation matrix through a random sampling consensus algorithm, and the affine transformation matrix is ​​used to align the secondary coordinates for transformation; Bilinear interpolation is performed to obtain the transformed pixel values ​​to achieve the transformation alignment of the images to be registered.

7. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: The step 5 is specifically as follows: When there are two registered photoacoustic images, a new composite image is created directly based on the two photoacoustic images, and the image features of the two images are complementary and fused; When there are three or more registered photoacoustic images, they are first fused in pairs to initially fuse the photoacoustic image features of different depths of field into the same image, and the differences between different input images are displayed through pseudo-color, that is, the color of each pixel of the processed images at different depths of field is adjusted to different proportions of the three primary colors of R, G, and B, and then the multiple images after preliminary fusion are superimposed to obtain the final fusion result.

8. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: The photoacoustic image is a photoacoustic microscopic image or a photoacoustic tomographic image.

9. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: The photoacoustic images with different depths of field are obtained by adjusting the optical focus or the focusing plane of the acoustic transducer, or by directly obtaining the photoacoustic imaging pictures of different layers.

10. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that: The different laser wavelengths include 266 nm, 532 nm, near infrared I region and II region.

Images