A multi-focus photoacoustic imaging registration fusion method based on blind deconvolution

Abstract

The application relates to a multi-focus photoacoustic imaging registration fusion method based on blind deconvolution, which comprises the following steps: acquiring photoacoustic images of different laser wavelengths and different depths of field by using a fast photoacoustic imaging device; establishing a blind deconvolution model and initializing a blind deconvolution blur kernel; based on the blind deconvolution model, deconvolving the photoacoustic image and updating the blur kernel to obtain an optimized photoacoustic image through blind deconvolution, and continuously optimizing by using an image quality judgment function; using an intensity-based photoacoustic image registration algorithm to determine optimal transformation parameters based on intensity information, and registering the photoacoustic images of different depths of field after blind deconvolution optimization; using an image fusion algorithm to fuse the registered photoacoustic images to generate a focused fusion high-quality super-resolution photoacoustic image. Compared with the prior art, the application has the advantages of non-invasiveness, non-destructiveness, convenience, fast realization of photoacoustic image feature registration and fusion of different depths of field, and recovery of high-resolution imaging.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of photoacoustic imaging, in particular to a multi-focus photoacoustic imaging registration fusion method based on blind deconvolution. BACKGROUND

[0002] Biomedical imaging originated in the late nineteenth century. Until the 21st century, although various biomedical imaging methods have been proposed, some are expensive to produce, and some affect human health, which cannot meet the requirements of low-cost, high-quality and harmless medical imaging. In recent years, photoacoustic imaging technology has developed rapidly and has the advantages of high image contrast and high penetration depth. However, for non-focusing photoacoustic imaging, there is a problem of poor effect in the out-of-focus area when imaging biological tissues with multi-layer structure or irregular structure, that is, there is a problem of resolution and sensitivity reduction in the area outside the acoustic and optical focus. Some devices can solve this problem, for example, a dynamic focusing optical resolution photoacoustic microscope can achieve focusing imaging of different depths by free focusing, but dynamic focusing involves liquid lens or Z-axis electric adjustment, and the liquid lens often changes the optical aperture, making it difficult to achieve large-scale super-resolution imaging, and the mechanical inertia of Z-axis adjustment will slow down the speed of dynamic focusing.

[0003] The traditional photoacoustic imaging device can only obtain longitudinal or transverse cross-sectional imaging results within a certain range. For example, a photoacoustic microscope can only obtain horizontal direction imaging at a certain depth, and cannot fully display the imaging information of biological tissues with multi-layer structure or irregular structure. At present, in the research on extending the depth of field of the traditional photoacoustic microscope, dynamic remote focusing, spatial and spectral multiplexing, decoupled illumination and detection in the microscope can be used, but they are not only very complex in design, but also very expensive in cost. The optical resolution of the synchronous zoom photoacoustic objective lens is quickly controllable, and the confocal photoacoustic microscope solves this problem and can obtain horizontal direction imaging of different depths of biological tissues. Therefore, it is necessary to consider how to quickly obtain multi-focus super-resolution registration fusion photoacoustic images of biological tissues. SUMMARY

[0004] The purpose of the present application is to provide a multi-focus photoacoustic imaging registration fusion method based on blind deconvolution. The blind deconvolution method can obtain an approximate system response function when the system response function is unknown, thereby improving the image resolution. The intensity-based image registration fusion algorithm can effectively fuse the features of photoacoustic images at different depths. This method is not only suitable for photoacoustic microscopic imaging, but also suitable for photoacoustic tomography, and can quickly obtain multi-focus super-resolution registration fusion images of biological tissues.

[0005] The purpose of the present application can be achieved by the following technical solutions:

[0006] A multi-focus photoacoustic imaging registration fusion method based on blind deconvolution, comprising the following steps:

[0007] Step 1, using a fast photoacoustic imaging device to obtain photoacoustic images of different laser wavelengths and different depths of field;

[0008] Step 2, based on the photoacoustic image, a blind deconvolution model is established and the blind deconvolution kernel is initialized by a blind deconvolution algorithm;

[0009] Step 3, based on the blind deconvolution model established in step 2, the photoacoustic image is deconvolved and the kernel is updated to obtain the photoacoustic image optimized by blind deconvolution, and the optimization is continuously carried out through an image quality judgment function;

[0010] Step 4, using an intensity-based photoacoustic image registration algorithm, the optimal transformation parameter is determined based on the intensity information of the image, and the photoacoustic images of different depths of field optimized by blind deconvolution are registered based on the optimal transformation parameter;

[0011] Step 5, using an image fusion algorithm, the photoacoustic images registered in step 4 are fused to generate a focused fused high-quality super-resolution photoacoustic image.

[0012] The blind deconvolution model is a linear blur mathematical model used to describe the image blurring process, which is represented as: g(x,y) = o(x,y)*h(x,y) + n(x,y), wherein g(x,y) is the photoacoustic image obtained by the photoacoustic imaging device, o(x,y) is the original clear photoacoustic image, h(x,y) is the system function of the photoacoustic imaging device, i.e. the blur kernel, and n(x,y) refers to the additive noise in the imaging process.

[0013] In step 2, the blind deconvolution kernel is initialized as a known uniform blur kernel.

[0014] In step 3, the initial estimated blur kernel is used to deconvolve the photoacoustic image by a frequency domain transformation method, and the estimated blur kernel is updated by iterative optimization according to the difference between the deconvolved image and the original image.

[0015] In step 3, the blur kernel is optimally estimated by a maximum likelihood function, the estimation of the blur kernel is gradually improved through repeated deconvolution and iterative process of kernel updating, and the iteration is terminated by setting the maximum number of iterations or setting the transformation of the blur kernel to be less than a certain threshold.

[0016] The step 4 comprises the following steps:

[0017] The intensity images related to the intensity information are obtained by projecting the amplitude or parameter of the fixed photoacoustic image and the photoacoustic image to be registered after blind deconvolution optimization, respectively;

[0018] extracting feature points and feature descriptors of the fixed intensity image and the to-be-registered intensity image based on a robust feature algorithm;

[0019] determining the best match between the features of the fixed intensity image and the to-be-registered intensity image using a K nearest match algorithm;

[0020] estimating an affine transformation matrix through a random sample consensus algorithm, converting pixel coordinates of the intensity image into homogeneous coordinates, and transforming the homogeneous coordinates using the affine transformation matrix;

[0021] performing bilinear interpolation to obtain pixel values after transformation, and realizing transformation alignment of the to-be-registered image.

[0022] The step 5 specifically comprises:

[0023] When the registered photoacoustic images are two, a new composite image is directly created based on the two photoacoustic images, and the image features of the two are complementarily fused;

[0024] When the registered photoacoustic images are greater than or equal to three, firstly, two-by-two fusion is performed, the photoacoustic image features of different depths are preliminarily fused into the same image, and the differences between different input images are displayed through pseudo-color, that is, the color of each pixel of the image under different depths after processing is adjusted to the intensity of the proportion of different R, G and B primary colors, and then the preliminarily fused multiple images are superimposed to obtain the final fusion result.

[0025] The photoacoustic image is a photoacoustic microscopic image or a photoacoustic tomographic image.

[0026] The photoacoustic images of different depths are obtained by adjusting optical focusing or acoustic transducer focusing surface, or directly obtaining photoacoustic imaging pictures of different layers.

[0027] The different laser wavelengths include 266nm wavelength, 532nm wavelength, near-infrared I region and II region.

[0028] Compared with the prior art, the present application has the following beneficial effects:

[0029] For a traditional photoacoustic imaging system, if a high-definition focused image at different depths is to be obtained, the biological tissue at different depths needs to be imaged by adjusting the focusing plane or the reconstruction principle. Thus, the overall appearance of the biological tissue cannot be obtained from a single imaging result. The method of the present application can realize the presentation of multi-depth focused imaging results of biological tissue from a single imaging result for multi-focused photoacoustic imaging results. First, a blind deconvolution algorithm is used to process photoacoustic images at different depths output by a photoacoustic imaging system to reduce the influence of the photoacoustic imaging system response function on the image definition, and to reduce the influence of light penetration distribution attenuation on the photoacoustic imaging definition to a certain extent. Then, an affine transformation model is used to perform initial registration of photoacoustic images at different depths by using an intensity-based photoacoustic image registration algorithm. Finally, a focused fusion high-quality photoacoustic image is generated by quickly extracting features at different focusing depths through an image fusion method. The present application has the advantages of non-invasiveness, non-destructiveness, easy operation, fast realization of feature registration, fusion and recovery of high-resolution imaging of photoacoustic images at different depths. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A flowchart of the method of the present application is shown.

[0031] Figure 2 A photoacoustic image registration fusion result diagram of the present application in an embodiment is shown. DETAILED DESCRIPTION

[0032] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. The present embodiment is implemented on the basis of the technical solution of the present application, and detailed implementation modes and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0033] The present embodiment provides a multi-focus photoacoustic imaging registration fusion method based on blind deconvolution (MIFBD), as shown in Figure 1 The method comprises the following steps:

[0034] Step 1: Use a fast photoacoustic imaging device to obtain photoacoustic images at different laser wavelengths and different depths.

[0035] In the present embodiment, the photoacoustic images are various photoacoustic imaging images such as photoacoustic microscopic images or photoacoustic tomographic images. The photoacoustic images at different depths are obtained by adjusting the optical focusing or the acoustic transducer focusing plane, or directly obtained by obtaining photoacoustic imaging pictures at different layers. The different laser wavelengths that can be processed include 266nm wavelength, 532nm wavelength, near-infrared I region and II region.

[0036] Step 2, based on the photoacoustic image, a blind deconvolution model is established and the blind deconvolution kernel is initialized by a blind deconvolution algorithm.

[0037] The blind deconvolution model is a linear blur mathematical model used to describe the blurring process of the image, which is expressed as: g(x, y) = o(x, y) * h(x, y) + n(x, y), wherein g(x, y) is the photoacoustic image obtained by the photoacoustic imaging device, o(x, y) is the original clear photoacoustic image, h(x, y) is the system function of the photoacoustic imaging device, i.e. the blur kernel, and n(x, y) refers to the additive noise in the imaging process.

[0038] In this embodiment, the blind deconvolution kernel is initialized as a known uniform blur kernel, which is to provide a starting point for iterative optimization estimation.

[0039] Step 3, based on the blind deconvolution model established in step 2, the photoacoustic image is deconvolved and the kernel is updated to obtain the photoacoustic image after blind deconvolution optimization, and the image quality judgment function is used for continuous optimization.

[0040] Specifically, the initial estimated blur kernel is used to deconvolve the photoacoustic image by the method of frequency domain transformation, and the estimated blur kernel is updated by iterative optimization according to the difference between the deconvolved image and the original image.

[0041] In this embodiment, the maximum likelihood function is used to estimate the blur kernel, and the estimation of the blur kernel is gradually improved through the iterative process of repeated deconvolution and kernel updating. With the increase of the number of iterations, the ringing effect may occur. The maximum number of iterations or the transformation of the blur kernel is set to be less than a certain threshold to determine whether the iteration is terminated.

[0042] Step 4, using the intensity-based photoacoustic image registration algorithm, the optimal transformation parameter is determined based on the intensity information of the image, and the photoacoustic images of different depths after blind deconvolution optimization are registered based on the optimal transformation parameter.

[0043] Step 4 includes the following steps:

[0044] Step 41, the amplitude or parameter of the fixed photoacoustic image and the to-be-registered photoacoustic image after blind deconvolution optimization is projected to obtain an intensity image related to the intensity information, and the intensity information is represented by different gray values of intensity;

[0045] Step 42, based on the robust feature extraction algorithm, the feature points and feature descriptors of the fixed intensity image and the to-be-registered intensity image are extracted;

[0046] Step 43, the K nearest matching algorithm is used to determine the best match between the features of the fixed intensity image and the to-be-registered intensity image;

[0047] Step 44, estimate the affine transformation matrix by the random sample consensus algorithm, convert the pixel coordinates of the intensity image into homogeneous coordinates, and transform the homogeneous coordinates using the affine transformation matrix;

[0048] Step 45, perform bilinear interpolation to obtain the pixel value after transformation, realize the transformation alignment of the image to be registered, and make the feature points on the photoacoustic image achieve one-to-one mapping.

[0049] Step 5, using an image fusion algorithm, fuse the photoacoustic images registered in step 4 to generate a focused fused high-quality super-resolution photoacoustic image.

[0050] When the registered photoacoustic images are two, a new composite image is created directly based on the two photoacoustic images, and the image features of the two are complementary fused.

[0051] When the registered photoacoustic images are greater than or equal to three, first, two-by-two fusion is performed, the photoacoustic image features of different depths are preliminarily fused into the same image, and the differences between different input images are displayed through pseudo-color, as shown in Figure 2 , that is, the color of each pixel of the processed image at different depths is adjusted to the intensity of the proportion of the three primary colors R, G and B, and then the preliminarily fused multiple images are superimposed to obtain the final fusion result, highlighting the feature differences.

[0052] In this embodiment, the peak intensity corresponding to the marked position in the photoacoustic image is normalized and compared, and they are used as one of the definition indicators of the registered fusion image to quantitatively evaluate the image.

[0053] The preferred embodiments of the application are described in detail above. It should be understood that those skilled in the art can make many modifications and changes without creative labor according to the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment on the basis of the prior art according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution, characterized in that, Includes the following steps: Step 1: Use a fast photoacoustic imaging device to acquire photoacoustic images with different laser wavelengths and depths of field; Step 2: Based on the photoacoustic image, establish a blind deconvolution model and initialize the blind deconvolution blur kernel using the blind deconvolution algorithm. The blind deconvolution model is a linear blur mathematical model used to describe the image blurring process, and it is expressed as follows: ,in g ( x , y ( ) is a photoacoustic image acquired through a photoacoustic imaging device. o ( x , y (This is) the original, clear photoacoustic image. h ( x , y ) is the system function of the photoacoustic imaging device, i.e., the blur kernel. n ( x , y This refers to additive noise during the imaging process; Step 3: Based on the blind deconvolution model established in Step 2, deconvolve and update the blur kernel of the photoacoustic image to obtain the photoacoustic image after blind deconvolution optimization, and continuously optimize it through the image quality judgment function; In step 3, the fuzzy kernel is optimally estimated using the maximum likelihood function. Through repeated deconvolution and iterative processes of updating the fuzzy kernel, the estimation of the fuzzy kernel is gradually improved. The iteration is terminated by setting a maximum number of iterations or setting the transformation of the fuzzy kernel to be less than a certain threshold. Step 4: Using an intensity-based photoacoustic image registration algorithm, determine the optimal transformation parameters based on the intensity information of the image, and register photoacoustic images of different depths of field after blind deconvolution optimization based on the optimal transformation parameters. Step 4 includes the following steps: Intensity images containing relevant intensity information are obtained by projecting the amplitude or parameters of the fixed photoacoustic image after blind deconvolution optimization and the photoacoustic image to be registered, respectively. Based on the robust feature extraction algorithm, feature points and feature descriptors are extracted from the fixed intensity image and the intensity image to be registered. The K-nearest match algorithm is used to determine the best match between features of the fixed intensity image and the intensity image to be registered. The affine transformation matrix is ​​estimated using the random sampling consensus algorithm, the pixel coordinates of the intensity image are converted into homogeneous coordinates, and the affine transformation matrix is ​​used to transform the homogeneous coordinates. Bilinear interpolation is used to obtain the transformed pixel values, thereby achieving transformation alignment of the image to be registered; Step 5: Using an image fusion algorithm, fuse the photoacoustic images registered in Step 4 to generate a high-quality, focused, fused photoacoustic image. Step 5 specifically involves: When there are two registered photoacoustic images, a new composite image is created directly based on the two photoacoustic images, and the image features of the two are complementary and fused. When there are three or more registered photoacoustic images, they are first fused in pairs to initially fuse the features of photoacoustic images at different depths of field into the same image. The differences between the different input images are displayed by pseudo-color, that is, the color of each pixel in the processed images at different depths of field is adjusted to the proportion of the three primary colors R, G, and B. Then, the multiple images after initial fusion are superimposed to obtain the final fusion result.

2. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that, In step 2, the blind deconvolution blur kernel is initialized as a known uniform blur kernel.

3. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that, Step 3 specifically involves: using the initially estimated blur kernel to perform deconvolution processing on the photoacoustic image through frequency domain transformation, and updating the estimated blur kernel through iterative optimization based on the difference between the deconvolutioned image and the original image.

4. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that, The photoacoustic image is a photoacoustic micrograph or a photoacoustic tomographic image.

5. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that, The photoacoustic images at different depths of field are obtained by adjusting the optical focus or the focusing surface of the acoustic transducer, or by directly acquiring photoacoustic imaging images of different layers.

6. The multi-focus photoacoustic imaging registration and fusion method based on blind deconvolution according to claim 1, characterized in that, The different laser wavelengths include 266nm wavelength, 532nm wavelength, and near-infrared regions I and II.

Images