An image super-resolution enhancement method and device based on point spread function deconvolution

Abstract

The application discloses an image super-resolution enhancement method and device based on point spread function deconvolution, which comprises the following steps: performing image preprocessing to obtain a reconstructed image; performing image segmentation on the reconstructed image, and performing convolution on the intensity distribution image and an identity matrix with the same pixel size of a target super-resolution image based on a point spread function intensity distribution image to obtain an intensity distribution image and perform edge compensation; converting the intensity distribution image after edge compensation into a second-order Hessian matrix and a one-dimensional vector after down-sampling respectively, and adjusting the second-order Hessian matrix according to a regularization coefficient; setting a constraint solution as non-negative based on the second-order Hessian matrix and the one-dimensional vector, and performing deconvolution solving through a solver to obtain a solution vector, transforming the solution vector to two dimensions to obtain a target super-resolution image corresponding to a segmented sub-image; and splicing all results into a complete target super-resolution image. The application can significantly improve the image resolution and other performances, and is suitable for image post-processing and super-resolution reconstruction application scenarios in a fluorescence microscopic imaging system.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an image super-resolution enhancement method and apparatus based on point spread function deconvolution. Background Technology

[0002] Optical microscopy, a crucial tool in life sciences and materials science research, determines the precision of microscopic structures observed through its spatial resolution. Traditional fluorescence microscopy, limited by the diffraction limit, typically has a spatial resolution no lower than 200 nm. This limitation prevents the clear observation of fine features in many subcellular structures and nanoscale biomolecules. To overcome this diffraction limit, researchers have developed a series of super-resolution imaging techniques in recent years, such as structured illumination (SIM), stimulated emission depletion (STED), and single-molecule localization microscopy (SMLM). These techniques have improved the resolution of fluorescence microscopy to some extent, but still suffer from significant limitations in practical applications, including high equipment costs, low imaging speed, and complex operation. Furthermore, existing image enhancement algorithms, such as Wiener or Richardson-Lucy deconvolution, suffer from noise amplification and artifact generation, making it difficult to consistently obtain high-quality results, especially under low signal-to-noise ratio conditions.

[0003] With the rise of compressed sensing and sparse modeling, reconstruction methods based on the sparsity of images have gradually become an effective path to improve image resolution. Chinese patent application CN109816600A discloses a confocal microscopic image restoration method based on sparse representation. By adding linear coefficient constraints, the objective function is made easier to solve, and an image degradation model is introduced into the objective function to improve the approximation of the sparse representation, thereby obtaining a higher quality original image. Chinese patent application CN110852945A discloses a method for acquiring high-resolution images of biological samples. It performs Fourier transform on the original low-resolution three-dimensional stacked image to sparsify the image, and then uses compressed sensing for high-quality high-resolution image computation. These methods can preserve image structural features while suppressing noise through optimized solutions, but there is still room for improvement in model design and parameter tuning. It is difficult to balance reconstruction quality and computational efficiency, which to some extent limits the performance and effectiveness of such methods in practical applications.

[0004] Furthermore, the rise of deep learning technology has provided new opportunities to overcome the aforementioned bottlenecks. Convolutional Neural Networks (CNNs) can establish complex nonlinear mappings from low-resolution observations to high-resolution images through end-to-end learning. Architectures such as U-Net and GAN have demonstrated performance surpassing traditional methods in image super-resolution tasks, but in applications involving the acquisition of high-resolution images of biological samples, challenges in data, models, and computational resources still need to be overcome.

[0005] In conclusion, current super-resolution microscopy techniques and image enhancement methods have not yet achieved an ideal balance in terms of resolution, speed, cost, and ease of use, and further innovation and optimization are still needed. Summary of the Invention

[0006] In view of the above, the purpose of this invention is to provide an image super-resolution enhancement method and apparatus based on point spread function deconvolution, so as to achieve image enhancement and improvement of actual image resolution (increase of high frequency information in the frequency domain) under background noise suppression in microscopic image processing scenarios. In particular, it can achieve an extremely high resolution of 30nm under high signal-to-noise ratio conditions, and can still maintain the continuity and accuracy of image structure under low signal-to-noise ratio conditions. It is suitable for image post-processing and super-resolution reconstruction application scenarios in fluorescence microscopy imaging systems.

[0007] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0008] This invention provides an image super-resolution enhancement method based on point spread function deconvolution, comprising the following steps:

[0009] Preprocessing a single image, including image extraction and rolling ball algorithm, or preprocessing multiple images, including uniform structure reconstruction, image extraction and rolling ball algorithm, yields a reconstructed image.

[0010] The reconstructed image is segmented to obtain several segmented sub-images, and the size of the segmented sub-image is used as the smallest computational unit. The intensity distribution image is obtained by convolving the point spread function intensity distribution image and the target super-resolution image with the same pixel size as the unit matrix (all-1 matrix). Edge compensation is then performed on the intensity distribution image.

[0011] The intensity distribution image after edge compensation is transformed into a second-order Hessian matrix. The intensity distribution image after edge compensation is downsampled and multiplied with the corresponding elements of the segmented sub-image, the negative is taken, and then it is expanded into a one-dimensional vector. The main diagonal values ​​of the second-order Hessian matrix are adjusted according to the set regularization coefficient (second-order norm term).

[0012] Based on the second-order Hessian matrix and a one-dimensional vector, the constraint solution is set to be non-negative and the solution vector is obtained by deconvolution of the point spread function through the solver (i.e., solving the second-order optimization problem). The solution vector is then transformed into two dimensions to obtain the target super-resolution image corresponding to the segmented sub-image.

[0013] The target super-resolution image is obtained by iterating through all the segmented sub-images and stitching them together.

[0014] Preferably, the preprocessing of a single image, including image extraction and a rolling ball algorithm, or the preprocessing of multiple images, including uniform structure reconstruction, image extraction, and a rolling ball algorithm, to obtain a reconstructed image, includes:

[0015] Preprocessing a single image includes: extracting images from the single image to generate several sub-images, and reconstructing each sub-image using the rolling ball algorithm to obtain a reconstructed image;

[0016] Alternatively, preprocessing multiple images may be performed, including: calculating the average pixel value of multiple images at the same window position in multiple images, and calculating the average pixel value of a single image at each window position in each image; resetting the pixels of each image based on the average pixel value of multiple images and the average pixel value of a single image; extracting pixels from the reset multiple images to generate a single overall image; and processing the single overall image according to the preprocessing method for a single image to obtain a reconstructed image.

[0017] Preferably, the pixels of each image are reset based on the average pixel value of multiple images and the average pixel value of a single image, as expressed by the formula:

[0018]

[0019] Among them, i ′ represents the pixel reset value, i represents the original pixel value, m represents the average pixel value across multiple images, and n represents the average pixel value across a single image.

[0020] Preferably, the step of performing image segmentation on the reconstructed image to obtain several segmented sub-images includes:

[0021] The reconstructed image is divided into squares with the number of pixels as the side length. Then, the dividing lines are translated up, down, left, and right to half the length of the side length for further division. Parts that do not meet the integer division requirement are filled by mirroring until the integer division requirement is met, resulting in several segmented sub-images.

[0022] Preferably, the edge compensation of the intensity distribution image includes:

[0023] When the edge of the point spread function intensity distribution image extends beyond the pixel value region of the target super-resolution image, edge compensation is performed on the intensity distribution image. The intensity is additionally compensated by a factor of α, that is, the pixel value of the intensity distribution image is multiplied by (1+α). The formula for the compensation coefficient α is expressed as:

[0024]

[0025] Among them, PSF out The point spread function (PSF) represents the total intensity of the excess portion. in This represents the total intensity of the point spread function within the portion that does not exceed the specified limits.

[0026] Preferably, the step of converting the edge-compensated intensity distribution image into a second-order Hessian matrix includes:

[0027] The intensity distribution image after edge compensation is expanded into a one-dimensional column vector, and then the one-dimensional column vector is multiplied by its transpose to obtain the second-order Hesse matrix.

[0028] Preferably, the step of downsampling the edge-compensated intensity distribution image, multiplying it with the corresponding elements of the segmented sub-image, taking the negative, and then expanding it into a one-dimensional vector includes:

[0029] The intensity distribution image after edge compensation is downsampled using the nearest neighbor method so that the size of the downsampled image is the same as the input segmentation sub-image. The corresponding elements of the downsampled image and the segmentation sub-image are multiplied and the result is negative, and the result is expanded into a one-dimensional vector.

[0030] Preferably, solving by a solver includes using the quadprog function in MATLAB.

[0031] Preferably, the step of traversing and solving the target super-resolution images corresponding to all segmented sub-images and stitching them together to obtain the complete target super-resolution image includes:

[0032] For each of the target super-resolution images corresponding to the segmented sub-images obtained by traversing and solving, take the pixel-sized square image of the center half as the center sub-image, discard the rest, so that all the center sub-images are stitched together to form a complete target super-resolution image.

[0033] To achieve the above-mentioned objectives, this invention also provides an image super-resolution enhancement device based on point spread function deconvolution, which is implemented using the above-mentioned image super-resolution enhancement method based on point spread function deconvolution, including: a preprocessing reconstruction module, a subgraph convolution modeling module, a matrix dimensionality reduction processing module, a deconvolution solving module, and a full-image super-resolution stitching module.

[0034] The preprocessing and reconstruction module is used to perform preprocessing on a single image, including image extraction and rolling ball algorithm, or to perform preprocessing on multiple images, including uniform structure reconstruction, image extraction and rolling ball algorithm, to obtain a reconstructed image.

[0035] The sub-image convolution modeling module is used to perform image segmentation on the reconstructed image to obtain several segmented sub-images and use the size of the segmented sub-image as the smallest calculation unit. It performs convolution based on the point spread function intensity distribution image and the target super-resolution image with the same pixel size as the unit matrix to obtain the intensity distribution image, and performs edge compensation on the intensity distribution image.

[0036] The matrix dimensionality reduction processing module is used to convert the edge-compensated intensity distribution image into a second-order Hessian matrix, and after downsampling the edge-compensated intensity distribution image, multiply it with the corresponding elements of the segmented sub-image, take the negative, and then expand it into a one-dimensional vector. The main diagonal values ​​of the second-order Hessian matrix are adjusted according to the set regularization coefficient.

[0037] The deconvolution solution module is used to obtain the solution vector by setting the constraint solution to be non-negative and performing deconvolution of the point spread function based on the second-order Hess matrix and a one-dimensional vector. The solution vector is then transformed to two dimensions to obtain the target super-resolution image corresponding to the segmented sub-image.

[0038] The full-image super-resolution stitching module is used to traverse and solve the target super-resolution images corresponding to all segmented sub-images and stitch them together to obtain the complete target super-resolution image.

[0039] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0040] High structural reconstruction accuracy: Resolution can be improved to 30nm in Airyscan and SIM images, outperforming many existing open-source algorithms; Excellent noise suppression performance: Accurate reconstruction of the true structure is achieved even under low signal-to-noise ratio conditions, effectively avoiding artifacts common in traditional deconvolution; Strong adaptability: Applicable to various image types and different imaging noise levels, exhibiting high robustness; Computationally controllable: The resolution and structural continuity of the reconstructed image can be adjusted using a single parameter, the regularization coefficient, facilitating user operation and parameter optimization; No hardware modification required: Can be directly used for post-processing of commercial microscopic images, improving the resolution of existing systems; Easy modular integration: Seamlessly integrated with image acquisition, background removal, and multi-image fusion processes, suitable for integration into microscopy systems or image processing software. This invention provides an efficient, universal, and computationally stable image super-resolution enhancement method, particularly suitable for biological images with complex backgrounds, high-density structures, or sparse signals, possessing broad prospects for scientific research and clinical applications. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the image super-resolution enhancement method based on point spread function deconvolution provided in an embodiment of the present invention.

[0043] Figure 2 This is a schematic diagram of uniform intensity processing in multi-image preprocessing provided in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the sampling and image processing in multi-image preprocessing provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the image segmentation, super-resolution calculation, and stitching process provided in an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of the image convolution and compensation process provided in an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of the image enhancement effect provided in an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of the structure of the image super-resolution enhancement device based on point spread function deconvolution provided in an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0050] The inventive concept of this invention is as follows: Addressing the shortcomings of existing super-resolution microscopy techniques and image enhancement methods in terms of resolution, speed, cost, and ease of use, this invention provides an image super-resolution enhancement method and apparatus based on point spread function deconvolution. For single or multiple input images, an adaptive preprocessing strategy is employed to generate reconstructed images, improving data compatibility and initial quality. Intensity distribution modeling is performed based on a unitary matrix convolution operation with the same pixel size as the target super-resolution image using the point spread function (PSF), and edge compensation is used to correct image edge distortion, ensuring detail integrity. Simultaneously, the Hessian matrix scale is dynamically adjusted according to the regularization coefficient to balance model complexity and generalization ability. Furthermore, deconvolution is performed based on non-negativity constraints to achieve a significant improvement in spatial resolution. Finally, a complete super-resolution target image is obtained through extraction and stitching, ensuring computational efficiency while maintaining global consistency and local detail accuracy.

[0051] like Figure 1 As shown, the embodiment provides an image super-resolution enhancement method based on point spread function deconvolution, including the following steps:

[0052] S1, perform preprocessing on a single image including image extraction and rolling ball algorithm, or perform preprocessing on multiple images including uniform structure reconstruction, image extraction and rolling ball algorithm, to obtain a reconstructed image.

[0053] Receive the two-dimensional grayscale image to be processed and perform preprocessing according to its different characteristics.

[0054] S1.1, Single image preprocessing:

[0055] S1.1.1, Image Extraction:

[0056] First, set the segmentation intensity parameter 'a' as input. Then, traverse the image using a square window with a side length of 'a' pixels, and extract pixels at the corresponding positions in each window to obtain 'a'. 2 Sub-image sampling. This method of sampling sub-images actually utilizes the spatial continuity of the image, allowing continuous signals to be preserved when the sampled sub-image is processed by the Rolling Ball algorithm. Compared to the traditional Rolling Ball algorithm, even with high-intensity processing (setting a very small rolling ball radius), it does not cause severe image loss.

[0057] S1.1.2, Rolling ball algorithm processing:

[0058] The sub-images extracted from each image are processed using a rolling ball algorithm. This processing is performed by the open-source image processing software Fiji using the Process-Substract Background function. After processing, a is obtained. 2 Zhang Zitu.

[0059] S1.1.3, Image Reconstruction:

[0060] a 2 The sub-image is reconstructed into a complete image according to the original pixel positions. This process is the complete inverse of S1.1.1.

[0061] S1.2, Preprocessing of multiple images:

[0062] S1.2.1, Perform uniform structure reconstruction preprocessing on multiple images:

[0063] This processing addresses multiple identical images that, due to variations in light intensity, detector position, or other factors, produce images with different signal-to-noise ratios or point spread functions, despite potentially having the same point spread function. A typical application is in the Airyscan mode of a Zeiss confocal microscope, where it can display independent signal images from each detector.

[0064] S1.2.1.1, Uniform strength:

[0065] Set a uniform intensity window value of b, and use a square window with a side length of b pixels. Calculate the average pixel value m for all images at the corresponding window positions. For example... Figure 2 As shown, when b is 2, m is represented as (x1+x2+x3+x4+y1+y2+y3+y4+z1+z2+z3+z4) / 12.

[0066] S1.2.1.2, Calculate the pixel values ​​for each image window:

[0067] Calculate the average pixel value of each window position in each image as n. Reset the pixels of each image based on the average pixel value of multiple images and the average pixel value of a single image. The formula is as follows:

[0068]

[0069] Among them, i ′ This represents the pixel reset value, where i represents the original pixel value, m represents the average pixel value across multiple images, and n represents the average pixel value per image. For example... Figure 2 As shown, the average pixel value n of a single image in the pink image is represented as (z1+z2+z3+z4) / 4, the pixel value of z1 is reset to z1*m / n, and so on.

[0070] S1.2.1.3, Window Traversal:

[0071] Translate the window according to the method in S1.2.1.2 until all image positions have been calculated.

[0072] S1.2.1.4, Sampled raw image:

[0073] From multiple images after pixel resetting, pixels are randomly selected to generate a single overall image or a stack of several images. The number of images can be specified by the user, such as... Figure 3 As shown, a new image is generated by randomly selecting pixels.

[0074] S1.2.2, Excluding background noise from the entire image:

[0075] The reconstructed image is obtained by processing a single overall image according to the preprocessing method of single image in S1.1.

[0076] S2, the reconstructed image is segmented to obtain several segmented sub-images, and the size of the segmented sub-image is used as the smallest computational unit. The intensity distribution image is obtained by convolving the point spread function intensity distribution image and the target super-resolution image with the same pixel size as the unit matrix, and edge compensation is performed on the intensity distribution image.

[0077] S2.1, Image Segmentation:

[0078] like Figure 4 As shown, input a single reconstructed image or a reconstructed image from an image stack. Based on the user input, the reconstructed image is divided into squares with a side length equal to the number of pixels (if the image pixel count is less than 128*128, no division is required). Then, the dividing lines are shifted up, down, left, and right to half the length of the side length for further division. Parts that do not meet the integer division criteria are filled by mirroring until integer division is achieved, resulting in several segmented sub-images.

[0079] S2.2, Calculate the shape of the point spread function:

[0080] Using the full width at half maximum (FWHM) and full width at half maximum (FWHM) of the user-input point spread function (PSF) and the user-input super-resolution pixel magnification factor amp, a corresponding high-pixel PSF intensity distribution image P is obtained by Gaussian fitting or Bessel function fitting. Compared with the PSF of the original input image (segmented sub-image), the pixel density of P should be (amp). 2 times.

[0081] S2.3, Calculate the intensity distribution image:

[0082] Based on the number of pixels in the input image (or the segmented sub-image if image segmentation has been performed as the smallest computational unit), multiplied by the user-specified super-resolution pixel magnification factor amp, the number of pixels in the target super-resolution image is obtained as j*k. Then, convolving a j*k matrix with all elements equal to 1 with the PSF intensity distribution image P yields the intensity distribution image Px (using zero-padding convolution). When the edges of the point spread function intensity distribution image extend beyond the pixel value region of the target super-resolution image, edge compensation is performed on the intensity distribution image, with the intensity being additionally compensated by a factor of α, i.e., the pixel values ​​of the intensity distribution image are multiplied by (1+α). The formula for the compensation factor α is expressed as:

[0083]

[0084] Among them, PSF out The point spread function (PSF) represents the total intensity of the excess portion. in This represents the total intensity of the point spread function within the portion not exceeding the specified limits. For example... Figure 5 As shown, the pink image in the lower right corner is the intensity distribution image Px, and the compensation coefficient for the star-marked pixel in the upper left corner is α. The sum of the green intensity of the convolution kernel image P is PSF. out The total intensity of the blue portion is PSF. in .

[0085] S3 transforms the edge-compensated intensity distribution image into a second-order Hesse matrix, downsamples the edge-compensated intensity distribution image, multiplies it with the corresponding elements of the segmented sub-image, takes the negative, and then expands it into a one-dimensional vector. The main diagonal values ​​of the second-order Hesse matrix are adjusted according to the set regularization coefficient.

[0086] S3.1, Calculate the input for quadratic optimization:

[0087] The edge-compensated intensity distribution image Px is expanded into a one-dimensional column vector, and then multiplied by its transpose to obtain the second-order Hessian matrix Hp. Simultaneously, the edge-compensated intensity distribution image is downsampled by a factor of amp using a nearest-neighbor method to obtain image DPx, making the size of the downsampled image the same as the input image (or segmentation subimage). The corresponding elements of the downsampled image DPx are multiplied by the corresponding elements of the input image (or segmentation subimage), and the result is negatively multiplied, expanding into a one-dimensional vector A, which is now the coefficient vector of the first-order term. This yields the standard input for the quadratic optimization problem: the Hessian matrix Hp with quadratic coefficients and the vector A with first-order coefficients.

[0088] S3.2, Set the second norm:

[0089] The introduction of the second-order norm in the quadratic optimization problem is equivalent to adjusting the main diagonal values ​​of the second-order Hessian matrix Hp. Specifically, a user-defined regularization coefficient λ amplifies the main diagonal coefficients of the second-order Hessian matrix Hp by a factor of (1+λ). This improves the solution properties, making the image smoother and more continuous.

[0090] S4, based on the second-order Hessian matrix and a one-dimensional vector, sets the constraint solution to be non-negative and obtains the solution vector by deconvolution of the point spread function through a solver. Transforming the solution vector to two dimensions yields the target super-resolution image corresponding to the segmented sub-image.

[0091] S4.1, Non-negative solutions to quadratic optimization problems:

[0092] Since the problem to be solved has a maximum of quadratic terms, a quadratic optimization problem solver tool is used. The adjusted second-order Hessian matrix Hp and a one-dimensional vector A are input, the constraint solution is set to be non-negative, and the solution vector x is obtained. Transforming the solution vector x into two dimensions yields the desired super-resolution image of the target.

[0093] When Hp and A are fixed, this problem is a standard quadratic optimization problem, solvable by any solver. A typical solver is the quadprog function in MATLAB, which is a quadratic optimization solver. The following statement can be used to solve it:

[0094] lb = zeros(size(Hp,1),1); % Sets the constraint solution to non-negative

[0095] [x]=quadprog(Hp,A,[],[],[],[],lb,[],[],options)

[0096] Here, lb represents a zero-based vector with the same size as the solution x, used to represent the lower bound of the constraint; size(Hp,1) represents the length of the first dimension of Hp, equivalent to the length of the solution x; zeros(·) generates a zero-based vector of size size(Hp,1); [x] represents the solution vector obtained from the solution; quadprog(·) represents the solver function for solving quadratic optimization problems; [] indicates that some inputs to quadprog(·) are not input (not needed in this example); options indicates that some advanced settings of the quadprog(·) function are stored in options, which can be set by the user.

[0097] S5: Iterate through all the target super-resolution images corresponding to the segmented sub-images and stitch them together to obtain the complete target super-resolution image.

[0098] like Figure 4 As shown, for each of the target super-resolution images corresponding to all the segmented sub-images obtained through traversal and solution, a square image of half the center pixel size is taken as the center sub-image, and the rest is discarded, so that all the center sub-images are stitched together to form a complete target super-resolution image. For the image stack, one image in the stack is processed each time (if the image is larger than 128*128, the image segmentation in step S2.1 is used). After processing all the stack images (according to steps S2 to S4), all images are saved as a new stack, and the average image of the stack images is given.

[0099] like Figure 6 The image shows the processing effect of the method of the present invention on the image of a pore structure. Figure 6 The image on the left is the original image, the image on the right is the processed image, and the rightmost sidebar is a magnified view. The images show that aperture patterns exceeding the diffraction limit can be correctly reproduced by the method of this invention, significantly improving image resolution.

[0100] In summary, the image super-resolution enhancement method based on point spread function deconvolution provided by this invention transforms the problem into a quadratic optimization problem, effectively controlling the condition number problem in the solution process while maintaining the image's non-negativity and sparse structure. Based on the assumptions of high-pixel PSF and high-pixel images (the solution to the quadratic optimization problem), and combined with a downsampling mechanism and preprocessing steps, this method achieves accurate recovery of high-frequency information in real images. Especially when using commercial confocal or SIM images as input, the method of this invention can improve the spatial resolution from the traditional 100-150nm to 30nm, which is superior to existing methods and has the following significant advantages: (1) Higher resolution: significantly higher than existing methods under high SNR conditions; (2) Superior sparsity: due to the assumption of high-pixel images (unknowns in the solution), which are much larger than the number of pixels in the input image (number of equations), the sparsity of the solution is automatically generated without the need to add parameters for adjustment; (3) Strong model universality: constructing a multi-module process, integrating background removal and multi-image reconstruction algorithms, which can be used for various imaging modes such as SIM, confocal, and single-molecule localization; (4) Good parameter controllability: by adjusting the regularization coefficient λ, a balance between resolution and continuity is achieved, and further combined with non-negative constraints, background noise is suppressed while maintaining structural information, which can reduce the risk of computational ill-posedness and improve the stability and interpretability of image reconstruction. Therefore, the method of this invention provides a unified, efficient and robust image enhancement approach for the field of optical microscopy imaging, which is especially suitable for biological image analysis tasks with complex structures, sparse signals, and strong background interference. While maintaining structural authenticity, it significantly improves the spatial resolution and visual clarity of the image, providing new ideas and practical paths for the development of super-resolution imaging technology.

[0101] Based on the same inventive concept, such as Figure 7 As shown, this embodiment of the invention also provides an image super-resolution enhancement device 700 based on point spread function deconvolution, including: a preprocessing reconstruction module 710, a subgraph convolution modeling module 720, a matrix dimensionality reduction processing module 730, a deconvolution solving module 740, and a full-image super-resolution stitching module 750.

[0102] The preprocessing and reconstruction module 710 is used to perform preprocessing on a single image, including image extraction and rolling ball algorithm, or to perform preprocessing on multiple images, including uniform structure reconstruction, image extraction and rolling ball algorithm, to obtain a reconstructed image.

[0103] The sub-image convolution modeling module 720 is used to perform image segmentation on the reconstructed image to obtain several segmented sub-images. The size of the segmented sub-image is used as the smallest calculation unit. The intensity distribution image is obtained by convolution based on the point spread function intensity distribution image and the target super-resolution image with the same pixel size as the matrix. Edge compensation is then performed on the intensity distribution image.

[0104] The matrix dimensionality reduction processing module 730 is used to convert the intensity distribution image after edge compensation into a second-order Hessian matrix, and after downsampling the intensity distribution image after edge compensation, multiply it with the corresponding elements of the segmented sub-image, take the negative, and then expand it into a one-dimensional vector. The main diagonal values ​​of the second-order Hessian matrix are adjusted according to the set regularization coefficient.

[0105] The deconvolution solver module 740 is used to solve the solution vector by setting the constraint solution to be non-negative and performing deconvolution of the point spread function based on the second-order Hessian matrix and a one-dimensional vector. The solution vector is then transformed to two dimensions to obtain the target super-resolution image corresponding to the segmented sub-image.

[0106] The full-image super-resolution stitching module 750 is used to traverse and solve the target super-resolution images corresponding to all segmented sub-images and stitch them together to obtain the complete target super-resolution image.

[0107] It should be noted that the image super-resolution enhancement device based on point spread function deconvolution provided in the above embodiments belongs to the same inventive concept as the image super-resolution enhancement method based on point spread function deconvolution. For details of its specific implementation process, please refer to the embodiment of the image super-resolution enhancement method based on point spread function deconvolution, which will not be repeated here.

[0108] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image super-resolution enhancement method based on point spread function deconvolution, characterized in that, Includes the following steps: Preprocessing a single image, including image extraction and rolling ball algorithm, or preprocessing multiple images, including uniform structure reconstruction, image extraction and rolling ball algorithm, yields a reconstructed image. The reconstructed image is segmented to obtain several segmented sub-images, and the size of the segmented sub-image is used as the smallest computational unit. The intensity distribution image is obtained by convolving the point spread function intensity distribution image and the target super-resolution image with the same pixel size as the matrix. Edge compensation is then performed on the intensity distribution image. The intensity distribution image after edge compensation is transformed into a second-order Hessian matrix. The intensity distribution image after edge compensation is downsampled and multiplied with the corresponding elements of the segmented sub-image, the negative is taken, and then it is expanded into a one-dimensional vector. The main diagonal values ​​of the second-order Hessian matrix are adjusted according to the set regularization coefficient. Based on the second-order Hessian matrix and a one-dimensional vector, the constraint solution is set to be non-negative and the solution vector is obtained by deconvolution of the point spread function through the solver. The solution vector is then transformed into two dimensions to obtain the target super-resolution image corresponding to the segmented sub-image. The target super-resolution image is obtained by iterating through all the segmented sub-images and stitching them together.

2. The image super-resolution enhancement method based on point spread function deconvolution according to claim 1, characterized in that, The preprocessing of a single image, including image extraction and rolling ball algorithm, or the preprocessing of multiple images, including uniform structure reconstruction, image extraction and rolling ball algorithm, to obtain a reconstructed image, includes: Preprocessing a single image includes: extracting images from the single image to generate several sub-images, and reconstructing each sub-image using the rolling ball algorithm to obtain a reconstructed image; Alternatively, preprocessing multiple images may be performed, including: calculating the average pixel value of multiple images at the same window position in multiple images, and calculating the average pixel value of a single image at each window position in each image; resetting the pixels of each image based on the average pixel value of multiple images and the average pixel value of a single image; extracting pixels from the reset multiple images to generate a single overall image; and processing the single overall image according to the preprocessing method for a single image to obtain a reconstructed image.

3. The image super-resolution enhancement method based on point spread function deconvolution according to claim 2, characterized in that, Pixel reset is performed on each image based on the average pixel value of multiple images and the average pixel value of a single image, expressed by the formula: Among them, i ′ represents the pixel reset value, i represents the original pixel value, m represents the average pixel value across multiple images, and n represents the average pixel value across a single image.

4. The image super-resolution enhancement method based on point spread function deconvolution according to claim 1, characterized in that, The process of segmenting the reconstructed image to obtain several segmented sub-images includes: The reconstructed image is divided into squares with the number of pixels as the side length. Then, the dividing lines are translated up, down, left, and right to half the length of the side length for further division. Parts that do not meet the integer division requirement are filled by mirroring until the integer division requirement is met, resulting in several segmented sub-images.

5. The image super-resolution enhancement method based on point spread function deconvolution according to claim 1, characterized in that, The edge compensation of the intensity distribution image includes: When the edge of the point spread function intensity distribution image extends beyond the pixel value region of the target super-resolution image, edge compensation is performed on the intensity distribution image. The intensity is additionally compensated by a factor of α, that is, the pixel value of the intensity distribution image is multiplied by (1+α). The formula for the compensation coefficient α is expressed as: Among them, PSF out The point spread function (PSF) represents the total intensity of the excess portion. in This represents the total intensity of the point spread function within the portion that does not exceed the specified limits.

6. The image super-resolution enhancement method based on point spread function deconvolution according to claim 1, characterized in that, The process of converting the edge-compensated intensity distribution image into a second-order Hessian matrix includes: The intensity distribution image after edge compensation is expanded into a one-dimensional column vector, and then the one-dimensional column vector is multiplied by its transpose to obtain the second-order Hesse matrix.

7. The image super-resolution enhancement method based on point spread function deconvolution according to claim 1, characterized in that, The process of downsampling the edge-compensated intensity distribution image, multiplying it with the corresponding elements of the segmented sub-image, taking the negative, and then expanding it into a one-dimensional vector includes: The intensity distribution image after edge compensation is downsampled using the nearest neighbor method so that the size of the downsampled image is the same as the input segmentation sub-image. The corresponding elements of the downsampled image and the segmentation sub-image are multiplied and the result is negative, and the result is expanded into a one-dimensional vector.

8. The image super-resolution enhancement method based on point spread function deconvolution according to claim 1, characterized in that, Solving using a solver includes using the quadprog function in MATLAB.

9. The image super-resolution enhancement method based on point spread function deconvolution according to claim 1, characterized in that, The process of traversing and solving for the target super-resolution images corresponding to all segmented sub-images and stitching them together to obtain the complete target super-resolution image includes: For each of the target super-resolution images corresponding to the segmented sub-images obtained by traversing and solving, take the pixel-sized square image of the center half as the center sub-image, discard the rest, so that all the center sub-images are stitched together to form a complete target super-resolution image.

10. An image super-resolution enhancement device based on point spread function deconvolution, implemented using the image super-resolution enhancement method based on point spread function deconvolution as described in any one of claims 1 to 9, characterized in that, include: The system includes a preprocessing and reconstruction module, a subgraph convolution modeling module, a matrix dimensionality reduction module, a deconvolution solving module, and a full-image super-resolution stitching module. The preprocessing and reconstruction module is used to perform preprocessing on a single image, including image extraction and rolling ball algorithm, or to perform preprocessing on multiple images, including uniform structure reconstruction, image extraction and rolling ball algorithm, to obtain a reconstructed image. The sub-image convolution modeling module is used to perform image segmentation on the reconstructed image to obtain several segmented sub-images and use the size of the segmented sub-image as the smallest calculation unit. It performs convolution based on the point spread function intensity distribution image and the target super-resolution image with the same pixel size as the unit matrix to obtain the intensity distribution image, and performs edge compensation on the intensity distribution image. The matrix dimensionality reduction processing module is used to convert the edge-compensated intensity distribution image into a second-order Hessian matrix, and after downsampling the edge-compensated intensity distribution image, multiply it with the corresponding elements of the segmented sub-image, take the negative, and then expand it into a one-dimensional vector. The main diagonal values ​​of the second-order Hessian matrix are adjusted according to the set regularization coefficient. The deconvolution solution module is used to obtain the solution vector by setting the constraint solution to be non-negative and performing deconvolution of the point spread function based on the second-order Hess matrix and a one-dimensional vector. The solution vector is then transformed to two dimensions to obtain the target super-resolution image corresponding to the segmented sub-image. The full-image super-resolution stitching module is used to traverse and solve the target super-resolution images corresponding to all segmented sub-images and stitch them together to obtain the complete target super-resolution image.

Images