Nano-film microscopic image enhancement method and system based on sparse deconvolution

Abstract

The invention provides a sparse deconvolution-based nano-film microscopic image enhancement method and system, and aims to solve the problems of image blurring, low edge contrast and the like caused by an imaging system and sample characteristics in in-situ microscopic imaging. According to the method, the local contrast is enhanced through adaptive histogram equalization, multi-scale information is extracted in combination with wavelet decomposition, the high-frequency component weight is improved by introducing a channel and a space attention mechanism, and the resolution is further improved. According to the method, high-frequency details are recovered in combination with sparse blind deconvolution under the condition that a prior blurring kernel is not needed, and non-local mean denoising is adopted to weaken damage to edges, so that efficient and accurate image recovery and enhancement are realized, and the definition and detail expression of the nano-film microscopic image are remarkably improved; the method is suitable for optical in-situ microscopic image processing of nano-film materials, can provide high-quality image support for real-time observation and microstructure characterization of nano-materials, and has wide application prospects and important scientific research values.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a nano-film microscopic image enhancement method and system based on sparse deconvolution. Background Art

[0002] Nanofilm microscopy technology has become a key tool in many cutting-edge fields such as materials science, nanotechnology, biomedicine, and microelectronics engineering, with its high resolution and rich details at the nanoscale. Through precise microscopic imaging, researchers can deeply characterize the surface morphology, microstructure, and physical properties of nanofilm materials, providing an important scientific basis for the design and application of new materials. However, in practical applications, nanofilm microscopic images are often interfered by a variety of factors, resulting in a significant decrease in image quality, which restricts the accuracy and reliability of subsequent image analysis and research work.

[0003] The resolution limit of the microscope imaging system is one of the main reasons for image blur. Especially at high magnification, the tiny and dense details on the surface of the nanofilm are more susceptible to factors such as aberrations, optical distortion and equipment vibration, resulting in loss of image details and blurred edges. In addition, the inherent limitations of the imaging system itself, such as the non-ideal characteristics of the point spread function (PSF), will further aggravate the image blur problem. Secondly, noise is another key factor affecting the quality of nanofilm microscopic images. When imaging under low light conditions, significant noise, ring artifacts and other interferences often appear in the image. These noises not only reduce the signal-to-noise ratio (SNR) of the image, but may also mask or mislead the actual microstructural details in the image. Especially in high-resolution imaging, the destructive effect of noise on details is more significant, seriously affecting the usability of the image and the accuracy of analysis. The inhomogeneity of the light source and changes in the imaging environment can also lead to non-uniform distribution of image brightness and contrast. This inhomogeneity not only makes the details of some areas of the image unclear, but also may cause errors in edge detection and feature extraction, thereby affecting the accuracy of quantitative analysis and subsequent processing steps.

[0004] Traditional image restoration techniques, such as blind deconvolution and image enhancement methods, have been widely used in microscopic image processing. However, these methods are usually based on the assumption of known blur kernels, which makes it difficult to deal with the unpredictable blur kernels caused by complex imaging conditions in in situ microscopic imaging. In addition, although local contrast enhancement techniques (such as histogram equalization and contrast-limited adaptive histogram equalization, CLAHE) can improve the brightness and contrast of images to a certain extent, their noise suppression capabilities are limited when processing images with high noise and complex details, and they may introduce artifacts, resulting in image quality degradation.

[0005] Therefore, the enhancement of nano-film microscopic images needs to be further studied to improve the clarity of nano-film microscopic images and restore the true detail performance of the images. Summary of the Invention

[0006] The problem to be solved by the present invention is to provide a nano-film microscopic image enhancement method and system based on sparse deconvolution. Through a multi-step image processing process, combined with wavelet edge enhancement and sparse blind deconvolution techniques, the clarity and detail performance of nano-film microscopic images are effectively improved, and the efficient restoration and enhancement of nano-film microscopic images are realized.

[0007] The present invention adopts the following technical solutions: A nano-film microscopic image enhancement method based on sparse deconvolution, comprising the following steps:

[0008] S1. Image acquisition and preprocessing: Collect the blurred original image of the nano-film material through an in-situ optical microscope as the input of the in-situ microscopic image, and preprocess the input in-situ microscopic image to obtain a grayscale image;

[0009] S2. Wavelet edge enhancement: Perform wavelet transform on the preprocessed grayscale image to enhance the edges and high-frequency details of the image;

[0010] S3. Blind deconvolution: Through the blind deconvolution algorithm, perform blur kernel estimation and potential clear image restoration on the image after wavelet edge enhancement, construct an optimization objective function, and use an iterative optimization algorithm to alternately update the blur kernel and the potential clear image;

[0011] S4. Non-blind deconvolution: According to whether the potential clear image restored by blind deconvolution is saturated, select a deconvolution method, construct a deconvolution objective function, and perform optimization and normalization of the potential clear image;

[0012] S5. Sparse reconstruction: Based on the sparse optimization theory, perform image reconstruction on the potential clear image optimized by non-blind deconvolution to obtain the restored nano-film microscopic image.

[0013] Preferably, the image acquisition and preprocessing in step S1 includes the following sub-steps:

[0014] S1.1. Image input and channel separation: Use the in-situ optical microscopy platform to collect the microscopic image video of the nano-film material in real time, and segment the video into image files of consecutive frames as the image input;

[0015] The microscopic image video includes RGBA images, RGB images, and grayscale image formats. For RGBA images, perform image separation to separate the image into RGB channels and Alpha channels; for RGB or grayscale images, directly proceed to the next step.

[0016] S1.2. Color space conversion: Convert the input image from the RGB color space to an enhanced color space, and separate the luminance information and color information in the input image;

[0017] S1.3. Contrast enhancement: Use the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm to enhance the local contrast of the image and improve the visibility of image details by restricting the degree of contrast enhancement in histogram equalization;

[0018] Specifically, apply the CLAHE algorithm to the red, green, and blue color channels in the RGB channels respectively. Use the CLAHE function in the image processing library to create a CLAHE object and apply it to each color channel. The CLAHE algorithm avoids the over-enhancement problem caused by global histogram equalization by restricting the increase in contrast, and effectively improves the local contrast of the image.

[0019] S1.4. Grayscale processing: Recombine the RGB channels after CLAHE processing with the original Alpha channel to form an enhanced RGBA image. Subsequently, convert the color image with enhanced contrast to a grayscale image, and retain the structural and texture information of the image;

[0020] S1.5. Noise suppression: Adopt denoising methods, including bilateral filtering and non-local means filtering, to suppress the high-frequency noise in the image and maintain the sharpness of edges and details.

[0021] Preferably, in step S2 wavelet edge enhancement, the edges and high-frequency details of the image are enhanced through multi-scale analysis, which specifically includes the following sub-steps:

[0022] S2.1. Wavelet decomposition: Use wavelet transform technology to perform two-dimensional wavelet decomposition on the preprocessed RGB image, and select an appropriate wavelet basis (such as the 'Haar' wavelet) and the number of decomposition levels (such as 2 levels). Wavelet decomposition decomposes the image into a low-frequency (approximate) sub-band and multiple high-frequency (detail) sub-bands.

[0023] S2.2. High-frequency enhancement: Through dynamic gain adjustment, calculate the average luminance value of the image, and dynamically adjust the gain coefficient of the high-frequency sub-band according to the luminance level.

[0024] Set a luminance threshold T. When the average luminance L of the image is lower than this threshold, use a larger gain coefficient α to significantly enhance the high-frequency details; otherwise, use a smaller gain coefficient β to avoid over-enhancement. The formula is expressed as:

[0025]

[0026] S2.3. Wavelet reconstruction: Recombine the adjusted high-frequency sub-band and low-frequency sub-band, and reconstruct the image with enhanced edges through inverse wavelet transform to improve the image details and sharpness.

[0027] Specifically, the enhanced high-frequency subband after gain adjustment is combined with the original low-frequency subband, and inverse wavelet transform is performed to reconstruct the enhanced RGB image. By cropping the pixel values of the image within the effective range, pixel value overflow is avoided. Finally, the enhanced RGB image is recombined with the original Alpha channel to obtain the edge-enhanced RGBA image.

[0028] Preferably, step S3, blind deconvolution, aims to simultaneously estimate the PSF (Point Spread Function, blur kernel) of the image and restore the clear image. The specific implementation steps are as follows:

[0029] S3.1. Grayscale image conversion and blur kernel generation: Convert the edge-enhanced image into a grayscale image, and generate a blur kernel through a blind deconvolution algorithm to estimate the blur kernel and restore the potential clear image of the wavelet edge-enhanced image;

[0030] Specifically, according to the optical parameters input by the user (such as wavelength λ, numerical aperture NA, and pixel size Pixel), a point spread function (PSF) kernel is generated. The generation of the blur kernel is based on the optical formula:

[0031]

[0032] where J1 is the first kind of first-order Bessel function, and r is the radial distance.

[0033] S3.2. Optimize the objective function: Construct a blind deconvolution objective function that includes a data fitting term and a gradient regularization term. By introducing the image gradient and the sparsity constraint of the image itself, estimate the blur kernel of the image and restore the clear image;

[0034] Specifically, adopt a blind deconvolution algorithm based on maximum a posteriori (MAP) estimation. By iterative optimization, simultaneously restore the image and the blur kernel. The blind deconvolution objective function includes: the fitting error after convolution of the image and the blur kernel, the sparsity constraint of the image gradient, and the sparsity constraint of the image itself, which are expressed as follows:

[0035]

[0036] where I is the restored potential clear image, k is the estimated blur kernel, Y is the preprocessed input image, is the gradient of the image I, λ grad and λ sparsity are the gradient regularization parameter and the sparsity regularization parameter respectively; ‖.‖1 and ‖.‖2 are the norm and norm respectively.

[0037] S3.3. Iterative Optimization: The Alternating Direction Method of Multipliers (ADMM) is used for iterative optimization to gradually update the estimated values of the image and the blur kernel until the preset convergence condition is reached. Each iteration includes two steps: optimizing the image with the blur kernel fixed and optimizing the blur kernel with the image fixed.

[0038] S3.4. Blur Kernel and Image Saving: The blind deconvolution algorithm outputs the estimated blur kernel and the intermediate restored image. The blur kernel is saved as an image file after normalization for subsequent analysis and visualization.

[0039] Preferably, in step S4 of non-blind deconvolution, in the case of a known blur kernel, the noise and ringing artifacts in the image are further removed. The specific steps are as follows:

[0040] S4.1. Selection of Deconvolution Method: Different deconvolution methods are selected according to the saturation degree of the image. For unsaturated images, the default sparse deconvolution method is used; for images with saturation, a specially designed ringing artifact removal algorithm is selected to effectively suppress the artifacts and noise in the image.

[0041] S4.2. Construction of Deconvolution Objective Function: Combining the estimated blur kernel and appropriate regularization parameters, a clear image is restored through iterative optimization. The objective function of deconvolution is defined as:

[0042]

[0043] where, is the gradient of the image, ∥I∥ TV is the total variation regularization term, is the L0 norm of the gradient, λ TV and λ L0 are regularization parameters.

[0044] S4.3. Image Normalization and Format Conversion: The processed image is normalized to ensure that the pixel values are within the valid range and converted into a format suitable for saving and display for subsequent saving and visualization operations.

[0045] Preferably, in step S5 of sparse reconstruction processing, the aforementioned CLAHE algorithm and wavelet edge enhancement algorithm are applied to obtain the preprocessed image. According to the parameters set by the user (such as the number of iterations, fidelity, sparsity, etc.), a sparse reconstruction algorithm is called to estimate the blur kernel and restore the clear image. The sparse deconvolution step utilizes the prior knowledge of the sparsity and continuity of the image to further improve the image restoration quality. The specific steps are as follows:

[0046] S5.1. Sparse Representation: A transform domain is selected to transform the potential clear image I into the sparse representation space Ψ(I), and the sparsity characteristics of the image in the transform domain are utilized to enhance the image structure and texture details;

[0047] Specifically, for sparse prior modeling, it is assumed that the image has a sparse representation in the gradient domain or wavelet domain, that is, most of the gradient or wavelet coefficients are close to zero, and only a few coefficients have significant values. Based on this assumption, the system constructs a sparse prior model and combines it with the continuity prior of the image to form a comprehensive optimization objective function.

[0048] S5.2. Construction of the optimization objective function: The optimization objective function includes a data fidelity term, a sparsity constraint term, and a continuity constraint term, and the expression is as follows:

[0049]

[0050] where A is the system matrix, Y is the input observed image; Ψ(I) is the sparse representation of image I in the selected transform domain, which is used to improve the accuracy and stability of image restoration while maintaining the image structure and texture information; ‖.‖1 and ‖.‖2 are the norm and norm respectively;

[0051] S5.3. Iterative optimization method: An efficient iterative optimization algorithm such as the Alternating Direction Method of Multipliers (ADMM) is used to gradually optimize the estimated values of the image and the blur kernel and minimize the above objective function. The algorithm decomposes the original problem into multiple sub-problems to optimize the image and the blur kernel respectively, ensuring the stability and efficiency of the optimization process.

[0052] S5.4. Implementation of sparse deconvolution: Using a deep learning framework (such as PyTorch) and combining GPU acceleration technology, perform sparse blind deconvolution calculations. And through parallel computing and optimized memory management, the execution efficiency of the algorithm is significantly improved to meet the real-time processing requirements of large-scale image data.

[0053] S5.5. Output of the restored image: The sparse deconvolution algorithm outputs a high-quality restored image, further improving the clarity and detail performance of the image.

[0054] The technical solution of the present invention also provides: A nano-film microscopic image enhancement system based on sparse deconvolution for implementing any of the above nano-film microscopic image enhancement methods, including: an image input and preprocessing module, a wavelet edge enhancement module, a blind deconvolution module, a non-blind deconvolution module, and a sparse reconstruction module;

[0055] The image input and preprocessing module collects a blurred original image through an in-situ optical microscope as an in-situ microscopic image input, preprocesses the input in-situ microscopic image, performs color space conversion, enhances the contrast through the CLAHE algorithm, and performs graying and noise suppression to obtain a gray image;

[0056] Wavelet edge enhancement module, which performs wavelet decomposition and wavelet reconstruction on the preprocessed grayscale image to enhance the edges and high-frequency details of the image;

[0057] Blind deconvolution module, which estimates the blur kernel and restores the latent sharp image for the image after wavelet edge enhancement through the blind deconvolution algorithm, performs data fitting and regularization, and adopts an iterative optimization algorithm to update the blur kernel and the latent sharp image;

[0058] Non-blind deconvolution module, which selects the deconvolution method according to whether the latent sharp image restored by blind deconvolution is saturated, constructs the objective function of deconvolution, and performs optimization and normalization of the latent sharp image;

[0059] Sparse reconstruction module, which performs image reconstruction on the latent image restored by blind deconvolution through sparse optimization transformation, constructs the optimization objective function, and obtains the restored nano-film microscopic image.

[0060] Compared with the prior art, the present invention adopts the above technical solutions and has the following technical effects:

[0061] 1. Multi-dimensional image enhancement and restoration technology: The method of the present invention integrates a variety of advanced image processing technologies such as CLAHE enhancement, blind deconvolution, wavelet transform, and sparse deconvolution to form a comprehensive image restoration framework, which can perform image restoration without a priori blur kernel, effectively remove blur, noise, and improve image details, and is especially suitable for complex and high-noise imaging environments, such as the restoration of nano-film images.

[0062] 2. Efficient image detail restoration and noise suppression: The method of the present invention combines an adaptive optimization algorithm with sparse deconvolution technology, which can accurately improve the resolution and clarity of the image and achieve efficient detail restoration. Especially in nano-film images, it can effectively eliminate the interference caused by blur and noise, restore the original information of fine structures, and thus obtain high-quality imaging results.

[0063] 3. Wide application potential and scalability: The method of the present invention not only performs excellently in the restoration and enhancement of nano-film microscopic images, but also has wide cross-field application potential, and can play an important role in multiple fields such as biomedical microscopic image processing, industrial microscopic inspection, materials science, and electron microscopic images, promoting the wide application of high-precision imaging technology.

[0064] 4. Multi-scale edge enhancement and detail restoration: The method of the present invention combines the multi-scale enhancement of wavelet transform on image edges and the accurate restoration of high-frequency details by sparse deconvolution, which can significantly improve the image resolution, clarity, and detail performance. Through this multi-level processing, the tiny structures and textures of the image are fully restored, providing richer and more accurate visual data support for subsequent analysis and research.

[0065] 5. Excellent robustness and adaptability: The method of the present invention demonstrates high robustness and adaptability in the face of various blurs, noises, and complex backgrounds. Whether it is in low-contrast images, non-uniform illumination environments, or in the case of poor image quality, it can provide stable and efficient restoration results, ensuring the maximization of image quality during the restoration process.

[0066] 6. Enhancement of image analysis and subsequent research capabilities: The method of the present invention significantly improves the image quality, providing high-quality image data for further image segmentation, pattern recognition, quantitative measurement, and scientific research. Whether it is in image quantitative analysis, image registration, target detection, or in more complex deep learning applications, the clear and accurate image data provided by the method of the present invention can provide strong support for subsequent work. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is a block diagram of the nano-film microscopic image enhancement system based on sparse deconvolution of the present invention;

[0068] Figure 2 is a flowchart of the steps of the nano-film microscopic image enhancement method based on sparse deconvolution of the present invention;

[0069] Figure 3 is a comparison diagram of the results before and after the nano-film microscopic image enhancement method based on sparse deconvolution of the present invention;

[0070] Figure 4 is a comparison diagram of spatial resolution in the embodiment of the present invention;

[0071] Figure 5 is the comparison result of spatial resolution using different classical algorithms in the embodiment of the present invention;

[0072] Figure 6 is a comparison diagram of the running score results on two reference image quality evaluation metrics, PSNR and SSIM, in the embodiment of the present invention;

[0073] Figure 7 is an evaluation result diagram of the perceptual loss of the original image and the enhanced image compared to the SEM image and four no-reference quality evaluation methods in the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0074] To make the objectives, technical solutions, and advantages of the present invention clearer, the following further elaborates on the technical solutions of the application in conjunction with the accompanying drawings. The described embodiments are only a part of the embodiments related to the present invention. All non-innovative embodiments of other researchers in this field belong to the protection scope of the present invention. At the same time, for the step numbers in the embodiments of the present invention, they are only set for the convenience of elaboration and explanation, and no limitation is imposed on the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0075] A method and system for enhancing nano-film microscopic images based on sparse deconvolution according to the present invention propose an all-round image restoration and enhancement solution by comprehensively applying various advanced image processing technologies such as CLAHE, blind deconvolution, wavelet transform enhancement, and sparse deconvolution.

[0076] Specifically, first, the local contrast of the image is enhanced by the CLAHE algorithm to improve the details in the dark areas, making each area of the image more uniformly visible; then, an unsupervised blind deconvolution algorithm is used to restore the blurred information in the image without prior knowledge of the blur kernel; next, combined with an advanced denoising algorithm, the noise and artifacts in the image are effectively removed to ensure the clarity of the image; finally, the wavelet transform is used to enhance the multi-scale details of the image edges, and the high-frequency details are further restored by the sparse deconvolution technology, thereby significantly improving the resolution and clarity of the image.

[0077] The application of this multi-step integrated method in nano-film microscopic image processing can effectively eliminate multiple interferences such as blur, noise, and uneven illumination, restore the true details of the image, and particularly show significant advantages in the restoration of edge regions and high-frequency details. This method not only improves the quality of nano-film microscopic images but also provides high-quality image support for the real-time observation and microstructural characterization of nano-materials, promoting the development of nano-scale imaging analysis technology, and has broad application prospects and important scientific research value.

[0078] In an embodiment of the present invention, a system for enhancing nano-film microscopic images based on sparse deconvolution, as Figure 1 shown, includes: an image input and preprocessing module, a wavelet edge enhancement module, a blind deconvolution module, and a sparse reconstruction module;

[0079] The image input and preprocessing module collects a blurred original image through an in-situ optical microscope as the in-situ microscopic image input, preprocesses the input in-situ microscopic image, performs color space conversion, enhances the contrast through the CLAHE algorithm, and performs grayscale conversion and noise suppression to obtain a grayscale image;

[0080] Wavelet edge enhancement module, which performs wavelet decomposition and wavelet reconstruction on the preprocessed grayscale image to enhance the edges and high-frequency details of the image;

[0081] Blind deconvolution module, which estimates the blur kernel and restores the latent sharp image of the image enhanced by wavelet edge through the blind deconvolution algorithm, performs data fitting and regularization, and uses the iterative optimization algorithm to update the blur kernel and the latent sharp image;

[0082] Non-blind deconvolution module, which selects the deconvolution method according to whether the latent sharp image restored by blind deconvolution is saturated, constructs the objective function of deconvolution, and performs optimization and normalization of the latent sharp image;

[0083] Sparse reconstruction module, which performs image reconstruction on the latent image optimized by non-blind deconvolution through sparse optimization transformation, constructs the optimization objective function, and obtains the restored nano-film microscopic image.

[0084] Furthermore, based on the above system, nano-film microscopic image enhancement based on sparse deconvolution is carried out, as Figure 2 shown, including: image acquisition and preprocessing, wavelet edge enhancement, blind deconvolution, non-blind deconvolution (denoising), sparse deconvolution, result enhancement and saving, and the implementation of the user interface (GUI). The specific steps are as follows:

[0085] Step 1: Image acquisition and preprocessing

[0086] In the in-situ optical microscopy video acquisition step, first integrate a microscope in the chemical vapor deposition (CVD) equipment, set appropriate parameters such as magnification, exposure time, and frame rate, and collect the microscopy video during the growth process of the nano-film in real time. Then segment the video into consecutive frame image files as the input for subsequent image processing. By ensuring the stability and high quality of video acquisition, the obtained image sequence can accurately reflect the microscopic structural changes of film growth.

[0087] Subsequently, the system enters the image preprocessing module. The image preprocessing step aims to enhance the local contrast of the image, especially the dark details, and improve the overall image quality.

[0088] First, the system receives the input image, which is in the RGBA four-channel format to ensure that the image contains transparency information. If the input image is in the RGB three-channel or grayscale format, it directly enters the subsequent processing steps. For RGBA images, the system separates them into three color channels of RGB and the Alpha channel.

[0089] Then, the Contrast Limited Adaptive Histogram Equalization (CLAHE) algorithm is used to preprocess the input microscopic image of the nanometer thin film to enhance the contrast of the image, especially the details in the dark areas, while keeping the transparency information of the image intact. The specific steps are as follows:

[0090] S1.1. Image loading and channel separation: Load the image containing four channels of RGBA, and separate the RGB and Alpha channels. S1.2. Color space conversion: Convert the RGB channels to the LAB color space and extract the luminance (L) channel:

[0091] L, A, B = split(LAB)

[0092] S1.3. CLAHE application: Apply the CLAHE algorithm to the luminance channel to avoid noise caused by over-enhancement by restricting contrast enhancement:

[0093] cl = CLAHE(L)

[0094] S1.4. Color space reconstruction and channel merging: Merge the processed luminance channel with the original A and B channels, convert back to the RGB color space, and finally recombine with the Alpha channel to obtain the enhanced RGBA image:

[0095] enhanced_image = dstack((enhanced_RGB, Alpha))

[0096] In this embodiment, the CLAHE algorithm is applied to each of the three RGB channels respectively, with the clipping limit set to clipLimit = 2.0 and the grid size to tileGridSize = (8, 8) to enhance the local contrast, especially the details in the dark areas of the image. The implementation of the CLAHE algorithm uses the cv2.createCLAHE function in the OpenCV library. For grayscale or RGB images, the CLAHE algorithm is directly applied without channel separation. The processed RGB image and the original Alpha channel are recombined to form the enhanced RGBA image. Subsequently, the system converts the enhanced RGB image to a grayscale image, which is completed using the Image.fromarray function in the Pillow library, preparing for the subsequent blind deconvolution step.

[0097] Step Two. Wavelet edge enhancement

[0098] The wavelet edge enhancement step enhances image edges and high-frequency details through multi-scale analysis. First, a two-dimensional wavelet decomposition is performed on the preprocessed RGB image. The 'Haar' wavelet basis is selected, and the decomposition level is set to 2. The wavelet decomposition is implemented using the pywt.wavedec2 function in the PyWavelets library, resulting in a low-frequency (approximate) subband and high-frequency (detail) subbands. Then, the system calculates the average brightness value of the image.

[0099] In this embodiment, if the average brightness is lower than 128, a larger gain coefficient gain_dark = 2.0 is used to amplify the high-frequency subband; otherwise, a smaller gain coefficient gain_bright = 1.2 is used to amplify the high-frequency subband. By adjusting the gain of the high-frequency subband, the system enhances the edges and texture details of the image. The adjusted high-frequency subband is combined with the original low-frequency subband, and the inverse wavelet transform is performed using the pywt.waverec2 function to reconstruct the enhanced RGB image. The reconstructed image undergoes pixel value clipping to ensure that the pixel values are in the range of 0 - 255, which is achieved using the numpy.clip function.

[0100] Finally, the enhanced RGB image is merged with the original Alpha channel again to obtain the edge-enhanced RGBA image.

[0101] Step 3: Blind deconvolution

[0102] The blind deconvolution step aims to simultaneously estimate the image blur kernel (PSF) and restore the clear image. First, the system converts the edge-enhanced RGBA image to a grayscale image, which is completed using the Image.convert('L') function in the Pillow library. Subsequently, a blind deconvolution algorithm based on maximum a posteriori (MAP) estimation is adopted and implemented using a custom blind_deconv function.

[0103] In this embodiment, the parameters for blind deconvolution are set as follows: the feature fidelity regularization parameter lambda_ftr = 3e-4, the dark enhancement regularization parameter lambda_dark = 0, the gradient fidelity regularization parameter lambda_grad = 4e-3, the prescaling factor prescale = 1, the number of iterations xk_iter = 5, the gamma correction parameter gamma_correct = 1.0, the blur kernel threshold k_thresh = 20, and the convolution kernel size kernel_size = 29. The blind deconvolution process estimates the blur kernel and restores the image step by step through iterative optimization, reducing image blur and artifacts. The system uses the time module in Python to record the time required for blind deconvolution processing.

[0104] Finally, the blind deconvolution algorithm outputs the estimated blur kernel and the intermediate restored image. After normalization, the blur kernel is saved as an image file for subsequent analysis and visualization.

[0105] Step 4: Non-blind deconvolution (de-blurring)

[0106] In the non-blind deconvolution step, given the known blur kernel, the noise and ringing artifacts in the image are further removed. First, the system selects different deconvolution methods based on whether the image is saturated.

[0107] In this embodiment, the saturation flag saturation = 0 is set, indicating that the image is not saturated, and the default deconvolution method is used. Specifically, the system calls the custom ringing_artifacts_removal function, combines the estimated blur kernel and the set regularization parameters, and performs non-blind deconvolution processing. The regularization parameters for deconvolution include the total variation regularization parameter lambda_tv = 0.001, the L0 regularization parameter lambda_l0 = 5e-4, and the ringing artifact weight weight_ring = 1.

[0108] Through iterative optimization, combining TV and L0 regularization, the system effectively suppresses the noise and ringing artifacts in the image, further enhancing the clarity and detail performance of the image. The processed image is normalized to ensure that the pixel values are in the range of 0 - 255 and converted to an 8-bit unsigned integer format for subsequent saving and display. The wavelet transform and de-blurring results are as Figure 3 shown.

[0109] Step 5: Sparse reconstruction

[0110] The sparse reconstruction step utilizes the prior knowledge of the sparsity and continuity of the image to further improve the image restoration quality. First, the system assumes that the image has a sparse representation in the gradient domain or wavelet domain, that is, most of the gradient or wavelet coefficients are close to zero, and only a few coefficients have significant values. Based on this assumption, the system constructs a sparse prior model and combines the continuity prior of the image to form a comprehensive optimization objective function.

[0111] In this embodiment, the optimization objective function includes a data fidelity term, a sparsity constraint term, and a continuity constraint term. The parameters for sparse deconvolution are set as follows: the fidelity parameter fidelity = 150, the continuity parameter zconti = 1, the sparsity parameter sparsity = 15, and the number of iterations SHIter = 100. Using the PyTorch framework and combining GPU acceleration technology, the system efficiently performs sparse blind deconvolution calculations to restore a high-quality clear image SHVideo.

[0112] Step 6: Result enhancement

[0113] The result enhancement and saving steps are used to further improve the details and edge sharpness of the restored image and save the processing result to a specified directory.

[0114] First, the system applies wavelet transform to the restored image SHVideo again for edge enhancement, using the same wavelet basis 'Haar' and decomposition level 2 as before. According to the average brightness of the image, the gain coefficient of the high-frequency subband is dynamically adjusted. If the brightness is lower than 128, a larger gain coefficient gain_dark = 2.0 is used to amplify the high-frequency subband; otherwise, a smaller gain coefficient gain_bright = 1.2 is used to amplify the high-frequency subband. The inverse wavelet transform is performed through the pywt.waverec2 function in the PyWavelets library to reconstruct the enhanced RGB image, and it is recombined with the original Alpha channel to obtain the final edge-enhanced image.

[0115] Subsequently, the system normalizes the enhanced image to ensure that the pixel values are in the range of 0 - 255, which is achieved using the numpy.clip function, and converts the image to an 8-bit unsigned integer format, which is completed using the astype('uint8') function. Then, using the Image.save function in the Pillow library, the finally enhanced image is saved to the specified directory results, and the file name contains the original file name and the enhancement identifier (such as _enhanced.tif). At the same time, the estimated blur kernel is normalized, which is achieved using array operations in the numpy library, and saved as a PNG format image (such as _kernel.png) for subsequent analysis and visualization.

[0116] Step 7. Image quality assessment

[0117] The edge spread function (ESF) describes the intensity transition between nanoparticles and the background and is defined as:

[0118] ESF(x) = I(x)

[0119] where I(x) is the intensity measured along the line profile analysis.

[0120] To reduce noise, the ESF is smoothed using a Gaussian filter (sigma = 1 pixel), which is a common method in spatial resolution measurement. To quantitatively analyze the blur degree of the nanoparticle edge, the ESF is further fitted with the cumulative distribution function (CDF) of the Gaussian function, and the model is:

[0121]

[0122] Wherein, A is the intensity contrast between the nanoparticles and the background, x is the edge position, σ is the standard deviation of the Gaussian function, which describes the width of the transition, and erf is the error function, which is related to the Gaussian integral.

[0123] The parameter σ is directly related to the resolution. A smaller value of σ indicates a clearer transition (higher resolution), while a larger value of σ indicates a greater blur (lower resolution). The comparison results are as Figure 4 and Figure 5 shown. It can be seen that the spatial resolution of the SEM (scanning electron microscope) image is the smallest, the processing effect of the method of the present invention is the second, and the original image is the worst, that is, the spatial resolution σ (unit: μm): Orig>AI>SEM. It can be judged from this that the quality of the AI processing result is second only to the SEM result and is much better than the quality of the original image. Figure 5 Shows the comparison of the spatial resolution of the processing results of different classical algorithms.

[0124] Peak Signal-to-Noise Ratio (PSNR) is a commonly used image quality evaluation index, especially in the fields of image compression, reconstruction, denoising, etc. PSNR measures the ratio between the signal intensity (the maximum value of the original image) and the noise intensity (the distortion of the reconstructed or compressed image), and its unit is decibels (dB).

[0125] PSNR is usually calculated based on MSE (Mean Squared Error), and the formula is as follows:

[0126]

[0127] Wherein, MAX I is the maximum possible value of the pixels in the image. For an 8-bit image (8 bits per pixel, that is, the range is 0-255), MAXI = 255. MSE is the mean squared error, which represents the average of the squares of the errors between the reconstructed image and the original image.

[0128] Structural Similarity Index Measure (SSIM for short) is an index used to measure the similarity degree between two images, which pays more attention to the structural information of the images, so it is more in line with human visual perception when evaluating image quality. SSIM evaluates the similarity between two images by considering the brightness, contrast and structure of the images simultaneously.

[0129] Specifically, SSIM divides the image into several small windows, calculates the similarity of the above three aspects within each window, and then performs weighted averaging on the results of all windows to obtain the overall similarity score. The formula is as follows:

[0130] SSIM(x,y) = [l(x,y)] α ·[c(x,y)] β ·[s(x,y)] γ

[0131] where: l(x,y) represents the luminance similarity, c(x,y) represents the contrast similarity, s(x,y) represents the structure similarity, and α, β, γ are parameters for adjusting the weights of each part. The comparison of the running scores of the two reference evaluation metrics, PSNR and SSIM, is as Figure 6 shown.

[0132] To further evaluate the image quality, the present invention uses several common reference-free quality assessment algorithms: niqe, piqe, MetaIQA, and RankIQA (reference-free image quality assessment means directly calculating the visual quality of the distorted image in the absence of a reference image), and a deep learning-based perceptual loss assessment method - LPIPS (Learned Perceptual Image Patch Similarity), which can be used to measure the perceptual similarity between two images and is closer to human subjective perception. This metric standard learns to generate the inverse mapping from the generated image to the ground truth, forcing the generator to learn the inverse mapping for reconstructing the real image from the fake image and preferentially processing the perceptual similarity between them. The lower the value of LPIPS, the smaller the perceptual difference between the two images. The evaluation results are as Figure 7 shown.

[0133] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A nanofilm microscopic image enhancement method based on sparse deconvolution, characterized in that: The steps include: S1. Image acquisition and preprocessing: The blurred original image of the nanofilm material is collected by an in-situ optical microscope as an in-situ microscopic image input, and the input in-situ microscopic image is preprocessed to obtain a grayscale image; S2, wavelet edge enhancement: perform wavelet transform on the preprocessed grayscale image to enhance the image edge and high-frequency details; S3, blind deconvolution: the blur kernel is estimated and the potential clear image is restored for the image after wavelet edge enhancement by blind deconvolution method, the optimization objective function is constructed, and the iterative optimization algorithm is used to alternately update the blur kernel and the potential clear image; S4, non-blind deconvolution: according to whether the potential clear image restored by blind deconvolution is saturated, a deconvolution method is selected, a deconvolution objective function is constructed, and the potential clear image is optimized and normalized; S5. Sparse reconstruction: Based on the sparse optimization theory, the potential clear image after non-blind deconvolution optimization is reconstructed to obtain the restored nanofilm microscopic image.

2. The nanofilm microscopic image enhancement method based on sparse deconvolution according to claim 1, characterized in that: The image acquisition and preprocessing described in step S1 includes the following sub-steps: S1.1, Image input and channel separation: Using the in-situ optical microscope platform, real-time microscopic image video of nanofilm materials is collected, and the video is divided into image files of continuous frames as image input; The microscopic image video includes RGBA image, RGB image and grayscale image formats, and the RGBA image is separated into an RGB channel and an Alpha channel; S1.2, color space conversion: convert the input image from the RGB color space to the enhanced processing color space, and separate the brightness information and color information in the input image; S1.3, Contrast enhancement: The CLAHE algorithm is used to enhance the local contrast of the image by limiting the degree of contrast enhancement in histogram equalization; S1.4, Grayscale processing: convert the contrast-enhanced color image into a grayscale image, and retain the structure and texture information of the image; S1.5, Noise suppression: Denoising methods, including bilateral filtering and non-local mean filtering, are used to suppress high-frequency noise in the image and maintain the clarity of edges and details.

3. The nanofilm microscopic image enhancement method based on sparse deconvolution according to claim 1, characterized in that: The wavelet transform in step S2 includes the following sub-steps: S2.1, wavelet decomposition: multi-scale wavelet transform is used to perform multi-level decomposition of the grayscale image to separate sub-bands of different frequencies, including high-frequency sub-bands and low-frequency sub-bands; S2.2, high frequency enhancement: For high frequency sub-bands, the gain factor of the high frequency coefficient is adaptively adjusted according to the overall brightness and contrast characteristics of the image to enhance the edge and detail information in the image; S2.3, wavelet reconstruction: The adjusted high-frequency sub-band is recombined with the low-frequency sub-band, and the edge-enhanced image is reconstructed through inverse wavelet transform to improve image details and sharpness.

4. The nanofilm microscopic image enhancement method based on sparse deconvolution according to claim 1, characterized in that: The blind deconvolution in step S3 includes the following sub-steps: S3.

1. Grayscale image conversion and blur kernel generation: Convert the edge-enhanced image to a grayscale image, generate blur kernels through blind deconvolution algorithm, and perform blur kernel estimation and potential clear image restoration on the image after wavelet edge enhancement; S3.2, Optimize the objective function: Construct a blind deconvolution objective function including data fitting terms and gradient regularization terms, and estimate the blur kernel of the image and restore the clear image by introducing the image gradient and the sparsity constraints of the image itself; The data fitting term is used to measure the difference between the restored potential clear image and the input image, and the gradient regularization term balances the smoothness and edge information of the image by controlling the gradient change of the image in multiple directions; S3.3, Iterative optimization: Using an iterative optimization algorithm, the blur kernel and the latent image are updated alternately in each iteration until the preset convergence criterion or the maximum number of iterations is reached.

5. The nanofilm microscopic image enhancement method based on sparse deconvolution according to claim 4, characterized in that: The size and shape of the blur kernel in step S3.1 are adaptively set according to the blur degree and expected resolution of the image after wavelet edge enhancement; based on the input optical parameters, including wavelength λ, numerical aperture NA, and pixel size Pixel, a point spread function kernel is generated. The optical formula for generating the blur kernel is expressed as follows: Wherein, J1 is the first-order Bessel function of the first kind, and r is the radial distance.

6. The nanofilm microscopic image enhancement method based on sparse deconvolution according to claim 4, characterized in that: The blind deconvolution objective function in step S3.2 includes: the fitting error after the image and blur kernel are convolved, the sparsity constraint of the image gradient, and the sparsity constraint of the image itself, which is expressed as follows: Where I is the restored potential clear image, k is the estimated blur kernel, and Y is the input preprocessed image. is the gradient of image I, λ grad and λ sparsity are the gradient regularization parameter and sparsity regularization parameter respectively; ‖.‖1 and ‖.‖2 are the l1 norm and l2 norm respectively.

7. The nanofilm microscopic image enhancement method based on sparse deconvolution according to claim 4, characterized in that: The non-blind deconvolution in step S4 includes the following sub-steps: S4.

1. Deconvolution method selection: Deconvolution method is selected according to whether the potential clear image restored by blind deconvolution is saturated. For unsaturated images, sparse deconvolution method is used; for images with saturation, ringing artifact removal algorithm is selected to suppress artifacts and noise in the image. S4.

2. Construct the deconvolution objective function: Combine the estimated blur kernel, set the regularization parameters, and restore the clear image through iterative optimization. The deconvolution objective function is expressed as follows: in, is the gradient of the image, ∥I∥ TV is the total variation regularization term, is the L0 norm of the gradient, λ TV and λ L0 is the regularization parameter; S4.3, Image normalization and format conversion: The processed image is normalized and converted into a preset saving and display format for storage and visualization.

8. The nanofilm microscopic image enhancement method based on sparse deconvolution according to claim 7, characterized in that: The sparse reconstruction in step S5 includes the following sub-steps: S5.1, sparse representation: select a transform domain to transform the latent clear image I into a sparse representation space Ψ(I), and use the sparsity characteristics of the image in the transform domain to enhance the image structure and texture details; S5.

2. Optimization objective function construction: Construct a sparse reconstruction objective function including data fitting terms and sparse regularization terms. S5.3, performing optimization and solution: introducing multi-scale wavelet transform, using sparse optimization algorithm, including: iterative shrinkage threshold algorithm, accelerated iterative shrinkage threshold algorithm, to optimize the restoration result of image I; S5.4, Sparse Deconvolution: Use deep learning frameworks combined with GPU acceleration methods to perform sparse blind deconvolution calculations; S5.

5. Restored image output: output the restored image to improve image clarity and detail.

9. The nanofilm microscopic image enhancement method based on sparse deconvolution according to claim 8, characterized in that: The sparse reconstruction objective function in step S5.2 is expressed as follows: Where A is the system matrix, Y is the input observation image; Ψ(I) is the sparse representation of image I in the selected transform domain, which is used to improve the image restoration accuracy and stability while maintaining the image structure and texture information; ‖.‖1 and ‖.‖2 are the l1 norm and l2 norm, respectively; The system matrix A is constructed by combining the point spread function of the optical system, and the sparse representation Ψ(I) adopts multi-scale wavelet transform or Hessian transform. The Hessian transform is expressed as follows: in, is the fidelity term, which represents the distance between the restored image x and the input image f, b is the background estimated based on the characteristics of the optical system, and A is the PSF of the optical system; R Hessian (x) is the continuity prior, λ L1 ‖x‖1 is the sparsity prior, λ and λ L1 Represents the weight factor that balances image fidelity and sparsity.

10. A nanofilm microscopic image enhancement system based on sparse deconvolution, used to implement the nanofilm microscopic image enhancement method according to any one of claims 1 to 9, characterized in that: include: Image input and preprocessing module, wavelet edge enhancement module, blind deconvolution module, non-blind deconvolution module, sparse reconstruction module; The image input and preprocessing module collects the blurred original image through an in-situ optical microscope as an in-situ microscopic image input, preprocesses the input in-situ microscopic image, performs color space conversion, performs contrast enhancement through a CLAHE algorithm, and performs grayscale conversion and noise suppression to obtain a grayscale image; The wavelet edge enhancement module performs wavelet decomposition and wavelet reconstruction on the preprocessed grayscale image to enhance the edge and high-frequency details of the image; The blind deconvolution module estimates blur kernels and restores potential clear images of the image after wavelet edge enhancement through a blind deconvolution algorithm, performs data fitting and regularization, and uses an iterative optimization algorithm to update blur kernels and potential clear images; The non-blind deconvolution module selects a deconvolution method and constructs a deconvolution objective function to optimize and normalize the potential clear image according to whether the potential clear image restored by blind deconvolution is saturated; The sparse reconstruction module reconstructs the potential image optimized by the non-blind deconvolution module through sparse optimization and conversion, constructs a sparse reconstruction objective function, and obtains a restored nanofilm microscopic image.

Images