A deep learning-based microscopic image real-time processing method

Abstract

The application discloses a kind of based on deep learning's microscopic image real-time processing method, comprising the following steps: step one: acquisition microscopic image frame, generates observation image frame;Step two: establish product space variable set and state cache, constrain point spread function parameter vector feasible region;Step three: based on microscopic system transfer function generates band-limited mask, decomposes recovered image variable;Step four: extract self-evidence point and calculate model mismatch, determine reflection intensity and operator order;Step five: execute Douglas-Rachford split iteration, sequentially complete data consistency, depth priori, feasible region reflection update;Step six: impose band-limited constraint on data consistency reflection update and project support domain outer energy upper bound;Step seven: calculate consistency certificate and determine termination, output recovered image frame and update state cache.The application improves the stability and real-time of microscopic image recovery.

Description

Technical Field

[0001] This invention relates to the field of microscopic image processing technology, and in particular to a real-time microscopic image processing method based on deep learning. Background Technology

[0002] Microscopic imaging is used in life sciences, materials analysis, and semiconductor testing to obtain details of microstructures. However, due to limitations such as optical diffraction limits, imaging system transfer characteristics, sample scattering, and noise superposition, raw microscopic images often exhibit blurring, reduced contrast, and loss of detail. Current techniques for microscopic image restoration typically revolve around deconvolution, inputting observed image frames and assuming the point spread function is known or can be obtained through calibration, then combining regularization constraints to solve for the restored image. To improve speed, engineering implementations often employ fixed-window frame-by-frame processing and parameter reuse, using the previous frame's result as initialization to reduce the number of iterations, and embedding the restoration module into the data link of the microscopic imaging device to meet online processing requirements.

[0003] One approach in existing technologies uses fixed point spread function (PSF) parameters or offline calibrated PSF parameter vectors, combined with classic iterative algorithms for reconstruction. However, when the transfer function of the microscopic system changes with imaging depth, temperature drift, and objective lens switching, the PSF parameter vector deviates from the true degradation. This leads to model mismatch during reconstruction, making it difficult to reduce residuals, resulting in overly smoothed details, enhanced artifacts, or structural streaking. Another approach introduces deep networks as priors to suppress noise and enhance texture. However, there is a lack of stable coordination mechanisms between the deep network output and data consistency constraints. Common practices involve alternating data consistency and prior updates in a fixed order, lacking a mechanism to adjust the update intensity and operator order based on the mismatch state of the observed image frames. This results in inconsistent convergence behavior across different samples and imaging states, leading to oscillations or premature termination during the iteration process.

[0004] Meanwhile, the frequency domain support domain corresponding to the transfer function of the microscopic system determines the bandwidth range of recoverable information. Common practices in existing technologies include adding a frequency domain penalty term to the loss function or implicitly learning bandwidth constraints during network training. However, these methods lack explicit band-limited decomposition and constraint control of the restored image variables during iterative updates. This leads to the continuous amplification of frequency components outside the support domain during iterations, introducing high-frequency noise and ring artifacts, affecting inter-frame consistency. On the other hand, real-time processing scenarios require controllable determination of termination conditions. However, existing technologies often use only a single residual threshold or a fixed number of iterations, making it difficult to simultaneously consider both data consistency residuals and the magnitude of changes in iterative variables. This results in insufficient iteration leading to inadequate restoration, or excessive iteration leading to increased latency and state buffer drift, affecting the stability of continuous processing of microscopic image frame sequences.

[0005] Therefore, how to provide a real-time microscopic image processing method based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a real-time microscopic image processing method based on deep learning. This invention achieves stable restoration and enhancement of microscopic image frame sequences by jointly optimizing the restored image variables and point spread function parameter vectors in the product spatial variable set, introducing a model mismatch-driven adaptive adjustment mechanism for data consistency reflection intensity and reflection operator execution order, and applying explicit band-limited constraints and consistency certificate judgments during the iteration process. It has the advantages of strong model adaptability, stable iterative convergence, high detail preservation capability, and meeting real-time processing requirements.

[0007] A real-time microscopic image processing method based on deep learning according to an embodiment of the present invention includes the following steps:

[0008] Step 1: Obtain the sequence of microscopic image frames and generate observation image frames;

[0009] Step 2: Establish a product space variable set, which includes the restored image variable and the point spread function parameter vector. The point spread function parameter vector satisfies the feasible region of the point spread function parameter vector. Establish a state cache, which includes the initial values ​​of the restored image variable and the initial values ​​of the point spread function parameter vector.

[0010] Step 3: Construct a band-limited mask based on the transfer function of the microscopic system, and decompose the restored image variables into in-support domain components and out-of-support domain components;

[0011] Step 4: Extract the set of evidence points based on the observed image frames, calculate the model mismatch based on the set of evidence points, and determine the data consistency reflection intensity and the execution order of the reflection operators based on the model mismatch.

[0012] Step 5: Perform Douglas-Rachford splitting iteration on the product space variable set, perform data consistency reflection update on the data consistency set according to the execution order of reflection operators, perform depth prior reflection update on the depth prior set, and perform feasible region reflection update on the feasible region of the point spread function parameter vector;

[0013] Step 6: Set a band-bound constraint for the data consistency reflection update. The band-bound constraint limits the data consistency reflection update to update the components within the support domain, and performs an energy upper bound projection on the components outside the support domain.

[0014] Step 7: Calculate the consistency certificate and determine the termination condition, output the restored image variable to obtain the restored image frame, and update the state cache.

[0015] Optionally, step one specifically includes:

[0016] Set the acquisition frame rate parameters, image height parameters, and image width parameters, and start the microscopic imaging equipment to acquire data;

[0017] When acquisition is triggered, a frame of raw microscopic image is read and the channel identifier is read. The raw microscopic image is composed of a pixel matrix determined by the image height parameter and the image width parameter.

[0018] Generate a frame index and update it incrementally according to the acquisition order; generate a timestamp, which is the acquisition clock reading corresponding to the acquisition trigger time.

[0019] The original microscopic image, channel identifier, frame index, and timestamp are written into the observed image frame, and then written into the microscopic image frame sequence in ascending order of frame index.

[0020] Optionally, step two specifically involves:

[0021] Establish a product space variable set, which includes restored image variables and point spread function parameter vectors;

[0022] Set a parameter component index sequence for the point spread function parameter vector, set a lower bound value and an upper bound value for each parameter component in the parameter component index sequence, and construct the feasible region of the point spread function parameter vector based on the lower bound value and the upper bound value of the parameter component;

[0023] Establish a state cache, which contains initial values ​​of the restored image variables and initial values ​​of the point spread function parameter vector;

[0024] Read the initial value of the restored image variable from the state buffer. If the initial value of the restored image variable is missing, assign the pixel matrix contained in the observed image frame as the initial value of the restored image variable.

[0025] Read the initial value of the point spread function parameter vector from the state cache. If the initial value of the point spread function parameter vector is missing, take the midpoint value of the lower bound value and the upper bound value of each parameter component according to the parameter component index sequence to generate the initial value of the point spread function parameter vector.

[0026] The initial value of the point spread function parameter vector is projected onto the feasible region of the point spread function parameter vector to obtain the point spread function parameter vector. The initial value of the restored image variable is assigned to the restored image variable, and the restored image variable and the point spread function parameter vector are written into the product space variable set.

[0027] Optionally, step three specifically includes:

[0028] Read the transfer function of the microscopic system and establish a frequency domain coordinate grid, which includes frequency horizontal axis coordinates and frequency vertical axis coordinates.

[0029] The support domain boundary is determined based on the transfer function of the microscopic system. A band-limited mask is generated on the frequency domain coordinate grid according to the support domain boundary. The band-limited mask is a mask matrix. The mask matrix takes the value of one inside the support domain and the value of zero outside the support domain.

[0030] Perform a Fourier transform on the restored image variables to obtain frequency domain restored image variables, and use a band-limited mask to perform point-by-point multiplication on the frequency domain restored image variables to obtain the frequency domain components in the support domain.

[0031] The frequency domain restored image variables are inverted using a band-limited mask to obtain an inverted mask. The inverted mask is then used to perform point-by-point multiplication on the frequency domain restored image variables to obtain the out-of-domain frequency components.

[0032] Perform an inverse Fourier transform on the frequency domain components within the support domain to obtain the components within the support domain, and perform an inverse Fourier transform on the frequency domain components outside the support domain to obtain the components outside the support domain.

[0033] Optionally, step four specifically involves:

[0034] Gaussian Laplace filtering is performed on the observed image frames to obtain the response map;

[0035] Non-maximum suppression is performed on the response map according to the neighborhood size parameter to obtain a candidate point set. The row coordinates and column coordinates of pixels whose response values ​​are greater than the peak threshold in the candidate point set are written into the evidence point set.

[0036] For each self-evidence point in the set of self-evidence points, a window block is extracted centered on the self-evidence point according to the window size parameter. Two-dimensional Gaussian fitting is performed on the point-like response distribution within the window block to obtain the local point diffusion scale parameter.

[0037] The predicted point diffusion scale parameter is calculated based on the point diffusion function parameter vector. The absolute difference between the local point diffusion scale parameter and the predicted point diffusion scale parameter is calculated to obtain the scale difference value.

[0038] The model mismatch is obtained by averaging the scale differences corresponding to the self-evidence point set.

[0039] Set a lower mismatch threshold, an upper mismatch threshold, a minimum strength value, and a maximum strength value. When the model mismatch is less than or equal to the lower mismatch threshold, the data consistency reflection strength is assigned the minimum strength value. When the model mismatch is greater than or equal to the upper mismatch threshold, the data consistency reflection strength is assigned the maximum strength value. When the model mismatch is greater than the lower mismatch threshold but less than the upper mismatch threshold, the data consistency reflection strength is calculated by performing linear interpolation on the minimum strength value and the maximum strength value according to the ratio of the model mismatch between the lower mismatch threshold and the upper mismatch threshold.

[0040] When the model mismatch is greater than or equal to the upper mismatch threshold, the execution order of the reflection operator is set to feasible region reflection update, data consistency reflection update, and deep prior reflection update. When the model mismatch is less than the upper mismatch threshold, the execution order of the reflection operator is set to data consistency reflection update, deep prior reflection update, and feasible region reflection update.

[0041] Optionally, step five specifically includes:

[0042] The restored image variables and the point spread function parameter vector are concatenated in a fixed order to form a joint variable, which is then used as the iteration variable for the Douglas–Rachford splitting iteration.

[0043] A point spread function is generated based on the point spread function parameter vector. A predicted observation image is calculated based on the restored image variables and the point spread function. A data consistency residual is calculated based on the predicted observation image and the observation image frame.

[0044] Based on the data consistency residuals, the restored image variables in the joint variables are updated by data consistency projection. The step size of the data consistency projection update is determined by the data consistency reflection intensity. Based on the data consistency projection update results, the joint variables are updated by data consistency reflection to obtain the data consistency reflection joint variables.

[0045] The data consistency reflection joint variables are input into the deep prior network to obtain the prior projection image. The prior projection image is written into the joint variables to obtain the prior projection joint variables. Based on the prior projection joint variables, the deep prior reflection update is performed to obtain the deep prior reflection joint variables.

[0046] Project the point spread function parameter vector in the depth prior reflection joint variable onto the feasible region of the point spread function parameter vector to obtain the feasible region projection vector. Write the feasible region projection vector into the joint variable to obtain the feasible region projection joint variable. Perform feasible region reflection update based on the feasible region projection joint variable to obtain the feasible region reflection joint variable.

[0047] According to the execution order of the reflection operator, the data consistency reflection joint variable, the deep prior reflection joint variable, and the feasible region reflection joint variable are concatenated to obtain the composite reflection joint variable. The difference vector between the composite reflection joint variable and the joint variable is calculated to obtain the update increment. The update increment is scaled according to the relaxation coefficient and added to the joint variable to obtain the updated joint variable. The relaxation coefficient is determined by the data consistency reflection intensity mapping.

[0048] The updated joint variables are split to obtain the updated restored image variables and the updated point spread function parameter vector, which are then written into the product space variable set.

[0049] Optionally, step six specifically includes:

[0050] Perform a Fourier transform on the restored image variables obtained from the data consistency reflection update to obtain the frequency domain restored image variables;

[0051] The frequency domain components within the support domain are obtained by performing point-by-point multiplication on the frequency domain restored image variables using a band-limited mask, and the frequency domain components outside the support domain are obtained by inverting the band-limited mask and then performing point-by-point multiplication on the frequency domain restored image variables using the inverted mask.

[0052] Perform an inverse Fourier transform on the frequency domain components within the support domain to obtain the components within the support domain, and perform an inverse Fourier transform on the frequency domain components outside the support domain to obtain the components outside the support domain.

[0053] Calculate the out-of-domain energy value of the out-of-domain component. The out-of-domain energy value is the sum of the squares of the pixel values ​​of the out-of-domain component. Set the upper limit parameter of the out-of-domain energy. When the out-of-domain energy value is greater than the upper limit parameter, the out-of-domain component is scaled by a scaling factor. The scaling factor is the ratio of the square root of the upper limit parameter to the square root of the out-of-domain energy value.

[0054] The in-support domain component is added to the scaled out-support domain component to obtain the band-limited constrained restored image variable, which is then used as the data consistency reflection update output.

[0055] Optionally, step seven specifically includes:

[0056] After each iteration of the Douglas–Rachford splitting iteration is completed, the data consistency residual norm is calculated based on the observed image frame and the predicted observed image. The incremental norm of the restored image variable is calculated based on the restored image variable of the current iteration and the restored image variable of the previous iteration. The incremental norm of the point spread function parameter vector is calculated based on the point spread function parameter vector of the current iteration and the point spread function parameter vector of the previous iteration.

[0057] The data consistency residual norm, the restored image variable increment norm, and the point spread function parameter vector increment norm are written into the certificate record in a fixed field order, and the certificate record serves as the consistency certificate.

[0058] Set residual threshold, image increment threshold, and parameter increment threshold. Based on the consistency certificate, compare the data consistency residual norm with the residual threshold, compare the restored image variable increment norm with the image increment threshold, and compare the point spread function parameter vector increment norm with the parameter increment threshold.

[0059] The termination condition is determined when the data consistency residual norm is less than or equal to the residual threshold, the restored image variable increment norm is less than or equal to the image increment threshold, and the point spread function parameter vector increment norm is less than or equal to the parameter increment threshold.

[0060] When the termination condition is met, the restored image variable is output to obtain the restored image frame. The restored image variable is written to the state buffer and marked as the initial value of the restored image variable. The point spread function parameter vector is written to the state buffer and marked as the initial value of the point spread function parameter vector.

[0061] The beneficial effects of this invention are:

[0062] This invention simultaneously models the restored image variables and the point spread function parameter vector within a unified set of product space variables. It also introduces a model mismatch-driven data consistency mechanism for adjusting reflection intensity and reflection operator execution order within the Douglas-Rachford split-iteration framework. This enables iterative updates to adaptively adjust the update path and magnitude based on the deviation between the observed image frame and the imaging model. Consequently, it maintains stable convergence behavior even under conditions of point spread function variation, noise level fluctuations, or unstable imaging conditions, avoiding the oscillation, oversmoothing, or artifact accumulation problems that easily occur in traditional fixed-parameter or fixed-operator-order methods.

[0063] Meanwhile, this invention explicitly introduces band-limited constraints based on the microscopic system transfer function in the data consistency reflection update, and applies an upper bound projection of energy to the out-of-domain components, effectively suppressing the amplification of unrecoverable frequency components during the iteration process, so that the restoration result achieves a stable balance between detail preservation and noise control; combined with the consistency certificate, the residual change and variable update magnitude are jointly judged, so that the iteration termination condition has interpretability and controllability, ensuring both the sufficiency of restoration and meeting the requirements of real-time processing of microscopic image frame sequences for computational delay and inter-frame consistency. Attached Figure Description

[0064] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0065] Fig. 1 This is a flowchart of a real-time microscopic image processing method based on deep learning proposed in this invention;

[0066] Fig. 2 This is a schematic diagram of the Douglas–Rachford split-iterative update structure of a real-time microscopic image processing method based on deep learning proposed in this invention. Detailed Implementation

[0067] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0068] refer to Figs. 1-2A real-time microscopic image processing method based on deep learning includes the following steps:

[0069] Step 1: Obtain the sequence of microscopic image frames and generate observation image frames;

[0070] Step 2: Establish a product space variable set, which includes the restored image variable and the point spread function parameter vector. The point spread function parameter vector satisfies the feasible region of the point spread function parameter vector. Establish a state cache, which includes the initial values ​​of the restored image variable and the initial values ​​of the point spread function parameter vector.

[0071] Step 3: Construct a band-limited mask based on the transfer function of the microscopic system, and decompose the restored image variables into in-support domain components and out-of-support domain components;

[0072] Step 4: Extract the set of evidence points based on the observed image frames, calculate the model mismatch based on the set of evidence points, and determine the data consistency reflection intensity and the execution order of the reflection operators based on the model mismatch.

[0073] Step 5: Perform Douglas-Rachford splitting iteration on the product space variable set, perform data consistency reflection update on the data consistency set according to the execution order of reflection operators, perform depth prior reflection update on the depth prior set, and perform feasible region reflection update on the feasible region of the point spread function parameter vector;

[0074] Step 6: Set a band-bound constraint for the data consistency reflection update. The band-bound constraint limits the data consistency reflection update to update the components within the support domain, and performs an energy upper bound projection on the components outside the support domain.

[0075] Step 7: Calculate the consistency certificate and determine the termination condition, output the restored image variable to obtain the restored image frame, and update the state cache.

[0076] In this embodiment, step one specifically includes:

[0077] Set the acquisition frame rate parameter, image height parameter, and image width parameter. The acquisition frame rate parameter is used to determine the acquisition trigger period, the image height parameter is used to determine the number of rows in the pixel matrix, and the image width parameter is used to determine the number of columns in the pixel matrix. After completing the parameter writing, start the microscopic imaging device to acquire data.

[0078] When acquisition is triggered, a frame of original microscopic image is read and the channel identifier is read. The original microscopic image is output in the form of a pixel matrix. The number of rows of the pixel matrix is ​​consistent with the image height parameter, and the number of columns of the pixel matrix is ​​consistent with the image width parameter. The channel identifier is output by the microscopic imaging device with each frame and is associated with the original microscopic image.

[0079] A frame index is generated and updated incrementally according to the acquisition order. The frame index is generated by a frame counter, which is incremented by one each time acquisition is triggered. A timestamp is generated, which is taken from the acquisition clock reading corresponding to the acquisition trigger time and associated with the frame index.

[0080] The original microscopic image, channel identifier, frame index, and timestamp are written into the observed image frame. The observed image frame is written into the microscopic image frame sequence as a recording unit. The microscopic image frame sequence stores the observed image frames in ascending order of the frame index.

[0081] In this embodiment, step two specifically includes:

[0082] A product space variable set is established to store the restored image variables and the point spread function parameter vector. The restored image variables are represented by a pixel matrix of the same size as the observed image frame, and the point spread function parameter vector is represented by a numerical sequence arranged according to the parameter components.

[0083] A parameter component index sequence is set for the point spread function parameter vector. The parameter component index sequence is used to fix the arrangement order of the parameter components. For each parameter component in the parameter component index sequence, a lower bound value and an upper bound value are set for the parameter component. The lower bound value and the upper bound value of the parameter component are written into the parameter boundary table. The feasible region of the point spread function parameter vector is constructed based on the parameter boundary table. The feasible region of the point spread function parameter vector is composed of the parameter value range defined by the parameter boundary table.

[0084] Establish a state cache, which is used to store the initial values ​​of the restored image variables and the initial values ​​of the point spread function parameter vector. Set a cache key for the state cache, which includes the key of the initial value of the restored image variables and the key of the initial value of the point spread function parameter vector.

[0085] The initial value of the restored image variable is read from the state cache. When the initial value of the restored image variable is missing, the pixel matrix is ​​read from the observed image frame and the pixel matrix is ​​assigned as the initial value of the restored image variable. The initial value of the restored image variable is written to the initial value key of the restored image variable in the state cache.

[0086] The initial value of the point spread function parameter vector is read from the state cache. When the initial value of the point spread function parameter vector is missing, the parameter boundary table is read. The lower boundary value and upper boundary value of each parameter component are taken in sequence according to the parameter component index sequence and the midpoint value is calculated. Each midpoint value is arranged in the parameter component index sequence to generate the initial value of the point spread function parameter vector. The initial value of the point spread function parameter vector is written to the initial value key of the point spread function parameter vector in the state cache.

[0087] The initial values ​​of the point spread function parameter vector are projected onto the feasible region of the point spread function parameter vector to obtain the point spread function parameter vector. During the projection process, boundary clipping is performed on each parameter component in the parameter component index sequence. Boundary clipping restricts the parameter component values ​​to the interval determined by the lower and upper bound values ​​of the parameter components. After the projection is completed, the point spread function parameter vector is obtained. The initial values ​​of the restored image variables are assigned to the restored image variables, and the restored image variables and the point spread function parameter vector are written into the product space variable set.

[0088] In this embodiment, step three specifically includes:

[0089] Read the transfer function of the microscopic system. The transfer function of the microscopic system is stored in the form of a frequency domain discrete array. The row and column indices of the frequency domain discrete array correspond to the positions of the frequency sampling points. Establish a frequency domain coordinate grid. The frequency domain coordinate grid consists of a frequency horizontal axis coordinate sequence and a frequency vertical axis coordinate sequence. The frequency horizontal axis coordinate sequence and the frequency vertical axis coordinate sequence correspond one-to-one with the row and column indices of the frequency domain discrete array.

[0090] The support domain boundary is determined based on the transfer function of the microscopic system. The support domain boundary is determined by the non-zero amplitude region of the transfer function of the microscopic system. A band-limited mask is generated on the frequency domain coordinate grid based on the support domain boundary. The band-limited mask is a mask matrix with the same size as the frequency domain discrete array. The sampling point position of the mask matrix inside the support domain takes a value of one, and the sampling point position of the mask matrix outside the support domain takes a value of zero.

[0091] The restored image variables are subjected to Fourier transform to obtain frequency domain restored image variables. The frequency domain restored image variables are complex spectrum matrices corresponding to the restored image variables. A band-limited mask is used to perform point-by-point multiplication on the frequency domain restored image variables. The point-by-point multiplication multiplies the complex value of the frequency domain restored image variables with the value of the mask matrix at each frequency sampling point to obtain the frequency domain components in the support domain. The frequency domain components in the support domain are zero at positions outside the support domain.

[0092] The frequency domain restored image variables are inverted using a band-limited mask to obtain an inverted mask. The inverted mask has a value of zero in the support domain and a value of one outside the support domain. The frequency domain restored image variables are multiplied point by point using the inverted mask to obtain the frequency domain components outside the support domain. The frequency domain components outside the support domain have a value of zero in the support domain.

[0093] Perform an inverse Fourier transform on the frequency domain components within the support domain to obtain the support domain components, which are the spatial domain pixel matrices corresponding to the support domain frequency domain components. Perform an inverse Fourier transform on the frequency domain components outside the support domain to obtain the support domain components, which are the spatial domain pixel matrices corresponding to the support domain frequency domain components.

[0094] In this embodiment, step four specifically includes:

[0095] Gaussian Laplacian filtering is performed based on the observed image frames. The pixel matrix of the observed image frames is first convolved and smoothed with a Gaussian kernel, and then the second difference is calculated on the convolution result using the Laplacian operator. The second difference result is used as the response map, and each pixel position in the response map corresponds to a response value.

[0096] Non-maximum suppression is performed in the response map according to the neighborhood size parameter. The neighborhood size parameter determines the comparison neighborhood range of the pixel position. For each pixel position, the response value in the comparison neighborhood is read and it is determined whether the response value of the pixel position is the maximum value in the comparison neighborhood. The maximum value position is written into the candidate point set. After the traversal is completed, the candidate point set is obtained. The row coordinates and column coordinates of the pixels in the candidate point set whose response values ​​are greater than the peak threshold are written into the evidence point set.

[0097] For each self-evidence point in the self-evidence point set, a window block is extracted centered on the self-evidence point according to the window size parameter. The window size parameter determines the number of rows and columns of the window block. The window block is composed of a pixel submatrix of the response map within the window range. A two-dimensional Gaussian fitting is performed on the point-like response distribution within the window block. The two-dimensional Gaussian fitting uses the least squares criterion to estimate the scale parameter of the two-dimensional Gaussian function. The scale parameter of the two-dimensional Gaussian function is used as the local point diffusion scale parameter.

[0098] The predicted point diffusion scale parameter is calculated based on the point diffusion function parameter vector. The predicted point diffusion scale parameter is obtained by mapping the point diffusion function parameter vector. The mapping rule is to select the parameter component corresponding to the scale according to the parameter component index sequence and perform scale transformation. The scale difference is obtained by calculating the absolute difference between the local point diffusion scale parameter and the predicted point diffusion scale parameter. The scale difference corresponds one-to-one with the self-evidence points in the self-evidence point set.

[0099] The mean of the scale difference corresponding to the self-evidence point set is calculated. The mean is obtained by summing the scale differences and dividing by the number of self-evidence points in the self-evidence point set. The mean is used as the model mismatch.

[0100] Set a lower mismatch threshold, an upper mismatch threshold, a minimum intensity value, and a maximum intensity value. When the model mismatch is less than or equal to the lower mismatch threshold, the data consistency reflection intensity is taken as the minimum intensity value. When the model mismatch is greater than or equal to the upper mismatch threshold, the data consistency reflection intensity is taken as the maximum intensity value. When the model mismatch is greater than the lower mismatch threshold but less than the upper mismatch threshold, calculate the scaling factor. The scaling factor is the ratio of the offset of the model mismatch relative to the lower mismatch threshold to the span of the upper mismatch threshold relative to the lower mismatch threshold. The data consistency reflection intensity is obtained by adding the scaling factor to the minimum intensity value and multiplying the difference between the minimum intensity value and the maximum intensity value.

[0101] When the model mismatch is greater than or equal to the upper mismatch threshold, the execution order of the reflection operator is set to feasible region reflection update, data consistency reflection update, and deep prior reflection update. When the model mismatch is less than the upper mismatch threshold, the execution order of the reflection operator is set to data consistency reflection update, deep prior reflection update, and feasible region reflection update. The execution order of the reflection operator is written into the iteration control parameters and called by the Douglas-Rachford split iteration.

[0102] In this embodiment, step five specifically includes:

[0103] The restored image variables and the point spread function parameter vector are concatenated in a fixed order to form a joint variable. The fixed order consists of the restored image variables first and the point spread function parameter vector last. The joint variable is used as the iteration variable of the Douglas-Rachford split iteration and written into the iteration variable cache.

[0104] A point spread function is generated based on the point spread function parameter vector. The point spread function is obtained by mapping the parameter components of the point spread function parameter vector. A predicted observation image is calculated based on the restored image variables and the point spread function. The predicted observation image is obtained by performing a convolution operation on the restored image variables and the point spread function. A data consistency residual is calculated based on the predicted observation image and the observed image frame. The data consistency residual is composed of the pixel-by-pixel difference between the predicted observation image and the observed image frame, and the residual norm is calculated as the residual scalar.

[0105] Based on the data consistency residual, the restored image variables in the joint variables are updated by data consistency projection. The data consistency projection update is achieved by applying a correction along the residual gradient direction to the restored image variables. The correction magnitude is determined by the data consistency reflection intensity. Based on the data consistency projection update result, the joint variables are updated by data consistency reflection. The data consistency reflection update mirrors the data consistency projection update result relative to the original restored image variables while keeping the point spread function parameter vector unchanged, thus obtaining the data consistency reflection joint variables.

[0106] The data consistency reflection joint variables are input into the deep prior network to obtain the prior projection image. The deep prior network takes the restored image variables corresponding to the data consistency reflection joint variables as input and outputs the prior projection image. The prior projection image is written into the joint variables to form the prior projection joint variables. Based on the prior projection joint variables, the deep prior reflection update is performed. The deep prior reflection update performs a mirror transformation of the prior projection image relative to the restored image variables of the data consistency reflection joint variables while keeping the point spread function parameter vector unchanged, thus obtaining the deep prior reflection joint variables.

[0107] Projecting the point spread function parameter vector in the depth prior reflection joint variable onto the feasible region of the point spread function parameter vector yields the feasible region projection vector. During the projection process, boundary clipping is performed on each parameter component in the parameter component index sequence. Boundary clipping restricts the parameter component values ​​to the interval determined by the lower and upper bounds of the parameter components. The feasible region projection vector is written into the joint variable to form the feasible region projection joint variable. Based on the feasible region projection joint variable, feasible region reflection update is performed. The feasible region reflection update mirrors the feasible region projection vector relative to the point spread function parameter vector of the depth prior reflection joint variable while keeping the restored image variables unchanged, thus obtaining the feasible region reflection joint variable.

[0108] According to the execution order of the reflection operators, the data consistency reflection joint variable, the depth prior reflection joint variable, and the feasible region reflection joint variable are subjected to concatenated reflection to obtain the composite reflection joint variable. The concatenated reflection takes the previous reflection joint variable as the input of the next reflection update and outputs the next reflection joint variable in the order of the reflection operators. The difference vector between the composite reflection joint variable and the joint variable is calculated to obtain the update increment. The difference vector is obtained by subtracting the corresponding components of the joint variable one by one. The update increment is scaled according to the relaxation coefficient and added to the joint variable to obtain the updated joint variable. The relaxation coefficient is determined by the data consistency reflection intensity mapping and written into the iteration control parameters.

[0109] The updated joint variables are split in a fixed order to obtain the updated restored image variables and the updated point spread function parameter vector. The updated restored image variables and the updated point spread function parameter vector are written into the product space variable set and the iteration variable cache is updated.

[0110] This invention introduces a joint variable representation of the restored image variables and the point spread function parameter vector in the Douglas–Rachford split iteration and sets up an iteration variable cache to achieve cross-iteration state reuse. It introduces a model mismatch-driven adaptive configuration of data consistency reflection intensity and reflection operator execution order, allowing the concatenation path of data consistency reflection update and feasible region reflection update to change with mismatch. It introduces data consistency projection update and mirror reflection update, with the step size and relaxation coefficient controlled by the data consistency reflection intensity. It introduces a deep prior network to output a prior projection image and constructs a depth prior reflection update using the prior projection image. It introduces feasible region projection and feasible region reflection update of the point spread function parameter vector to achieve constrained self-correction of the point spread function parameter vector. It introduces concatenated reflection to generate a composite reflection joint variable and completes the iterative update with the update increment, simultaneously converging the point spread function parameter vector and reducing the burden of manual parameter tuning. Thus, under point spread function mismatch and noise disturbance conditions, it reduces iterative oscillation and improves convergence stability. Under the same computational budget, it reduces the number of iterations and improves the detail preservation and artifact suppression capabilities of the restored image frame, meeting the requirements of real-time microscopic image processing for latency and consistency.

[0111] In this embodiment, step six specifically includes:

[0112] A Fourier transform is performed on the restored image variables obtained by the data consistency reflection update. The Fourier transform takes the pixel matrix of the restored image variables as input and outputs a complex spectrum matrix, which serves as the frequency domain restored image variable.

[0113] A band-limited mask is used to perform point-by-point multiplication on the frequency domain restored image variables to obtain the frequency domain components within the support domain. For each frequency sampling point, the complex value of the frequency domain restored image variable is multiplied by the value of the band-limited mask. Frequency sampling points with a band-limited mask value of one retain the original complex value, while frequency sampling points with a band-limited mask value of zero are set to zero. The band-limited mask is then inverted to obtain an inverted mask. The inverted mask has a value of zero within the support domain and a value of one outside the support domain. The inverted mask is then used to perform point-by-point multiplication on the frequency domain restored image variables to obtain the frequency domain components outside the support domain.

[0114] Perform an inverse Fourier transform on the frequency domain components within the support domain to obtain the support domain components. The support domain components are the spatial domain pixel matrices corresponding to the support domain frequency domain components. Perform an inverse Fourier transform on the frequency domain components outside the support domain to obtain the support domain components outside the support domain. The support domain components are the spatial domain pixel matrices corresponding to the support domain frequency domain components outside the support domain.

[0115] The out-of-domain energy value of the out-of-domain component is calculated by traversing each pixel position of the out-of-domain component and summing the squared pixel values. An upper bound parameter for the out-of-domain energy is set, which is a preset energy limit. When the out-of-domain energy value is greater than the upper bound parameter, a scaling factor is calculated. The scaling factor is obtained by dividing the square root of the upper bound parameter by the square root of the out-of-domain energy value. The scaling factor is used to perform multiplicative scaling on each pixel value of the out-of-domain component to obtain the scaled out-of-domain component. When the out-of-domain energy value is less than or equal to the upper bound parameter, the out-of-domain component is used as the scaled out-of-domain component.

[0116] The in-support-domain components are added pixel by pixel to the scaled out-support-domain components to obtain the band-limited restored image variable. The band-limited restored image variable is written into the data consistency reflection update output and returned to the Douglas–Rachford split iteration update process.

[0117] In this embodiment, step seven specifically includes:

[0118] After each iteration of the Douglas–Rachford splitting iteration is completed, the data consistency residual norm is calculated based on the observed image frame and the predicted observed image. The data consistency residual is composed of the pixel-by-pixel difference between the predicted observed image and the observed image frame. The data consistency residual norm is obtained by summing the squares of the pixel-by-pixel differences and taking the square root. The restoration image variable increment norm is calculated based on the restored image variable of the current iteration and the restored image variable of the previous iteration. The restoration image variable increment is composed of the pixel-by-pixel difference between the restored image variables of the two iterations and is obtained by summing the squares and taking the square root. The point spread function parameter vector increment norm is calculated based on the point spread function parameter vector of the current iteration and the point spread function parameter vector of the previous iteration. The point spread function parameter vector increment is composed of the component-by-component difference between the point spread function parameter vectors of the two iterations and is obtained by summing the squares and taking the square root.

[0119] The data consistency residual norm, the restored image variable increment norm, and the point spread function parameter vector increment norm are written into the certificate record in a fixed field order. The fixed field order consists of the data consistency residual norm field, the restored image variable increment norm field, and the point spread function parameter vector increment norm field. The certificate record is stored as a consistency certificate in the consistency certificate cache and associated with the iteration count.

[0120] Set residual threshold, image increment threshold, and parameter increment threshold. Based on the consistency certificate, compare the data consistency residual norm with the residual threshold, compare the restored image variable increment norm with the image increment threshold, and compare the point spread function parameter vector increment norm with the parameter increment threshold. The comparison results are output as residual judgment flag, image increment judgment flag, and parameter increment judgment flag, respectively.

[0121] The termination condition is determined to be met when the data consistency residual norm is less than or equal to the residual threshold, the restored image variable increment norm is less than or equal to the image increment threshold, and the point spread function parameter vector increment norm is less than or equal to the parameter increment threshold. When the termination condition is met, the certificate record in the consistency certificate cache is marked as a termination certificate record.

[0122] When the termination condition is met, the restored image variable is output to obtain the restored image frame. The restored image variable is written to the state buffer and marked as the initial value of the restored image variable. The point spread function parameter vector is written to the state buffer and marked as the initial value of the point spread function parameter vector.

[0123] Example 1:

[0124] To verify the feasibility of this invention in practice, it was applied to a live-cell fluorescence microscopy continuous imaging scenario. During the acquisition process, the imaging object exhibits slight drift and brightness fluctuations. The transfer function of the microscopy system is affected by objective lens switching and temperature drift, resulting in bandwidth contraction. The shape of the point spread function changes with each frame, causing diffusion blurring, blurred cell boundaries, and weak textures being submerged by noise in the original microscopic image. When using a fixed point spread function for deconvolution, ring artifacts are easily generated. When using simple depth prior enhancement, details are easily "fabricated" and inter-frame flicker is easily caused. The real-time processing stability of continuous frame sequences is difficult to guarantee.

[0125] During scene access, the microscopic imaging device outputs a sequence of microscopic image frames according to the acquisition frame rate. Each frame generates an observation image frame carrying a channel identifier, frame index, and timestamp. The product spatial variable set maintains the restored image variables and point spread function parameter vectors, while the state cache provides initial values ​​for the restored image variables and point spread function parameter vectors for warm-up. On the frequency domain side, a band-limited mask is generated based on the microscopic system transfer function, decomposing the restored image variables into in-support domain and out-of-support domain components. The observation image frames are filtered by Laplace-Gaussian to obtain a response map. The response map is then subjected to non-maximum suppression to form a set of self-evidence points. A two-dimensional Gaussian fitting is performed on the self-evidence point window block to obtain local point spread scale parameters. These parameters are then compared with the predicted point spread scale parameters calculated from the point spread function parameter vector to form a scale difference, which is summarized as the model mismatch. The model mismatch is mapped to obtain the data consistency reflection intensity and to determine the execution order of the reflection operators. Subsequently, Douglas-Rachford splitting iterations are performed on the product space variable set. Data consistency reflection updates are combined with data consistency reflection intensity adjustment step size, and depth prior reflection updates are constructed from the prior projection image output by the depth prior network. Feasible region reflection updates project the point spread function parameter vector onto the feasible region of the point spread function parameter vector. Band-limited constraints are applied at the data consistency reflection update exit point, supporting the retention of in-domain components and performing energy upper bound projection on out-of-domain components to suppress the amplification of unrecoverable frequency components. During each frame iteration, a consistency certificate is calculated. The certificate record includes the data consistency residual norm, the restored image variable increment norm, and the point spread function parameter vector increment norm. When the threshold condition is met, the process terminates and the restored image frame is output, while the state buffer is updated for the initialization of the next frame.

[0126] To quantify the beneficial effects, a series of consecutive frames were selected for comparative experiments. Baseline 1 used classical deconvolution iteration with fixed-point spread function parameter vectors, while baseline 2 used single-inference enhancement with a deep prior network. The method of this invention is the scheme defined in the claims. Evaluation metrics include single-frame end-to-end latency, average number of iterations, data consistency residual norm, and the ratio of inter-frame flicker amplitude to out-of-support energy. The inter-frame flicker amplitude is the absolute value of the mean difference between the restored image variables of adjacent frames, and the out-of-support energy ratio is the out-of-support energy value divided by the in-support energy value, as shown in Table 1.

[0127] Table 1. Comparison Test Results of Real-Time Microscopic Image Processing Methods

[0128] Method Single frame end-to-end latency (ms) Average number of iterations (iterations / frame) Data consistency residual norm (normalized) Inter-frame flicker amplitude (normalized) Support domain out-of-energy ratio Fixed point spread function deconvolution 42.8 18.6 0.087 0.061 0.214 Deep prior network single pass enhancement 16.3 1.0 0.132 0.074 0.198 The method of the invention 21.7 7.4 0.052 0.029 0.083

[0129] The comparative results show that, under the model mismatch scenario caused by the change of the point spread function, the data consistency residual norm is significantly reduced, the inter-frame flicker amplitude is reduced, and the out-of-domain energy ratio is effectively suppressed, indicating that the band-limited constraint and the upper energy bound projection suppress the accumulation of high-frequency artifacts in the iterative process. At the same time, the average number of iterations is significantly reduced and the latency is kept within the range available for real-time processing. The state buffer and the adaptive reflection intensity mechanism stabilize the convergence process, and the frame boundaries of the restored images of continuous frame sequences are clearer and the weak textures are more coherent, meeting the requirements of stability and consistency for real-time processing of microscopic images.

[0130] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A real-time microscopic image processing method based on deep learning, characterized in that, Includes the following steps: Step 1: Obtain the sequence of microscopic image frames and generate observation image frames; Step 2: Establish a product space variable set, which includes the restored image variable and the point spread function parameter vector. The point spread function parameter vector satisfies the feasible region of the point spread function parameter vector. Establish a state cache, which includes the initial values ​​of the restored image variable and the initial values ​​of the point spread function parameter vector. Step 3: Construct a band-limited mask based on the transfer function of the microscopic system, and decompose the restored image variables into in-support domain components and out-of-support domain components; Step 4: Extract the set of evidence points based on the observed image frames, calculate the model mismatch based on the set of evidence points, and determine the data consistency reflection intensity and the execution order of the reflection operators based on the model mismatch. Step 5: Perform Douglas-Rachford splitting iteration on the product space variable set, perform data consistency reflection update on the data consistency set according to the execution order of reflection operators, perform depth prior reflection update on the depth prior set, and perform feasible region reflection update on the feasible region of the point spread function parameter vector; Step 6: Set a band-bound constraint for the data consistency reflection update. The band-bound constraint limits the data consistency reflection update to update the components within the support domain, and performs an energy upper bound projection on the components outside the support domain. Step 7: Calculate the consistency certificate and determine the termination condition, output the restored image variable to obtain the restored image frame, and update the state cache.

2. The real-time microscopic image processing method based on deep learning according to claim 1, characterized in that, Step one specifically involves: Set the acquisition frame rate parameters, image height parameters, and image width parameters, and start the microscopic imaging equipment to acquire data; When acquisition is triggered, a frame of raw microscopic image is read and the channel identifier is read. The raw microscopic image is composed of a pixel matrix determined by the image height parameter and the image width parameter. Generate a frame index and update it incrementally according to the acquisition order; generate a timestamp, which is the acquisition clock reading corresponding to the acquisition trigger time. The original microscopic image, channel identifier, frame index, and timestamp are written into the observed image frame, and then written into the microscopic image frame sequence in ascending order of frame index.

3. The real-time microscopic image processing method based on deep learning according to claim 1, characterized in that, Step two specifically involves: Establish a product space variable set, which includes restored image variables and point spread function parameter vectors; Set a parameter component index sequence for the point spread function parameter vector, set a lower bound value and an upper bound value for each parameter component in the parameter component index sequence, and construct the feasible region of the point spread function parameter vector based on the lower bound value and the upper bound value of the parameter component; Establish a state cache, which contains initial values ​​of the restored image variables and initial values ​​of the point spread function parameter vector; Read the initial value of the restored image variable from the state buffer. If the initial value of the restored image variable is missing, assign the pixel matrix contained in the observed image frame as the initial value of the restored image variable. Read the initial value of the point spread function parameter vector from the state cache. If the initial value of the point spread function parameter vector is missing, take the midpoint value of the lower bound value and the upper bound value of each parameter component according to the parameter component index sequence to generate the initial value of the point spread function parameter vector. The initial value of the point spread function parameter vector is projected onto the feasible region of the point spread function parameter vector to obtain the point spread function parameter vector. The initial value of the restored image variable is assigned to the restored image variable, and the restored image variable and the point spread function parameter vector are written into the product space variable set.

4. The real-time microscopic image processing method based on deep learning according to claim 1, characterized in that, Step three specifically involves: Read the transfer function of the microscopic system and establish a frequency domain coordinate grid, which includes frequency horizontal axis coordinates and frequency vertical axis coordinates. The support domain boundary is determined based on the transfer function of the microscopic system. A band-limited mask is generated on the frequency domain coordinate grid according to the support domain boundary. The band-limited mask is a mask matrix. The mask matrix takes the value of one inside the support domain and the value of zero outside the support domain. Perform a Fourier transform on the restored image variables to obtain frequency domain restored image variables, and use a band-limited mask to perform point-by-point multiplication on the frequency domain restored image variables to obtain the frequency domain components in the support domain. The frequency domain restored image variables are inverted using a band-limited mask to obtain an inverted mask. The inverted mask is then used to perform point-by-point multiplication on the frequency domain restored image variables to obtain the out-of-domain frequency components. Perform an inverse Fourier transform on the frequency domain components within the support domain to obtain the components within the support domain, and perform an inverse Fourier transform on the frequency domain components outside the support domain to obtain the components outside the support domain.

5. The real-time microscopic image processing method based on deep learning according to claim 1, characterized in that, Step four specifically involves: Gaussian Laplace filtering is performed on the observed image frames to obtain the response map; Non-maximum suppression is performed on the response map according to the neighborhood size parameter to obtain a candidate point set. The row coordinates and column coordinates of pixels whose response values ​​are greater than the peak threshold in the candidate point set are written into the evidence point set. For each self-evidence point in the set of self-evidence points, a window block is extracted centered on the self-evidence point according to the window size parameter. Two-dimensional Gaussian fitting is performed on the point-like response distribution within the window block to obtain the local point diffusion scale parameter. The predicted point diffusion scale parameter is calculated based on the point diffusion function parameter vector. The absolute difference between the local point diffusion scale parameter and the predicted point diffusion scale parameter is calculated to obtain the scale difference value. The model mismatch is obtained by averaging the scale differences corresponding to the self-evidence point set. Set a lower mismatch threshold, an upper mismatch threshold, a minimum strength value, and a maximum strength value. When the model mismatch is less than or equal to the lower mismatch threshold, the data consistency reflection strength is assigned the minimum strength value. When the model mismatch is greater than or equal to the upper mismatch threshold, the data consistency reflection strength is assigned the maximum strength value. When the model mismatch is greater than the lower mismatch threshold but less than the upper mismatch threshold, the data consistency reflection strength is calculated by performing linear interpolation on the minimum strength value and the maximum strength value according to the ratio of the model mismatch between the lower mismatch threshold and the upper mismatch threshold. When the model mismatch is greater than or equal to the upper mismatch threshold, the execution order of the reflection operator is set to feasible region reflection update, data consistency reflection update, and deep prior reflection update. When the model mismatch is less than the upper mismatch threshold, the execution order of the reflection operator is set to data consistency reflection update, deep prior reflection update, and feasible region reflection update.

6. The real-time microscopic image processing method based on deep learning according to claim 1, characterized in that, Step five specifically involves: The restored image variables and the point spread function parameter vector are concatenated in a fixed order to form a joint variable, which is then used as the iteration variable for the Douglas–Rachford splitting iteration. A point spread function is generated based on the point spread function parameter vector. A predicted observation image is calculated based on the restored image variables and the point spread function. A data consistency residual is calculated based on the predicted observation image and the observation image frame. Based on the data consistency residuals, the restored image variables in the joint variables are updated by data consistency projection. The step size of the data consistency projection update is determined by the data consistency reflection intensity. Based on the data consistency projection update results, the joint variables are updated by data consistency reflection to obtain the data consistency reflection joint variables. The data consistency reflection joint variables are input into the deep prior network to obtain the prior projection image. The prior projection image is written into the joint variables to obtain the prior projection joint variables. Based on the prior projection joint variables, the deep prior reflection update is performed to obtain the deep prior reflection joint variables. Project the point spread function parameter vector in the depth prior reflection joint variable onto the feasible region of the point spread function parameter vector to obtain the feasible region projection vector. Write the feasible region projection vector into the joint variable to obtain the feasible region projection joint variable. Perform feasible region reflection update based on the feasible region projection joint variable to obtain the feasible region reflection joint variable. According to the execution order of the reflection operator, the data consistency reflection joint variable, the deep prior reflection joint variable, and the feasible region reflection joint variable are concatenated to obtain the composite reflection joint variable. The difference vector between the composite reflection joint variable and the joint variable is calculated to obtain the update increment. The update increment is scaled according to the relaxation coefficient and added to the joint variable to obtain the updated joint variable. The relaxation coefficient is determined by the data consistency reflection intensity mapping. The updated joint variables are split to obtain the updated restored image variables and the updated point spread function parameter vector, which are then written into the product space variable set.

7. The real-time microscopic image processing method based on deep learning according to claim 1, characterized in that, Step six specifically involves: Perform a Fourier transform on the restored image variables obtained from the data consistency reflection update to obtain the frequency domain restored image variables; The frequency domain components within the support domain are obtained by performing point-by-point multiplication on the frequency domain restored image variables using a band-limited mask, and the frequency domain components outside the support domain are obtained by inverting the band-limited mask and then performing point-by-point multiplication on the frequency domain restored image variables using the inverted mask. Perform an inverse Fourier transform on the frequency domain components within the support domain to obtain the components within the support domain, and perform an inverse Fourier transform on the frequency domain components outside the support domain to obtain the components outside the support domain. Calculate the out-of-domain energy value of the out-of-domain component. The out-of-domain energy value is the sum of the squares of the pixel values ​​of the out-of-domain component. Set the upper limit parameter of the out-of-domain energy. When the out-of-domain energy value is greater than the upper limit parameter, the out-of-domain component is scaled by a scaling factor. The scaling factor is the ratio of the square root of the upper limit parameter to the square root of the out-of-domain energy value. The in-support domain component is added to the scaled out-support domain component to obtain the band-limited constrained restored image variable, which is then used as the data consistency reflection update output.

8. The real-time microscopic image processing method based on deep learning according to claim 1, characterized in that, Step seven specifically involves: After each iteration of the Douglas–Rachford splitting iteration is completed, the data consistency residual norm is calculated based on the observed image frame and the predicted observed image. The incremental norm of the restored image variable is calculated based on the restored image variable of the current iteration and the restored image variable of the previous iteration. The incremental norm of the point spread function parameter vector is calculated based on the point spread function parameter vector of the current iteration and the point spread function parameter vector of the previous iteration. The data consistency residual norm, the restored image variable increment norm, and the point spread function parameter vector increment norm are written into the certificate record in a fixed field order, and the certificate record serves as the consistency certificate. Set residual threshold, image increment threshold, and parameter increment threshold. Based on the consistency certificate, compare the data consistency residual norm with the residual threshold, compare the restored image variable increment norm with the image increment threshold, and compare the point spread function parameter vector increment norm with the parameter increment threshold. The termination condition is determined when the data consistency residual norm is less than or equal to the residual threshold, the restored image variable increment norm is less than or equal to the image increment threshold, and the point spread function parameter vector increment norm is less than or equal to the parameter increment threshold. When the termination condition is met, the restored image variable is output to obtain the restored image frame. The restored image variable is written to the state buffer and marked as the initial value of the restored image variable. The point spread function parameter vector is written to the state buffer and marked as the initial value of the point spread function parameter vector.

Images