Deconvolution method of image with spatial non-uniform point spread function, medium and equipment

Abstract

The invention provides a deconvolution method for an image with a spatial non-uniform point spread function, a medium and equipment, and the method comprises the steps: 1, constructing a point spread function PSF (i, j) at each point (i, j) on an image G (i, j) with the spatial non-uniform point spread function; and step 2, based on the representation of the PSF (i, j), adopting a Richardson-Lucy method to carry out PSF iterative deconvolution so as to obtain estimation of a real image after iteration. According to the method, the form of the target object in the image after PSF deconvolution is continuously changed without jumping, the image before and after PSF deconvolution keeps self-similarity in form, and the flow conservation of the relatively extended PSF in the deconvolution process is better ensured. The method provided by the invention is very beneficial to deconvolution of a space non-uniform PSF field which changes gently, a PSF image which can be relatively completely described by a few PSF components (such as principal components) at any position, and an image of which the PSF is very extended.

Description

Technical Field

[0001] The present invention belongs to the technical field of image acquisition and processing, and relates to a deconvolution method, medium and device for an image with a spatially non-uniform point spread function. Background Art

[0002] During the image acquisition process, affected by the manufacturing process of optical instruments, the diffraction law of light, etc., it is very difficult to acquire an image with a point spread function (PSF) that is completely consistent across the entire focal plane. For example, for an observation device with a large field of view, the PSFs at the center and edges of the field of view often have very large differences; for a large-aperture device with active optical technology, it is very difficult to adjust its integrated mirror surface to a perfect paraboloid; for ground-based observation devices, they are easily affected by atmospheric turbulence, thus changing the spatial distribution of the PSF.

[0003] This spatial non-uniformity of the PSF poses the following challenges to traditional PSF deconvolution techniques:

[0004] 1) It is impossible to perform PSF deconvolution on the entire image. Traditional PSF deconvolution requires a PSF that does not vary with space. As a compromise, to perform PSF deconvolution on a spatially non-uniform PSF, generally several sub-images are divided and the traditional method is used to perform PSF deconvolution on the sub-images (since the sub-image area is small enough, it can be assumed that the PSF is in the area where the sub-image is located). Of course, it can be expected that there must be jumps and discontinuities in the PSF at the overlapping parts of adjacent sub-images. This discontinuity can only be reduced by increasing the sub-image sampling, but cannot be completely eliminated.

[0005] 2) Increasing the sub-image sampling will reduce its size, which results in the PSF extended on this sub-image being unable to affect adjacent sub-images, thus making it difficult to ensure the flow conservation and the morphological accuracy of the extended target (source) during the PSF deconvolution process. For example, the James Webb Telescope has a very prominent iconic hexagonal starburst (PSF). Although the core part of this starburst drops rapidly, the periphery drops very slowly, which results in a very extended PSF for the Webb Telescope. In this case, increasing the sub-image sampling will not be able to correctly deconvolve the PSF on the sub-image, resulting in a relatively serious non-conservation of the flow on each sub-image.

[0006] Therefore, traditional PSF deconvolution has exposed many defects in dealing with spatially non-uniform PSFs, and there is an urgent need for innovative technical means to solve this problem. Summary of the Invention

[0007] Aiming at the deficiencies in the prior art, the present invention provides a deconvolution method, medium and device for an image with a spatially non-uniform point spread function.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a deconvolution method for an image with a spatially non-uniform point spread function, comprising the following steps: Step 1: For an image G(i, j) with a spatially non-uniform point spread function, construct a point spread function PSF(i, j) at each point (i, j) on the image G(i, j).

[0010] Step 2: Based on the representation of PSF(i, j), perform PSF iterative deconvolution using the Richardson-Lucy method, and obtain an estimate of the true image after iteration.

[0011] Optionally, in Step 1, the point spread function PSF(i, j) is expressed as a linear superposition of a series of PSFs that do not depend on the position (i, j), and PSF(i, j) is normalized everywhere.

[0012] Optionally, in Step 1, the point spread function PSF(i, j) is represented as follows:

[0013]

[0014] where P k is the k-th component of PSF(i, j), k is an integer from 0 to M, and C i，j，k is the coefficient matrix of P k at the position (i, j).

[0015] Optionally, in Step 2, the formula for the PSF iterative deconvolution is as follows:

[0016]

[0017] where f t (i, j) represents the t-th estimate of the true image, P is the PSF that does not depend on the position (i, j), P T is the transpose matrix of P, represents the convolution symbol, and * and / are dot multiplication and dot division respectively.

[0018] Optionally, in Step 2, the specific process of the PSF iterative deconvolution is as follows:

[0019] Step 2.1: Multiply the t-th estimate f t (i, j) of the true image by the coefficient matrix C k of P i，j，k respectively to obtain a new matrix, and then convolve P k to obtain M + 1 convolutions. Calculate the dot sum of these M + 1 convolutions and scale by dividing by M + 1 to obtain the matrix V(i, j);

[0020] Step 2.2: Divide the matrix G(i, j) by the matrix V(i, j) to obtain the residual term R(i, j).

[0021] Step 2.3: Multiply the residual term R(i, j) element-wise by the coefficient matrix C k of P i，j，k to obtain a new matrix, and then convolve with the transposed matrix P k of P k T , to obtain M + 1 convolutions. Calculate the element-wise sum of these M + 1 convolutions and scale by dividing by M + 1 to obtain the iterative update term U(i, j).

[0022] Step 2.4: Multiply the t-th estimate f t (i, j) of the real image by the iterative update term U(i, j) to obtain the (t + 1)-th estimate f t+1 (i, j) of the real image.

[0023] Loop through steps 2.1 - 2.4 until an estimated result that meets the threshold condition is output.

[0024] In a second aspect, the present invention provides a computer-readable storage medium storing a computer program, which causes a computer to execute the deconvolution method for an image with a spatially non-uniform point spread function as described in the first aspect.

[0025] In a third aspect, the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the deconvolution method for an image with a spatially non-uniform point spread function as described in the first aspect.

[0026] The beneficial effects of the present invention are:

[0027] The morphology of the target object in the image after deconvolving the PSF of the present invention also changes continuously without jumps; the images before and after deconvolving the PSF maintain self-similarity in morphology; any PSF component can affect the entire field of view. Therefore, it better ensures the flux conservation of a relatively extended PSF during the deconvolution process.

[0028] The present invention is very beneficial for the deconvolution of a smoothly varying spatially non-uniform PSF field, or an image of an arbitrary position PSF that can be relatively completely described by a small number of PSF components (such as principal components), or an image with a very extended PSF, and can be effectively applied to the image processing of large field-of-view telescopes, microscopes, or other optical devices that can generate spatially non-uniform PSF images. Description of the Drawings

[0029] Figure 1 are 4 PSF components constructed from the original image in this embodiment.

[0030] Figure 2 are the coefficients corresponding to each PSF component constructed from the original image at each point in the image.

[0031] Figure 3 is the result obtained by using the traditional sub - image sampling method to cut out the sub - image of the central star and performing PSF de - convolution using the Richardson - Lucy de - convolution method.

[0032] Figure 4 is the result of cutting out the central star after performing PSF de - convolution using the method of the present invention.

[0033] Figure 5 is the output result of the overall image after performing PSF de - convolution using the method of the present invention. Detailed implementation manners

[0034] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application.

[0035] In one embodiment, the present invention proposes a de - convolution method for an image with a spatially non - uniform point spread function, which mainly includes the following steps:

[0036] Step 1: Obtain the PSF(i, j) of an image with a spatially non - uniform point spread function.

[0037] For an image G(i, j) with a spatially non - uniform point spread function, where i and j are integers from 1 to N, it is necessary to construct the point spread function PSF(i, j) at each point (i, j) on the image G(i, j), and PSF(i, j) can be expressed as a linear superposition of a series of PSFs that do not depend on the position (i, j), as shown in formula (1).

[0038]

[0039] where, P k is the k - th component of PSF(i, j) (k is an integer from 0 to M) and does not depend on the position (i, j); C i，j，k is the coefficient of P k at the position (i, j). In addition, it is required here that PSF(i, j) is normalized everywhere. Figure 1 are the 4 PSF components constructed from the original image in this embodiment. Figure 2 are the coefficients corresponding to each PSF component constructed from the original image at each point in the image.

[0040] Step 2: Iterative processing.

[0041] Perform PSF iterative deconvolution in combination with the Richardson-Lucy method, as shown in Equation (2). Here, it can also be the PSF deconvolution method of Landwebber. Among them, f t (i, j) is the t-th estimate of the real image, where t is an integer from 0 to L, P is the PSF that does not depend on the position (i, j), and P T is the transpose matrix of P, represents the convolution symbol, and * and / are dot multiplication and dot division respectively.

[0042]

[0043] Since the spatially non-uniform PSF depends on the position, the PSF iterative deconvolution cannot be directly performed using Equation (2). The PSF iterative deconvolution process of this embodiment is specifically as follows:

[0044] Step 2.1: Multiply the t-th estimate f t (i, j) of the real image by the coefficient matrix C k of P i，j，k respectively to obtain a new matrix, and then convolve P k . In this way, M + 1 convolutions are obtained. Calculate the dot sum of these M + 1 convolutions and divide by M + 1 for scaling to obtain the matrix V(i, j).

[0045] Step 2.2: Divide the matrix G(i, j) by the matrix V(i, j) to obtain the residual term R(i, j).

[0046] Step 2.3: Multiply the residual term R(i, j) by the coefficient matrix C k of P i，j，k respectively to obtain a new matrix, and then convolve the transpose matrix P k of P k T . In this way, M + 1 convolutions are obtained. Calculate the dot sum of these M + 1 convolutions and divide by M + 1 for scaling to obtain the iterative update term U(i, j).

[0047] Step 2.4: Multiply the t-th estimate f t (i, j) by the iterative update term U(i, j) to obtain the (t + 1)-th estimate f t+1 (i, j).

[0048] Loop through Steps 2.1 - 2.4 until the output satisfies the threshold condition.

[0049] In the iterative calculation of this embodiment, the Richardson-Lucy method (Formula 2) is ingeniously combined with the representation of the spatially non-uniform point spread function (Formula 1). In each iteration, the estimated image is first multiplied by the coefficient matrix and then convolved with the PSF component. And then scaling is required to perform division or multiplication with the original image or the estimated image.

[0050] Figure 3 It is the result obtained by using the traditional subgraph sampling method to cut out the subgraph of the central star and performing PSF deconvolution using the Richardson-Lucy deconvolution method (where the deconvolved PSF is the PSF of this central point). The upper left figure is the original input image, the upper right is the output after 20 iterations, the lower left is the output after 40 iterations, and the lower right is the output after 60 iterations.

[0051] Figure 4 It is the result after cutting out the central star after performing PSF deconvolution using the method of the present invention. The upper left figure is the original input image, the upper right is the output after 20 iterations, the lower left is the output after 40 iterations, and the lower right is the output after 60 iterations.

[0052] Figure 5 It is the output result of the overall image after performing PSF deconvolution using the method of the present invention. The upper left figure is the original input image, the upper right is the output after 20 iterations, the lower left is the output after 40 iterations, and the lower right is the output after 60 iterations.

[0053] Comparison Figure 3 and 4 It can be clearly seen that the method of the present invention has a higher peak and better flux conservation than the traditional subgraph sampling and then deconvolution. Comparing Figure 3 , 4 and 5, it can be found that the stellar morphology output by the method of the present invention has better similarity with the initial image, while Figure 3 changes this self-similarity, which is not friendly to research fields that focus on morphological measurement, such as the deformation caused by gravitational lensing, galaxy morphological classification, etc.

[0054] In another embodiment, the present invention proposes a computer-readable storage medium storing a computer program, and the computer program causes a computer to execute the deconvolution method of the image with a spatially non-uniform point spread function in the foregoing embodiment.

[0055] In another embodiment, the present invention proposes an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the deconvolution method of the image with a spatially non-uniform point spread function in the foregoing embodiment.

[0056] In the embodiments disclosed in the present application, the computer storage medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the computer storage medium would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0057] Those of ordinary skill in the art will appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed in the present application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present application.

[0058] The above are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, several improvements and refinements made without departing from the principle of the present invention should be regarded as within the protection scope of the present invention.

Claims

1. A deconvolution method for an image with a spatially non-uniform point spread function, characterized in that: The steps include: Step 1: For an image G(i, j) with a spatially non-uniform point spread function, construct a point spread function PSF(i, j) at each point (i, j) on the image G(i, j); Step 2: Based on the representation of PSF(i, j), the Richardson-Lucy method is used to perform PSF iterative deconvolution, and an estimate of the real image is obtained after iteration.

2. A deconvolution method for an image with a spatially non-uniform point spread function as claimed in claim 1, characterized in that: In step 1, the point spread function PSF (i, j) is expressed as a linear superposition of a series of PSFs that are independent of the position (i, j), and PSF (i, j) is normalized everywhere.

3. A deconvolution method for an image with a spatially non-uniform point spread function as claimed in claim 2, characterized in that: In step 1, the point spread function PSF(i, j) is expressed as follows: Where P k is the kth component of PSF(i, j), k is an integer from 0 to M, C i，j，k is the P at position (i, j) k The coefficient matrix of .

4. A deconvolution method for an image with a spatially non-uniform point spread function as claimed in claim 3, characterized in that: In step 2, the formula for the PSF iterative deconvolution is as follows: In the formula, f t (i, j) represents the t-th estimate of the true image, P is the PSF that is independent of position (i, j), P T is the transposed matrix of P, represents the convolution symbol, * and / represent dot multiplication and dot division respectively.

5. A deconvolution method for an image with a spatially non-uniform point spread function as claimed in claim 4, characterized in that: In step 2, the specific process of the PSF iterative deconvolution is as follows: Step 2.1: Set the t-th estimate f of the real image t (i, j) dot product P k The coefficient matrix C i，j，k Get the new matrix and then convolve P k , we get M+1 convolutions, we calculate the dot sum of these M+1 convolutions and divide them by M+1 to scale them, and get the matrix V(i, j); Step 2.2: Divide the matrix G(i, j) by the matrix V(i, j) to obtain the residual term R(i, j); Step 2.3: Multiply the residual term R(i, j) by P k The coefficient matrix C i，j，k Get the new matrix and then convolve P k The transposed matrix P k T , we get M+1 convolutions, we calculate the dot sum of these M+1 convolutions and divide them by M+1 to scale them, and get the iterative update term U(i, j); Step 2.4: Set the t-th estimate f of the real image t (i, j) point multiplication iteratively updates the term U(i, j) to obtain the t+1th estimate f of the real image t+1 (i, j); Repeat steps 2.1-2.4 until an estimation result that meets the threshold condition is output.

6. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables a computer to execute the deconvolution method of an image with a spatially non-uniform point spread function as claimed in any one of claims 1 to 5.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the deconvolution method for an image with a spatially non-uniform point spread function as described in any one of claims 1 to 5 is implemented.

Images