Method of determining deblurring filter coefficients

Abstract

A method for determining deblurring filter coefficients is disclosed. The method includes the steps of generating an edge data table based on data of an image, estimating a blurring kernel based on the edge data table, wherein the blurring kernel represents how an imaging process blurs original image data to generate blurred image data, and determining coefficients of a deblurring filter such that a result of processing the blurred image data by the deblurring filter approximates the original image data.

Description

Technical Field

[0001] This invention relates to a filter design, and more particularly to a method for determining the coefficients of a deblurring filter. Background Technology

[0002] There are two main methods for image sharpening. One method is to directly amplify the high-frequency components of the image signal, but this method often suffers from over-sharpening, resulting in unnatural black and white borders in the sharpened image. The other method is based on deconvolution to reverse the blurring process of an imaging system, thereby sharpening the image; however, this method requires obtaining a blur kernel representing the blurring characteristics of the imaging system beforehand, which is usually obtained through expensive metrology equipment. Summary of the Invention

[0003] One of the purposes of this disclosure is to provide a method for determining deblurring filter coefficients to avoid the problems of prior art.

[0004] An embodiment of the method disclosed herein includes the following steps: generating an edge data table based on data of an image; estimating a blurring core based on the edge data table, wherein the blurring core represents how an imaging process blurs the original image data to produce blurred image data; and determining the coefficients of a deblurring filter such that a processing result of the blurred image data processed by the deblurring filter approximates the original image data.

[0005] As described above, one embodiment of generating the edge data table includes: identifying the edge positions of multiple columns of a region of interest in the image, wherein each of the multiple columns contains multiple neighboring edge pixels, and the distance between each of the multiple neighboring edge pixels and the edge position of the column is less than a preset distance; aligning the edge positions of the multiple columns; and after aligning the edge positions of the multiple columns, averaging all neighboring edge pixels located at the same horizontal position in the multiple columns to obtain a column average value as the edge data table.

[0006] Another embodiment of generating the edge data table includes generating the edge data table based on a Gaussian smoothed step edge method.

[0007] The features, implementation, and effects of this invention are described in detail below with reference to the accompanying drawings, using preferred embodiments. Attached Figure Description

[0008] Figure 1 An embodiment of the method of this disclosure is shown;

[0009] Figure 2 show Figure 1 A practical example of step S1 10;

[0010] Figure 3 show Figure 1 A practical example of step S120;

[0011] Figure 4 show Figure 1 Another practical example of step S120; and

[0012] Figure 5 show Figure 1 A practical example of step S130. Detailed Implementation

[0013] This specification discloses a method for determining the coefficients of a deblurring filter. This method is based on deconvolution. While deconvolution-based methods require obtaining a blurring kernel representing the blurring characteristics of an imaging system, the method disclosed herein does not use expensive metrological equipment to obtain this blurring kernel. Instead, it estimates the blurring kernel based on an image edge data table, and then obtains the coefficients of the deblurring filter according to the method described in the applicant's U.S. Patent No. 10,762,610B2 ​​(hereinafter referred to as '610 Patent').

[0014] Figure 1 An embodiment of the method shown in this disclosure includes the following steps:

[0015] S110: Generate an edge data table based on the data of an image. For example, the edge data table can be generated using a known / self-developed slanted edge method.

[0016] S120: Estimate a blurring core based on the edge data table, wherein the blurring core represents how an imaging process blurs the original image data to produce blurred image data. In this embodiment, the sum of the coefficients of the blurring core is 1; however, this feature is not a limitation of the present invention as may be implemented. It is worth noting that the image data described in step S110 is equivalent to the blurred image data.

[0017] S130: Determine the coefficients of a deblurring filter so that the processing result of the blurred image data after being processed by the deblurring filter approximates the original image data. In this embodiment, the sum of the coefficients of the deblurring filter is 1; however, the above feature is not a limitation of the present invention as it may be implemented.

[0018] Figure 2 show Figure 1 An example of a practical implementation of step S110 includes:

[0019] S210: Find the edge positions of multiple columns of a region of interest in the image, wherein each of the multiple columns contains multiple neighboring edge pixels, and the distance between each of the multiple neighboring edge pixels and the edge position of the column is less than a preset distance (e.g., a distance of 3 pixels), the preset distance depending on the implementation requirements.

[0020] For example, step S210 includes: determining the region of interest, wherein the region of interest contains the plurality of columns, and each of the plurality of columns contains a plurality of pixels (e.g., 100 pixels); finding the pixel position with a maximum absolute gradient value in each of the plurality of columns and treating the pixel position as an edge position to obtain the edge positions of the plurality of columns; and aligning the edge positions of the plurality of columns so that the edge positions of the plurality of columns have the same horizontal position.

[0021] It is worth noting that the term "level" is used in this specification for ease of understanding, and those skilled in the art will understand that it does not refer to "level" in physical space. It is also worth noting that "finding the pixel location with a maximum absolute gradient value" can be achieved through a known / self-developed method.

[0022] S220: After aligning the edge positions of the multiple columns, average all the neighboring edge pixels in the multiple columns that are at the same horizontal position to obtain a column average value as the edge data table.

[0023] It is worth noting that this edge data table can be generated in other ways. For example, a Gaussian smoothed step edge method can be used to describe this edge data table, as described in publicly available materials (https: / / en.wikipedia.org / wiki / Edge_detection#A_simple_edge_model).

[0024] Figure 3 show Figure 1 An example of a practical implementation of step S120 includes:

[0025] S310: Take the first derivative of the edge data table in a positive direction to obtain a series of positive derivative values.

[0026] For example, the edge data table is [x1 x2 x3 x4 x5 x6], and the positive differential value of this column is [(x2-x1)(x3-x2)(x4-x3)(x5-x4)(x6-x5)] = [y1 y2 y3 y4 y5].

[0027] It is worth noting that the size of this edge data table can be expanded or shrunk depending on implementation needs.

[0028] S320: Take the first derivative of the edge data table in the opposite direction to obtain a series of reverse derivative values.

[0029] For example, the edge data table is [x1 x2 x3 x4 x5 x6], and the reverse derivative of this column is [(x6-x5)(x5-x4)(x4-x3)(x3-x2)(x2-x1)] = [z1 z2 z3 z4 z5].

[0030] S330: Add the forward differential value of the column to the reverse differential value of the column to obtain a total value, and then normalize the total value of the column to obtain the one-dimensional form of the fuzzy core.

[0031] For example, the forward differential value of this column is [y1 y2 y3 y4 y5], and the backward differential value is [z1 z2 z3 z4z5]. The normalized sum of this column (i.e., the one-dimensional form of the fuzzy core) is: in

[0032] Figure 4 show Figure 1 Another practical example of step S120 includes:

[0033] S410: Use a sigmoid edge model to approximate the data in the edge data table, wherein the sigmoid edge model uses a sigmoid function and includes an undetermined parameter.

[0034] For example, one embodiment of this S-shaped curve edge model can be represented as follows: The definition of the S-function S(w) is as follows: According to publicly available information (https: / / en.wikipedia.org / wiki / Sigmoid_function), parameter c is the parameter to be determined.

[0035] S420: Define an error function to represent an error of the parameter to be determined.

[0036] For example, one embodiment of this error function can be expressed as follows:

[0037]

[0038] S430: Use a gradient descent method to find an optimal value for the undetermined parameter that minimizes the error, wherein the gradient descent method is as described in the published documentation (https: / / www.seas.ucla.edu / ~vandenbe / 236C / lectures / gradient.pdf).

[0039] For example, the optimal value (c) opt ) can be represented as c opt =arg min c＞0 E(c).

[0040] S440: The fuzzification core is obtained by using the optimal value and the first derivative function of the S-function (i.e., S(w)·(1-S(w))).

[0041] For example, the fuzzification core can be represented as follows:

[0042]

[0043]

[0044] Figure 5 show Figure 1 An example of a practical implementation of step S130 includes

[0045] S510: Calculate based on the collected data and / or specified data such that the difference between the processing result and the original image data is less than a threshold, thereby determining the coefficients of the deblurring filter.

[0046] For example, this step includes: assuming the sum of the coefficients of the deblurring filter is 1, determining which coefficients of the deblurring filter minimize the difference between the processed result and the original image data. Since the details of this step can be inferred from the '610 patent, these details are omitted here.

[0047] The following is an explanation Figure 1 An example of an implementation of the embodiment.

[0048] A one-dimensional fuzzification kernel can be represented as follows:

[0049] [s1 s2 s3 s4 s5]

[0050] A one-dimensional signal (or original one-dimensional signal) of an ideal graph can be represented as follows:

[0051]

[0052] The blurred one-dimensional signal obtained by an imaging system processing the original one-dimensional signal can be represented as follows:

[0053]

[0054] The gradient of this fuzzy one-dimensional signal can be calculated as follows:

[0055]

[0056] The above formula is equivalent to first calculating the gradient of the original one-dimensional signal, and then blurring the gradient using the blurring core, which can be expressed as follows:

[0057]

[0058] The basis of the above formula is:

[0059]

[0060] The above explanation can also be interpreted as follows. First, this fuzzy one-dimensional signal can be represented as follows:

[0061] b*x

[0062] In the above formula, b is the fuzzification kernel, and x is the original one-dimensional signal. The gradient of the fuzzy one-dimensional signal can be expressed as follows:

[0063] g*(b*x)

[0064] In the above equation, g represents the calculation of the gradient. Based on the commutative law (https: / / en.wikipedia.org / wiki / Convolution#Algebraic_properties) and the associative law (https: / / en.wikipedia.org / wiki / Convolution#Differentiation), the above equation can be rewritten as follows:

[0065] g*(b*x)=g*(x*b)

[0066] g*(x*b)=(g*x)*b

[0067] If the gradient of the original one-dimensional signal is an ideal step edge, the gradient of the original one-dimensional signal will be a pulse signal as shown below:

[0068] g*x=[… 0 0 1 0 0 …]

[0069] The convolution of the pulse signal with the blurring kernel will still result in the same blurring kernel, as shown below:

[0070] (g*x)*b=[… 0 0 1 0 0 …]*b=b

[0071] The blur kernel is normalized so that the sum of its coefficients is 1. This normalization is based on the premise that the blur kernel does not change the overall brightness of an image.

[0072] In summary, by calculating the gradient of a blurred image, the blurring characteristics (i.e., the blurring kernel) of an imaging system can be obtained.

[0073] The following explanation further clarifies how to determine the coefficients of a defuzzification filter based on a fuzzification core.

[0074] First, the aforementioned one-dimensional fuzzification core [s1 s2 s3 s4 s5] = s T The fuzzy core can be expanded into a 5x5 two-dimensional matrix u (i.e., the two-dimensional fuzziness kernel) as shown below:

[0075] u=ss T

[0076] In the above formula, the (i, j)th term of u (u i，j This can be represented as:

[0077] u i，j =s i s j

[0078] In the above formula, i and j are restricted to [1, 5], and s i Represents the i-th term of s, s j Let represent the j-th term of s. The two-dimensional matrix u can be represented as follows:

[0079]

[0080] For example, an example of u is shown in the table below:

[0081] 0.0133136 0.035503 0.0177515 0.035503 0.0133136 0.035503 0.0946746 0.0473373 0.0946746 0.035503 0.0177515 0.0473373 0.0236686 0.0473373 0.0177515 0.035503 0.0946746 0.0473373 0.0946746 0.035503 0.0133136 0.035503 0.0177515 0.035503 0.0133136

[0082] Table 1

[0083] In the table above, all coefficients u i，j The sum is 1.

[0084] Next, the relationship between the blurred pixels and the original pixels can be represented as follows:

[0085]

[0086]

[0087] In the above formula, x is the 81-dimensional vector that expands the original 9×9 image packet, with the pixel values ​​in the original image packet arranged in column-major order as a column vector; y is the 25-dimensional vector that expands the 5×5 blurred pixels located at the center of the 9×9 image range; H is a 25×81 matrix; N is a 25-dimensional vector with zero average noise, which can be represented in column-major order as [n 1，1 … n 5，5 ] T In matrix H, each column has only 25 points representing the coefficient u. i，j The values ​​at other points are 0, and matrix H can be represented as follows:

[0088]

[0089] In the aforementioned formula, Hx means: convolving the original 9×9 image (x) with a 5×5 blurring kernel (i.e., the aforementioned two-dimensional matrix u) to obtain a 5×5 filtered result (y).

[0090] According to patent '610, a deblurring filter g (i.e., the deblurring core) can be found that satisfies the following equation:

[0091] y T g≈x T f

[0092] In the above formula, the deblurring filter g (i.e., the gradient calculation) is a 25-dimensional vector; f is the same filter (i.e., x). T f = x T ).

[0093] In the first scenario (e.g., when sufficient data has been collected beforehand), a dataset Q containing n data points is collected in advance. This dataset Q can be represented as follows:

[0094] Q={(x i y i |0≤i≤n-1}

[0095] Where x i and y i These correspond to the original image and the blurred image with noise interference, respectively.

[0096] According to patent '610, the aforementioned y T g≈x T The f relation can be transformed into the following operations:

[0097] Under the condition that Cg = d,

[0098] Minimize() is a minimization algorithm. Based on the above, this algorithm performs a minimization operation under the condition that Cg = d to find yg, which is the closest approximation of xf. Since the sum of the coefficients of the deblurring filter g is 1, in the above equation, C is a 25-dimensional column vector where each term is 1, and d is 1, as shown below:

[0099] C = [1 … 1]

[0100] d=1

[0101] According to patent '610, A and b can be defined as follows:

[0102]

[0103] The coefficients of the deblurring filter g are the solution of the following equation:

[0104]

[0105] The definition of Z can be found on pages 15-16 of the publicly available document "Constrained least squares" (http: / / www.seas.ucla.edu / ~vandenbe / 133A / lectures / cls.pdf) and on pages 3-4 of the publicly available document "Derivative, Gradient, and Lagrange Multipliers" (http: / / ee263.stanford.edu / notes / gradient-lagrange.pdf).

[0106] According to patent '610, since the aforementioned N is zero average noise, the following formula holds:

[0107]

[0108]

[0109] In the above formula, E[xx] T ] is a symmetric and positive semi-definite matrix, and n is [n] as mentioned before. 1，1 … n 5，5 When N is white noise, the following equation holds:

[0110]

[0111] Where σ is the standard deviation of the noise, and I is the identity matrix. According to the '610 patent, eigenvalue decomposition can be used to process E[xx]. T ], as shown below:

[0112]

[0113] In the above formula, e0, ..., e m-1 These are m-dimensional eigenvectors, and they form a set of orthogonal normalized basis vectors, whose corresponding eigenvalues ​​are λ0≥…≥λ. m-1 ≥0. In the aforementioned formula, matrix E[xx T The coefficients of the deblurring filter g can be derived from the known collected data Q and the short matrix H is a matrix derived from the fuzzing core.

[0114] Furthermore, in a second scenario (e.g., when insufficient data has been collected beforehand), according to the '610 patent, a diagonal matrix can be specified to approximate E[xx]. T ], as shown below:

[0115]

[0116] In the above formula, d 0，0 , ..., d i，j It is an m-dimensional discrete cosine transform orthogonal normalization basis. Since x is an 81-dimensional vector, m = 81; furthermore, The index value (i+j) 2 The larger the value, the more it represents w. i，j The higher the corresponding frequency. k T It can be represented as follows:

[0117]

[0118] And d i，j It can be represented as follows: It is arranged into a column vector in column order.

[0119] Regarding this diagonal matrix, the intensity of the discrete cosine transform based on the natural image is inversely proportional to the frequency, therefore w i，j The index value (i+j) 2 The larger w is i，j The smaller. For example, w i，j A non-restrictive example is Where i is between Between, and j is between Between these two values, m represents the dimension of the aforementioned data Q, and σ represents the standard deviation of the data distribution. Since this diagonal matrix is ​​user-defined, the coefficients of the deblurring filter g can be calculated accordingly.

[0120] Please note that, where possible, those skilled in the art may selectively implement some or all of the technical features in any of the foregoing embodiments, or selectively implement a combination of some or all of the technical features in the foregoing multiple embodiments, thereby increasing the flexibility in implementing the present invention.

[0121] In summary, the present invention can determine the deblurring filter coefficients without using expensive measurement equipment.

[0122] Although the embodiments of the present invention have been described above, these embodiments are not intended to limit the present invention. Those skilled in the art can make changes to the technical features of the present invention based on the explicit or implicit content of the present invention. All such changes may fall within the scope of patent protection sought by the present invention. In other words, the scope of patent protection of the present invention shall be determined by the scope of the patent application in this specification.

[0123] [Symbol Explanation]

[0124] S110~S130: Steps

[0125] S210~S220: Steps

[0126] S310~S330: Steps

[0127] S410~S440: Steps

[0128] S510: Steps.

Claims

1. A method for determining deblurring filter coefficients, comprising the following steps: Generate an edge data table based on the data of an image; Based on the edge data table, a blurring core is estimated. This blurring core represents how an imaging process blurs the original image data to produce blurred image data. Based on this edge data table, a blurring core is estimated through at least one of the following methods: Method A: Take the first derivative of the edge data table in a positive direction to obtain a column of positive derivative values, and take the first derivative of the edge data table in a negative direction to obtain a column of negative derivative values. Add the column of positive derivative values ​​and the column of negative derivative values ​​to obtain a column of sum values, and then normalize the column of sum values ​​to obtain the one-dimensional form of the fuzzy core. Method B: An S-curve edge model is used to approximate the data in the edge data table. This S-curve edge model employs a sigmoid function and includes an undetermined parameter. An error function is defined to represent the error of this undetermined parameter. A gradient descent method is used to find the optimal value of the undetermined parameter that minimizes this error. The fuzzing kernel is then determined using this optimal value and the first derivative of the sigmoid function. The coefficients of a deblurring filter are determined so that the processed image data, after being processed by the deblurring filter, approximates the original image data. The steps for generating this edge data table include: Find the edge positions of multiple columns of a region of interest in the image, wherein each of the multiple columns contains multiple neighboring edge pixels, and the distance between each of the multiple neighboring edge pixels and the edge position of the column is less than a preset distance; Align the edge positions of these multiple columns; and After aligning the edge positions of the multiple columns, average all neighboring edge pixels at the same horizontal position in the multiple columns to obtain a column of average values ​​as the edge data table.

2. The method of claim 1, wherein the step of aligning the edge positions of the plurality of columns comprises: Find the pixel position with the largest absolute gradient value in each of the multiple columns, and treat this pixel position as an edge position, thus obtaining multiple pixel positions in the multiple columns as edge positions of the multiple columns; and Align the edge positions of the multiple columns so that the edge positions of the multiple columns have the same horizontal position.

3. The method of claim 1, wherein the step of generating the edge data table comprises: generating the edge data table based on a Gaussian smoothed step edge method.

4. The method according to claim 1, wherein the sum of the coefficients of the fuzzification core is 1.

5. The method according to claim 1, wherein the sum of the coefficients of the deblurring filter is 1.

6. The method of claim 1, wherein the step of determining the coefficients of the deblurring filter comprises: calculating based on collected data and / or specified data such that the difference between the processing result and the original image data is less than a threshold, thereby determining the coefficients of the deblurring filter.

7. The method according to claim 6, wherein the step of determining the coefficients of the deblurring filter comprises: calculating the coefficients of the deblurring filter, provided that the sum of the coefficients of the deblurring filter is 1, so as to minimize the difference between the processing result and the original image data.

Images