A focused topography recovery method using minimal size windows

Abstract

This invention provides a focused topography restoration method using a minimum-size window. First, the window matrix is ​​transformed to obtain a position vector. Then, the distribution radius of all positions in the position vector and the minimum distance between each position and other positions are calculated, and the variance between the minimum values ​​is calculated. Finally, the distribution radius and variance are weighted and summed to obtain the focus value, thus determining the final depth map. The smaller the window size, the more accurate the calculated focus value. In other words, this invention can calculate a more accurate depth map under the premise of using the minimum image window size, which greatly reduces the dependence of calculation accuracy on a large-size window.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional reconstruction technology, and in particular relates to a method for focused topography recovery using a minimum-size window. Background Technology

[0002] Shape Form Focus (SFF) is a monocular passive reconstruction technique that uses the sharpness level of each pixel in an image as a reconstruction cue to determine the depth of each pixel in real space. First, a partially focused image sequence is acquired by varying the distance between the imaging system and the object to be reconstructed. Then, focus measurement (FM) is used to determine the sharpness level of each pixel position in the image sequence. The calculated sharpness level value is called the focus value, and all the calculated focus values ​​in the image sequence are called the focus volume (FV). The height of the position with the maximum focus value along the Z-axis of the FV is used as the depth, ultimately determining the depth map of the object.

[0003] Currently, improving depth map accuracy mainly involves three aspects: improving the accuracy of the FM operator calculation results, refining the FV (Film View), and optimizing the depth map. The depth map is generated by selecting the position of the maximum focal value along the optical axis in the FV; therefore, the accuracy of the FV directly determines the accuracy of the reconstruction result. When there are many errors in the FV, error propagation occurs, leading to a large number of incorrect depth estimates, which limits the effectiveness of depth map optimization.

[0004] Currently, commonly used focus operators are calculated primarily in the spatial and transform domains. In the spatial domain, operators based on first and second derivatives are frequently used, employing image edge information as a sharpness evaluation criterion. For small window regions, statistical operators are predominantly used, calculating the variance within the window region. In sharp locations, the large differences in grayscale values ​​result in significant variance. In the transform domain, discrete cosine transforms and Fourier transforms are performed on the image to obtain high-frequency components. Many new focus operators have also been proposed, extracting more useful information by selecting windows of different shapes. For specific objects, the image is transformed from Cartesian coordinates to polar coordinates, utilizing the properties of different coordinate systems to obtain accurate focus values. To avoid the limitations of a single focus operator, principal component analysis is used to process a large number of focus operators to determine the optimal focus value.

[0005] Regarding improving focus measurement accuracy: Narayanan et al. used 39 focus measurement operators and selected the optimal focus position using maximum consensus and MRF consensus. Mahmood et al. developed an optimal composite depth function using genetic encoding, extracting key information to achieve accurate depth estimation by optimizing one or more focus measurement methods. Lee et al. proposed a new focus measurement operator that transforms the coordinate system to a log-polar coordinate system, utilizing the spatial characteristics of the polar coordinate system to achieve good robustness to noise. Pradeep et al. proposed an improved SFF method that estimates the final shape of the reconstructed object by calculating the degree of defocusing blur in the image sequence.

[0006] In summary, improvements to the FM operator mainly involve using more FM operators to determine the final focus value or performing spatial transformations on the image. Some scholars have also studied the impact of image window size on reconstruction accuracy, suggesting that the most suitable window size is 5×5. However, few studies have explored achieving higher reconstruction accuracy with the smallest window size. Summary of the Invention

[0007] To address the aforementioned issues, this invention provides a focused topography recovery method using a minimum-size window. This method obtains a better three-dimensional topography of the object by selecting the minimum window size, which significantly reduces the dependence of computational accuracy on a large-size window.

[0008] A method for recovering a focused topography using a minimum-size window includes the following steps:

[0009] Obtain the image sequence of the object to be reconstructed;

[0010] The focus value corresponding to each pixel in each frame of the image sequence is obtained respectively. The method for obtaining the focus value corresponding to any pixel is as follows: the pixel region centered on the current pixel is extracted using the smallest window size of 3×3, and the focus value acquisition operation is performed on the pixel region to obtain the focus value.

[0011] The maximum value of the focal point corresponding to all pixels at the same position is used as the depth value at each position to obtain the depth map of the object to be reconstructed.

[0012] Furthermore, the focus value acquisition operation is as follows:

[0013] Assume the extracted pixel region R1 is:

[0014]

[0015] Where, r 11 ~r 33 These are the pixel values ​​of each pixel point in pixel region R1;

[0016] Transpose R1 to obtain the transpose matrix R2:

[0017]

[0018] Expanding R1 and R2 into column vectors respectively, we obtain column vectors RW and RC:

[0019] RW = [r 11 r 21 r 31 r 12 r 22 r 32 r 13 r 23 r 33 ] T

[0020] RC = [r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] T

[0021] Combining column vectors RW and RC, we obtain the position vector RV = [RW, RC];

[0022] Obtain the center position P of the position vector RV;

[0023] The maximum distance between all positions in the position vector RV and the center position P is taken as the distribution radius d. max ;

[0024] Obtain the nearest distance between each position in the position vector RV and its nearest neighbor.

[0025] Obtain the variance σ of the minimum nearest neighbor distance for all positions corresponding to the position vector RV;

[0026] For the distribution radius d max The sum is calculated by weighting the variance σ, and the resulting sum is the focus value corresponding to the current pixel.

[0027] Furthermore, the method for obtaining the center position P of the position vector RV is as follows:

[0028]

[0029] in, The average position of column vector RW. RW is the positional average of column vector RC.i For the i-th position in column vector RW, RC j Let be the i-th position in the column vector RC.

[0030] Furthermore, the distance d between any position in the position vector RV and the center position P i The calculation method is as follows:

[0031]

[0032] Where i = 1, 2, ..., 9, The average position of column vector RW. RW is the positional average of column vector RC. i For the i-th position in column vector RW, RC j Let i be the i-th position in the column vector RC.

[0033] Furthermore, the distance D between any position in the position vector RV and its neighboring positions... ij The calculation method is as follows:

[0034]

[0035] Among them, RW i Let RW be the i-th position in the column vector RW. j For the j-th position in column vector RW, RC i Let RC be the i-th position in the column vector RC. j Let i be the j-th position in the column vector RC, where i ≠ j.

[0036] Furthermore, the variance σ is calculated as follows:

[0037]

[0038] Among them, D min,i Let μ be the minimum nearest neighbor distance corresponding to the i-th position in column vector RW or column vector RC, and μ be the nearest neighbor distance for all D. min,i The mean.

[0039] Furthermore, the distribution radius d max The weighted sum of the variance σ is 1.

[0040] Beneficial effects:

[0041] This invention provides a focused topography restoration method using a minimum-size window. First, the window matrix is ​​transformed to obtain a position vector. Then, the distribution radius of all positions in the position vector and the minimum distance between each position and other positions are calculated, along with the variance between these minimum values. Finally, the weighted sum of the distribution radius and variance is used as the focal value to determine the final depth map. This method avoids the influence of abnormal grayscale values ​​of the image on the calculated focal value. The smaller the window size, the more accurate the calculated focal value. In other words, this invention can calculate a more accurate depth map using only the minimum image window size, significantly reducing the dependence of computational accuracy on large-size windows. Attached Figure Description

[0042] Figure 1 A flowchart of the focused topography restoration method using a minimum-size window provided by the present invention. Detailed Implementation

[0043] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0044] This invention proposes a focused topography restoration method using a minimum-size window. First, the image window matrix is ​​transformed into a position vector. Then, the distribution radius of all positions in the position vector and the minimum distance between each position and other positions are calculated, along with the variance of the minimum distance. Finally, the distribution radius and variance are weighted and summed to obtain the focus value, thus yielding the object's depth map. This method includes the following steps:

[0045] Acquire the image sequence of the object to be reconstructed; for example, use a monocular vision system to acquire a partially focused image sequence of the object at different height positions at equal intervals, and convert the acquired image sequence from RGB format to grayscale image;

[0046] The focus value corresponding to each pixel in each frame of the image sequence is obtained respectively. The method for obtaining the focus value corresponding to any pixel is as follows: the pixel region centered on the current pixel is extracted using the smallest window size of 3×3, and the focus value acquisition operation is performed on the pixel region to obtain the focus value. It should be noted that the window size of the present invention is an odd number and is not equal to 1, so 3×3 is the minimum window size.

[0047] The maximum value of the focal point corresponding to all pixels at the same position is used as the depth value at each position to obtain the depth map of the object to be reconstructed.

[0048] Furthermore, the focus value acquisition operation is as follows:

[0049] Assume the extracted pixel region R1 is:

[0050]

[0051] Where, r 11 ~r 33 These are the pixel values ​​of each pixel point in pixel region R1;

[0052] Transpose R1 to obtain the transpose matrix R2:

[0053]

[0054] Expanding R1 and R2 into column vectors respectively, we obtain column vectors RW and RC:

[0055] RW = [r 11 r 21 r 31 r 12 r 22 r 32 r 13 r 23 r 33 ] T

[0056] RC = [r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] T

[0057] Combining column vectors RW and RC, we obtain the position vector RV = [RW, RC];

[0058] It should be noted that the focus value calculated by a smaller window is less accurate than that calculated by a larger window. This is mainly because a smaller window contains less information and has a lower degree of distinction between sharp and blurry areas. Therefore, the window matrix is ​​transformed, with each value of the window matrix as x and each value of the transpose of the window matrix as y. The resulting vector is RV = [RW, RC] = (x, y), where RW is x and RC is y.

[0059] Obtain the center position P of the position vector RV:

[0060]

[0061] in, The average position of column vector RW. RW is the positional average of column vector RC.i For the i-th position in column vector RW, RC j This refers to the i-th position in the column vector RC;

[0062] The maximum distance between all positions in the position vector RV and the center position P is taken as the distribution radius d. max Wherein, d is the distance between any position in the position vector RV and the center position P. i The calculation method is as follows:

[0063]

[0064] Where i = 1, 2, ..., 9, The average position of column vector RW. RW is the positional average of column vector RC. i For the i-th position in column vector RW, RC j This refers to the i-th position in the column vector RC;

[0065] It should be noted that the distribution radius can determine the degree of dispersion between each position. The gray values ​​of clear areas are more different, indicating a high degree of dispersion, while the gray values ​​of blurred areas are similar, indicating a low degree of dispersion. Therefore, the center position of all positions is determined, and then the maximum distance from each position to the center position is calculated as the distribution radius.

[0066] Obtain the nearest neighbor distances between all positions in the position vector RV and their neighboring positions; obtain the distance D between any position in the position vector RV and its neighboring positions. ij The calculation method is as follows:

[0067]

[0068] Among them, RW i Let RW be the i-th position in the column vector RW. j For the j-th position in column vector RW, RC i Let RC be the i-th position in the column vector RC. j Let i be the j-th position in the column vector RC, and i ≠ j;

[0069] It should be noted that in clear areas, the grayscale values ​​fluctuate greatly, resulting in inconsistent distances between the generated position vectors. In blurred areas, the distances between position vectors are smaller and the changes are not significant. Therefore, this invention needs to calculate the shortest distance between each position and other positions, and determine the variance of the shortest distance. Specifically, the method for obtaining the variance σ of the minimum neighbor distance corresponding to all positions of position vector RV is as follows:

[0070]

[0071] Among them, D min,i Let μ be the minimum nearest neighbor distance corresponding to the i-th position in column vector RW or column vector RC, and μ be the nearest neighbor distance for all D. min,i The mean;

[0072] For the distribution radius d max The value is obtained by weighted summation of the variance σ and the value of the sum, which is the focal value of the current pixel. Distribution radius d max The weighted sum of the variance σ is 1.

[0073] It should be noted that when there is abnormal noise in the blurred image, the outlier will deviate from the center of the position vector, resulting in a larger calculated distribution radius, which is inconsistent with the blurred state of the window. The method of finding the shortest distance between each position and other positions does not have this problem because the center position is close to the abnormal noise and other positions. However, when the image is clear, there are cases where the distance between each position and the center position is close, resulting in a smaller calculated variance, which is inconsistent with the clear state of the image, but the distribution radius is still larger. Therefore, this invention uses a weighted sum of the variances of the distribution radius and the shortest distance as the focus value.

[0074] In summary, this invention relates to a focus shape restoration method using a minimum-size window. A focus evaluation function for calculating window sharpness level is designed. This method uses a minimum-size window to calculate the window's sharpness level, and the result is called the focus value. Using a minimum window yields optimal results while improving computational speed. The minimum window size is determined to be 3×3. First, the window matrix is ​​transformed by transposing it, forming a set of position vectors at the corresponding positions in the original matrix. These position vectors contain nine position coordinates. Then, the center point coordinates of these nine positions are calculated, and the distance from each position to the center point is calculated. The largest distance from the center point is taken as the distribution radius of these nine positions. Next, the minimum distance from each position to other positions is calculated, and the variance of these minimum distances is calculated. This variance is then weighted and summed with the previously calculated distribution radius to obtain the calculated focus value, avoiding the influence of abnormal grayscale values ​​of the image on the calculated focus value. This invention uses two indicators to calculate the focus value; the smaller the window size, the more accurate the calculated focus value, which significantly reduces the dependence of computational accuracy on a large-size window.

[0075] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A method of focus topography recovery using a minimum size window, characterized by, The method comprises the following steps: obtaining an image sequence of an object to be reconstructed; obtaining a focus value corresponding to each pixel point on each frame image in the image sequence respectively, wherein the focus value corresponding to any pixel point is obtained by: extracting a pixel region with the current pixel point as the center of the window by using the smallest window size 3*3, performing a focus value obtaining operation on the pixel region, and obtaining the focus value; taking the maximum value of the focus values corresponding to all pixel points at the same position as the depth value at each position to obtain a depth map of the object to be reconstructed; the focus value obtaining operation is as follows: assuming that the extracted pixel region R1 is as follows: wherein, is a pixel value of each pixel point in the pixel region R1; transposing R1 to obtain a transposed matrix R2 as follows: Each R 1 and R 2. Expand into column vectors, and obtain the corresponding column vectors. and column vectors : The column vector and the column vector are combined to obtain the position vector RV [ RW , RC ]. Acquiring a position vector RV The center position of P ; The position vector RV The maximum distance between all positions in the center position P as the distribution radius d max ; Respective acquisition position vector RV of all positions and the adjacent distance between the adjacent positions thereof; Acquiring a position vector RV of all positions corresponding to the minimum value of the proximity distance ; The distribution radius d max and variance The weighted sum is the focus value corresponding to the current pixel.

2. A focused topography recovery method using a minimum size window as recited in claim 1, wherein, Position vector RV The center position of the P The acquisition method is: wherein is the position average of the column vector , is the position average of the column vector , is the position average of the column vector , i is the position average of the column vector , is the position average of the column vector i .

3. A focused topography recovery method using a minimum size window as recited in claim 1, wherein, The distance between any one of the position vectors RV and the center position P The calculation method is: wherein i = 1, 2, … 9, is the position average of the column vector , is the position average of the column vector , is the position average of the column vector , i is the position average of the column vector , is the position average of the column vector i .

4. A focused topography recovery method using a minimum size window as recited in claim 1, wherein, The distance between any one position of the position vector RV and its adjacent position The calculation method is: wherein is the jth position in the column vector is the jth position in the column vector i is the jth position in the column vector is the jth position in the column vector is the jth position in the column vector j is the jth position in the column vector is the jth position in the column vector is the jth position in the column vector i is the jth position in the column vector is the jth position in the column vector is the jth position in the column vector j is the jth position in the column vector .

5. A focused topography recovery method using a minimum size window as recited in claim 1, wherein, Variance The calculation method is: in, column vector or column vector The first in i The minimum nearest neighbor distance for each location μ For all D min,i The mean.

6. A focused topography recovery method using a minimum size window as recited in claim 1, wherein, Distribution radius d max and variance The weight and value of 1.

Images