Focusing morphology recovery method using minimum-size window

Abstract

The invention provides a focusing morphology recovery method using a window with a minimum size, which comprises the following steps: firstly, transforming a window matrix to obtain a position vector, then calculating the distribution radiuses of all positions in the position vector and the minimum values of the distances between each position and other positions, and calculating the variance between the minimum values to obtain the focusing morphology recovery result. And finally, performing weighted summation on the distribution radius and the variance to serve as a focus value, and determining a final depth map. The smaller the window size is, the more accurate the calculated focus value is, that is, the more accurate depth map can be obtained through calculation under the precondition that the minimum image window size is used, and the dependence of calculation precision on a large-size window is reduced to a great extent.

Description

Technical Field

[0001] The invention belongs to the technical field of three-dimensional reconstruction, and in particular relates to a focusing morphology restoration method using a minimum-size window. Background Art

[0002] Shape Form Focus (SFF) is a monocular passive reconstruction technology that uses the clarity level of each pixel in the image as a reconstruction clue to determine the depth of each pixel in real space. First, a partially focused image sequence is acquired by changing the distance between the imaging system and the reconstructed object. Then, the focus measurement method (FM) is used to determine the clarity level of each pixel position in the image sequence. The calculated clarity level value is called the focus value. All the focus values ​​of the calculated image sequence are called the focus volume (Focus Volume, FV). The height of the maximum focus value at each position along the Z-axis direction of the FV is found as the depth, and finally the depth map of the object is determined.

[0003] At present, there are three main aspects to improve the accuracy of depth maps: improving the accuracy of the calculation results of the FM operator, refining the FV, and optimizing the depth map. The generation of the depth map is determined by selecting the position of the maximum focus value along the optical axis in the FV. Therefore, the accuracy of the FV directly determines the accuracy of the reconstruction result. When there are many errors in the FV, error transmission will occur, resulting in a large number of erroneous depth estimates, which leads to limited optimization of the depth map.

[0004] The commonly used focus operators are mainly calculated in the spatial domain and the transform domain. In the spatial domain, the first-order derivative and second-order derivative operators are commonly used. The edge information of the image is used as the clarity evaluation standard. For small window areas, statistical operators are mostly used to calculate the variance in the window area. When the gray value is large in the clear position, a large variance value will be generated. In the transform domain, the image is subjected to discrete cosine transform and Fourier transform to obtain the high-frequency components. Many new focus operators have also been proposed. By selecting windows of different shapes, more useful information can be obtained. For specific objects, the image is transformed from the Cartesian coordinate system to the polar coordinate system, and the properties of different coordinate systems are used to obtain accurate focus values. In order to avoid the limitations of a single focus operator, the principal component analysis method is used to process a large number of focus operators to determine the optimal focus value.

[0005] In terms of improving the accuracy of focus measurement: Narayanan et al. used 39 focus measurement operators and selected the best focus position using maximum consensus and MRF consensus on the measurement results. Mahmood et al. used genetic coding to develop an optimal composite depth function, which extracts key information to achieve accurate depth estimation by optimizing one or more focus measurement methods. Lee et al. proposed a new focus measurement operator that transforms the coordinate system into a logarithmic polar coordinate system and uses the spatial characteristics of the polar coordinate system, which has good robustness to noise. Pradeep et al. proposed an improved SFF method that estimates the final shape of the reconstructed object by calculating the blur degree of defocus in the image sequence.

[0006] In summary, the improvement of the FM operator mainly lies in using more FM operators to determine the final focus value, or performing spatial transformation on the image. Some scholars have also studied the influence of image window size on reconstruction accuracy and believed that the most suitable window size is 5×5. Few people have studied how to obtain higher reconstruction accuracy at the minimum window size. Summary of the invention

[0007] To solve the above problems, the present invention provides a focused morphology recovery method using a minimum size window, which obtains a better three-dimensional morphology of the object under the condition of selecting the minimum window size, which greatly reduces the dependence of calculation accuracy on large-size windows.

[0008] A method for restoring a focused shape using a minimum size window comprises the following steps:

[0009] Acquire an image sequence of the object to be reconstructed;

[0010] The focus value corresponding to each pixel point on each frame image in the image sequence is obtained respectively, wherein the method for obtaining the focus value corresponding to any pixel point is: using the smallest window size 3×3 to extract a pixel area with the current pixel point as the window center, performing a focus value acquisition operation on the pixel area to obtain the focus value;

[0011] The maximum value of the focus values ​​corresponding to all pixels at the same position is taken as the depth value at each position to obtain the depth map of the object to be reconstructed.

[0012] Furthermore, the focus value acquisition operation is as follows:

[0013] Assume that the extracted pixel area R1 is:

[0014]

[0015] Among them, r 11 ~r 33 is the pixel value of each pixel in the pixel area R1;

[0016] Transpose R1 to obtain the transposed matrix R2:

[0017]

[0018] Expand R1 and R2 into column vectors respectively, and get column vector RW and column vector RC accordingly:

[0019] RW=[r 11 r 21 r 31 r 12 r 22 r 32 r 13 r 23 r 33 ] T

[0020] RC = [r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] T

[0021] Combine the column vector RW and the column vector RC to obtain the position vector RV = [RW, RC];

[0022] Get the center position P of the position vector RV;

[0023] The maximum distance between all positions in the position vector RV and the central position P is taken as the distribution radius d max ;

[0024] Obtain the adjacent distances between all positions of the position vector RV and their adjacent positions respectively;

[0025] Obtain the variance σ of the minimum value of the neighboring distance corresponding to all positions of the position vector RV;

[0026] Distribution radius d max The weighted sum of the variance σ is obtained, and the sum is the focus value corresponding to the current pixel.

[0027] Furthermore, the method for obtaining the center position P of the position vector RV is:

[0028]

[0029] in, is the positional average of the column vector RW, is the position average of the column vector RC, RWi is the i-th position in the column vector RW, RC j is the i-th position in the column vector RC.

[0030] Furthermore, the distance d between any position in the position vector RV and the center position P is i The calculation method is:

[0031]

[0032] Where i = 1, 2, ... 9, is the positional average of the column vector RW, is the position average of the column vector RC, RW i is the i-th position in the column vector RW, RC j is the i-th position in the column vector RC.

[0033] Furthermore, the distance D between any position in the position vector RV and its adjacent position ij The calculation method is:

[0034]

[0035] Among them, RW i is the i-th position in the column vector RW, RW j is the jth position in the column vector RW, RC i is the i-th position in the column vector RC, RC j is the j-th position in the column vector RC, and i≠j.

[0036] Furthermore, the variance σ is calculated as:

[0037]

[0038] Among them, D min,i is the minimum value of the neighboring distance corresponding to the i-th position in the column vector RW or column vector RC, and μ ... min,i The mean of .

[0039] Furthermore, the distribution radius d max The sum of the weights of and variance σ is 1.

[0040] Beneficial effects:

[0041] The present invention provides a focus morphology recovery method using a minimum-size window. First, the window matrix is ​​transformed to obtain a position vector. Then, the distribution radius of all positions in the position vector and the minimum value of the distance between each position and other positions are calculated, and the variance between the minimum values ​​is calculated. Finally, the weighted sum of the distribution radius and the variance is used as the focus value to determine the final depth map, thereby avoiding the influence of abnormal grayscale values ​​of the image on the calculated focus value. The smaller the window size, the more accurate the calculated focus value. That is to say, the present invention can calculate a more accurate depth map under the premise of using the minimum image window size, which greatly reduces the dependence of the calculation accuracy on the large-size window. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of a focusing morphology recovery method using a minimum size window provided by the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0044] The present invention proposes a focusing morphology restoration method using a minimum size window. First, the image window matrix is ​​transformed into a position vector. Then, the distribution radius of all positions in the position vector and the minimum distance between each position and other positions are calculated, and the variance of the minimum distance is calculated. Finally, the distribution radius and the variance are weighted and summed as the focus value to obtain a depth map of the object. The method includes the following steps:

[0045] Acquire an image sequence of the object to be reconstructed; for example, use a monocular vision system to collect a partially focused image sequence of the object at different height positions at equal intervals, and convert the collected image sequence from RGB format to a grayscale image;

[0046] The focus value corresponding to each pixel point on each frame image in the image sequence is obtained respectively, wherein the method for obtaining the focus value corresponding to any pixel point is: using the minimum window size 3×3 to extract a pixel area with the current pixel point as the window center, performing a focus value acquisition operation on the pixel area, and obtaining the focus value; it should be noted that the window size of the present invention is an odd number and is not equal to 1, then 3×3 is the minimum window size;

[0047] The maximum value of the focus values ​​corresponding to all pixels at the same position is taken as the depth value at each position to obtain the depth map of the object to be reconstructed.

[0048] Furthermore, the focus value acquisition operation is as follows:

[0049] Assume that the extracted pixel area R1 is:

[0050]

[0051] Among them, r 11 ~r 33 is the pixel value of each pixel in the pixel area R1;

[0052] Transpose R1 to obtain the transposed matrix R2:

[0053]

[0054] Expand R1 and R2 into column vectors respectively, and get column vector RW and column vector RC accordingly:

[0055] RW=[r 11 r 21 r 31 r 12 r 22 r 32 r 13 r 23 r 33 ] T

[0056] RC = [r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] T

[0057] Combine the column vector RW and the column vector RC to obtain the position vector RV = [RW, RC];

[0058] It should be noted that the focus value calculated by a smaller window is less accurate than that calculated by a larger window. The main reason is that a small window contains less information and the degree of distinction between clear and blurred areas is small. Therefore, the window matrix is ​​transformed, and the value of each position of the window matrix is ​​taken as x, and the value of each position of the transposed matrix of the window matrix is ​​taken as y. The position vector RV = [RW, RC] = (x, y) is obtained by combining them, where RW is taken as x and RC is taken as y.

[0059] Get the center position P of the position vector RV:

[0060]

[0061] in, is the positional average of the column vector RW, is the position average of the column vector RC, RWi is the i-th position in the column vector RW, RC j is the i-th position in the column vector RC;

[0062] The maximum distance between all positions in the position vector RV and the central position P is taken as the distribution radius d max ; Among them, the distance d between any position in the position vector RV and the center position P i The calculation method is:

[0063]

[0064] Where i = 1, 2, ... 9, is the positional average of the column vector RW, is the position average of the column vector RC, RW i is the i-th position in the column vector RW, RC j is the i-th position in the column vector RC;

[0065] It should be noted that the distribution radius can determine the degree of discreteness between each position. The grayscale values ​​of the clear area are quite different, and the discreteness is large. The grayscale values ​​of the blurred area are close, and the discreteness is small. Therefore, the center position of all positions is determined, and then the maximum distance from each position to the center position is calculated as the distribution radius.

[0066] Get the adjacent distances between all positions of the position vector RV and its adjacent positions respectively; the distance D between any position in the position vector RV and its adjacent positions ij The calculation method is:

[0067]

[0068] Among them, RW i is the i-th position in the column vector RW, RW j is the jth position in the column vector RW, RC i is the i-th position in the column vector RC, RC j is the j-th position in the column vector RC, and i≠j;

[0069] It should be noted that in a clear area, the gray value fluctuates greatly, and the distances between the generated position vectors vary. In a blurred area, the distances between the position vectors are small and the changes are not obvious. Therefore, the present invention needs to calculate the shortest distance between each position and other positions, and calculate the variance of the shortest distance. Specifically, the method for obtaining the variance σ of the minimum value of the neighboring distance corresponding to all positions of the position vector RV is as follows:

[0070]

[0071] Among them, D min,i is the minimum value of the neighboring distance corresponding to the i-th position in the column vector RW or column vector RC, and μ ... min,i The mean of

[0072] Distribution radius d max The weighted sum of the variance σ is obtained, and the sum is the focus value corresponding to the current pixel. Distribution radius d max The sum of the weights of and variance σ is 1.

[0073] It should be noted that when there is abnormal noise in the blurred image, the abnormal value will deviate from the center position of the position vector, resulting in a larger calculated distribution radius, which is inconsistent with the blurred state of the window. The method of the shortest distance between each position and other positions will not have this problem, because the center position is close to the abnormal noise and other positions. However, when the image is clear, there is a situation where the distance between each position and the center position is close, and the calculated variance is small, which is inconsistent with the clear state of the image, but the distribution radius is still large. Therefore, the present invention takes the weighted sum of the variance of the distribution radius and the shortest distance as the focus value.

[0074] In summary, the present invention relates to a focusing morphology restoration method using a minimum size window, and designs a focusing evaluation function for calculating the sharpness level of the window. The method uses a minimum size window to calculate the sharpness level of the window. The result of the calculation is called the focus value. The minimum window calculation is used to obtain the optimal result while improving the operation speed. The size of the minimum window used is determined to be 3×3. First, the window matrix is ​​transformed, the window matrix is ​​transposed, and a set of position vectors are formed at the corresponding positions of the original matrix. The position vector contains 9 position coordinates, and then the center point coordinates of the 9 positions are calculated, and the distance from each position to the center point is calculated. The maximum distance from the center point is taken as the distribution radius of the 9 positions. After that, the minimum distance from each position to other positions is calculated, and the variance of these minimum distances is calculated. The variance is weighted and summed with the previously calculated distribution radius as the calculated focus value, avoiding the influence of the abnormal gray value of the image on the calculated focus value. The method of the present invention uses two indicators to calculate the focus value. The smaller the window size, the more accurate the calculated focus value, which greatly reduces the dependence of the calculation accuracy on the large size window.

[0075] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may certainly make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for restoring focused morphology using a minimum size window, characterized in that: The following steps are involved: Acquire an image sequence of the object to be reconstructed; The focus value corresponding to each pixel point on each frame image in the image sequence is obtained respectively, wherein the method for obtaining the focus value corresponding to any pixel point is: using the smallest window size 3×3 to extract a pixel area with the current pixel point as the window center, performing a focus value acquisition operation on the pixel area to obtain the focus value; The maximum value of the focus values ​​corresponding to all pixels at the same position is taken as the depth value at each position to obtain the depth map of the object to be reconstructed.

2. A method for restoring focused morphology using a minimum size window as claimed in claim 1, characterized in that: The focus value acquisition operation is as follows: Assume that the extracted pixel area R1 is: Among them, r 11 ~r 33 is the pixel value of each pixel in the pixel area R1; Transpose R1 to obtain the transposed matrix R2: Expand R1 and R2 into column vectors respectively, and get column vector RW and column vector RC accordingly: RW=[r 11 r 21 r 31 r 12 r 22 r 32 r 13 r 23 r 33 ] T RC=[r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 ] T Combine the column vector RW and the column vector RC to obtain the position vector RV = [RW, RC]; Get the center position P of the position vector RV; The maximum distance between all positions in the position vector RV and the central position P is taken as the distribution radius d max ; Obtain the adjacent distances between all positions of the position vector RV and their adjacent positions respectively; Obtain the variance σ of the minimum value of the neighboring distance corresponding to all positions of the position vector RV; Distribution radius d max The weighted sum of the variance σ is obtained, and the sum is the focus value corresponding to the current pixel.

3. A method for restoring focused morphology using a minimum size window as claimed in claim 2, characterized in that: The method for obtaining the center position P of the position vector RV is: in, is the positional average of the column vector RW, is the position average of the column vector RC, RW i is the i-th position in the column vector RW, and RCj is the i-th position in the column vector RC.

4. A method for restoring focused morphology using a minimum size window as claimed in claim 2, characterized in that: The distance d between any position in the position vector RV and the center position P i The calculation method is: Where i = 1, 2, ... 9, is the positional average of the column vector RW, is the position average of the column vector RC, RW i is the i-th position in the column vector RW, RC j is the i-th position in the column vector RC.

5. A method for restoring focused morphology using a minimum size window as claimed in claim 2, characterized in that: The distance D between any position in the position vector RV and its neighboring positions ij The calculation method is: Among them, RW i is the i-th position in the column vector RW, RW j is the jth position in the column vector RW, RC i is the i-th position in the column vector RC, RC j is the j-th position in the column vector RC, and i≠j.

6. A method for restoring focused morphology using a minimum size window as claimed in claim 2, characterized in that: The calculation method of variance σ is: Among them, D min,i is the minimum value of the neighboring distance corresponding to the i-th position in the column vector RW or column vector RC, and μ ... min，i The mean of .

7. A method for restoring focused morphology using a minimum size window as claimed in claim 2, characterized in that: Distribution radius d max The sum of the weights of and variance σ is 1.

Images