A dual-modal focusing analysis method based on re-blurring theory
The dual-modal focusing analysis method based on re-blurring theory addresses sensitivity and robustness issues in existing methods by combining spatial and frequency domain edges, improving focus evaluation accuracy and efficiency.
Patent Information
- Application Number
- CN202210519790.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-13
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2042-05-13
AI Technical Summary
Existing image focusing methods lack sensitivity, robustness, and computational efficiency, particularly in low-contrast and high scenarios, and generalization ability, and are limited by the inability to perceive edge changes and are not effectively solve the sensitivity and robustness issues of low-contrast images, and have high time complexity and limited generalization ability.
A dual-modal focusing analysis method based on re-blurring theory, which involves blurring the grayscale image to be analyzed, and the high-frequency subbands of the blurred image, and calculating the variance matrices and mean variance differences of the spatial and frequency domain edges to determine the focus position.
The method improves the sensitivity, robustness, and accuracy of focus evaluation by combining spatial and frequency domain edges, enhancing noise resistance and computational efficiency, and achieving exponential performance improvements in clarity ratio and peak sensitivity.
Smart Images
Figure CN114972141B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of image analysis, and in particular to a dual-mode focusing analysis method of re-blurring theory. Background Art
[0002] Focused images are generally rich in details, with obvious edge changes in the spatial domain and rich high-frequency information in the frequency domain. Defocused images are relatively blurred and difficult to observe details. The edge changes in the spatial domain are not obvious, and the high-frequency components in the frequency domain are relatively small. At present, the focus of an image is roughly studied from four directions: statistics, grayscale difference, frequency domain, and machine learning. The statistical focus evaluation method is mainly based on the statistical law of the grayscale value of the image. The evaluation functions used are variance function and entropy function. This type of method lacks the ability to perceive edge changes and is limited by the grayscale of the image. It has poor sensitivity and lacks robustness in the evaluation of low-contrast images. The evaluation function based on grayscale difference adopts the calculation method of grayscale gradient. The functions used are mainly Sobel, Robert, SML, Laplacian, SMD, and Brenner functions. This type of method solves the shortcomings of statistical methods in edge transformation sensitivity to a certain extent, but on low-contrast images, due to the small grayscale difference value, it also lacks a certain robustness. Frequency domain-based evaluation methods mainly include Fourier transform, wavelet transform, and discrete cosine, etc. They use high-frequency information of the image to determine whether it is focused, solving the sensitivity and robustness problems of statistics and grayscale differences on low-contrast images, but there is a problem of low computational efficiency. Machine learning-based methods require the construction of a network model in advance. Although the robustness of the focusing method has been improved to a certain extent, the time complexity is also relatively high. Secondly, the application scenario is related to the training set, and the generalization ability needs to be strengthened. Summary of the invention
[0003] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art and propose a dual-modal focusing analysis method based on re-blurring theory, which has better sensitivity and robustness than traditional single-modal functions, improves the universality of the focusing evaluation method, has good sensitivity and noise resistance, and improves the accuracy of focusing analysis.
[0004] The present invention adopts the following technical solution:
[0005] A dual-modal focusing analysis method of re-blurring theory, including:
[0006] The grayscale image to be analyzed is blurred using convolution:
[0007] Performing wavelet transform on the image to be analyzed and the blurred image respectively, obtaining the low-frequency subband of the image to be analyzed, the high-frequency subband of the image to be analyzed, the low-frequency subband of the blurred image and the high-frequency subband of the blurred image, and superimposing the high-frequency subband of the image to be analyzed to obtain the high-frequency edge of the image to be analyzed, and superimposing the high-frequency subband of the blurred image to obtain the high-frequency edge of the blurred image;
[0008] Extracting the spatial domain edges of the image to be analyzed and the blurred image, and calculating the edge map of the image to be analyzed and the edge map of the blurred image;
[0009] The image to be analyzed and the re-blurred image are subjected to similarity calculation, and the variance matrix of the extracted low-frequency texture of the image to be analyzed, the low-frequency texture of the blurred image, the high-frequency edge of the image to be analyzed, the high-frequency edge of the blurred image, the edge map of the image to be analyzed, and the edge map of the blurred image is calculated using a sliding window of a mean matrix;
[0010] Perform difference operation on the variance matrices of the low-frequency texture of the image to be analyzed, the low-frequency texture of the blurred image, the high-frequency edge of the image to be analyzed, the high-frequency edge of the blurred image, the edge map of the image to be analyzed, and the edge map of the blurred image to obtain the low-frequency texture variance difference matrix, the high-frequency edge variance difference matrix, and the spatial edge variance difference matrix of the image to be analyzed and the blurred image, and sum and calculate the mean of the low-frequency texture variance difference matrix, the high-frequency edge variance difference matrix, and the spatial edge variance difference matrix to obtain the low-frequency texture mean variance difference, the high-frequency edge mean variance difference, and the spatial edge mean variance difference, and finally calculate the focusing score;
[0011] According to the focus score, the focus position of the image is obtained.
[0012] Specifically, the grayscale image to be analyzed is blurred by convolution, specifically:
[0013] P2(x,y)=P1(x,y)*h(x,y)
[0014] Where: P2(x,y) is the blurred image; P1(x,y) is the image to be analyzed; h(x,y) is the convolution kernel, x is the horizontal coordinate, and y is the vertical coordinate.
[0015] Specifically, the convolution kernel includes but is not limited to a Gaussian kernel, a Cauchy kernel, and a Gaussian-Cauchy kernel.
[0016] Specifically, the spatial edges of the image to be analyzed and the blurred image are extracted, and the edge map of the image to be analyzed and the edge map of the blurred image are calculated, wherein the spatial edges of the image to be analyzed and the blurred image are extracted using methods including but not limited to sobel operator, canny operator or threshold calculation.
[0017] Specifically, wavelet transform is performed on the image to be analyzed and the blurred image respectively, wherein the wavelet feature decomposition is primary decomposition or multi-level decomposition.
[0018] Specifically, the similarity calculation is performed on the image to be analyzed and the blurred image, and the similarity calculation method is not limited to a cosine similarity method, a histogram method or SSIM.
[0019] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:
[0020] (1) The present invention adopts the re-blurring theory to realize the self-comparison of reference-free images, reduces the actual requirements of the evaluation method on the application scenarios and measurement samples, and improves the universality of the focus evaluation method.
[0021] (2) The present invention adopts wavelet decomposition, which can better resist the influence of factors such as uneven illumination, mine the frequency domain information on the grayscale gradient, and has good sensitivity and noise resistance.
[0022] (3) The present invention combines the edges of two different modalities, spatial domain and frequency domain, and combines the spatial edge changes of the image to mine the grayscale gradient information of the image itself, thereby improving the accuracy of focus evaluation.
[0023] (4) The method proposed in the present invention can obtain an evaluation effect that is superior to various existing single-modal evaluation methods in a microscopic measurement system, with an exponential performance improvement in clarity ratio, and a significant improvement in peak sensitivity and steepness, indicating that the present method has excellent sensitivity and resolution. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 A flow chart of a method provided by an embodiment of the present invention;
[0025] Figure 2 A schematic diagram of a test sample provided by an embodiment of the present invention;
[0026] FIG. 3 is a diagram showing the comparison results between the method of the present invention and the prior art method provided by an embodiment of the present invention.
[0027] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION
[0028] like Figure 1 The following is a dual-mode focusing analysis method (Reblur_2Mode) based on a reblurring theory according to an embodiment of the present invention. The main steps are as follows:
[0029] S101: The grayscale image to be evaluated (if it is color, it is grayscaled first) is blurred by convolution:
[0030] P2(x,y)=P1(x,y)*h(x,y)
[0031] Where: P2(x,y) is the blurred image; P1(x,y) is the image to be evaluated; h(x,y) is the convolution kernel.
[0032] S102: Perform wavelet transform on the image to be evaluated P1(x, y) and the blurred image P2(x, y) respectively to obtain the low-frequency subband (low-frequency texture) P of the image to be evaluated 1LL (x, y), high-frequency subband and low-frequency subband of blurred image (low-frequency texture) P 2LL (x, y), high-frequency sub-bands, and superimpose each high-frequency sub-band to obtain the high-frequency edge P of the image to be evaluated 1HH (x,y) and the high-frequency edge P of the blurred image 2HH (x,y).
[0033] S103: Extract the spatial edge of the original image and the blurred image, and calculate the edge map P of the original image 1edge (x,y) and the edge map P of the blurred image 2edge (x,y).
[0034] S104: Calculate the similarity between the image to be analyzed and the blurred image, and use a mean matrix of a certain size and a sliding window to calculate the low-frequency texture P of the image to be evaluated in the second step. 1LL (x,y) and the low-frequency texture P of the blurred image 2LL (x, y), the high-frequency edge P of the image to be evaluated in the second step 1HH (x,y) and the high-frequency edge P of the blurred image 2HH (x, y), the third step is the spatial edge P of the image to be evaluated 1edge (x,y) and the edge map P of the blurred image 2edge (x,y), the variance matrix (sliding window is not repeated) σ 1LL (x,y),σ 2LL (x,y),σ 1HH (x,y),σ 2HH (x,y),σ 1edge (x,y),σ 2edge (x, y), if the original 253x253 image matrix is converted into a 23x23 variance matrix after the 11x11 sliding window operation.
[0035] There are many methods to calculate image similarity, such as cosine similarity, histogram, SSIM, etc.
[0036] S105: σ obtained by combining the original image and the blurred image 1LL (x,y),σ 2LL(x,y),σ 1HH (x,y),σ 2HH (x,y),σ 1edge (x,y),σ 2edge The six variance matrices (x, y) are interpolated to obtain the variance difference matrix Δσ of the two images regarding low-frequency texture, high-frequency edge, and spatial edge. LL (x,y),Δσ HH (x,y),Δσ edge (x, y), and sum the three matrices to calculate the mean, and get the mean variance difference Δσ about low-frequency texture, high-frequency edge, and spatial edge LL_mean , Δσ HH_mean , Δσ edge_mean , and finally calculate the focus score.
[0037] S106: Determine the focus position of the image according to the focus score.
[0038] During the image focusing process, an image is acquired at certain step intervals, and each image is evaluated once, and an evaluation curve of the focusing process is obtained. When the score decreases after passing through the maximum point, the image index number corresponding to the maximum point is the focusing position.
[0039] Specifically, the convolution kernel includes but is not limited to a Gaussian kernel, a Cauchy kernel, and a Gaussian-Cauchy kernel.
[0040] Specifically, the spatial edges of the image to be analyzed and the blurred image are extracted, and the edge map of the image to be analyzed and the edge map of the blurred image are calculated, wherein the spatial edges of the image to be analyzed and the blurred image are extracted using methods including but not limited to sobel operator, canny operator or threshold calculation.
[0041] Specifically, wavelet transform is performed on the image to be analyzed and the blurred image respectively, wherein the wavelet feature decomposition is primary decomposition or multi-level decomposition. Specific embodiment:
[0043] First, a series of equal-step images are obtained, including two types of images, one is a single background image and the other is a large step change image.
[0044] In the actual measurement process, if the surface morphology of the measured object does not change much, the background can be considered to be single, and the best focusing surface has a unique value, that is, the focusing score curve has only a maximum value of one.
[0045] When there are non-negligible large step changes on the surface of the measured object, that is, when the focusing surface does not have a unique value, the measured object can be considered as a complex background, that is, the obtained focusing score curve has multiple maximum points.
[0046] The convolution kernel uses Gaussian kernel, the wavelet uses first-level decomposition, the spatial edge uses a combination of Sobel and Robert, and compares Sobel, Robert, SML, Laplacian, SMD, SMD2, DCT, Energy, and Brenner functions. Figure 2 A schematic diagram of a test sample provided by an embodiment of the present invention.
[0047] In the image sequence of a single background, the evaluation values of each analysis method are calculated to obtain Figure 3 (a). It can be seen that, except for Brenner, all other focus evaluation functions can accurately determine the image focus position. For example, in a single background image sequence of 50 images, each focus evaluation function has an extreme value in the 24th image, showing a good evaluation result, but the method proposed in the present invention has a faster convergence speed, a shorter half-height width, and is easier to determine the image focus position. As shown in Figure 3 (b), in the boundary image with large step changes, SMD, SMD2, and Brenner do not obtain a maximum value on the first focus plane, and Laplacian does not obtain a maximum value on the second focus plane. At the same time, the Laplacian focus evaluation function has multiple peaks on the large defocus scene image, which is not suitable for cross-scale step dual-surface focus evaluation. Other methods can accurately determine the dual-plane focus position of the step, but in the defocus area, the normalized result of the evaluation method proposed in the present invention is infinitely close to zero, which is better than other methods. The evaluation of each analysis method is analyzed using a special evaluation index.
[0048] Performance evaluation indicators include:
[0049] 1. Clarity ratio
[0050] The clarity ratio is an indicator that reflects the difference in clarity recognition ability. The larger the clarity ratio, the more significant the difference between the curve height and the lower the curve, the stronger the algorithm's ability to distinguish between clear and blurred images, and the easier it is to achieve accurate focus. The clarity ratio calculation formula is:
[0051]
[0052] 2. Sensitivity factor
[0053] The expression of sensitivity factor is shown in the formula, sensitivity factor S sensitivity It is used to indicate the drastic degree of change of the vertical coordinate function value near the maximum value.
[0054]
[0055] Where: Score max(i) represents the maximum point in the curve, and i represents the position of the image in the sequence; Score(i+1) represents the clarity score value of the next clearest image.
[0056] 3. Steepness
[0057] The expression of steepness is shown in the formula, S abruptness It is used to indicate the steepness of the curve and can characterize the resolution ability of the focus evaluation function for images with different defocus.
[0058]
[0059] Where: Score(i1) represents the evaluation value of the last image whose left side is less than the fixed threshold, and its index number is i1; Score(i2) represents the evaluation value of the first image whose right side is less than the fixed threshold, and its index number is i2. In the single-peak gradient evaluation curve, the threshold is generally 0.1.
[0060] The clarity ratio, peak sensitivity and steepness of the focus evaluation in Figure 3 are calculated, and the results in Table 1-3 are obtained. For a bimodal image, the clarity ratio and sensitivity factor are calculated separately on the left side of the left peak and the right side of the right peak, while the steepness is calculated on one side of the peak. The specific results are as follows:
[0061] Table 1 Clarity ratio
[0062]
[0063] Table 2 Peak sensitivity
[0064]
[0065]
[0066] Table 3 Steepness
[0067]
[0068] It can be seen from the data in the table that under different measurement conditions, the clarity ratio, peak sensitivity and steepness of the focus evaluation curve of the proposed Reblur_2Mode method are the largest. In a single background image, the sensitivity ratio of the Reblur_2Mode method is at least several orders of magnitude higher than that of Sobel, SML, Laplacian, SMD, SMD2, DCT, Robert, Energy, and Brenner. The peak sensitivity is 2.18, 1.89, 1.07, 2.28, 2.27, 1.88, 2.17, 2.15, and 1.71 higher than that of Sobel, SML, Laplacian, SMD, SMD2, DCT, Robert, Energy, and Brenner, respectively. The steepness is 0.28, 0.01, 0.33, 0.33, and 0.36 higher than that of SML, Laplacian, DCT, Energy, and Brenner, respectively. At the same time, in the focused image sequence with step transformation, the proposed Reblur_2Mode method still maintains good sensitivity and convergence characteristics compared with Sobel, Robert, SML, Laplacian, SMD, SMD2, DCT, Energy, and Brenner, and can quickly detect the focus position.
[0069] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.
Claims
1. A dual-modal focusing analysis method based on re-blurring theory, characterized in that: include: The grayscale image to be analyzed is blurred using convolution; Performing wavelet transform on the image to be analyzed and the blurred image respectively, obtaining the low-frequency subband of the image to be analyzed, the high-frequency subband of the image to be analyzed, the low-frequency subband of the blurred image and the high-frequency subband of the blurred image, and superimposing the high-frequency subband of the image to be analyzed to obtain the high-frequency edge of the image to be analyzed, and superimposing the high-frequency subband of the blurred image to obtain the high-frequency edge of the blurred image; Extracting the spatial domain edges of the image to be analyzed and the blurred image, and calculating the edge map of the image to be analyzed and the edge map of the blurred image; The similarity between the image to be analyzed and the blurred image is calculated, and the variance matrix of the extracted low-frequency texture of the image to be analyzed, the low-frequency texture of the blurred image, the high-frequency edge of the image to be analyzed, the high-frequency edge of the blurred image, the edge map of the image to be analyzed, and the edge map of the blurred image is calculated using a sliding window of a mean matrix; Perform difference operation on the variance matrices of the low-frequency texture of the image to be analyzed, the low-frequency texture of the blurred image, the high-frequency edge of the image to be analyzed, the high-frequency edge of the blurred image, the edge map of the image to be analyzed, and the edge map of the blurred image to obtain the low-frequency texture variance difference matrix, the high-frequency edge variance difference matrix, and the spatial edge variance difference matrix of the image to be analyzed and the blurred image, and sum and calculate the mean of the low-frequency texture variance difference matrix, the high-frequency edge variance difference matrix, and the spatial edge variance difference matrix to obtain the low-frequency texture mean variance difference, the high-frequency edge mean variance difference, and the spatial edge mean variance difference, and finally calculate the focusing score; According to the focus score, the focus position of the image is obtained.
2. The dual-modal focusing analysis method of the re-blurring theory according to claim 1, characterized in that: The grayscale image to be analyzed is blurred by convolution, specifically: P2(x,y)=P1(x,y)*h(x,y) Where: P2(x,y) is the blurred image; P1(x,y) is the image to be analyzed; h(x,y) is the convolution kernel, x is the horizontal coordinate, and y is the vertical coordinate.
3. The dual-modal focusing analysis method of the re-blurring theory according to claim 2, characterized in that: The convolution kernel includes a Gaussian kernel, a Cauchy kernel or a Gaussian-Cauchy kernel.
4. The dual-modal focusing analysis method of the re-blurring theory according to claim 1, characterized in that: The spatial domain edges of the image to be analyzed and the blurred image are extracted, and the edge map of the image to be analyzed and the edge map of the blurred image are calculated, wherein the spatial domain edges of the image to be analyzed and the blurred image are extracted using a sobel operator, a canny operator or a threshold calculation.
5. The dual-modal focusing analysis method of the re-blurring theory according to claim 1, characterized in that: Wavelet transform is performed on the image to be analyzed and the blurred image respectively, wherein the wavelet feature decomposition is one-level decomposition or multi-level decomposition.
6. The dual-modal focusing analysis method of the re-blurring theory according to claim 1, characterized in that: The similarity calculation is performed on the image to be analyzed and the blurred image, and the similarity calculation method includes a cosine similarity method, a histogram method or SSIM.