An edge detection method and apparatus based on adaptive gradient threshold Canny operator
By using the adaptive gradient thresholding Canny operator, combined with median filtering, Gaussian filtering, and gray-level histogram, the Otsu method calculates the gray-level threshold, solving the problems of numerous false edges and blurred edges in lithographic SEM images, and achieving more accurate edge detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-06
- Publication Date
- 2026-03-06
AI Technical Summary
The existing Canny operator suffers from problems such as numerous false edges, low edge extraction accuracy, and blurred edges when detecting edges in photolithographic SEM images. Furthermore, the blurring effect of Gaussian filtering on edges is not fully considered.
An adaptive gradient thresholding Canny operator is adopted, and noise is removed by median filtering and Gaussian filtering. The gray-level threshold is calculated by combining gray-level histogram and Otsu's method. The high and low gradient thresholds are automatically determined, and the Canny operator is used for edge recognition, taking into account the blurring effect of Gaussian filtering on the edges.
It improves the accuracy of edge detection, reduces false edges, avoids edge blurring, and achieves better edge detection results.
Smart Images
Figure CN115797300B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of edge detection technology, and more specifically to an edge detection method based on the adaptive gradient threshold Canny operator and an apparatus for implementing the method. Background Technology
[0002] In the field of semiconductor integrated circuits, with the continuous advancement of photolithography technology, the size of key circuit boards or devices is constantly shrinking. Scanning electron microscopy (SEM), as a high-resolution observation method, provides crucial metrological data for evaluating photolithography quality through CD-SEM images. Image contours, as an important part of characterizing image information, are helpful for model-based OPC and defect detection techniques. However, relatively simple edge operators such as Sobel, Prewitt, and Log often fail to achieve the expected results in edge detection. Compared to these simpler single edge operators, the Canny operator, proposed by John Canny in 1986, has become one of the most recognized and effective methods in traditional edge detection algorithms. Therefore, edge detection based on the Canny operator has good applicability in contour extraction from SEM images. However, since the high and low gradient threshold settings of the Canny operator depend on the operator's experience, multiple trials are needed for different images to determine the final threshold, making the processing of photolithography SEM images with large sample sizes quite complex.
[0003] Due to the limitations of the Canny operator's gradient threshold setting, a series of algorithms have been developed for automatically selecting the Canny operator's threshold:
[0004] [1] Wang Zhi, He Saixian. An adaptive edge detection method based on Canny theory [J]. Journal of Image and Graphics, 2004(08):65-70. The method proposes to divide the whole image into several sub-images and generate a dynamic threshold adaptively based on the edge gradient information of each sub-image and the global edge gradient feature information, which improves the automation of edge detection.
[0005] [2] Duan Jun, Gao Xiang. An adaptive double threshold improved Canny operator edge detection algorithm based on statistical filtering [J]. Laser Journal, 2015, 36(01):10-12. DOI:10.14016 / j.cnki.jgzz.2015.01.010. A threshold selection algorithm based on simple statistics is proposed.
[0006] [3] Hu Jingyi, Kuang Yao, Yu Zheng, Wang Jingjing, Zhang Lanlan, Liu Yunfei. Threshold-adaptive Canny operator for images with large luminance variations [J]. Mechanical & Electronic, 2022, 40(08): 18-22+27. The Otsu method was used to calculate the high and low thresholds.
[0007] [4] Li Mu, Yan Jihong, Li Ge, Zhao Jie. Adaptive Canny operator edge detection technology [J]. Journal of Harbin Engineering University, 2007(09):1002-1007. The existence of high and low thresholds obtained by the above OTSU algorithm is theoretically proven.
[0008] [5] Wang Xiaojun, Liu Xumin, Guan Yong. Image edge detection algorithm based on improved Canny operator [J]. Computer Engineering, 2012, 38(14):196-198+202. The optimal high and low thresholds are calculated by iterative algorithm.
[0009] In summary, the existing technology divides an image into two parts—target and background—using an iterative algorithm to calculate a threshold. The average grayscale values of the target and background are then used as the high and low thresholds for the Canny operator. The specific algorithm can be described as follows: The average of the minimum and maximum grayscale values is obtained through histogram statistics and used as the initial threshold. All grayscale values are then divided into two parts: one part higher than the initial threshold and the other lower. The average of these two parts is calculated, and the average of the two averages is calculated again to obtain the threshold after one iteration. This threshold is compared with the initial threshold. If the two thresholds are equal or differ by a certain relationship, the iteration ends, and the obtained threshold is the optimal threshold. If not, this threshold is used to divide all grayscale values, and the above steps are repeated. Finally, the average of the two parts separated by this threshold is the gradient high and low threshold.
[0010] The shortcomings of existing technologies: In existing improved algorithms for the Canny operator, the mainstream approach is to use the gradient magnitude histogram of the image and Otsu's method to divide the gradient magnitude into three categories, finding the high and low thresholds that maximize the inter-class variance. However, in practical applications, because the number of pixels of non-contour points in lithographic SEM images is much greater than the number of pixels of contour points, its gradient histogram only shows a single-peak distribution with low gradient peaks. If Otsu's method is used to calculate based on this histogram, the results will be concentrated near the peak and generate a large number of false edges, greatly affecting the accuracy of edge extraction. As mentioned above, the image edge detection algorithm based on the improved Canny operator uses the method of obtaining high and low thresholds based on the gray-level average of the background and the target. However, since the high and low thresholds of the Canny operator are different from the gray-level thresholds of the image, directly using the average of the target and the background as the high and low thresholds of the Canny operator has not been theoretically justified. Furthermore, while image filtering can reduce noise to some extent, the edge and noise information in an image are both high-frequency information. Therefore, the denoised image will cause the edges to be blurred, which will affect the subsequent edge detection effect. The current algorithm does not take this factor into account. Summary of the Invention
[0011] This invention provides an edge detection method and apparatus based on the adaptive gradient threshold Canny operator, which solves the problems of numerous false edges, low edge extraction accuracy, blurred edges, and poor edge detection effect in the prior art. It can make full use of the gray-level distribution information of the image and unify the inherent relationship between the gray-level distribution and gradient distribution of the image with the built-in method of the Canny operator.
[0012] In a first aspect, an edge detection method based on the adaptive gradient threshold Canny operator is provided, which includes the following steps: acquiring the original image; preprocessing the original image and removing noise from the image using median filtering and Gaussian filtering; converting the filtered image into a grayscale image and obtaining the grayscale histogram of the grayscale image, and calculating the grayscale threshold based on the grayscale histogram; calculating the grayscale mean of the target and the background with the grayscale threshold T as the boundary; determining the high gradient threshold and the low gradient threshold based on the grayscale mean of the target and the background; and performing edge recognition using the Canny operator and the calculated high and low gradient thresholds to obtain an edge detection image.
[0013] Based on the above scheme, this invention proposes a reasonable adaptive thresholding Canny algorithm, which requires that the grayscale threshold for thresholding and the high and low gradients of the Canny operator be automatically selected. Furthermore, since the Canny edge detection method requires pre-processing the image with Gaussian filtering, the gradient threshold of this invention's algorithm takes into account the blurring effect of Gaussian filtering on edges.
[0014] In one possible implementation, the method further includes: a preprocessing step of the original image, specifically performing 3D processing on the original image. Median filtering is applied within a 3-region area to obtain the median-filtered image; a closing operation is performed on the filtered image; an opening operation is performed on the filtered image; a Gaussian filter function is constructed to apply a 3-dimensional filtering operation to the image after the opening and closing operations. The image is then filtered and denoised again within region 3 to obtain the Gaussian noise-processed image.
[0015] In one possible implementation, the method further includes: calculating a grayscale threshold, specifically drawing a grayscale histogram of the grayscale image, where the horizontal axis represents the grayscale value and the vertical axis represents the number of pixels; obtaining the peak value of the grayscale histogram; and setting initial left and right boundaries [L]. i ,R i ], where i is the iteration number, initially i = 0, i.e. L0 = 0, R0 = 255; the Otsu method is applied to the grayscale image to obtain the threshold T0.
[0016] In one possible implementation, the method further includes: dividing the grayscale image into [L] segments based on grayscale level, using a threshold T0 as a boundary. i Ti and [T i , R i are divided into two parts, denoted as the target and the background. The means u1 i , u2 i of these two parts are obtained respectively; determine the next iteration boundary [L i , R i ; perform Otsu's method on the grayscale image within [L i+1 , R i+1 to obtain the threshold T i ; set the threshold difference to T min , and judge |T i - T i-1 | >= T i . If not, the program terminates and outputs T i-1 .
[0017] In a possible implementation, the method further includes: the process of obtaining the peak of the grayscale histogram. Specifically, it can be: assume the peak grayscale is k, and the corresponding number of pixel points is p(k), which includes: p(k) is a maximum value, that is: p(k) > p(k - 1) and p(k) > p(k + 1); p(k) is the maximum value within the grayscale [k - rang1: k + rang2]. If there is another grayscale within the value range of rang1 and rang2 whose number of pixels is equal to p(k), then take the smaller grayscale value k as the peak grayscale; the proportion of p(k) in the total number of pixels is greater than 0.2%.
[0018] In a possible implementation, the method further includes: the process of determining the next iteration boundary [L i , R i . Specifically, it can be: if i > [2 / m], L i+1 = L i , R i+1 = R i ; if i <= [2 / m], then further judgments include: if T i < u1, then L i+1 = T i , otherwise L i+1 = u1; if T m-(i+1) < u1, then R i+1 = T m-(i+1) , otherwise R i+1 = u2.
[0019] In one possible implementation, the method further includes: determining a high gradient threshold and a low gradient threshold, specifically: dividing the grayscale image into a target image and a background image based on the threshold, and setting the grayscale mean values of the target image and the background image to u1 and u2; in the Canny operator, the eight-neighbor gradient Sobel operator used is Gy, Gx; assuming that the edges within the small region are straight edges, the magnitude of the gradient vectors is obtained by adding the gradient vectors, expressed as formula (1):
[0020] G= (1)
[0021] Its maximum amplitude is expressed by formula (2):
[0022] G max = 4 |u1-u2| (2)
[0023] The maximum amplitude G max =4 0.726 |u1-u2| as the high gradient threshold T h , set the low gradient threshold T l Set to 0.726 |u1-u2|.
[0024] In one possible implementation, the method further includes: an edge recognition process using the Canny operator and calculated high and low gradient thresholds, specifically calculating the gradient magnitude and gradient direction of edge pixels; performing non-maximum suppression on the gradient magnitude based on the gradient direction, wherein the gradient magnitude of the eight-neighbor center pixel of each pixel is compared with the gradient magnitude of the two pixels along its corresponding gradient direction, and if the eight-neighbor center pixel is the maximum value, it is retained; otherwise, the center pixel is set to 0 to obtain a refined edge;
[0025] In one possible implementation, the method further includes: setting the gradient magnitude to a value greater than a high gradient threshold T. h Pixels with gradient magnitudes less than the low gradient threshold T are used as edge points. l Pixels that are not edge points are considered edge points. Other pixels are considered edge points if there are edge points in their eight neighborhoods, otherwise they are considered non-edge points.
[0026] Based on the above solution, the present invention has the following effects:
[0027] 1. This invention can accurately determine the peak value of the gray-level histogram applicable to iteration conditions;
[0028] 2. This invention can avoid the difference between the left and right boundaries being too small in two iterations;
[0029] 3. This invention can automatically calculate the grayscale threshold for threshold segmentation;
[0030] 4. This invention can automatically calculate the high and low gradient thresholds of the Canny operator;
[0031] 5. This invention avoids the blurring effect of Gaussian filtering on edges.
[0032] Secondly, the present invention also provides an edge detection device based on the adaptive gradient threshold Canny operator, which includes: an image acquisition unit for acquiring an original image;
[0033] The image processing unit is used to preprocess the original image and remove noise from the image using median filtering and Gaussian filtering.
[0034] The grayscale thresholding unit is used to convert the filtered image into a grayscale image, obtain the grayscale histogram of the grayscale image, and calculate the grayscale threshold based on the grayscale histogram.
[0035] The grayscale mean unit is used to calculate the grayscale mean of the target and the background, with the grayscale threshold T as the boundary.
[0036] High and low threshold units are used to determine high and low gradient thresholds based on the gray-scale mean of the target and background.
[0037] The edge detection unit is used to perform edge recognition using the Canny operator and the calculated high and low gradient thresholds to obtain an edge detection image.
[0038] Based on the above scheme, the present invention uses an edge detection device composed of an image acquisition unit, an image processing unit, a gray level thresholding unit, a gray level mean unit, a high and low thresholding unit, and an edge detection unit to unify the inherent correlation between the gray level distribution and gradient distribution of an image and the built-in method of the Canny operator, thereby solving the problems of numerous false edges, low edge extraction accuracy, blurred edges, and poor edge detection effect in the prior art.
[0039] Thirdly, an electronic device is provided, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the edge detection method based on the adaptive gradient threshold Canny operator described above.
[0040] Fourthly, a storage medium is provided, on which a computer program is stored, which, when executed by a processor, is used to implement the steps of the edge detection method based on the adaptive gradient threshold Canny operator described above.
[0041] Based on the above solution, the present invention provides an electronic device and a storage medium, comprising: one or more memories and one or more processors. The memories are used to store program code and intermediate data generated during program execution, storage of model output results, and storage of the model and model parameters; the processors are used for the processor resources occupied by the code execution and multiple processor resources occupied when training the model.
[0042] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description
[0043] Figure 1 This is a flowchart of an embodiment of an edge detection method based on the adaptive gradient threshold Canny operator of the present invention.
[0044] Figure 2 This is a schematic diagram of the ideal target and background grayscale distribution in an embodiment of an edge detection method based on the adaptive gradient threshold Canny operator of the present invention.
[0045] Figure 3 This is a schematic diagram of Gy, representing an embodiment of the edge detection method based on the adaptive gradient threshold Canny operator of the present invention.
[0046] Figure 4 This is a schematic diagram of Gx in an embodiment of an edge detection method based on the adaptive gradient threshold Canny operator of the present invention.
[0047] Figure 5 This is about 3 in an embodiment of the edge detection method based on the adaptive gradient threshold Canny operator of the present invention. A schematic diagram of a Gaussian filter kernel.
[0048] Figure 6 This is a schematic diagram of the gray-level distribution of the first eight neighborhoods in an embodiment of an edge detection method based on the adaptive gradient threshold Canny operator of the present invention.
[0049] Figure 7 This is a schematic diagram of the eight-neighbor distribution after filtering in an embodiment of an edge detection method based on the adaptive gradient threshold Canny operator of the present invention.
[0050] Figure 8 This is a schematic diagram of an embodiment of an edge detection device based on the adaptive gradient threshold Canny operator of the present invention. Detailed Implementation
[0051] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0052] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0053] First, some of the terms and related technologies used in this application will be explained in conjunction with the accompanying drawings to facilitate understanding by those skilled in the art.
[0054] Canny operator: The Canny edge detection operator is a multi-level edge detection algorithm developed by John F. Canny in 1986. Its goal is to find an optimal edge detection algorithm. This algorithm is considered by many to be the optimal edge detection algorithm, and its accuracy in identifying image edges is much higher than other edge detection algorithms.
[0055] Gray-level histogram: A gray-level histogram is a function of the gray-level distribution, representing a statistical representation of the gray-level distribution in an image. It calculates the frequency of occurrence of all pixels in a digital image according to their gray-level values. In essence, a gray-level histogram is a function of gray levels, indicating the number of pixels in an image with a specific gray level, reflecting the frequency of that particular gray level.
[0056] Median filtering: Median filtering is a non-linear smoothing technique that sets the gray value of each pixel to the median of the gray values of all pixels within a certain neighborhood window of that pixel.
[0057] Gaussian filtering: Gaussian filtering is a linear smoothing filter suitable for eliminating Gaussian noise and widely used in image processing noise reduction. Gaussian filtering is essentially a weighted average of the entire image; the value of each pixel is obtained by weighted averaging of its own value and the values of its neighboring pixels. Specifically, Gaussian filtering involves scanning each pixel in the image with a template (or convolution, mask), and replacing the value of the center pixel of the template with the weighted average gray value of the pixels in the neighborhood defined by the template.
[0058] An example of an edge detection method based on the adaptive gradient threshold Canny operator:
[0059] See Figure 1 The present invention provides an edge detection method based on the adaptive gradient threshold Canny operator, which may include the following steps:
[0060] Step S1: Obtain the original image;
[0061] Step S2: Preprocess the original image and remove noise from the image using median filtering and Gaussian filtering;
[0062] Step S3: Convert the filtered image into a grayscale image, obtain the grayscale histogram of the grayscale image, and calculate the grayscale threshold based on the grayscale histogram.
[0063] Step S4: Calculate the average grayscale values of the target and the background, using the grayscale threshold T as the boundary.
[0064] Step S5: Determine the high gradient threshold and low gradient threshold based on the grayscale mean of the target and background;
[0065] Step S6: Use the Canny operator and the calculated high and low gradient thresholds to perform edge recognition and obtain an edge detection image.
[0066] Based on the above scheme, this embodiment proposes a reasonable adaptive thresholding Canny algorithm, which requires that the grayscale threshold for thresholding and the high and low gradients of the Canny operator be automatically selected. Furthermore, since the Canny edge detection method requires pre-processing the image with Gaussian filtering, the gradient threshold of this invention takes into account the blurring effect of Gaussian filtering on the edges.
[0067] Optionally, this embodiment may also perform the following operations:
[0068] The process of preprocessing the original image can specifically involve performing 3D processing on the original image. Median filtering is applied within a 3-region area to obtain the median-filtered image; a closing operation is performed on the filtered image; an opening operation is performed on the filtered image; a Gaussian filter function is constructed to apply a 3-dimensional filtering operation to the image after the opening and closing operations. The image is then filtered and denoised again within region 3 to obtain the Gaussian noise-processed image.
[0069] Alternatively, the following operations can also be performed:
[0070] The process of calculating the grayscale threshold can specifically involve drawing a grayscale histogram of the grayscale image, with the horizontal axis representing the grayscale value and the vertical axis representing the number of pixels; and then obtaining the peak value of the grayscale histogram.
[0071] Set initial left and right boundaries [L] i ,R i], where i is the iteration number, initially i = 0, i.e. L0 = 0, R0 = 255;
[0072] The threshold T0 is obtained by applying Otsu's method to the grayscale image.
[0073] Optionally, this embodiment may also perform the following operations:
[0074] Using a threshold T0 as the boundary, the grayscale image is divided into [L... i T i ] and [T i R i The target and background are denoted as two parts, and the mean u1 of each part is calculated. i u2 i ;
[0075] Determine the boundary of the next iteration [L] i ,R i ];
[0076] For grayscale images in [L i+1 , R i+1 By performing Otsu's method, the threshold T is obtained. i ;
[0077] Set the threshold difference to T min Determine |T i - T i-1 |>= T i If not, the program terminates and outputs T. i-1 .
[0078] Optionally, this embodiment may also perform the following operations:
[0079] The process of finding the peak value of the grayscale histogram can be specifically as follows:
[0080] Assuming the peak gray level is k, and the corresponding number of pixels is p(k), it includes:
[0081] p(k) is a local maximum, that is: p(k)>p(k-1) and p(k)>p(k+1);
[0082] p(k) is the maximum value of grayscale [k-rang1:k+rang2]. If there is another grayscale value within the range of rang1 and rang2 whose number of pixels is equal to p(k), then the smaller grayscale value k is taken as the peak grayscale.
[0083] p(k) accounts for more than 0.2% of the total number of pixels.
[0084] Optionally, this embodiment may also perform the following operations:
[0085] Determine the boundary of the next iteration [L]i , R i process, specifically it can be: if i > [2 / m], L i+1 = L i , R i+1 = R i ;
[0086] If i <= [2 / m], then further judgments include:
[0087] If T i < u1, then L i+1 = T i , otherwise L i+1 = u1;
[0088] If T m-(i+1) < u1, then R i+1 = T m-(i+1) , otherwise R i+1 = u2.
[0089] Optionally, this embodiment can also perform the following operations:
[0090] The process of determining the high-gradient threshold and the low-gradient threshold can specifically be:
[0091] Taking the threshold as the boundary, dividing the grayscale image into the target and background images, and setting the grayscale means of the target and background images as u1 and u2;
[0092] In the canny operator, the eight-neighborhood gradient sobel operators used are Gy, Gx;
[0093] Assuming that the edge in the small area is a straight edge, after adding the gradient vectors and taking the magnitude, it is expressed as formula (1):
[0094] G = (1)
[0095] Its maximum magnitude is expressed as formula (2):
[0096] G max = 4 |u1 - u2| (2)
[0097] Taking the maximum magnitude G max = 4 0.726 |u1 - u2| as the high-gradient threshold T h , and setting the low-gradient threshold T l to 0.726 |u1 - u2|.
[0098] Optionally, this embodiment can also perform the following operations:
[0099] The process of edge recognition using the Canny operator and the calculated high and low gradient thresholds can specifically involve calculating the gradient magnitude and gradient direction of edge pixels.
[0100] Non-maximum suppression is performed on the gradient magnitude based on the gradient direction. The gradient magnitude of the eight-neighbor center pixel of each pixel is compared with the gradient magnitude of the two pixels along its corresponding gradient direction. If the eight-neighbor center pixel is the maximum value, it is retained; otherwise, the center pixel is set to 0 to obtain a thinned edge.
[0101] Optionally, this embodiment may also perform the following operations:
[0102] The gradient magnitude is greater than the high gradient threshold T. h Pixels with gradient magnitudes less than the low gradient threshold T are used as edge points. l Pixels that are not edge points are considered edge points. Other pixels are considered edge points if there are edge points in their eight neighborhoods, otherwise they are considered non-edge points.
[0103] In practical applications, this application provides an edge detection method based on the adaptive gradient threshold Canny operator, which specifically includes:
[0104] 1. Read the SEM image.
[0105] II. Preprocessing of SEM images, specifically including:
[0106] 1. Perform 3 Median filtering within a 3-region area;
[0107] 2. Perform a closing operation on the image;
[0108] 3. Perform an opening operation on the image;
[0109] 4. Perform 3 Gaussian filtering within the 3-region area.
[0110] III. Calculating the grayscale threshold, which specifically includes:
[0111] 1. Obtain the grayscale histogram of the grayscale image, where the horizontal axis represents the grayscale value and the vertical axis represents the number of pixels.
[0112] 2. Find the peak value (maximum) T of the grayscale histogram. m Where m is the number of peak values, and the judgment condition (assuming the peak gray level is k and the corresponding number of pixels is p(k)) is as follows:
[0113] (1) p(k) is a local maximum, that is: p(k)>p(k-1) and p(k)>p(k+1);
[0114] (2) p(k) is the maximum value within the gray level range [k - rang1:k + rang2], where rang1 and rang2 take values from 5 to 8. If there is another gray level within this range whose number of pixels is equal to p(k), then the smaller gray level value of k is taken as the peak gray level;
[0115] (3) The proportion of p(k) in the total number of pixels is greater than 0.2%.
[0116] It can be seen that the above three conditions can be used for the method of judging the peak of the gray level histogram, which ensures that the peak of the gray level histogram can be effectively and accurately detected, and avoids the problems of too close distance between two peaks and too low peak values.
[0117] 3. Set the initial left and right boundaries [L i , R i (i = 0), that is, L0 = 0, R0 = 255, where i is the iteration number.
[0118] 4. Perform the Otsu method on the gray level image to obtain the threshold T0.
[0119] 5. Taking the threshold T0 as the boundary, divide the gray level image into two parts according to the gray level as [L i , T i and [T i , R i , denoted as the target and the background respectively, and calculate the means u1 i , u2 i .
[0120] 6. Determine the next iteration boundary [L i , R i as follows:
[0121] (1) If i > [2 / m], L i+1 = L i , R i+1 = R i ;
[0122] (2) If i <= [2 / m], make a further judgment:
[0123] A: If T i < u1, then L i+1 = T i , otherwise L i+1 = u1;
[0124] B: If T m-(i+1) < u1 then R i+1 = T m-(i+1) , otherwise R i+1 = u2.
[0125] 7. For grayscale images in [L] i+1 , R i+1 By performing Otsu's method, the threshold T is obtained. i .
[0126] 8. Set the threshold difference T min If |T i - T i-1 |>= T i Return to step 5; otherwise, the program terminates and outputs T. i-1 .
[0127] IV. Calculate the average grayscale values of the target and background:
[0128] Using a threshold T as the boundary, the grayscale image is divided into two parts according to grayscale: [0, T] and [T, 255], denoted as the target and the background, respectively. The grayscale mean values u1 and u2 of these two parts are then calculated.
[0129] V. Calculate the high and low gradient thresholds:
[0130] Low threshold T l =0.726 |u1-u2|; High threshold T h =4 0.726 |u1-u2|.
[0131] The calculation approach is as follows:
[0132] 1. The image is divided into target and background using a threshold. The grayscale mean values of the two parts are u1 and u2, respectively. Therefore, in the ideal case, the target grayscale value is u1 and the background grayscale value is u2. Figure 2 As shown.
[0133] 2. In the Canny operator, the eight-neighbor gradient Sobel operators used are Gy and Gx, as follows: Figure 3 and Figure 4 As shown:
[0134] 3. Assumption: In this embodiment, the edges within the small region are considered to be straight edges. Therefore, the magnitude of the sum of the gradient vectors is taken, i.e., G = Its maximum value G max = 4 |u1-u2|.
[0135] 4. Due to the filtering effect, pixels near the edges will change, specifically as follows: Figure 5-7 As shown.
[0136] Therefore, after such calculation, G = The maximum value Gmax4 0.726 |u1-u2| determines the high threshold. The low threshold is considered to be the minimum value that could be an edge point, and |u1-u2| is the minimum grayscale difference between the target and the background. Considering the edge blurring caused by filtering, the low threshold is set to 0.726. |u1-u2|.
[0137] VI. Edge recognition is performed using the Canny operator and high / low thresholds. The specific steps are as follows:
[0138] 1. Gaussian filter, with a kernel size of 3. 3.
[0139] 2. Calculate the gradient magnitude and direction using the eight-neighbor Sobel operator. The Sobel operator is as follows: Figure 2 , Figure 3 As shown, the total gradient is the magnitude of the vector sum of the gradients in the x and y directions.
[0140] 3. Perform non-maximum suppression on gradient magnitude based on gradient direction. Specifically, the gradient magnitude of the center pixel of each pixel's eight neighborhoods is compared with the gradient magnitudes of the two pixels along its corresponding gradient direction. If the center pixel has the maximum value, it is retained; otherwise, the center pixel is set to 0. This suppresses non-maximum pixels and retains the points with the largest local gradients, thus obtaining a refined edge.
[0141] 4. Lag threshold judgment and edge connection. Specifically, thresholds with gradient magnitudes greater than T are used for edge connections. h Points with gradient magnitudes less than T are considered edge points. l Points with a certain gradient are considered non-edge points, while other points are considered edge points if they have an edge point within their eight neighborhoods; otherwise, they are non-edge points. It is evident that the above approach to calculating high and low gradient thresholds takes into account the built-in methods of the Canny operator and combines the effect of Gaussian filtering for threshold selection.
[0142] Therefore, this application's embodiments combine the characteristics of SEM images, using the gray-level histogram with relatively obvious features as the starting point, avoiding the analysis of gradient histograms, which in most cases exhibit a unimodal distribution. Regarding the gray-level threshold, iterative Otsu's method is used for calculation, improving accuracy. Based on the gray-level threshold, the average gray-level values of the target and background are calculated. Furthermore, considering the Canny operator's gradient calculation method, the average gray-level values are used to reasonably estimate the gradient threshold. In addition, since the Canny operator requires a Gaussian filtering step, its blurring effect on edges affects the selection of the gradient threshold; this invention also takes this effect into account, effectively improving the problem of edge breakage.
[0143] The present invention also provides an edge detection device based on the adaptive gradient threshold Canny operator, comprising:
[0144] The image acquisition unit is used to acquire the original image;
[0145] The image processing unit is used to preprocess the original image and remove noise from the image using median filtering and Gaussian filtering.
[0146] The grayscale thresholding unit is used to convert the filtered image into a grayscale image, obtain the grayscale histogram of the grayscale image, and calculate the grayscale threshold based on the grayscale histogram.
[0147] The grayscale mean unit is used to calculate the grayscale mean of the target and the background, with the grayscale threshold T as the boundary.
[0148] High and low threshold units are used to determine high and low gradient thresholds based on the gray-scale mean of the target and background.
[0149] The edge detection unit is used to perform edge recognition using the Canny operator and the calculated high and low gradient thresholds to obtain an edge detection image.
[0150] Based on the above scheme, the present invention uses an edge detection device composed of an image acquisition unit, an image processing unit, a gray level thresholding unit, a gray level mean unit, a high and low thresholding unit, and an edge detection unit to unify the inherent correlation between the gray level distribution and gradient distribution of an image and the built-in method of the Canny operator, thereby solving the problems of numerous false edges, low edge extraction accuracy, blurred edges, and poor edge detection effect in the prior art.
[0151] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an edge detection method based on the adaptive gradient threshold Canny operator.
[0152] Those skilled in the art will understand that the computer device structure shown in this embodiment is only a partial structure related to the solution of this application and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than shown in this embodiment, or combine certain components, or have different component arrangements.
[0153] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.
[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.
[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.
[0156] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0157] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.
Claims
1. An edge detection method based on an adaptive gradient threshold canny operator, characterized by, The method comprises the following steps: obtaining an original image; preprocessing the original image and removing noise from the image by median filtering and Gaussian filtering; converting the filtered image into a gray-scale image and obtaining a gray-scale histogram of the gray-scale image, and calculating a gray-scale threshold value according to the gray-scale histogram; calculating the gray-scale mean values of the target and the background based on the gray-scale threshold value T; determining a high gradient threshold value and a low gradient threshold value based on the gray-scale mean values of the target and the background; performing edge recognition by using a canny operator and the calculated high and low gradient threshold values to obtain an edge detection image. The calculation of the gray-scale threshold value comprises: drawing a gray-scale histogram of the gray-scale image, with the abscissa representing the gray-scale value and the ordinate representing the number of pixel points; obtaining the peak value of the gray-scale histogram; Set initial left and right boundary [L i ,R i ], where i is the iteration number, initially i = 0, i.e. L0 = 0, R0 = 255; obtaining the threshold value T0 by performing Otsu method on the gray-scale image. With threshold T i as the boundary, threshold T i is initially set as threshold T0, and the gray scale image is divided into two parts [L i , T i ] and [T i , R i ] according to the gray scale, which are recorded as target and background, and the average values u1 i and u2 i of the two parts are respectively obtained; determining the next iteration boundary [L i ,R i ]; The Otsu method is performed on the gray-scale image in [L i+1 , R i+1 ] to obtain a threshold T i ; Judge |T i - T i-1 | >= T i If not, the program terminates and outputs T i-1； The obtaining of the peak value of the gray-scale histogram comprises: assuming that the peak gray-scale value is k and the corresponding number of pixel points is p(k), which comprises: p(k) is a maximum value, i.e., p(k)>p(k-1) and p(k)>p(k+1); p(k) is the maximum value in the gray-scale range [k-rang1:k+rang2], and if there is another gray-scale value in the range of rang1 and rang2, whose number of pixel points is equal to p(k), then the smaller gray-scale value k is taken as the peak gray-scale value; p(k) accounts for more than 0.2% of the total number of pixel points.
2. The method of claim 1, wherein, The preprocessing of the original image comprises: 3 3 median filtering in the region to obtain a median filtered image; performing closed operation on the filtered image; performing open operation on the filtered image; Constructing a Gaussian filter function to filter the image after the open-close operation 3 3 Re-filtering noise in the region to obtain a Gaussian noise processed image.
3. The method of claim 1, wherein, The determination of the high and low gradient threshold values comprises: dividing the gray-scale image into a target image and a background image based on the threshold value, and setting the gray-scale mean values of the target image and the background image as u1 and u2; in the canny operator, the eight-neighborhood gradient sobel operator used is Gy, Gx; assuming that the edge in a small area is a straight edge, the gradient vector is added and the amplitude is taken, which is represented by formula (1): G= （1） the maximum amplitude is represented by formula (2): G max = 4 |u1-u2| (2) The maximum amplitude G max = 4 0.726 | u1 - u2 | as a high gradient threshold T h , and the low gradient threshold T l is set to 0.726 | u1 - u2 |.
4. The method of claim 3, wherein, The determining the next iteration boundary [L i ,R i ] comprises: if i > [2 / m], L i+1 = L i , R i+1 = R i ; if i <= [2 / m], then further judgment comprises: If T i < u1, then L i+1 = T i , otherwise L i+1 = u1; If T m-(i+1) < u1, then R i+1 = T m-(i+1) , otherwise R i+1 = u2.
5. The method of claim 4, wherein, The edge recognition by using the canny operator and the calculated high and low gradient threshold values comprises: calculating the gradient amplitude and gradient direction of the edge pixel points; performing non-maximum suppression on the gradient amplitude according to the gradient direction, wherein, if the gradient amplitude of the center pixel of the eight-neighborhood of each pixel point is the maximum value compared with the gradient amplitudes of the two pixels along its corresponding gradient direction, the center pixel is retained, otherwise, the center pixel is set to 0, to obtain the thinned edge.
6. The method of claim 5, wherein, It comprises: pixels with gradient magnitude greater than a high gradient threshold T h pixels with gradient magnitude less than a low gradient threshold T l pixels with gradient magnitude less than a low gradient threshold T 7. An edge detection apparatus based on an adaptive gradient threshold canny operator, characterized by, It comprises: an image acquisition unit configured to acquire an original image; an image processing unit configured to preprocess the original image and remove noise from the image by median filtering and Gaussian filtering; a gray-scale threshold unit configured to convert the filtered image into a gray-scale image, obtain a gray-scale histogram of the gray-scale image, and calculate a gray-scale threshold value according to the gray-scale histogram; a gray-scale mean value unit configured to calculate the gray-scale mean values of a target and a background based on the gray-scale threshold value T; a high and low threshold unit configured to determine a high gradient threshold value and a low gradient threshold value based on the gray-scale mean values of the target and the background; and a high and low threshold unit configured to determine a high gradient threshold value and a low gradient threshold value based on the gray-scale mean values of the target and the background. An edge detection unit is configured to perform edge recognition by using a Canny operator and the calculated high and low gradient thresholds to obtain an edge detection image. The calculation of the gray threshold comprises: A gray histogram of the gray image is drawn, with the horizontal coordinate being a gray value and the vertical coordinate being a pixel number; A peak value of the gray histogram is obtained; Set initial left and right boundaries [L i ,R i ], where i is the iteration number, initially i = 0, i.e. L0 = 0, R0 = 255; Otsu method is performed on the gray image to obtain a threshold value T0. With threshold T i as the boundary, threshold T i is initially set as threshold T0, and the gray scale image is divided into two parts [L i , T i ] and [T i , R i ] according to the gray scale, which are recorded as target and background, and the average values u1 i and u2 i of the two parts are respectively obtained; determining the next iteration boundary [L i ,R i ] Laplace filtering is performed on the gray-scale image in [L i+1 , R i+1 ] to obtain a threshold T i ; Judge |T i - T i-1 | >= T i If not, the program terminates and outputs T i-1； The calculation of the peak value of the gray histogram comprises: Suppose that a peak gray value is k, and the corresponding pixel number is p(k), which comprises: p(k) is a maximum value, i.e., p(k)>p(k-1) and p(k)>p(k+1); p(k) is a maximum value in the gray range [k-rang1:k+rang2], and if there is another gray value in the value range of rang1 and rang2, and the pixel number thereof is equal to p(k), then the smaller gray value k is taken as the peak gray value; The proportion of p(k) in the total pixel number is greater than 0.2%.
Citation Information
Patent Citations
Goods edge detection method and device in hazardous chemical substance warehouse and terminal equipment
CN111815660A