A method for extracting a circular hole edge at a sub-pixel level under complex illumination background conditions

Abstract

The application discloses a circular hole edge sub-pixel level extraction method under complex illumination background conditions. First, an attention mechanism is integrated into a Deeplabv3+ semantic segmentation network to construct a model; second, based on a hole-containing part image dataset with labeled circular hole features and combined with additional information extracted based on phase consistency, the model is trained; third, a to-be-measured part image is input, a semantic segmentation mask of a target circular hole is output, and edge information of the to-be-measured part image is extracted, and the above two are post-processed together; finally, based on the edge information of the extracted annular outer edge region, the extracted full-image sub-pixel points are intersected, and the sub-pixel point fitting of the target circular hole edge is completed. The application reduces the interference of changing illumination, complex background noise and the like in the actual measurement process, improves the precision of edge extraction and the stability of the edge extraction algorithm, improves the segmentation precision, and further improves the precision and stability of circular hole edge extraction.

Description

Technical Field

[0001] This invention relates to a method for extracting the edge of a circular hole, and more particularly to a sub-pixel level extraction method for the edge of a circular hole under complex lighting background conditions. Background Technology

[0002] The automated assembly and docking process of components with holes requires real-time detection of the target circular hole's pose, and the basis for accurate measurement of the circular hole's pose is the high-precision extraction of the circular hole's edge.

[0003] In real-world measurement environments, complex background conditions such as illumination variations and noise interference can affect edge extraction accuracy. Furthermore, the Robert et al. differential operator method is highly sensitive to noise, leading to poor stability in traditional differential operator-based edge extraction algorithms. This is especially true in actual industrial production environments, where varying illumination and unstable operating conditions constantly constrain image acquisition. These adverse environmental factors can result in poor image quality and low contrast, making accurate edge extraction impossible or even producing false edges with numerous noise points. This significantly impacts subsequent ellipse fitting processes. Therefore, fast and high-precision edge extraction of circular holes remains a challenging task.

[0004] Tabatabai first established an edge step model in his 1984 paper "Edge location to subpixel values ​​in digital imagery" published in *IEEE Transactions on Pattern Analysis and Machine Intelligence*, using gray-level moment variables to obtain four parameters of the edge variables. Ghosal S et al., in their 1993 paper "Orthogonal moment operators for subpixel detection" published in *Pattern Recognition*, proposed fitting an ideal two-dimensional edge using four parameters: background gray level, step size, distance from the edge to the mask center, and edge direction. They used discrete Zernike moments to obtain three complex masks and calculated all edge parameters for subpixel detection. However, moment-based methods and their improvements are computationally intensive and sensitive to image noise. Kris Jensen et al., in their 1995 paper "Subpixel localization and the interpolation of still images" published in *IEEE Transactions on Image*, studied nonlinear interpolation and used this method for subpixel detection; however, these methods are also susceptible to noise. In his 2014 paper "Gaussian-basrd approach to subpixel detection of blurred and unsharp edges" published at the IEEE Federated Conference on Computer Science and Information Systems, Fabi first used the Sobel operator for pixel-level edge detection. Then, he used Gaussian fitting near the pixel-level edges to locate subpixels, and finally used a cubic spline interpolation method to fit and connect the detected subpixel edge points. However, this method is limited by the accuracy of edge region detection. In their 2019 paper "Richer Convolutional Features for Edge Detection" published at IEEE TPAMI, Liu et al. utilized multi-scale features from deep convolutional neural networks and improved the calculation method of the loss function, significantly improving edge detection accuracy under the ODS metric.

[0005] The above research has improved the accuracy of circular hole pose measurement, but it still faces problems such as blurred circular hole edges, inaccurate high-frequency information, and long processing time. Summary of the Invention

[0006] To address the problems existing in the background technology, this invention proposes a sub-pixel level extraction method for the edge of a circular hole under complex lighting background conditions, which has the advantages of being fast, efficient, and highly accurate.

[0007] This invention first incorporates an attention mechanism into the commonly used Deeplabv3+ semantic segmentation model in the industry, training the model with sample images of annotated circular holes and additional information extracted by phase consistency. Second, it detects parts containing holes, outputs a semantic segmentation mask of the target circular hole, and extracts edges from the original image to obtain an edge information map, using both as inputs for post-processing. It further extracts edge information of the annular edge region to obtain accurate edges, and finally performs sub-pixel fitting on the accurate edges of the target circular hole.

[0008] The specific technical solution adopted in this invention is as follows:

[0009] 1) Collect multiple images of perforated parts, and use the edge consistency method to extract phase feature information from each image of perforated parts to obtain the corresponding phase-consistent part images; the dataset consists of multiple images of perforated parts and the corresponding phase-consistent part images.

[0010] 2) Input the dataset into the improved Deeplabv3+ semantic segmentation network to train the network and obtain a semantic segmentation model for hole-type parts;

[0011] 3) Input the image of the part containing holes to be predicted into the semantic segmentation model for the part with holes for prediction, and output the edge mask of the circular hole;

[0012] 4) Use the edge extraction operator to extract the edges of the image of the hole-containing part to be predicted, and obtain the initial edge image of the part. Multiply the initial edge image of the part with the circular hole edge mask to obtain the coarse edge image of the part.

[0013] 5) Perform fine edge extraction on the coarse edge image of the part to obtain an accurate edge image;

[0014] 6) Based on the image of the part with holes to be predicted and the accurate edge image, calculate the sub-pixel level edge image.

[0015] In step 1), each image of a part with holes is augmented using a data augmentation method to obtain multiple augmented images of the part with holes.

[0016] In step 2), the improved Deeplabv3+ semantic segmentation network adds an attention mechanism module before the pyramid pooling ASPP module of the Deeplabv3+ semantic segmentation network, and the pyramid pooling ASPP module and the attention mechanism module are cascaded.

[0017] In step 4), the edge extraction operator is specifically the Roberts operator.

[0018] Step 5) specifically involves:

[0019] The rough edge image of the part is sequentially fitted, connected, and thinned to obtain an accurate edge image.

[0020] Step 6) specifically involves:

[0021] The image of the part containing holes to be predicted is subjected to full-image sub-pixel extraction to obtain full-image sub-pixel points. The intersection operation of the full-image sub-pixel points and the accurate edge image is performed to obtain the sub-pixel points of the circular hole edge, thereby obtaining the sub-pixel level edge image.

[0022] The beneficial effects of this invention are:

[0023] 1) By combining edge extraction methods with phase consistency features, the interference from changing lighting and complex background noise during actual measurement is reduced, thereby improving the accuracy and stability of edge extraction algorithms.

[0024] 2) By incorporating an attention mechanism module into the semantic segmentation network, the segmentation accuracy is improved, that is, it helps the network to accurately segment the target circular hole region, thereby capturing more refined edge information of the target circular hole region.

[0025] 3) The above-mentioned sub-pixel extraction process for the circular hole edge proposed in this invention, namely, first training the network and obtaining the target circular hole output, then using the edge information map for post-processing to obtain the fine edge of the circular hole, and finally performing sub-pixel fitting, further improves the accuracy and stability of the circular hole edge extraction. Attached Figure Description

[0026] Figure 1 This is a flowchart of the present invention.

[0027] Figure 2 These are examples of images from a sample library of perforated parts, with features extracted using phase consistency. The left image is the original image of the perforated part, and the right image is the result of phase consistency extraction.

[0028] Figure 3 This is the mask result of the target circular hole output from the image of a part containing holes. The left image is the output target edge mask, and the right image is the result of pasting the edge mask onto the original image in red.

[0029] Figure 4 It is an edge information map extracted through various operators.

[0030] Figure 5 It is the fine edge of the target circular hole in the part containing the hole.

[0031] Figure 6 It is the sub-pixel extraction result of the edge contour of the target circular hole in the part containing holes. Detailed Implementation

[0032] The present invention will be further described below with reference to the accompanying drawings and examples.

[0033] The embodiments of the present invention and their implementation process are as follows:

[0034] like Figure 1 As shown, the present invention includes the following steps:

[0035] 1) Acquire multiple images of parts containing holes, such as Figure 2 As shown in (a), the edge consistency method is used to extract phase feature information from each image of a hole-type part to obtain the corresponding phase-consistent part image, such as... Figure 2 As shown in (b); the dataset consists of multiple images of perforated parts and corresponding phase-consistent part images;

[0036] In step 1), each image of a part with holes has been labeled with the target original holes using tools such as Labelme. Each image of a part with holes is augmented using a data augmentation method to obtain multiple augmented images of the part with holes. The data augmentation method includes operations such as random cropping, rotation, and resizing. Both the augmented images of the part with holes and the original images of the part with holes are used as images of the part with holes, and phase feature information is extracted to obtain corresponding phase-consistent part images.

[0037] The phase coherence function at any point x is defined as follows:

[0038]

[0039] Where PC(x) represents the phase consistency information at pixel X in the image, PC(x)∈(0,1); n is the Fourier series value; A n φ is the amplitude coefficient. n (x) represents the phase of the nth Fourier component at point x; It is the weighted average of the local phase angles of each harmonic when the function reaches its maximum value at x.

[0040] Searching for peaks in the local energy function allows for the equivalent calculation of the point of maximum phase coherence. The local energy function E(x) is defined as a one-dimensional brightness distribution, and its energy is equal to the phase coherence scaled by the sum of Fourier amplitudes. The formula for calculating the phase coherence of each pixel in a two-dimensional image is shown below:

[0041]

[0042] Where 'o' specifies the direction of the pixel, typically 30°, 60°, or 90°; T o This refers to compensation noise in a specified direction; e is typically taken as 1e. -5 PC(x) represents the phase consistency information at pixel X in the image, E o (x) represents the energy function value of a point on a two-dimensional image in a certain direction, A no (x) represents the magnitude of a point on a two-dimensional image in a certain direction, and n represents the sum of the magnitudes of all two-dimensional points on the entire image.

[0043] 2) Input the dataset into the improved Deeplabv3+ semantic segmentation network to train the network. Each image of a hole-type part is used as the first input of the network, and the corresponding phase-consistent part image is used as the second input of the network to obtain the semantic segmentation model of hole-type parts.

[0044] In step 2), the improved Deeplabv3+ semantic segmentation network adds an attention mechanism module before the pyramid pooling ASPP module of the Deeplabv3+ semantic segmentation network. The pyramid pooling ASPP module and the attention mechanism module are cascaded. The module in the Deeplabv3+ semantic segmentation network that is placed before the pyramid pooling ASPP module and cascaded with it is also cascaded with the attention mechanism module. Images of parts containing holes first undergo feature information filtering by the attention mechanism module, and then semantic segmentation is performed on the results accordingly.

[0045] 3) Input the image of the part containing holes to be predicted into the semantic segmentation model for hole-type parts for prediction, and output the edge mask of the circular hole; such as Figure 3 As shown, Figure 3 (a) is the output circular hole edge mask. Figure 3 (b) is the result of applying the edge mask of the circular hole to the image of the part containing the hole to be predicted.

[0046] 4) Use the edge extraction operator to extract the edges of the image of the hole-containing part to be predicted, and obtain the initial edge image of the part. Multiply the initial edge image of the part with the circular hole edge mask to obtain the coarse edge image of the part.

[0047] In step 4), the edge extraction operator is specifically the Roberts operator. The edge extraction results of each typical operator are as follows: Figure 4 As shown. Figure 4 (a)-(e) are the Canny operator, the MH operator, the Prewitt operator, the Sobel operator, and the Roberts operator, respectively.

[0048] 5) Perform fine edge extraction on the coarse edge image of the part to obtain an accurate edge image;

[0049] Step 5) specifically involves:

[0050] The coarse edge image of the part is sequentially fitted, connected, and thinned to obtain an accurate edge image, such as... Figure 5 As shown.

[0051] 6) Based on the image of the part with holes to be predicted and the accurate edge image, calculate the sub-pixel level edge image.

[0052] Step 6) specifically involves:

[0053] To obtain sub-pixel points in the image of the part containing holes to be predicted, sub-pixel points are extracted from the entire image. Then, the intersection of these sub-pixel points with the accurate edge image is performed to obtain the sub-pixel points of the hole edge, thus obtaining a sub-pixel level edge image. Figure 6 As shown.

[0054] To test the algorithm's efficiency and verify the sub-pixel-level edge image extraction effect of this invention, this example conducts relevant testing and verification processes, as shown in Tables 1 and 2. First, several commonly used edge extraction operators are selected, and the percentage of edge integrity in a single image is tested under default parameters. The advantages and disadvantages are compared to determine whether the algorithm is suitable for relevant scenarios and facilitates sub-pixel-level extraction of the target circular hole's edge.

[0055] Table 1 Edge extraction effect of different operators

[0056]

[0057]

[0058] Table 2 Edge extraction results of different operators (continued)

[0059]

[0060] Table 3 shows the results of edge segmentation of the circular hole image using the Deeplabv3+ model directly and the semantic segmentation model constructed using the method of this invention. By comparing the commonly used evaluation metrics in semantic segmentation, such as Mean Intersection over Union (MIOU) and Pixel Accuracy, the segmentation performance of the semantic segmentation model of this invention in the background region and the target circular hole region is verified. As shown in Table 3, the method of this invention exhibits better semantic segmentation performance than the original Deeplabv3+ model in both the background region and the target circular hole region.

[0061] Table 3 Image segmentation results

[0062]

[0063] The experimental results above demonstrate that the method of this invention achieves sub-pixel-level extraction of the edge of a circular hole under complex background conditions such as varying illumination and noise. According to this method, the semantic segmentation accuracy of the background region and the target circular hole region in images of parts containing holes is significantly improved compared to current mainstream models, meeting the high-speed, high-efficiency, and high-precision measurement requirements in practical applications.

[0064] The above embodiments should not be considered as limitations on the present invention, but any improvements made based on the spirit of the present invention should be within the protection scope of the present invention.

Claims

1. A method for sub-pixel level extraction of the edge of a circular hole under complex lighting background conditions, characterized in that, Includes the following steps: 1) Collect multiple images of perforated parts, and use the edge consistency method to extract phase feature information from each image of perforated parts to obtain the corresponding phase-consistent part images; the dataset consists of multiple images of perforated parts and the corresponding phase-consistent part images. 2) Input the dataset into the improved Deeplabv3+ semantic segmentation network to train the network and obtain the semantic segmentation model for hole-type parts; the improved Deeplabv3+ semantic segmentation network adds an attention mechanism module before the pyramid pooling ASPP module of the Deeplabv3+ semantic segmentation network, and the pyramid pooling ASPP module and the attention mechanism module are cascaded. 3) Input the image of the part containing holes to be predicted into the semantic segmentation model for the part with holes for prediction, and output the edge mask of the circular hole; 4) Use the edge extraction operator to extract the edges of the image of the hole-containing part to be predicted, and obtain the initial edge image of the part. Multiply the initial edge image of the part with the circular hole edge mask to obtain the coarse edge image of the part. 5) Perform fine edge extraction on the coarse edge image of the part to obtain an accurate edge image; 6) Based on the image of the part with holes to be predicted and the accurate edge image, calculate the sub-pixel level edge image.

2. The method for sub-pixel level extraction of circular hole edges under complex lighting background conditions according to claim 1, characterized in that, In step 1), each image of a part with holes is augmented using a data augmentation method to obtain multiple augmented images of the part with holes.

3. The method for sub-pixel level extraction of circular hole edges under complex lighting background conditions according to claim 1, characterized in that, In step 4), the edge extraction operator is specifically the Roberts operator.

4. The method for sub-pixel level extraction of circular hole edges under complex lighting background conditions according to claim 1, characterized in that, Step 5) specifically involves: The rough edge image of the part is sequentially fitted, connected, and thinned to obtain an accurate edge image.

5. The method for sub-pixel level extraction of circular hole edges under complex lighting background conditions according to claim 1, characterized in that, Step 6) specifically refers to: The image of the part containing holes to be predicted is subjected to full-image sub-pixel extraction to obtain full-image sub-pixel points. The intersection operation of the full-image sub-pixel points and the accurate edge image is performed to obtain the sub-pixel points of the circular hole edge, thereby obtaining the sub-pixel level edge image.

Images