Circular hole positioning method and device under complex background, electronic equipment and storage medium

Abstract

The invention relates to a round hole positioning method and device under a complex background, electronic equipment and a storage medium. The circular hole positioning method under the complex background comprises the following steps: sequentially performing contrast enhancement and denoising on an original image; sequentially carrying out negation, contrast enhancement and denoising on the denoised image to obtain a highlight image; carrying out frequency domain conversion on the highlight image, filtering the frequency domain of the highlight image by adopting a Gaussian filter, and converting the highlight image back to a spatial domain to obtain a background image; background elimination is carried out on the highlight image according to the background image, and round hole area enhancement is carried out on the background-removed image to obtain a significant round hole area; and carrying out contour extraction on the significant circular hole region, and carrying out circular fitting on coordinates of the extracted edge contour by adopting a least square method to obtain a target frame of the circular hole, thereby completing positioning of the circular hole. The circular hole positioning method under the complex background has the advantage of accurately positioning the circular hole in the complex background image.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to a circular hole positioning method, device, electronic equipment and computer storage medium under complex background. Background Art

[0002] In the field of industrial image detection, it is often necessary to identify and locate highly dense small circular holes in complex backgrounds, such as: pin detection of IC chips on circuit boards in the 3C industry, pin detection of circuit terminals, and pinhole identification and size detection of lithium battery poles. Figure 1 As shown, Figure 1 The following is a schematic diagram of a circular hole in a complex background. Since small circular holes are often limited by product materials, equipment conditions or process differences, their imaging effect is often poor, especially when the grayscale difference between the background and the target circular hole is small, the detection difficulty increases significantly.

[0003] Traditional algorithms usually detect small circular holes through image enhancement, contrast improvement and edge detection. However, in complex backgrounds, due to the interference of noise and background texture, such methods can easily lead to an increase in false edge detections and fail to accurately extract the boundaries of the target circular holes. In the end, the fitting positioning of the circular holes often results in errors, including overlap, false detection and missed detection, and the recognition accuracy is low.

[0004] Based on this, the existing technology generally adopts the semantic segmentation method, which extracts and fuses the pre-processed image multiple times to identify the significant features of the circular hole, so as to accurately locate the circular hole. However, semantic segmentation has high requirements on data and computing resources, and in dense target scenes, the accuracy of segmentation is often affected by background noise or the proximity of target circular holes. Therefore, when dealing with the positioning of dense circular targets in complex backgrounds, the existing technology still has the problem of inaccurate positioning. Summary of the invention

[0005] Based on this, the object of the present invention is to provide a circular hole positioning method under complex background.

[0006] A circular hole positioning method under complex background, characterized by comprising the following steps:

[0007] S1, performing contrast enhancement and denoising on the original image in sequence; and performing inversion, contrast enhancement and denoising on the denoised image in sequence to obtain a highlighted image;

[0008] S2, performing frequency domain conversion on the highlight image, filtering the frequency domain information of the highlight image using a Gaussian filter, and converting the filtered frequency domain back to the spatial domain to obtain a background image;

[0009] S3, removing the background of the highlight image according to the background image, and enhancing the circular hole area of ​​the image after background removal to obtain a significant circular hole area;

[0010] S4. Extract the contour of the significant circular hole area, and use the least square method to perform circular fitting on the coordinates of the extracted edge contour to obtain the target frame of the circular hole and complete the positioning of the circular hole.

[0011] Compared with the prior art, the circular hole positioning method under complex background described in the present invention uses frequency domain filtering to extract the background image from the highlight image as a template, effectively removing the interference noise in the complex background, and ensuring that the circular hole area is clearer. In addition, after eliminating the background, the present invention extracts the edge of the significant circular hole area, and then uses the least squares method to perform circular fitting on the extracted edge coordinates, thereby achieving accurate fitting in a low-noise environment, thereby significantly improving the accuracy of circular hole positioning, and effectively reducing the possible false detection and missed detection in complex backgrounds.

[0012] Furthermore, the step S1 includes the following sub-steps:

[0013] S11, performing grayscale stretching and morphological opening operations on the original image in sequence to obtain a preliminary denoised image;

[0014] Among them, the specific calculation of grayscale stretching is as follows:

[0015]

[0016] Where G is the image after grayscale drawing of the grayscale image f of the original image; (x, y) is the pixel coordinate; f max and f min They are represented as the minimum grayscale value and the maximum grayscale value respectively; the morphological operation is used for denoising, which is specifically expressed as follows:

[0017]

[0018] In the formula, G f is the denoised image; S is a rectangular structural element; It is a grayscale corrosion operation, which is specifically expressed as follows:

[0019]

[0020] In the formula, G t is the image after grayscale corrosion; min u,v∈S G(x+u,y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x,y) within the coverage of the rectangular structure element S5, and taking the minimum value as the current pixel coordinate (x,y);

[0021] It is a grayscale expansion operation, which is used to restore the original edge of the eroded image. The specific expression is as follows:

[0022]

[0023] In the formula, max u,v∈S G t (x+u, y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x, y) within the coverage of the rectangular structure element S, and taking the minimum value as the current pixel coordinate (x, y);

[0024] S12, inverting the preliminary denoised image, and performing grayscale stretching and morphological opening operations to obtain a highlight image.

[0025] Based on this, the present invention preliminarily improves the grayscale contrast of the image, and further achieves effective denoising through corrosion and expansion operations, so that the edge of the circular hole area is clearer and the interference of the complex background is significantly reduced. Then, the present invention further enhances the brightness of the circular hole area by performing an inversion operation on the denoised image, performing grayscale stretching and corrosion and expansion processing again, making the contrast between the circular hole area and the background more obvious, thereby ensuring that the background image can be accurately extracted when filtering in the frequency domain later.

[0026] Furthermore, the step S3 includes the following sub-steps:

[0027] S31, subtracting the highlight image from the background image to obtain a circular hole image;

[0028] S32, performing threshold processing and morphological restoration on the circular hole image to obtain a significant circular hole area;

[0029] Wherein, the morphological repair comprises the following sub-steps:

[0030] S321, using rectangular structural elements to process the image G after threshold processing bi Perform morphological closing operation to obtain the preliminary repaired image. The specific calculation is as follows:

[0031]

[0032] In the formula, G closed represents the initial repair image; G bi represents the image after threshold processing, and They represent expansion operation and corrosion operation respectively, and S is a rectangular structural element;

[0033] S322, performing an internal filling operation on the preliminary repaired image to obtain a closed repaired image, which is specifically expressed as follows:

[0034] G filled=Fill(G closed )

[0035] In the formula, G filled Indicates the closed repair image, and Fill(·) is the filling operation;

[0036] S323, using a rectangular structure to perform a morphological opening operation on the closed repaired image to obtain a morphologically repaired image, which is specifically expressed as follows:

[0037]

[0038] In the formula, G completed represents the image after morphological restoration, and They represent expansion operation and corrosion operation respectively, and S is a rectangular structural element;

[0039] S324, performing area filtering on the image after morphological restoration to obtain a significant circular hole area.

[0040] Accordingly, the present invention ensures the integrity and smoothness of the circular hole area through multi-level area repair, such as closing operation, filling, opening operation and area filtering, effectively avoiding the positioning error of the circular hole caused by defects in the area near the circular hole, thereby significantly improving the accuracy and robustness of the circular hole positioning.

[0041] Furthermore, the step S4 includes the following sub-steps:

[0042] S41, using an edge detection operator to extract the contour of the significant circular hole area to obtain the coordinates of the circular hole edge;

[0043] Wherein, the edge detection operator is a Prewitt operator;

[0044] S42, using the least square method to perform circular fitting on the edge coordinates of the circular hole to obtain a target frame of the circular hole;

[0045] The circular fitting comprises the following sub-steps:

[0046] S421, dividing the circular hole edge coordinate Edge into connected domains to obtain several circular hole edge coordinate subsets, wherein the subsets are specifically expressed as follows:

[0047] Edge={edge1,edge2,…,edge k}

[0048] In the formula, edge k is the kth circular hole edge coordinate subset, which contains the edge coordinate points belonging to the same connected domain;

[0049] S422, performing least squares fitting on each circular hole edge coordinate subset edge, generating the circular hole center and radius, and forming a target frame of the circular hole;

[0050] Among them, for any circular hole edge coordinate subset edge k , and its fitting process is expressed as follows:

[0051] Set edge k There are several circular hole edge coordinates {(x i ,y i )|i∈N}, where N is the edge k The total number of circular hole edge coordinates;

[0052] Assume that the equation of the fitted circle is expressed as:

[0053] (x) 2 +(yb) 2 =R 2

[0054] Where (a, b) is the coordinate of the center of the circle, and R is the radius of the circle;

[0055] Then the fitted circle equation is linearized and algebraically transformed, and the linearized fitted circle equation is expressed as follows:

[0056] x 2 +y 2 +a′x+b′y+c=0

[0057] In the formula, a' is -2a, b' is -2b, and c is a 2 +b 2 -R 2 ;

[0058] According to the linearized fitting circle equation and several circular hole edge coordinates {(x i ,y i )|i∈N}, construct the linear error sum of squares S, which is specifically expressed as follows:

[0059]

[0060] Taking partial derivatives of a′, b′, and c respectively and setting their partial derivatives to zero, we obtain the following system of linear equations:

[0061]

[0062] The linear equations are solved to obtain the optimal values ​​of a′, b′, and c, and an algebraic inverse transformation is performed to obtain the center (a, b) and radius R of the circular hole to form the corresponding target frame of the circular hole.

[0063] Since the edge interference of the current image has been effectively removed in the aforementioned morphological restoration step, the present invention adopts a linearized fitting circle equation and combines it with the least squares method to perform circle fitting, thereby significantly improving the real-time performance of circular hole detection and the ability to locate high-density circular holes. At the same time, through the least squares method solution optimization, the high precision of the fitting circle is ensured, and the robustness of circular hole positioning under complex backgrounds is improved, thereby effectively improving the efficiency and accuracy of circular hole detection and positioning under complex backgrounds.

[0064] A circular hole positioning device under complex background, comprising an image enhancement unit, a background extraction unit, a circular hole image extraction unit and a circular hole positioning unit;

[0065] The image enhancement unit is used to sequentially perform contrast enhancement and denoising on the original image; and sequentially perform inversion, contrast enhancement and denoising on the denoised image to obtain a highlighted image;

[0066] The background extraction unit is used to perform frequency domain conversion on the highlight image, filter the frequency domain information of the highlight image using a Gaussian filter, and convert the filtered frequency domain back to the spatial domain to obtain a background image;

[0067] The circular hole image extraction unit is used to remove the background of the highlight image according to the background image, and to enhance the circular hole area of ​​the image after background removal to obtain a significant circular hole area;

[0068] The circular hole positioning unit is used to extract the contour of the significant circular hole area, and use the least square method to perform circular fitting on the coordinates of the extracted edge contour to obtain the target frame of the circular hole and complete the positioning of the circular hole.

[0069] Further, the image enhancement unit includes a contrast enhancement module and a reverse contrast enhancement module;

[0070] The contrast enhancement module is used to perform grayscale stretching and morphological opening operations on the original image in sequence to obtain a preliminary denoised image;

[0071] Among them, the specific calculation of grayscale stretching is as follows:

[0072]

[0073] Wherein, R is the image after grayscale drawing of the grayscale image f of the original image; (x, y) is the pixel coordinate; f max and f min They are represented as the minimum grayscale value and the maximum grayscale value respectively; the morphological operation is used for denoising, which is specifically expressed as follows:

[0074]

[0075] In the formula, Gf is the denoised image; S is a rectangular structural element; It is a grayscale corrosion operation, which is specifically expressed as follows:

[0076]

[0077] In the formula, G t is the image after grayscale corrosion; min u,v∈S G(x+u,y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x,y) within the coverage of the rectangular structure element S5, and taking the minimum value as the current pixel coordinate (x,y);

[0078] It is a grayscale expansion operation, which is used to restore the original edge of the eroded image. The specific expression is as follows:

[0079]

[0080] In the formula, max u,v∈S G t (x+u, y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x, y) within the coverage of the rectangular structure element S, and taking the minimum value as the current pixel coordinate (x, y);

[0081] The reverse contrast enhancement module is used to invert the preliminary denoised image, and perform grayscale stretching and morphological opening operations to obtain a highlighted image.

[0082] Furthermore, the circular hole image extraction unit includes a background elimination module and a circular hole enhancement module;

[0083] The background elimination module is used to subtract the highlight image from the background image to obtain a circular hole image;

[0084] The circular hole enhancement module is used to perform threshold processing and morphological restoration on the circular hole image to obtain a significant circular hole area;

[0085] Wherein, the morphological repair comprises the following sub-steps:

[0086] S321, using rectangular structural elements to process the image G after threshold processing bi Perform morphological closing operation to obtain the preliminary repaired image. The specific calculation is as follows:

[0087]

[0088] In the formula, G closed represents the initial repair image; G bi represents the image after threshold processing, and They represent expansion operation and corrosion operation respectively, and S is a rectangular structural element;

[0089] S322, performing an internal filling operation on the preliminary repaired image to obtain a closed repaired image, which is specifically expressed as follows:

[0090] G filled =Fill(G closed )

[0091] In the formula, G filled Indicates the closed repair image, and Fill(·) is the filling operation;

[0092] S323, using a rectangular structure to perform a morphological opening operation on the closed repaired image to obtain a morphologically repaired image, which is specifically expressed as follows:

[0093]

[0094] In the formula, G completed represents the image after morphological restoration, and They represent expansion operation and corrosion operation respectively, and S is a rectangular structural element;

[0095] S324, performing area filtering on the image after morphological restoration to obtain a significant circular hole area.

[0096] Furthermore, the circular hole positioning unit includes an edge extraction module and a circular hole fitting module:

[0097] The edge extraction module is used to extract the contour of the significant circular hole area using an edge detection operator to obtain the coordinates of the circular hole edge; wherein the edge detection operator is a Prewitt operator;

[0098] The circular hole fitting module is used to perform circular fitting on the circular hole edge coordinates using the least square method to obtain a target frame of the circular hole;

[0099] The circular fitting comprises the following sub-steps:

[0100] S421, dividing the circular hole edge coordinate Edge into connected domains to obtain several circular hole edge coordinate subsets, wherein the subsets are specifically expressed as follows:

[0101] Edge={edge1,edge2,…,edge k}

[0102] In the formula, edge k is the kth circular hole edge coordinate subset, which contains the edge coordinate points belonging to the same connected domain;

[0103] S422, performing least squares fitting on each circular hole edge coordinate subset edge, generating the circular hole center and radius, and forming a target frame of the circular hole;

[0104] Among them, for any circular hole edge coordinate subset edge k , and its fitting process is expressed as follows:

[0105] Set edge k There are several circular hole edge coordinates {(x i ,y i )|i∈N}, where N is the edge k The total number of circular hole edge coordinates;

[0106] Assume that the equation of the fitted circle is expressed as:

[0107] (x) 2 +(yb) 2 =R 2

[0108] Where (a, b) is the coordinate of the center of the circle, and R is the radius of the circle;

[0109] Then the fitted circle equation is linearized and algebraically transformed, and the linearized fitted circle equation is expressed as follows:

[0110] x 2 +y 2 +a′x+b′y+c=0

[0111] In the formula, a' is -2a, b' is -2b, and c is a 2 +b 2 -R 2 ;

[0112] According to the linearized fitting circle equation and several circular hole edge coordinates {(x i ,y i )|i∈N}, construct the linear error sum of squares S, which is specifically expressed as follows:

[0113]

[0114] Taking partial derivatives of a′, b′, and c respectively and setting their partial derivatives to zero, we obtain the following system of linear equations:

[0115]

[0116] The linear equations are solved to obtain the optimal values ​​of a′, b′, and c, and an algebraic inverse transformation is performed to obtain the center (a, b) and radius R of the circular hole to form the corresponding target frame of the circular hole.

[0117] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0118] Figure 1 This is a schematic diagram of a circular hole example in a complex background;

[0119] Figure 2 It is a schematic diagram of the overall structure of the fine circular hole positioning device under complex background of the present invention;

[0120] Figure 3 It is a schematic diagram of the overall process of the circular hole positioning method under a complex background described in the present invention;

[0121] Figure 4 This is a sample diagram of a preliminary denoised image;

[0122] Figure 5 This is a sample diagram of a highlighted image;

[0123] Figure 6 Example diagram for inverting a preliminary denoised image;

[0124] Figure 7 This is a simple example diagram of a circular hole image;

[0125] Figure 8 A simplified example schematic diagram of a significant circular hole area;

[0126] Fig. 9 A schematic diagram of the effect of circular hole positioning using the circular hole positioning method under a complex background of the present invention;

[0127] Fig.10 It is a schematic diagram of an example of the prior art using Hough transform to fit a circle and not using morphological restoration. DETAILED DESCRIPTION

[0128] In order to solve the problem of inaccurate circular hole positioning in the existing technology when processing the positioning of dense circular holes under complex background, the present invention performs contrast enhancement, double filtering and inversion processing on the original image in sequence, and then performs contrast enhancement and double filtering again to obtain a highlight image; then, the highlight image is subjected to frequency domain Gaussian filtering and inverse transformation to obtain a background image, and the background is removed by subtracting the highlight image from the background image to obtain a circular hole image; then, the circular hole image is subjected to threshold processing and morphological restoration to extract the significant circular hole area, and the edge detection is performed on the area to obtain the edge coordinates; finally, the least squares method is used to fit the edge coordinates into a circle, so as to accurately locate the circular hole. Accordingly, the present invention effectively removes the complex background and noise in the image that are easy to interfere with the circular hole positioning through background extraction and morphological restoration, and uses the least squares method to fit the circular hole edge, so that the circular hole can be accurately detected and located in the image with a complex background.

[0129] Based on the above design, the present invention proposes a circular hole recognition and positioning method under a complex background, and based on the method, proposes a circular hole recognition and positioning device under a complex background.

[0130] Please also see Figure 2 and Figure 3 , Figure 2 is a schematic diagram of the overall structure of the fine circular hole positioning device under complex background according to the present invention, Figure 3 It is a schematic diagram of the overall process of the circular hole positioning method under complex background described in the present invention.

[0131] The device for locating fine circular holes under complex background comprises an image enhancement unit 1 , a background extraction unit 2 , a circular hole image extraction unit 3 and a circular hole locating unit 4 .

[0132] The image enhancement unit 1 is used to execute step S1: sequentially perform contrast enhancement and denoising on the original image; and sequentially perform inversion, contrast enhancement and denoising on the denoised image to obtain a highlighted image.

[0133] Specifically, the image enhancement unit 1 includes a contrast enhancement module 11 and a reverse contrast enhancement module 12 .

[0134] The contrast enhancement module 11 is used to execute step S11: sequentially perform grayscale stretching and morphological opening operations on the original image to obtain a preliminary denoised image.

[0135] Specifically, calculate all the image grayscale values ​​in the original image, and obtain the minimum grayscale value and the maximum grayscale value in the image grayscale values; then, linearly map all the image grayscale values ​​according to the minimum grayscale value and the maximum grayscale value to obtain a high-contrast grayscale image, which can be referred to Figure 4 , Figure 4The following is a schematic diagram of a grayscale image example of a high-contrast image. For any grayscale pixel coordinate f(x, y), the specific calculation of its linear mapping is as follows:

[0136]

[0137] Where G(x,y) is the value of the high-contrast grayscale image at the pixel coordinate (x,y); f max and f min Represented as the minimum grayscale value and the maximum grayscale value respectively.

[0138] A 5x5 rectangular structural element is used to perform a morphological opening operation on the high-contrast grayscale image to obtain a preliminary denoised image. Figure 4 , Figure 4 This is an example diagram of a preliminary denoised image. The specific calculation process of its morphological opening operation is shown as follows:

[0139]

[0140] In the formula, G f is the preliminary denoised image; S is a 5x5 rectangular structure element; It is a grayscale corrosion operation, which is used to remove small bright spots and noise in the image. The specific expression is as follows:

[0141]

[0142] In the formula, G t is the image after grayscale corrosion; min u,v∈S G(x+u,y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x,y) within the coverage of the rectangular structure element S, and taking the minimum value as the current pixel coordinate (x,y), which is used to "shrink" the highlighted pixel values ​​in the image and remove the small highlighted noise in the image;

[0143] It is a grayscale expansion operation, which is used to restore the original edge of the eroded image. The specific expression is as follows:

[0144]

[0145] In the formula, max u,v∈s G t (x+u,y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x,y) within the coverage of the rectangular structure element S, and using the minimum value as the current pixel coordinate (x,y) to "expand" the highlighted pixel values ​​in the image and restore the image to its original boundary.

[0146] The reverse contrast enhancement module 12 is used to execute step S12: inverting the preliminary denoised image, and performing grayscale stretching and morphological opening operations to obtain a highlight image.

[0147] Specifically, the specific calculation expression for obtaining the highlight image is as follows:

[0148]

[0149] In the formula, G light For highlighted images, please refer to Figure 5 , Figure 5 This is an example diagram of a highlighted image; (255-G f ) represents the initial denoising image G f Perform the inversion operation. The inverted image can be found in Figure 6 , Figure 6 This is an example diagram of inverting a preliminary denoised image. and They are grayscale corrosion and grayscale expansion operations respectively, which are specifically consistent with the aforementioned descriptions of grayscale corrosion and grayscale expansion operations, and the present invention will not go into details here.

[0150] Based on this, the highlight image can increase the brightness of the circular hole part, thereby enhancing the contrast between the circular hole and the background, making the circular hole area more prominent against a complex background. In addition, by enhancing the brightness of the circular hole after negation, the circular hole area is made more obvious in the image, which helps to accurately identify the circular hole area in the subsequent frequency domain filtering, and based on this, filter out the circular hole area, thereby accurately extracting the background image.

[0151] The background extraction unit 2 is used to execute step S2: perform frequency domain conversion on the highlight image, filter the frequency domain information of the highlight image using a Gaussian filter, and convert the filtered frequency domain back to the spatial domain to obtain a background image.

[0152] Specifically, the background extraction unit 2 includes a frequency domain conversion module 21 and a background filtering module 22 .

[0153] The frequency domain conversion module 21 is used to execute step S21: perform fast Fourier transform on the highlight image to obtain the frequency domain of the highlight image.

[0154] Specifically, the fast Fourier transform is specifically expressed as follows:

[0155]

[0156] Where F(u,v) is the frequency spectrum of the highlight image in the frequency domain, u and v are the frequency coordinates; M and N are the width and height of the highlight image, respectively, which are used for normalization calculation; j is an imaginary unit, specifically Index term Used to convert spatial coordinates (x, y) to frequency coordinates.

[0157] Accordingly, by converting the highlight image into the frequency domain, its frequency characteristics can be analyzed more effectively so as to distinguish the low-frequency background information and the high-frequency circular hole details in the image, especially the highlight image with improved contrast after processing in step S1, the spectrum corresponding to the circular hole details is clearer in the frequency domain, and the corresponding spectrum information can be accurately identified and extracted.

[0158] The background filtering module 22 is used to execute step S22: filtering and inverse fast Fourier transforming the frequency domain of the highlight image in sequence through a Gaussian filter to obtain a background image.

[0159] Specifically, a Gaussian filter in the frequency domain is constructed, which is specifically expressed as follows:

[0160]

[0161] Where (u, v) is the frequency coordinate in the frequency domain; σ is the standard deviation, which is used to control the bandwidth of the filter. The larger its value, the wider the low-frequency range retained.

[0162] Next, the frequency domain of the highlight image is multiplied element by element by the Gaussian filter to obtain the filtered frequency domain F flitered (u,v), specifically expressed as follows:

[0163] F flitered (u,v)=F(u,v)·H(u,v)

[0164] Finally, the filtered frequency domain is subjected to an inverse fast Fourier transform to obtain a background image. The specific representation of the inverse fast Fourier transform is as follows:

[0165]

[0166] Based on this, filtering the frequency domain of the highlight image with a Gaussian filter can effectively filter out the spectral information belonging to the circular holes in the highlight image spectrum, thereby retaining the spectral information belonging to the background of the image and obtaining a smooth background image, providing a basis for subsequent circular hole detail extraction and enhancement.

[0167] The circular hole image extraction unit 3 is used to execute step S3: removing the background of the highlight image according to the background image, and enhancing the circular hole area of ​​the image after background removal to obtain a significant circular hole area.

[0168] Specifically, the circular hole image extraction unit 3 includes a background removal module 31 and a circular hole enhancement module 32 .

[0169] The background removal module 31 is used to execute step S31: subtract the highlight image from the background image to obtain a circular hole image.

[0170] For details, see Figure 7 , Figure 7 is a simple example diagram of a circular hole image, where the circular hole image G target The calculation is expressed as follows:

[0171] G target =G BG -G light

[0172] By taking the highlight image as the main body and subtracting it from the background image, most of the background information can be removed, thereby highlighting the target circular hole and making the circular hole area more obvious.

[0173] The circular hole enhancement module 32 is used to execute step S32: performing threshold processing and morphological restoration on the circular hole image to obtain a significant circular hole area.

[0174] Specifically, the circular hole image is subjected to threshold processing to enhance the contrast of the circular hole image and obtain a binary image, which is specifically identified as follows:

[0175]

[0176] In the formula, G bi is a binary image, T is the binarization threshold, which can be adjusted according to needs. The default value is the grayscale mean of the current image.

[0177] Next, a 7x7 rectangular structure element is used to perform a morphological closing operation on the binary image to repair the breaks in the binary image and obtain a preliminary repaired image. The specific calculation is expressed as follows:

[0178]

[0179] In the formula, G closed represents the initial repair image, and They represent expansion and corrosion operations respectively, S is a rectangular structure element with a size of 7x7;

[0180] Perform internal filling operation on the preliminary repair image to obtain a closed repair image, which is specifically expressed as follows:

[0181] G filled =Fill(G closed )

[0182] In the formula, G filledrepresents a closed repaired image, and Fill(·) is a filling operation for filling unfilled internal holes in the closed area. Specifically, the cv2.floodFill function in the OpenCV library can be used, and the present invention does not specifically limit its filling method.

[0183] A 7x7 rectangular structure is used to perform a morphological opening operation on the closed repaired image to eliminate small branches and noise in the image and obtain a morphologically repaired image. The specific expression is as follows:

[0184]

[0185] In the formula, G completed represents the image after morphological restoration, and They represent expansion and corrosion operations respectively, S is a rectangular structure element with a size of 7x7;

[0186] Finally, the image after morphological restoration is subjected to area filtering to further remove small interference areas and obtain significant circular hole areas. For details, please refer to Figure 8 , Figure 8 This is a simplified example schematic diagram of a significant circular hole area, which is specifically represented as follows:

[0187] G hyperTarget =Area fliter (G completed ,minArea)

[0188] In the formula, G hyperTarget Represented as a significant circular hole area, Area fliter It is an area filter function, which is used to filter out the areas in the image that are smaller than the area threshold minArea.

[0189] Accordingly, the present invention ensures the integrity and smoothness of the circular hole area through a refined morphological operation chain, effectively eliminates the interference caused by minor defects, prevents errors in subsequent edge extraction, thereby reducing the positioning deviation that may occur in the fitting process, and further improves the accuracy of circular hole detection.

[0190] The circular hole positioning unit 4 is used to execute step S4: extracting the contour of the significant circular hole area, and performing circular fitting on the coordinates of the extracted edge contour using the least square method to obtain the target frame of the circular hole, thereby completing the positioning of the circular hole.

[0191] Specifically, the circular hole positioning unit 4 includes an edge extraction module 41 and a circular hole fitting module 42 .

[0192] The edge extraction module 41 is used to execute step S41: using an edge detection operator to extract the contour of the significant circular hole area to obtain the coordinates of the circular hole edge.

[0193] Specifically, the edge detection operator is the Prewitt operator, which is specifically expressed as follows:

[0194]

[0195] In the formula, G x is the horizontal gradient operator, which is used to calculate the gradient of the image in the horizontal direction; G y It is a vertical gradient operator, used to calculate the gradient of the image in the vertical direction.

[0196] And use the horizontal gradient operator G x and the vertical gradient operator G y For the salient circular hole area image G hyperTarget Perform convolution to obtain the horizontal gradient Grad in the circular hole area x and vertical gradient Grad y .

[0197] Next, according to the horizontal gradient Grad of the circular hole area x and vertical gradient Grad y , calculate the edge strength G of the circular hole edge , specifically expressed as follows:

[0198]

[0199] The edge strength G of the circular hole is measured by a preset threshold T. edge (x, y) is used to extract the edge contour and obtain the edge coordinates of the circular hole, which are specifically expressed as follows:

[0200] Edge={(x,y)|G edge (x,y)>T}

[0201] Where Edge is the set of coordinates of the edge of the circular hole; when G edge If (x, y) is greater than the preset threshold T, the pixel coordinate (x, y) is considered to belong to the edge point of the circular hole and is included in Edge.

[0202] Among them, the threshold T can be rotated to a fixed threshold according to the histogram characteristics of the circular hole edge intensity, or an adaptive threshold method can be used. Accordingly, the selection of the threshold can be selected according to the specific situation. The present invention does not specifically limit the selection of the threshold T.

[0203] The circular hole fitting module 42 is used to execute step S42: performing circular fitting on the circular hole edge coordinates using the least square method to obtain a target frame of the circular hole.

[0204] Specifically, the connected domain of the circular hole edge coordinates is divided to distinguish the edges of different independent circular hole regions, and several subsets of circular hole edge coordinates are obtained. The specific representation of the subsets is as follows:

[0205] Edge={edge1,edge2,…,edge k}

[0206] In the formula, edge k is the kth circular hole edge coordinate subset, which includes edge coordinate points belonging to the same connected domain; wherein, the connected domain division is used to divide continuous pixel coordinates into independent subsets, which can be specifically executed by the cv2.findContours function in the OpenCV library, and the present invention does not specifically limit the specific implementation method of its connected domain division.

[0207] Next, the least squares method is used to fit each subset of circular hole edge coordinates to generate the center and radius of the circular hole and form the target frame of the circular hole. k , and its fitting process is expressed as follows:

[0208] Set edge k There are several circular hole edge coordinates {(x i ,y i )|i∈N}, where N is the edge k The total number of circular hole edge coordinates; let the equation of the fitting circle be expressed as:

[0209] (x) 2 +(yb) 2 =R 2

[0210] Where (a, b) is the coordinate of the center of the circle, and R is the radius of the circle;

[0211] Next, in order to simplify the solution process and computational complexity, the fitted circle equation is linearized and algebraically transformed. The linearized fitted circle equation is expressed as follows:

[0212] x 2 +y 2 +a′x+b′y+c=0

[0213] In the formula, a' is -2a, b' is -2b, and c is a 2 +b 2 -R 2 ;

[0214] Based on this, according to the linearized fitting circle equation and several circular hole edge coordinates {(x i ,yi )|i∈N}, construct the linear error sum of squares S, which is specifically expressed as follows:

[0215]

[0216] In order to minimize the linear error sum of squares S, partial derivatives of a′, b′, and c are taken respectively, and their partial derivatives are set to zero, and the following linear equations are obtained:

[0217]

[0218] Finally, the linear equations are solved to obtain the optimal values ​​of a′, b′, and c, and an algebraic inverse transformation is performed to obtain the center (a, b) and radius R of the circular hole, forming the corresponding target frame of the circular hole; wherein the specific expression of the algebraic inverse transformation is as follows:

[0219]

[0220] Since the above steps eliminate the noise and interference around the circular hole through morphological restoration, the extracted circular hole edge contour is clearer. Therefore, the present invention linearizes the fitting circle equation and then uses the least squares method for fitting, so that the fitting circular hole positioning is more accurate. At the same time, it also improves the robustness of the algorithm, effectively reduces the fitting error caused by noise and edge irregularity, and thus ensures the stability and consistency of circular hole positioning under complex backgrounds.

[0221] It should be noted that the edge contour of the circular hole is not linear data in nature. Therefore, if the morphological restoration is not sufficient, resulting in the appearance of additional protruding points or discontinuous edge points, it is easy to introduce large errors when performing linear fitting, thereby greatly reducing the fitting precision and positioning accuracy.

[0222] Compared with the prior art, the present invention adopts a multi-level image processing method, which significantly improves the contrast of the circular hole target and effectively removes noise and background interference by sequentially performing contrast enhancement, inversion and morphological restoration operations on the image under the complex background. Based on this, the present invention further filters the spectrum information of the circular hole in the frequency domain, accurately extracts the background image, and uses it as a template to remove the background area from the high-contrast image, thereby obtaining a pure, interference-free significant circular hole area, thereby making the target area more prominent, and greatly improving the robustness of locating the circular hole under the complex background, effectively reducing the interference of the complex background.

[0223] In addition, on the basis of fully reducing background interference as mentioned above, the present invention further adopts a linearized fitting circle equation and combines it with the least squares method for optimization and solution, thus achieving high precision and high fitting of the circular hole edge. Accordingly, the present invention can ensure the stability and accuracy of circular hole positioning in a complex background with high density and multiple circular holes, and significantly improve the efficiency and robustness of circular hole detection and positioning.

[0224] In order to show the difference between its final effect and the prior art, please refer to Fig. 9 and Fig.10 ,That Fig. 9 This is a schematic diagram of the effect of circular hole positioning using the circular hole positioning method under a complex background of the present invention, wherein the circular holes positioned are represented by colored circles; Fig.10 The figure is a schematic diagram of an example of the prior art using Hough transform to fit a circle and not using morphological restoration. Not using morphological restoration causes the connected domain analysis and Hough circle algorithm to be interfered by background noise, resulting in false detection and low accuracy.

[0225] Based on the same inventive concept, the present application also provides an electronic device, which may be a terminal device such as a server, a desktop computing device or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement the circular hole positioning method under a complex background of an embodiment of the present invention; and the memory is used to store a computer program executable by the processor.

[0226] Based on the same inventive concept, the present application also provides a computer-readable storage medium, corresponding to the embodiment of the aforementioned circular hole locating method under a complex background, and the computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the circular hole locating method under a complex background recorded in any of the aforementioned embodiments.

[0227] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0228] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.

Claims

1. A circular hole positioning method under complex background, characterized in that: The following steps are involved: S1, performing contrast enhancement and denoising on the original image in sequence; and performing inversion, contrast enhancement and denoising on the denoised image in sequence to obtain a highlighted image; S2, performing frequency domain conversion on the highlight image, filtering the frequency domain information of the highlight image using a Gaussian filter, and converting the filtered frequency domain back to the spatial domain to obtain a background image; S3, removing the background of the highlight image according to the background image, and enhancing the circular hole area of ​​the image after background removal to obtain a significant circular hole area; S4. Extract the contour of the significant circular hole area, and use the least square method to perform circular fitting on the coordinates of the extracted edge contour to obtain the target frame of the circular hole and complete the positioning of the circular hole.

2. The circular hole positioning method under complex background according to claim 1, characterized in that: The step S1 comprises the following sub-steps: S11, performing grayscale stretching and morphological opening operations on the original image in sequence to obtain a preliminary denoised image; Among them, the specific calculation of grayscale stretching is as follows: Where G is the image after grayscale drawing of the grayscale image f of the original image; (x, y) is the pixel coordinate; f max and f min They are represented as the minimum grayscale value and the maximum grayscale value respectively; the morphological operation is used for denoising, which is specifically expressed as follows: In the formula, G f is the denoised image; S is a rectangular structural element; It is a grayscale corrosion operation, which is specifically expressed as follows: In the formula, G t is the image after grayscale corrosion; min u,v∈S G(x+u,y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x,y) within the coverage of the rectangular structure element S5, and taking the minimum value as the current pixel coordinate (x,y); It is a grayscale expansion operation, which is used to restore the original edge of the eroded image. The specific expression is as follows: In the formula, max u,v∈S G t (x+u, y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x, y) within the coverage of the rectangular structure element S, and taking the minimum value as the current pixel coordinate (x, y); S12, inverting the preliminary denoised image, and performing grayscale stretching and morphological opening operations to obtain a highlight image.

3. The circular hole positioning method under complex background according to claim 2, characterized in that: The step S3 comprises the following sub-steps: S31, subtracting the highlight image from the background image to obtain a circular hole image; S32, performing threshold processing and morphological restoration on the circular hole image to obtain a significant circular hole area; Wherein, the morphological repair comprises the following sub-steps: S321, using rectangular structural elements to process the image G after threshold processing bi Perform morphological closing operation to obtain the preliminary repaired image. The specific calculation is as follows: In the formula, G closed represents the initial repair image; G bi represents the image after threshold processing, and They represent expansion operation and corrosion operation respectively, and S is a rectangular structural element; S322, performing an internal filling operation on the preliminary repaired image to obtain a closed repaired image, which is specifically expressed as follows: G filled =Fill(G closed ) In the formula, G filled Indicates the closed repair image, and Fill(·) is the filling operation; S323, using a rectangular structure to perform a morphological opening operation on the closed repaired image to obtain a morphologically repaired image, which is specifically expressed as follows: In the formula, G completed represents the image after morphological restoration, and They represent expansion operation and corrosion operation respectively, and S is a rectangular structural element; S324, performing area filtering on the image after morphological restoration to obtain a significant circular hole area.

4. The circular hole positioning method under complex background according to claim 3, characterized in that: The step S4 comprises the following sub-steps: S41, using an edge detection operator to extract the contour of the significant circular hole area to obtain the coordinates of the circular hole edge; Wherein, the edge detection operator is a Prewitt operator; S42, using the least square method to perform circular fitting on the edge coordinates of the circular hole to obtain a target frame of the circular hole; The circular fitting comprises the following sub-steps: S421, dividing the circular hole edge coordinate Edge into connected domains to obtain several circular hole edge coordinate subsets, wherein the subsets are specifically expressed as follows: Edge={edge1,edge2,…,edge k } In the formula, edge k is the kth circular hole edge coordinate subset, which contains the edge coordinate points belonging to the same connected domain; S422, performing least squares fitting on each circular hole edge coordinate subset edge, generating the circular hole center and radius, and forming a target frame of the circular hole; Among them, for any circular hole edge coordinate subset edge k , and its fitting process is expressed as follows: Set edge k There are several circular hole edge coordinates {(x i ,y i )|i∈N}, where N is the edge k The total number of circular hole edge coordinates; Assume that the equation of the fitted circle is expressed as: (x-a) 2 +(y-b) 2 =R 2 Where (a, b) is the coordinate of the center of the circle, and R is the radius of the circle; Then the fitted circle equation is linearized and algebraically transformed, and the linearized fitted circle equation is expressed as follows: x 2 +y 2 +a ′ x+b ′ y+c=0 In the formula, a ′ is -2a, b′ is -2b, c is a 2 +b 2 -R 2 ; According to the linearized fitting circle equation and several circular hole edge coordinates {(x i ,y i )|i∈N}, construct the linear error sum of squares S, which is specifically expressed as follows: For a ′ , b′, c partial derivatives, and set their partial derivatives to zero, then we get the following linear equations: Solve the linear equations to obtain a ′ , b′, c, and perform an inverse algebraic transformation to obtain the center (a, b) and radius R of the circular hole, forming the corresponding target frame of the circular hole.

5. A circular hole positioning device under complex background, characterized in that: It includes an image enhancement unit, a background extraction unit, a circular hole image extraction unit and a circular hole positioning unit; The image enhancement unit is used to sequentially perform contrast enhancement and denoising on the original image; and sequentially perform inversion, contrast enhancement and denoising on the denoised image to obtain a highlighted image; The background extraction unit is used to perform frequency domain conversion on the highlight image, filter the frequency domain information of the highlight image using a Gaussian filter, and convert the filtered frequency domain back to the spatial domain to obtain a background image; The circular hole image extraction unit is used to remove the background of the highlight image according to the background image, and to enhance the circular hole area of ​​the image after background removal to obtain a significant circular hole area; The circular hole positioning unit is used to extract the contour of the significant circular hole area, and use the least square method to perform circular fitting on the coordinates of the extracted edge contour to obtain the target frame of the circular hole and complete the positioning of the circular hole.

6. The circular hole positioning device under complex background according to claim 5, characterized in that: The image enhancement unit includes a contrast enhancement module and a reverse contrast enhancement module; The contrast enhancement module is used to perform grayscale stretching and morphological opening operations on the original image in sequence to obtain a preliminary denoised image; Among them, the specific calculation of grayscale stretching is as follows: Where G is the image after grayscale drawing of the grayscale image f of the original image; (x, y) is the pixel coordinate; f max and f min They are represented as the minimum grayscale value and the maximum grayscale value respectively; the morphological operation is used for denoising, which is specifically expressed as follows: In the formula, G f is the denoised image; S is a rectangular structural element; It is a grayscale corrosion operation, which is specifically expressed as follows: In the formula, G t is the image after grayscale corrosion; min u,v∈S G(x+u,y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x,y) within the coverage of the rectangular structure element S5, and taking the minimum value as the current pixel coordinate (x,y); It is a grayscale expansion operation, which is used to restore the original edge of the eroded image. The specific expression is as follows: In the formula, max u,v∈S G t (x+u, y+v) means taking the minimum value in the neighborhood of the current pixel coordinate (x, y) within the coverage of the rectangular structure element S, and taking the minimum value as the current pixel coordinate (x, y); The reverse contrast enhancement module is used to invert the preliminary denoised image, and perform grayscale stretching and morphological opening operations to obtain a highlighted image.

7. The circular hole positioning device under complex background according to claim 6, characterized in that: The circular hole image extraction unit includes a background elimination module and a circular hole enhancement module; The background elimination module is used to subtract the highlight image from the background image to obtain a circular hole image; The circular hole enhancement module is used to perform threshold processing and morphological restoration on the circular hole image to obtain a significant circular hole area; Wherein, the morphological repair comprises the following sub-steps: S321, using rectangular structural elements to process the image G after threshold processing bi Perform morphological closing operation to obtain the preliminary repaired image. The specific calculation is as follows: In the formula, G closed represents the initial repair image; G bi represents the image after threshold processing, and They represent expansion operation and corrosion operation respectively, and S is a rectangular structural element; S322, performing an internal filling operation on the preliminary repaired image to obtain a closed repaired image, which is specifically expressed as follows: G filled =Fill(G closed ) In the formula, G filled Indicates the closed repair image, and Fill(·) is the filling operation; S323, using a rectangular structure to perform a morphological opening operation on the closed repaired image to obtain a morphologically repaired image, which is specifically expressed as follows: In the formula, G completed represents the image after morphological restoration, and They represent expansion operation and corrosion operation respectively, and S is a rectangular structural element; S324, performing area filtering on the image after morphological restoration to obtain a significant circular hole area.

8. The circular hole positioning device under complex background according to claim 7, characterized in that: The circular hole positioning unit includes an edge extraction module and a circular hole fitting module: The edge extraction module is used to extract the contour of the significant circular hole area using an edge detection operator to obtain the coordinates of the circular hole edge; wherein the edge detection operator is a Prewitt operator; The circular hole fitting module is used to perform circular fitting on the circular hole edge coordinates using the least square method to obtain a target frame of the circular hole; The circular fitting comprises the following sub-steps: S421, dividing the circular hole edge coordinate Edge into connected domains to obtain several circular hole edge coordinate subsets, wherein the subsets are specifically expressed as follows: Edge={edge1,edge2,…,edge k } In the formula, edge k is the kth circular hole edge coordinate subset, which contains the edge coordinate points belonging to the same connected domain; S422, performing least squares fitting on each circular hole edge coordinate subset edge, generating the circular hole center and radius, and forming a target frame of the circular hole; Among them, for any circular hole edge coordinate subset edge k , and its fitting process is expressed as follows: Set edge k There are several circular hole edge coordinates {(x i ,y i )|i∈N}, where N is the edge k The total number of circular hole edge coordinates; Assume that the equation of the fitted circle is expressed as: (x-a) 2 +(y-b) 2 =R 2 Where (a, b) is the coordinate of the center of the circle, and R is the radius of the circle; Then the fitted circle equation is linearized and algebraically transformed, and the linearized fitted circle equation is expressed as follows: x 2 +y 2 +a ′ x+b ′ y+c=0 In the formula, a ′ is -2a, b′ is -2b, c is a 2 +b 2 -R 2 ; According to the linearized fitting circle equation and several circular hole edge coordinates {(x i ,y i )|i∈N}, construct the linear error sum of squares S, which is specifically expressed as follows: For a ′ , b′, c partial derivatives, and set their partial derivatives to zero, then we get the following linear equations: Solve the linear equations to obtain a ′ , b′, c, and perform an inverse algebraic transformation to obtain the center (a, b) and radius R of the circular hole, forming the corresponding target frame of the circular hole.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a circular hole locating method under a complex background as described in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by a processor, a circular hole positioning method in a complex background as described in any one of claims 1 to 4 is implemented.

Images