Tower drum flange bolt hole center positioning method, system and equipment based on improved Faster-RCNN coarse positioning and sub-pixel-level Canny edge detection fine positioning

Abstract

The invention discloses a tower drum flange bolt hole center positioning method, system and equipment based on improved Faster-RCNN coarse positioning and sub-pixel level Canny edge detection fine positioning, which are applied to the technical field of power transmission equipment assembly, and the method comprises the following steps: carrying out coarse positioning on a tower drum flange bolt hole through improved Faster-RCNN; the image in the coarse positioning frame is preprocessed, a sub-pixel-level Canny edge detection operator is adopted to extract the edge features of the bolt holes, and interference edges are screened and removed based on the roundness and length of the edges of the bolt holes; and carrying out ellipse fitting on the edge of the bolt hole by adopting a least square method, solving to obtain a two-dimensional coordinate of the center of the bolt hole in an image coordinate system, acquiring depth information of the center of the bolt hole through a depth camera, and converting the two-dimensional coordinate into a three-dimensional coordinate in a camera coordinate system by combining an internal reference matrix of the camera, so as to complete center positioning of the bolt hole. The positioning precision of the center of the flange bolt hole of the tower drum is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of power transmission equipment assembly technology, and more specifically to a method, system and equipment for locating the center of bolt holes in tower flanges based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization. Background Technology

[0002] As a critical infrastructure of power transmission networks, the quality of tower assembly directly affects the safety and reliability of the power transmission system. Currently, tower assembly relies heavily on manual operation, requiring workers to manually adjust the position of the towers to be assembled to align the flange bolt holes. This process suffers from low efficiency (long time required for each assembly), high technical requirements (dependent on operator experience), and significant safety hazards (risks associated with working at heights).

[0003] To achieve automated docking, the center coordinates of the tower flange bolt holes need to be accurately obtained first, but existing positioning technologies have significant drawbacks: (1) Limitations of traditional image processing methods: such as edge detection and Hough transform, are prone to edge extraction errors under complex outdoor lighting (strong light, shadow) and noise interference (dust, background debris), resulting in bolt hole center positioning deviation exceeding 2mm, which cannot meet the docking accuracy requirements.

[0004] (2) Insufficient single deep learning model: Although existing object detection models (such as traditional Faster-RCNN and YOLO-V5) have certain environmental adaptability, they have shortcomings. Traditional Faster-RCNN uses VGG16 backbone network, which has weak feature extraction ability, and the anchor box size does not match the circular characteristics of the bolt hole, making the predicted box easy to shift (the accuracy is only 48.3% when IOU=0.75); Although YOLO-V5 has a fast detection speed, its fine localization ability is insufficient, making it difficult to achieve sub-millimeter level coordinate acquisition.

[0005] (3) Poor adaptability to outdoor scenarios: Existing technologies are not optimized for outdoor scenarios of tower docking, and cannot simultaneously take into account "anti-interference" and "high precision", resulting in insufficient positioning robustness and difficulty in supporting the subsequent automatic tower docking posture adjustment requirements.

[0006] Therefore, how to provide a tower flange bolt hole center positioning method, system, and equipment based on improved Faster-RCNN coarse positioning and sub-pixel level Canny edge detection fine positioning that can effectively solve the pain points of the existing positioning technology and achieve high-precision and robust positioning of the tower flange bolt hole center in complex outdoor environments is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a method, system and equipment for locating the center of bolt holes in tower flanges based on improved Faster-RCNN coarse positioning and sub-pixel level Canny edge detection fine positioning. It aims to solve the technical problems of existing tower flange bolt hole positioning methods having low positioning accuracy, poor robustness and difficulty in adapting to automatic docking requirements under complex outdoor lighting and noise interference. It achieves sub-millimeter level positioning of bolt hole centers and provides accurate coordinate data for automatic tower docking.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: A method for locating the center of bolt holes in a tower flange based on coarse localization using an improved Faster-RCNN and fine localization using sub-pixel-level Canny edge detection includes: Step 1: Perform coarse localization of tower flange bolt holes by improving Faster-RCNN; the optimization of Faster-RCNN includes: replacing the traditional VGG16 network with ResNet50 network as the feature extraction network and using K-means clustering algorithm to cluster the bolt hole labeled dataset.

[0009] Step 2: Preprocess the image within the coarse positioning box, and use the subpixel-level Canny edge detection operator to extract the bolt hole edge features. Based on the roundness and length of the bolt hole edge, filter and remove interfering edges.

[0010] Step 3: Use the least squares method to fit the edges of the filtered and rejected bolt holes to an ellipse, and solve for the two-dimensional coordinates of the bolt hole center in the image coordinate system. Then, use a depth camera to obtain the depth information of the bolt hole center. Combine the calibrated camera intrinsic parameter matrix to convert the two-dimensional coordinates into three-dimensional coordinates in the camera coordinate system, and complete the bolt hole center positioning.

[0011] Optionally, in step 1, the ResNet50 network includes: one input module, four residual modules, and one output module.

[0012] Optionally, in step 1, the K-means clustering algorithm is used to cluster the bolt hole annotation dataset, specifically as follows: With the goal of minimizing Euclidean distance, the number of clusters k=3 was set, resulting in three anchor frames suitable for circular bolt hole shapes. The aspect ratio of the anchor frames is close to 1:1, and the dimensions are as follows: , , Pixel.

[0013] Optionally, in step 2, preprocessing includes: grayscale conversion, median filtering, and linear grayscale transformation.

[0014] Optionally, in step 2, the sub-pixel level Canny edge detection operator is used to extract the bolt hole edge features, specifically: Gaussian filtering is applied to the preprocessed image; Calculate the pixel horizontal gradient using the Sobel operator. with vertical gradient And based on the formula Calculate the gradient magnitude based on the formula Calculate the gradient direction; Perform non-maximum suppression along the gradient direction, determine whether a pixel is a local maximum, and preserve candidate edges; Double threshold detection is performed by setting a high threshold and a low threshold. Edges with amplitudes higher than the high threshold are retained, while edges with amplitudes lower than the low threshold are removed. Edges with amplitudes between the two are only retained when connecting to edges with the high threshold.

[0015] Optionally, in step 2, interfering edges are filtered and removed based on the roundness and length of the bolt hole edges, specifically as follows: Set roundness and length thresholds to remove interference edges whose roundness and length do not meet the thresholds.

[0016] Optionally, in step 3, the least squares method is used to fit an ellipse to the edges of the filtered and rejected bolt holes, and the two-dimensional coordinates of the bolt hole center in the image coordinate system are obtained, specifically: Construct the equation of the ellipse as follows:

[0017] in, These are the edge pixel coordinates; Construct the objective function as follows:

[0018] Solve for the elliptic parameters A, B, C, D, and E by deriving the linear equations using the principle of extrema; The two-dimensional coordinates of the bolt hole center in the image coordinate system are calculated based on the ellipse parameters, as follows:

[0019] Then convert the two-dimensional coordinates ( Convert to pixel coordinates .

[0020] Optionally, in step 3, the depth information of the bolt hole center is obtained through a depth camera, and combined with the calibrated camera intrinsic parameter matrix, the two-dimensional coordinates are converted into three-dimensional coordinates in the camera coordinate system, specifically: The depth information is the distance from the center of the bolt hole to the optical center of the camera; Camera intrinsic parameter matrix, including: focal length, image center coordinates; The transformation formula for three-dimensional coordinates is as follows:

[0021] in, This represents the distance from the center of the bolt hole to the optical center of the camera, and also the depth coordinate of the three-dimensional coordinates of the bolt hole center in the camera coordinate system. , The focal length of the camera; , The coordinates of the image center; It is a two-dimensional coordinate system.

[0022] This invention also provides a system for locating the center of bolt holes in tower flanges using an improved Faster-RCNN coarse localization and sub-pixel-level Canny edge detection fine localization method, comprising: Coarse localization module: used to coarsely locate the bolt holes of the tower flange by improving Faster-RCNN; the optimization of Faster-RCNN includes: using ResNet50 network to replace the traditional VGG16 network as the feature extraction network and using K-means clustering algorithm to cluster the bolt hole labeled dataset; Fine positioning module: used to preprocess the image within the coarse positioning box, and use the sub-pixel level Canny edge detection operator to extract the bolt hole edge features, and filter out interfering edges based on the roundness and length of the bolt hole edge; Bolt hole center positioning module: It is used to perform ellipse fitting on the edges of the filtered and rejected bolt holes using the least squares method, solve for the two-dimensional coordinates of the bolt hole center in the image coordinate system, and obtain the depth information of the bolt hole center through a depth camera. Combined with the calibrated camera intrinsic parameter matrix, the two-dimensional coordinates are converted into three-dimensional coordinates in the camera coordinate system to complete the bolt hole center positioning.

[0023] The present invention also provides an electronic device, comprising: Memory, used to store computer programs; A processor, configured to execute a computer program, implement the steps of a tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization as claimed in any one of claims 1-8.

[0024] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a method, system and equipment for locating the center of bolt holes in tower flanges based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization, achieving the following beneficial effects: (1) The dual-layer architecture of deep learning coarse localization and traditional image processing fine localization improves the anti-interference ability from the source and adapts to complex outdoor environments.

[0025] (2) Overcome the problem of low detection accuracy under high IOU threshold of mainstream deep learning models and reduce the offset of prediction boxes.

[0026] (3) Balancing efficiency and robustness to adapt to engineering application scenarios. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the method flow provided by the present invention.

[0029] Figure 2 A schematic diagram of the improved Faster-RCNN structure provided by this invention.

[0030] Figure 3 This is a schematic diagram of the ResNet50 network structure provided by the present invention.

[0031] Figure 4 This is a schematic diagram of the optimized anchor frame provided by the present invention.

[0032] Figure 5 This diagram illustrates the performance comparison between the improved Faster-RCNN and the traditional model provided by this invention.

[0033] Figure 6(a) is a schematic diagram of the original threaded hole provided by the present invention.

[0034] Figure 6(b) is a schematic diagram of the threaded hole after image preprocessing provided by the present invention.

[0035] Figure 6(c) is a schematic diagram of the threaded hole after edge detection provided by the present invention.

[0036] Figure 6(d) is a schematic diagram of the threaded hole after edge screening provided by the present invention.

[0037] Figure 7This is a schematic diagram illustrating the ellipse fitting effect provided by the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Example 1: Embodiment 1 of this invention discloses a method for locating the center of bolt holes in tower flanges based on coarse localization using improved Faster-RCNN and fine localization using sub-pixel-level Canny edge detection, such as... Figure 1 As shown, it includes: Preparation for image acquisition and transmission: Build a visual positioning hardware system, using an Intel RealSense D405 binocular depth camera as the image acquisition device to obtain RGB images and depth information of the tower flange; fix the depth camera to the camera bracket to ensure that its field of view fully covers the bolt hole area of ​​the tower flange, and no auxiliary light source is required (suitable for outdoor scenarios without additional light sources); connect the camera to the computer through the Intel RealSense SDK (pyrealsense2) to realize real-time image acquisition and data transmission.

[0040] Step 1: By means of Figure 2 The improved Faster-RCNN shown performs coarse localization of bolt holes on tower flanges, that is, it selects the approximate location of bolt holes in the image. This operation can eliminate the influence of the background on subsequent processing and improve localization efficiency. Among them, the optimizations of the improved Faster-RCNN include: backbone network replacement: using a ResNet50 network to replace the traditional VGG16 network as the feature extraction network and anchor box optimization: using the K-means clustering algorithm to cluster the bolt hole labeled dataset.

[0041] The ResNet50 network avoids deep gradient vanishing through residual connections, improving its ability to distinguish bolt holes from the background. Figure 3 As shown, it includes: 1 input module, 4 residual modules and 1 output module, with a total network structure of 50 layers. It can progressively extract low-level texture features and high-level semantic features of bolt holes, improving the ability to distinguish between background (such as tower main pipe, grass) and bolt holes.

[0042] The K-means clustering algorithm was used to cluster the bolt hole labeled dataset (including 272 tower flange images with 3416 bolt hole instances), specifically as follows: With the goal of minimizing Euclidean distance, the number of clusters, k=3 (balancing accuracy and computational efficiency), is set to obtain three anchor frames suitable for the circular shape of bolt holes, such as... Figure 4 As shown, the aspect ratios of the anchor frames are all close to 1:1 (specifically 1:1, 1:1.06, and 1.1:1), and their dimensions are as follows: , , , To address the issue of prediction frame offset caused by the mismatch between the traditional anchor frame and bolt hole shape, an additional 384² pixels are added to cover bolt holes of larger diameters, improving the model's adaptability to detecting bolt holes of different tower specifications.

[0043] Coarse localization execution: The acquired tower flange images are scaled to a uniform size before being fed into the ResNet50 feature extraction network. ResNet50 extracts feature information from the input image through a series of convolutions, residual connections, pooling, and other operations to form a feature map. Then, the feature map is fed into the RPN network, which introduces a sliding window on the feature map to generate a series of anchor boxes of different sizes. Candidate regions are generated by evaluating the classification score and bounding box regression score of each anchor box. Next, after non-maximum suppression, the remaining candidate regions are projected back onto the feature map to obtain the corresponding feature matrix. Each feature matrix is ​​scaled to a uniform 7×7 feature map through region of interest pooling. Finally, the feature map is flattened and passed through a series of fully connected layers to obtain the predicted position of the flange bolt holes.

[0044] Performance verification such as Figure 5 As shown, when IOU=0.75, the improved model mAP=0.950, which is better than the traditional Faster-RCNN (0.483) and YOLO-V5 (0.906), with a detection time of 0.147s, providing an efficient coarse localization basis for accuracy experiments.

[0045] Step 2: Preprocess the image within the coarse positioning box to reduce the impact of noise, highlight the edge features of the bolt holes, and use the sub-pixel level Canny edge detection operator to extract the edge features of the bolt holes. Based on the roundness and length of the bolt hole edges, filter and remove interfering edges.

[0046] Preprocessing includes: grayscale conversion (converting the RGB image to a single-channel grayscale image to simplify subsequent processing), median filtering (3×3 kernels) to remove salt-and-pepper noise from the image and preserve the edge details of the bolt holes, and image enhancement techniques to perform linear grayscale transformation in the image spatial domain to improve the contrast between the bolt hole edges and the background, ensuring the effectiveness of subsequent edge detection.

[0047] The bolt hole edge features are extracted using a sub-pixel level Canny edge detection operator, specifically: Gaussian filtering (5×5 Gaussian kernel) was applied to the preprocessed image to further reduce noise; Calculate the pixel horizontal gradient using the Sobel operator. with vertical gradient And based on the formula Calculate the gradient magnitude based on the formula Calculate the gradient direction; Perform non-maximum suppression along the gradient direction, determine whether a pixel is a local maximum, and preserve candidate edges; Set a high threshold (150) and a low threshold (50) to perform double threshold detection. Retain edges with amplitudes higher than the high threshold and remove edges with amplitudes lower than the low threshold. Edges with amplitudes between the two are only retained when connecting edges with the high threshold.

[0048] Interfering edges are filtered and eliminated based on the roundness (approaching a circle) and length of the bolt hole edges, specifically: Set a roundness threshold (≥ (0.8-1.0)) and a length threshold (≥ 50 pixels) to remove recessed or burr-like interference edges on the bolt hole edges that do not meet the roundness and length thresholds.

[0049] The edge extraction results are shown in Figures 6(a)-6(d); where Figure 6(a) is the original image of the threaded hole, Figure 6(b) is the image of the threaded hole after image preprocessing, Figure 6(c) is the image of the threaded hole after edge detection, and Figure 6(d) is the image of the threaded hole after edge filtering.

[0050] Step 3: Due to camera perspective deviation, the bolt holes appear elliptical in the image. Therefore, the least squares method is used to fit the edges of the filtered and removed bolt holes to an ellipse. The fitting result is as follows: Figure 7 As shown, the two-dimensional coordinates of the bolt hole center in the image coordinate system are obtained by solving the problem. The depth information of the bolt hole center is obtained by using a depth camera. Combined with the calibrated camera intrinsic parameter matrix, the two-dimensional coordinates are converted into three-dimensional coordinates in the camera coordinate system to complete the positioning of the bolt hole center.

[0051] The least squares method was used to fit an ellipse to the edges of the filtered and rejected bolt holes, and the two-dimensional coordinates of the bolt hole center in the image coordinate system were obtained. Specifically: Construct the equation of the ellipse as follows:

[0052] in, These are the edge pixel coordinates; Construct the objective function as follows:

[0053] Solve for the elliptic parameters A, B, C, D, and E by deriving the linear equations using the principle of extrema; The two-dimensional coordinates of the bolt hole center in the image coordinate system are calculated based on the ellipse parameters, as follows:

[0054] Then convert the two-dimensional coordinates ( Convert to pixel coordinates .

[0055] The depth information of the bolt hole center is obtained by using a depth camera. Combined with the calibrated camera intrinsic parameter matrix, the two-dimensional coordinates are converted into three-dimensional coordinates in the camera coordinate system. Specifically: The depth information is the distance from the center of the bolt hole to the optical center of the camera; The camera intrinsic parameter matrix was obtained using a traditional checkerboard calibration method. The checkerboard grid has a side length of 25mm, a pattern array of 12×9, and an accuracy of ±0.01mm. During calibration, checkerboard images were captured from multiple angles to ensure the accuracy of the intrinsic parameters. The camera intrinsic parameter matrix includes: focal length and image center coordinates. The transformation formula for three-dimensional coordinates is as follows:

[0056] in, This represents the distance from the center of the bolt hole to the optical center of the camera, and also the depth coordinate of the three-dimensional coordinates of the bolt hole center in the camera coordinate system. , The focal length of the camera; , The coordinates of the image center; It is a two-dimensional coordinate system.

[0057] Example 2: Embodiment 2 of this invention discloses a system for locating the center of bolt holes in tower flanges using an improved Faster-RCNN coarse localization and sub-pixel-level Canny edge detection fine localization method, comprising: Coarse localization module: used to coarsely locate the bolt holes of the tower flange by improving Faster-RCNN; the optimization of Faster-RCNN includes: using ResNet50 network to replace the traditional VGG16 network as the feature extraction network and using K-means clustering algorithm to cluster the bolt hole labeled dataset; Fine positioning module: used to preprocess the image within the coarse positioning box, and use the sub-pixel level Canny edge detection operator to extract the bolt hole edge features, and filter out interfering edges based on the roundness and length of the bolt hole edge; Bolt hole center positioning module: It is used to perform ellipse fitting on the edges of the filtered and rejected bolt holes using the least squares method, solve for the two-dimensional coordinates of the bolt hole center in the image coordinate system, and obtain the depth information of the bolt hole center through a depth camera. Combined with the calibrated camera intrinsic parameter matrix, the two-dimensional coordinates are converted into three-dimensional coordinates in the camera coordinate system to complete the bolt hole center positioning.

[0058] Example 3: Embodiment 3 of the present invention discloses an electronic device, comprising: Memory, used to store computer programs; A processor, configured to execute a computer program, implement the steps of a tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization as claimed in any one of claims 1-8.

[0059] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0060] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for locating the center of bolt holes in a tower flange based on coarse localization using an improved Faster-RCNN and fine localization using sub-pixel-level Canny edge detection, characterized in that... include: Step 1: Perform coarse localization of tower flange bolt holes using an improved Faster-RCNN; wherein, the optimization of the improved Faster-RCNN includes: replacing the traditional VGG16 network with a ResNet50 network as the feature extraction network and using the K-means clustering algorithm to cluster the bolt hole labeled dataset; Step 2: Preprocess the image within the coarse positioning box, and use the subpixel-level Canny edge detection operator to extract the bolt hole edge features, and filter out interfering edges based on the roundness and length of the bolt hole edges; Step 3: Use the least squares method to fit the edges of the filtered and rejected bolt holes to an ellipse, and solve for the two-dimensional coordinates of the bolt hole center in the image coordinate system. Then, use a depth camera to obtain the depth information of the bolt hole center. Combine the calibrated camera intrinsic parameter matrix to convert the two-dimensional coordinates into three-dimensional coordinates in the camera coordinate system, and complete the bolt hole center positioning.

2. The tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization according to claim 1, characterized in that, In step 1, the ResNet50 network includes: one input module, four residual modules, and one output module.

3. The tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization according to claim 1, characterized in that, In step 1, the K-means clustering algorithm is used to cluster the bolt hole annotation dataset, specifically as follows: With the goal of minimizing Euclidean distance, the number of clusters, k=3, was set to obtain three anchor frames suitable for the circular shape of bolt holes. The aspect ratio of the anchor frames is close to 1:1, and the dimensions are as follows: , , Pixel.

4. The tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization according to claim 1, characterized in that, In step 2, the preprocessing includes: grayscale conversion, median filtering, and linear grayscale transformation.

5. The tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization according to claim 1, characterized in that, In step 2, the sub-pixel level Canny edge detection operator is used to extract the edge features of the bolt holes, specifically: Gaussian filtering is applied to the preprocessed image; Calculate the pixel horizontal gradient using the Sobel operator. with vertical gradient And based on the formula Calculate the gradient magnitude based on the formula Calculate the gradient direction; Perform non-maximum suppression along the gradient direction, determine whether a pixel is a local maximum, and preserve candidate edges; Double threshold detection is performed by setting a high threshold and a low threshold. Edges with amplitudes higher than the high threshold are retained, while edges with amplitudes lower than the low threshold are removed. Edges with amplitudes between the two are only retained when connecting to edges with the high threshold.

6. The tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization according to claim 1, characterized in that, In step 2, interfering edges are filtered and removed based on the roundness and length of the bolt hole edges, specifically as follows: Set roundness and length thresholds to remove interference edges whose roundness and length do not meet the thresholds.

7. The tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization according to claim 1, characterized in that, In step 3, the least squares method is used to fit an ellipse to the edges of the filtered and rejected bolt holes, and the two-dimensional coordinates of the bolt hole center in the image coordinate system are obtained, specifically: Construct the equation of the ellipse as follows: in, These are the edge pixel coordinates; Construct the objective function as follows: Solve for the elliptic parameters A, B, C, D, and E by deriving the linear equations using the principle of extrema; Based on the ellipse parameters, the two-dimensional coordinates of the bolt hole center in the image coordinate system are calculated as follows: Then the two-dimensional coordinates ( Convert to pixel coordinates .

8. The tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization according to claim 1, characterized in that, In step 3, the depth information of the bolt hole center is obtained through a depth camera, and combined with the calibrated camera intrinsic parameter matrix, the two-dimensional coordinates are converted into three-dimensional coordinates in the camera coordinate system, specifically as follows: The depth information is the distance from the center of the bolt hole to the optical center of the camera; The camera intrinsic parameter matrix includes: focal length and image center coordinates; The transformation formula for the three-dimensional coordinates is as follows: in, This represents the distance from the center of the bolt hole to the optical center of the camera, and also the depth coordinate of the three-dimensional coordinates of the bolt hole center in the camera coordinate system. , The focal length of the camera; , The coordinates of the image center; The coordinates are the two-dimensional coordinates.

9. A system for locating the center of a tower flange bolt hole based on an improved Faster-RCNN coarse localization and sub-pixel-level Canny edge detection fine localization method according to any one of claims 1-8, characterized in that, include: Coarse localization module: used to coarsely locate the bolt holes of the tower flange using an improved Faster-RCNN; wherein, the optimization of the improved Faster-RCNN includes: replacing the traditional VGG16 network with a ResNet50 network as the feature extraction network and using the K-means clustering algorithm to cluster the bolt hole labeled dataset; Fine positioning module: used to preprocess the image within the coarse positioning box, and use the sub-pixel level Canny edge detection operator to extract the bolt hole edge features, and filter out interfering edges based on the roundness and length of the bolt hole edge; Bolt hole center positioning module: It is used to perform ellipse fitting on the edges of the filtered and rejected bolt holes using the least squares method, solve for the two-dimensional coordinates of the bolt hole center in the image coordinate system, and obtain the depth information of the bolt hole center through a depth camera. Combined with the calibrated camera intrinsic parameter matrix, the two-dimensional coordinates are converted into three-dimensional coordinates in the camera coordinate system to complete the bolt hole center positioning.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of a tower flange bolt hole center positioning method based on improved Faster-RCNN coarse localization and sub-pixel level Canny edge detection fine localization as described in any one of claims 1-8.

Images