Bolt positioning method, device and equipment

Abstract

The invention discloses a bolt positioning method, device and equipment, and the method comprises the steps: S1, inputting an initial bolt image obtained by a depth camera into a preset YOLOv11 model for bolt detection, and obtaining a bolt positioning block diagram; s2, extracting the ROI of the bolt according to the initial bolt image and the bolt positioning block diagram based on a Canny edge detection algorithm, and obtaining a bolt ROI map; s3, calculating the coordinates of the current center point of the bolt according to the ROI map of the bolt by adopting a Hough transform algorithm; s4, if the current center point coordinate does not meet the preset deviation condition, the shooting posture of the depth camera is adjusted, the step S1 is executed until the optimized center point coordinate is obtained, and the shooting posture comprises the shooting direction and the shooting angle. The technical problem that the bolt positioning result is lack of accuracy and robustness due to the fact that the noise influence of complex illumination or perspective distortion in the environment is difficult to eliminate in the prior art can be solved.

Description

Technical Field

[0001] This application relates to the technical field of power grid equipment, and particularly to a bolt positioning method, device, and equipment. Background Art

[0002] In live high-voltage operations, the disassembly and assembly operations of distribution line insulators play a crucial role and currently mainly rely on manual operations. In this process, workers are exposed to a high-voltage environment and undertake high-intensity labor, facing relatively high safety risks. With the development of industrial automation technology, robotic arms are increasingly widely used in the field of bolt detection and operation; however, since bolts are usually installed at high altitudes or in complex outdoor environments, traditional bolt detection and positioning technologies still have significant deficiencies in terms of accuracy and stability.

[0003] Therefore, the prior art usually uses a camera to take pictures above the bolts to be operated, quickly detects and locates the bolts through a neural network model, and then uses some image processing technologies to obtain the center point coordinates of the bolts as the positioning result. However, currently, such bolt positioning methods are still difficult to adapt to the noise effects caused by complex lighting or perspective distortion, resulting in the lack of accuracy and robustness of the actual positioning results and being difficult to apply in actual scenarios. Summary of the Invention

[0004] This application provides a bolt positioning method, device, and equipment to solve the technical problem that the prior art is difficult to eliminate the noise effects of complex lighting or perspective distortion in the environment, resulting in the lack of accuracy and robustness of the bolt positioning results.

[0005] In view of this, the first aspect of this application provides a bolt positioning method, including:

[0006] S1: Input the initial bolt image obtained by a depth camera into a preset YOLOv11 model for bolt detection to obtain a bolt positioning block diagram;

[0007] S2: Based on the Canny edge detection algorithm, extract the ROI of the bolt according to the initial bolt image and the bolt positioning block diagram to obtain a bolt ROI image;

[0008] S3: Use the Hough transform algorithm to calculate the current center point coordinates of the bolt based on the bolt ROI image;

[0009] S4: If the current center point coordinates do not meet the preset deviation condition, adjust the shooting posture of the depth camera and return to step S1 until the optimized center point coordinates are obtained. The shooting posture includes the shooting direction and shooting angle.

[0010] Preferably, before step S1, it further includes:

[0011] Obtain the bolt video stream in a complex environment in real time by adjusting the camera parameters of the depth camera;

[0012] Slice the bolt video stream into frame images to obtain initial bolt images.

[0013] Preferably, step S2 includes:

[0014] After converting the initial bolt image into a grayscale bolt image, use a Gaussian filter to denoise the grayscale bolt image to obtain a denoised bolt image;

[0015] Perform binarization processing on the denoised bolt image based on an adaptive threshold segmentation algorithm to generate a binary bolt image;

[0016] Based on the Canny edge detection algorithm, combine the bolt positioning block diagram and the binary bolt image to extract the ROI of the bolt to obtain a bolt ROI map.

[0017] Preferably, step S3 includes:

[0018] Determine multiple target line segments in the bolt ROI map based on the Hough transform algorithm, and the target line segments include endpoint coordinates;

[0019] Parametrize the straight line where the target line segment is located according to the endpoint coordinates to obtain a straight line equation;

[0020] Simultaneously solve the straight line equations corresponding to two adjacent target line segments and solve the intersection coordinates of the two lines to obtain an intersection coordinate sequence;

[0021] Based on the centroid calculation method, calculate the centroid of the polygon according to the intersection coordinate sequence to obtain the current center point coordinates of the bolt.

[0022] Preferably, step S4 includes:

[0023] Calculate the distance deviation value between the current center point coordinates and the reference center point coordinates;

[0024] If the distance deviation value is less than the distance threshold, the preset deviation condition is satisfied, and the current center point coordinates are used as the optimized center point coordinates;

[0025] If the distance deviation value is not less than the distance threshold, the preset deviation condition is not satisfied, adjust the shooting posture of the depth camera according to the distance deviation value, and return to step S1 until the optimized center point coordinates are obtained.

[0026] A second aspect of the present application provides a bolt positioning device, including:

[0027] A model positioning unit, used for inputting the initial bolt image acquired by the depth camera into a preset YOLOv11 model for bolt detection to obtain a bolt positioning block diagram;

[0028] An edge detection unit, configured to extract the ROI of the bolt according to the initial bolt image and the bolt positioning block diagram based on a Canny edge detection algorithm, and obtain a bolt ROI diagram;

[0029] A center calculation unit, used to calculate the current center point coordinates of the bolt according to the bolt ROI map using a Hough transform algorithm;

[0030] The deviation optimization unit is used to adjust the shooting posture of the depth camera and trigger the model positioning unit if the current center point coordinates do not meet the preset deviation conditions until the optimized center point coordinates are obtained, and the shooting posture includes a shooting direction and a shooting angle.

[0031] Preferably, the edge detection unit is specifically used for:

[0032] After converting the initial bolt image into a grayscale bolt image, a Gaussian filter is used to perform denoising on the grayscale bolt image to obtain a denoised bolt image;

[0033] Binarization is performed on the denoised bolt image based on an adaptive threshold segmentation algorithm to generate a binary bolt image;

[0034] Based on the Canny edge detection algorithm, the ROI of the bolt is extracted in combination with the bolt positioning block diagram and the binary bolt image to obtain a bolt ROI map.

[0035] Preferably, the central computing unit is specifically used for:

[0036] Determine a plurality of target line segments in the bolt ROI map based on a Hough transform algorithm, wherein the target line segments include endpoint coordinates;

[0037] Performing parameterization processing on the straight line where the target line segment is located according to the endpoint coordinates to obtain a straight line equation;

[0038] The straight line equations corresponding to two adjacent target line segments are combined, and the coordinates of the intersection of the two lines are solved to obtain a sequence of intersection coordinates;

[0039] The polygon centroid calculation is performed based on the intersection coordinate sequence based on the centroid calculation method to obtain the current center point coordinates of the bolt.

[0040] Preferably, the deviation optimization unit is specifically used for:

[0041] Calculating the distance deviation between the current center point coordinates and the reference center point coordinates;

[0042] If the distance deviation value is less than the distance threshold, the preset deviation condition is satisfied, and the current center point coordinates are used as the optimized center point coordinates;

[0043] If the distance deviation value is not less than the distance threshold, the preset deviation condition is not satisfied. Adjust the shooting pose of the depth camera according to the distance deviation value, and trigger the model positioning unit until the optimized center point coordinates are obtained.

[0044] A third aspect of the present application provides a bolt positioning device, which includes a processor and a memory;

[0045] The memory is used to store program code and transmit the program code to the processor;

[0046] The processor is used to execute the bolt positioning method described in the first aspect according to the instructions in the program code.

[0047] From the above technical solutions, it can be seen that the embodiments of the present application have the following advantages:

[0048] In the present application, a bolt positioning method is provided, including: S1: Input the initial bolt image obtained by the depth camera into a preset YOLOv11 model for bolt detection to obtain a bolt positioning block diagram; S2: Based on the Canny edge detection algorithm, extract the ROI of the bolt according to the initial bolt image and the bolt positioning block diagram to obtain a bolt ROI map; S3: Use the Hough transform algorithm to calculate the current center point coordinates of the bolt based on the bolt ROI map; S4: If the current center point coordinates do not meet the preset deviation condition, adjust the shooting pose of the depth camera and return to step S1 until the optimized center point coordinates are obtained. The shooting pose includes the shooting direction and the shooting angle.

[0049] A bolt positioning method provided by the present application optimizes bolt positioning from the data source through an image acquisition iterative optimization method; in this process, not only is it necessary to perform bolt positioning detection based on a preset YOLOv11 model, but it is also necessary to calculate clear center point coordinates based on the Canny edge detection algorithm and the Hough transform algorithm; then perform deviation analysis on this coordinate, and the specific optimized center point coordinates can be obtained only when the preset deviation condition is met, otherwise the shooting pose of the depth camera is adjusted to improve the lighting problem and distortion problem caused by the shooting angle and shooting direction of the bolt image, thereby ensuring the accuracy and robustness of the positioning result. Therefore, the present application can solve the technical problem that it is difficult for the prior art to eliminate the noise influence of complex lighting or perspective distortion in the environment, resulting in the lack of accuracy and robustness of the bolt positioning result. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic flowchart of a bolt positioning method provided by an embodiment of the present application;

[0051] Figure 2 Schematic structural diagram of a bolt positioning device provided by an embodiment of the present application;

[0052] Figure 3 Example of a bolt coordinate image obtained by combining lines and intersection coordinates provided by an embodiment of the present application Figure 1 ;

[0053] Figure 4 Example of a bolt coordinate image obtained by combining lines and intersection coordinates provided by an embodiment of the present application Figure 2 ;

[0054] Figure 5 Schematic diagram of the overall bolt positioning process provided by an embodiment of the present application;

[0055] Figure 6 Comparison example diagram of bolt positioning effects under different weather conditions provided by an application example of the present application. Detailed implementation manners

[0056] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0057] For ease of understanding, please refer to Figure 1 , an embodiment of a bolt positioning method provided by the present application, including:

[0058] Step 101: Input the initial bolt image obtained by a depth camera into a preset YOLOv11 model for bolt detection to obtain a bolt positioning block diagram.

[0059] Further, before step 101, it further includes:

[0060] Real-time acquisition of a bolt video stream in a complex environment by adjusting the camera parameters of the depth camera;

[0061] Segment the bolt video stream into frame images to obtain an initial bolt image.

[0062] It should be noted that the depth camera in this embodiment is installed on the robotic arm or fixed bracket, and can collect the bolt video stream in real time, which can ensure the coverage of all surface features of the bolt, and can also obtain information on different lighting, occlusion and angles, ensuring that the bolt video stream and the initial bolt image fully conform to the actual engineering conditions and environmental situations, thereby ensuring the reliability of the positioning result. In order to obtain a high-quality bolt video stream, the parameters of the depth camera, such as the exposure rate and focal length, can be dynamically adjusted. The bolt video stream can be divided into frame images in time sequence, and multiple frames of initial bolt images are obtained.

[0063] The preset YOLOv11 model is a bolt detection model pre-constructed and trained, with good detection and positioning performance, and can be directly used for bolt detection tasks in real-time scenarios; the data for training the model is also bolt images. Historical bolt images can be obtained and preprocessing operations can be performed to optimize the image quality and enhance the quantity to ensure the accuracy of model training, which is not specifically limited and can be designed according to the actual situation. Moreover, the bolt positioning block diagram output by the model includes the coordinates of key corner points, which can provide basic calculation data for subsequent center coordinate calculation.

[0064] Step 102: Based on the Canny edge detection algorithm, extract the ROI of the bolt according to the initial bolt image and the bolt positioning block diagram to obtain the bolt ROI map.

[0065] Furthermore, step 102 includes:

[0066] After converting the initial bolt image into a grayscale bolt image, use a Gaussian filter to denoise the grayscale bolt image to obtain a denoised bolt image;

[0067] Based on the adaptive threshold segmentation algorithm, perform binary processing on the denoised bolt image to generate a binary bolt image;

[0068] Based on the Canny edge detection algorithm, combine the bolt positioning block diagram and the binary bolt image to extract the ROI of the bolt to obtain the bolt ROI map.

[0069] The obtained initial bolt image is a color image extracted from the bolt video stream. It can be first converted into a grayscale bolt image; then use a Gaussian filter to denoise the grayscale bolt image to generate a denoised bolt image; then, based on the adaptive threshold segmentation algorithm, design a 25×25 moving window, and perform binary processing on the denoised bolt image through this window to generate a high-quality binary image, that is, a binary bolt image; then, based on the Canny edge detection algorithm, extract the bolt edge contour in the binary bolt image, and optimize the continuity of the edge through morphological operations; finally, combine the bounding box in the bolt positioning block diagram to extract the region of interest ROI of the bolt to generate the bolt ROI map.

[0070] When extracting the edge contour of the bolt based on the Canny edge detection algorithm, the binary bolt image needs to be calculated first. The horizontal and vertical gradients of

[0071]

[0072] the image are as follows: and the direction can be expressed as:

[0073] ,

[0074] Retain the maximum value in the local gradient direction, suppress other pixels to 0, and then define the high threshold and the low threshold to generate the edge contour image:

[0075]

[0076] Among them, is the edge contour image, is the gradient magnitude.

[0077] Step 103: Use the Hough transform algorithm to calculate the current center point coordinates of the bolt based on the bolt ROI image.

[0078] Further, step 103 includes:

[0079] Determine multiple target line segments in the bolt ROI image based on the Hough transform algorithm. The target line segments include endpoint coordinates;

[0080] Parameterize the straight line where the target line segment is located according to the endpoint coordinates to obtain the straight line equation;

[0081] Simultaneously solve the straight line equations corresponding to two adjacent target line segments, and solve the intersection coordinates of the two lines to obtain the intersection coordinate sequence;

[0082] Based on the centroid calculation method, calculate the centroid of the polygon according to the intersection coordinate sequence to obtain the current center point coordinates of the bolt.

[0083] It should be noted that the Hough transform algorithm can effectively detect straight lines in the image by converting straight lines in the image space into points in the parameter space. In the parameter space, the voting number of each straight line is statistically calculated by an accumulator, and the straight line with the voting number exceeding the preset threshold is selected as the detection result. However, due to the bolt fillet and the distortion of the bolt edge caused by the shooting angle, the six side lines obtained by the Hough transform cannot form a complete closed hexagon. Therefore, in this embodiment, further analysis is required based on the straight lines obtained by the Hough transform to obtain the intersection points of adjacent straight lines.

[0084] See also Figure 3 , after performing line extraction based on the Hough transform algorithm, a matrix is output. The first row of the matrix corresponds to line1, and its two endpoints are point1 and point2 respectively; starting from point1, the endpoints of other lines are determined in a counterclockwise direction. Specifically: line2 is determined by point3 and point5; line3 is determined by point7 and point9; line4 is determined by point11 and point12. Starting from point12, the endpoints of the remaining lines are determined in a counterclockwise direction: line5 is determined by point10 and point8; line6 is determined by point6 and point4. It can be found that the multiple target line segments obtained all include two endpoints and their corresponding endpoint coordinates.

[0085] Based on the endpoint coordinates, straight line fitting can be performed, that is, the straight line where the target line segment is located is parameterized to determine the standard expression of the straight line, that is, , taking the target line segment line1 as an example, if the coordinates of its two endpoints are , ,So , , .

[0086] For two adjacent target line segments, the corresponding two straight line equations can be combined to calculate the coordinates of the intersection of the two lines. Assume that the linear equation parameters corresponding to the target line segment line1 are , the straight line equation parameters corresponding to the target line segment line2 are , then calculate the determinant:

[0087]

[0088] like , then the intersection coordinates The calculation is as follows:

[0089] ,

[0090] Similarly, we can analyze the intersection points of all the adjacent target line segments in turn, get the intersection points of all the lines, and generate the intersection point coordinate sequence; combining the line and intersection point coordinates, we can get the bolt coordinate image. For details, please refer to Figure 3 and Figure 4 .

[0091] Then, the centroid calculation method is used to calculate the centroid of this polygon in the bolt coordinate image, and the current center point coordinates can be obtained. Specifically, the vertex sequence of the polygon needs to be determined first, that is, the intersection point coordinate sequence calculated above , where is the number of vertices; then, according to this sequence, in the clockwise or counterclockwise vertex order, calculate the polygon area:

[0092]

[0093] Among them, it can be set that , to close the polygon. If it is defined in the counterclockwise direction of the polygon, then the calculated area A is negative, and the absolute value can be taken for centroid calculation, which has no impact. Using the polygon area A and the vertex sequence, the coordinates of the centroid of the polygon can be calculated:

[0094]

[0095]

[0096] These centroid coordinates are regarded as the current center point coordinates of the bolt .

[0097] Step 104: If the current center point coordinates do not meet the preset deviation condition, adjust the shooting posture of the depth camera and return to step 101 until the optimized center point coordinates are obtained. The shooting posture includes the shooting direction and the shooting angle.

[0098] Furthermore, step 104 includes:

[0099] Calculate the distance deviation value between the current center point coordinates and the reference center point coordinates;

[0100] If the distance deviation value is less than the distance threshold, the preset deviation condition is met, and the current center point coordinates are used as the optimized center point coordinates;

[0101] If the distance deviation value is not less than the distance threshold, the preset deviation condition is not met. Adjust the shooting posture of the depth camera according to the distance deviation value and return to step S1 until the optimized center point coordinates are obtained.

[0102] It should be noted that the reference center point coordinates in this embodiment refer to the center point coordinates calculated last time; the center point coordinates calculated for the first time can be denoted as , which is also the reference center point coordinates relative to the second calculation. Moreover, since there is no reference for the first time, the shooting posture of the depth camera is adjusted directly based on this reference center point coordinates. After that, the current center point coordinates calculated All can calculate the distance deviation value from the reference center point coordinates , and perform distance deviation threshold judgment and analysis; if the distance threshold this preset deviation condition is not met, the shooting attitude of the depth camera is adjusted according to the distance deviation value until the current center point coordinates that meet the conditions are obtained, that is, the optimized center point coordinates. For the overall bolt positioning process of this embodiment, please refer to Figure 5 .

[0103] Among them, the adjustment of the shooting attitude of the depth camera specifically means: gradually aligning the optical axis of the depth camera with the normal direction of the reference point; then adjusting the pitch angle and yaw angle of the depth camera. The adjustment of the shooting attitude can reduce the perspective distortion and the influence of different illuminations on the bolt image, ensure that the projection of the image is as close to a perfect circle as possible, and facilitate the optimized calculation of the center point coordinates.

[0104] For the sake of easy understanding, this application has carried out simulation and actual scene tests, collected bolt images under sunny and cloudy days respectively, and carried out bolt positioning through the method proposed in this application. The positioning results obtained can be referred to Figure 6 ; according to the detection results, it can be found that the method proposed in this application can adapt to the influence of different illuminations in different environments on the accuracy and robustness of bolt positioning, can achieve better positioning, and meet the requirements of actual working conditions.

[0105] A bolt positioning method provided by an embodiment of this application optimizes bolt positioning from the data source through the way of image acquisition and iterative optimization; in this process, not only the bolt positioning detection needs to be carried out based on the preset YOLOv11 model, but also the clear center point coordinates need to be calculated based on the Canny edge detection algorithm and the Hough transform algorithm; then the deviation analysis of this coordinate is carried out, and the specific optimized center point coordinates can be obtained only when the preset deviation conditions are met, otherwise the attitude of the depth camera is adjusted to improve the illumination problem and distortion problem of the bolt image caused by the shooting angle and shooting direction, so as to ensure the accuracy and robustness of the positioning result. Therefore, the embodiment of this application can solve the technical problem that it is difficult for the existing technology to eliminate the noise influence of complex illumination or perspective distortion in the environment, resulting in the lack of accuracy and robustness of the bolt positioning result.

[0106] For the sake of easy understanding, please refer to Figure 2 , this application provides an embodiment of a bolt positioning device, including:

[0107] A model positioning unit 201, configured to input the initial bolt image obtained by the depth camera into a preset YOLOv11 model for bolt detection to obtain a bolt positioning block diagram;

[0108] The edge detection unit 202 is used to extract the ROI of the bolt according to the initial bolt image and the bolt positioning block diagram based on the Canny edge detection algorithm to obtain a bolt ROI map;

[0109] A center calculation unit 203, used to calculate the current center point coordinates of the bolt according to the bolt ROI map using a Hough transform algorithm;

[0110] The deviation optimization unit 204 is used to adjust the shooting posture of the depth camera and trigger the model positioning unit if the current center point coordinates do not meet the preset deviation conditions until the optimized center point coordinates are obtained. The shooting posture includes the shooting direction and the shooting angle.

[0111] Further, the edge detection unit 202 is specifically configured to:

[0112] After converting the initial bolt image into a grayscale bolt image, a Gaussian filter is used to denoise the grayscale bolt image to obtain a denoised bolt image;

[0113] The denoised bolt image is binarized based on the adaptive threshold segmentation algorithm to generate a binary bolt image.

[0114] Based on the Canny edge detection algorithm, the bolt ROI is extracted by combining the bolt positioning block diagram and the binary bolt image to obtain the bolt ROI map.

[0115] Furthermore, the central computing unit 203 is specifically configured to:

[0116] Determine multiple target line segments in the bolt ROI map based on the Hough transform algorithm, and the target line segments include the coordinates of the endpoints;

[0117] According to the endpoint coordinates, the straight line where the target line segment is located is parameterized to obtain the straight line equation;

[0118] Combine the straight line equations corresponding to the two adjacent target line segments, and solve the coordinates of the intersection of the two lines to obtain the intersection coordinate sequence;

[0119] Based on the centroid calculation method, the polygon centroid is calculated according to the intersection coordinate sequence to obtain the current center point coordinates of the bolt.

[0120] Furthermore, the deviation optimization unit 204 is specifically configured to:

[0121] Calculate the distance deviation between the current center point coordinates and the reference center point coordinates;

[0122] If the distance deviation value is less than the distance threshold, the preset deviation condition is met and the current center point coordinates are used as the optimized center point coordinates;

[0123] If the distance deviation value is not less than the distance threshold, the preset deviation condition is not satisfied. Adjust the shooting pose of the depth camera according to the distance deviation value, and trigger the model positioning unit 201 until the optimized center point coordinates are obtained.

[0124] This application also provides a bolt positioning device, which includes a processor and a memory;

[0125] The memory is used to store program code and transmit the program code to the processor;

[0126] The processor is used to execute the bolt positioning method in the above method embodiment according to the instructions in the program code.

[0127] In several embodiments provided by this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional unit in each embodiment of this application can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0130] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (English full name: Read-Only Memory, English abbreviation: ROM), random access memories (English full name: Random Access Memory, English abbreviation: RAM), magnetic disks, or optical discs.

[0131] As described above, the above embodiments are only used to illustrate the technical solution of this application, rather than to limit it; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of various embodiments of this application.

Claims

1. A bolt positioning method, characterized in that, Including: S1: Input the initial bolt image obtained by the depth camera into a preset YOLOv11 model for bolt detection to obtain a bolt positioning block diagram; S2: Based on the Canny edge detection algorithm, extract the ROI of the bolt according to the initial bolt image and the bolt positioning block diagram to obtain a bolt ROI image; S3: Use the Hough transform algorithm to calculate the current center point coordinates of the bolt based on the bolt ROI image; S4: If the current center point coordinates do not meet the preset deviation condition, adjust the shooting posture of the depth camera and return to step S1 until the optimized center point coordinates are obtained. The shooting posture includes the shooting direction and the shooting angle.

2. The bolt positioning method according to claim 1, wherein Before step S1, it also includes: Obtain the bolt video stream in a complex environment in real time by adjusting the camera parameters of the depth camera; Slice the bolt video stream into frame images to obtain the initial bolt image.

3. The bolt positioning method according to claim 1, wherein, Step S2 includes: After converting the initial bolt image into a grayscale bolt image, use a Gaussian filter to denoise the grayscale bolt image to obtain a denoised bolt image; Perform binarization processing on the denoised bolt image based on the adaptive threshold segmentation algorithm to generate a binary bolt image; Based on the Canny edge detection algorithm, combine the bolt positioning block diagram and the binary bolt image to extract the ROI of the bolt to obtain a bolt ROI image.

4. The bolt positioning method according to claim 1, characterized in that, Step S3 includes: Based on the Hough transform algorithm, determine multiple target line segments in the bolt ROI image. The target line segments include endpoint coordinates; Parameterize the straight line where the target line segment is located according to the endpoint coordinates to obtain a straight line equation; Simultaneously solve the straight line equations corresponding to two adjacent target line segments and solve the intersection coordinates of the two lines to obtain an intersection coordinate sequence; Based on the centroid calculation method, perform polygon centroid calculation according to the intersection coordinate sequence to obtain the current center point coordinates of the bolt.

5. The bolt positioning method according to claim 1, wherein Step S4 includes: Calculate the distance deviation value between the current center point coordinates and the reference center point coordinates; If the distance deviation value is less than the distance threshold, it meets the preset deviation condition, and use the current center point coordinates as the optimized center point coordinates; If the distance deviation value is not less than the distance threshold, it does not meet the preset deviation condition. Adjust the shooting posture of the depth camera according to the distance deviation value and return to step S1 until the optimized center point coordinates are obtained.

6. A bolt positioning device, characterized in that, Including: A model positioning unit for inputting the initial bolt image obtained by the depth camera into a preset YOLOv11 model for bolt detection to obtain a bolt positioning block diagram; An edge detection unit for extracting the ROI of the bolt based on the Canny edge detection algorithm according to the initial bolt image and the bolt positioning block diagram to obtain a bolt ROI image; A center calculation unit for calculating the current center point coordinates of the bolt using the Hough transform algorithm based on the bolt ROI image; A deviation optimization unit for adjusting the shooting posture of the depth camera and triggering the model positioning unit if the current center point coordinates do not meet the preset deviation condition until the optimized center point coordinates are obtained. The shooting posture includes the shooting direction and the shooting angle.

7. The bolt positioning device according to claim 6, wherein, The edge detection unit is specifically used for: After converting the initial bolt image into a grayscale bolt image, a Gaussian filter is used to denoise the grayscale bolt image to obtain a denoised bolt image; Based on the adaptive threshold segmentation algorithm, the denoised bolt image is binarized to generate a binary bolt image; Based on the Canny edge detection algorithm, combined with the bolt positioning block diagram and the binary bolt image, the ROI of the bolt is extracted to obtain a bolt ROI map.

8. The bolt positioning device according to claim 6, characterized in that, The central calculation unit is specifically used for: Based on the Hough transform algorithm, multiple target line segments are determined in the bolt ROI map, and the target line segments include endpoint coordinates; According to the endpoint coordinates, parameterize the straight line where the target line segment is located to obtain a straight line equation; Simultaneously solve the straight line equations corresponding to two adjacent target line segments, and solve the intersection coordinates of the two lines to obtain an intersection coordinate sequence; Based on the centroid calculation method, calculate the centroid of the polygon according to the intersection coordinate sequence to obtain the current center point coordinates of the bolt.

9. The bolt positioning device according to claim 6, characterized in that The deviation optimization unit is specifically used for: Calculate the distance deviation value between the current center point coordinates and the reference center point coordinates; If the distance deviation value is less than the distance threshold, the preset deviation condition is satisfied, and the current center point coordinates are used as the optimized center point coordinates; If the distance deviation value is not less than the distance threshold, the preset deviation condition is not satisfied, and the shooting posture of the depth camera is adjusted according to the distance deviation value, and the model positioning unit is triggered until the optimized center point coordinates are obtained.

10. A bolt positioning device, characterized in that, The device includes a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the bolt positioning method according to any one of claims 1-5 according to the instructions in the program code.

Images