High-altitude nut center positioning method based on machine vision

Through the improved YOLOv5 network model and Zero-DCE brightness enhancement algorithm, combined with RCF edge detection and Hough circle-line detection, the problem of low positioning accuracy of high-altitude nuts is solved, and high-precision, fast, and environmentally adaptable positioning effects are achieved.

CN118982648APending Publication Date: 2024-11-19HEFEI INSTITUTE OF PHYSICAL SCIENCE CHINESE ACADEMY OF SCIENCES

Patent Information

Application Number
CN202411052919.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The positioning accuracy of high-altitude nuts is low, the positioning rate is low, and is easily affected by the environment, especially in the case of insufficient ambient light and nut wear and stains, resulting in low detection accuracy.

Method used

The improved YOLOv5 network model is used to combine Zero-DCE brightness enhancement algorithm and RCF edge detection algorithm, and weighted average is performed through Hoff circle detection and Hoff straight line detection to determine the center coordinates of the nuts, improving positioning accuracy and robustness.

Benefits of technology

It significantly improves the accuracy and speed of the center positioning of the nut, reduces the environmental impact, enhances the processing capacity of low-brightness images, and reduces the impact of wear and stains on detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982648A_ABST
    Figure CN118982648A_ABST
Patent Text Reader

Abstract

The invention relates to a high-altitude nut center positioning method based on machine vision. The method comprises the following steps: acquiring image data of a nut and preprocessing the image data; obtaining an identification image of the nut, and cutting the identification image of the nut to obtain a cut nut picture; obtaining a nut picture after brightness enhancement; obtaining a nut picture after edge detection; performing Hough circle detection to obtain a center coordinate C1 of an inner circle of the nut; performing Hough straight line detection to obtain a nut center coordinate C2; and performing weighted average to obtain a final nut center coordinate. According to the method, the interference of images around the nut on positioning is greatly reduced, useful information is screened, and useless information is removed; the brightness of the image is improved, and dark part details are increased; the method greatly improves the contour of six edges and an inner circle of a nut to be detected, reduces the contour of other images, greatly reduces the influence of nut wear and stains on subsequent Hough circle-straight line detection, and improves the precision of nut center positioning. And the robustness is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning image recognition and high-altitude nut detection, and in particular to a high-altitude nut center positioning method based on machine vision. Background Art

[0002] High-altitude work usually refers to work performed by people at a certain height, which is often accompanied by danger. The detection of high-altitude nuts is also a type of high-altitude work. The precise positioning of high-altitude nuts is used in the automatic nut tightening maintenance of robots, such as angle steel towers in power transmission projects and tower cranes in construction sites. The nuts on angle steel towers and tower cranes may be damaged or loose during use and require regular maintenance. At present, the maintenance of high-altitude nuts is basically carried out by workers with live maintenance, which is inefficient and very dangerous.

[0003] Compared with the human eye, machine vision has better fault tolerance, is not affected by subjective factors, and has high detection efficiency. With the continuous development of detection algorithms, the detection accuracy has been greatly improved, the real-time performance is good, and it can adapt to the high-speed detection tasks of modern industry. Using machines instead of manual labor to complete detection and tightening operations has greatly improved the automation and intelligence of high-altitude nut maintenance. At present, the main problems in the research of high-altitude nut positioning are: insufficient ambient light causes the nut edge and its own features to be blurred, resulting in too little nut image information, affecting the accuracy of subsequent positioning algorithms; the wear and stains of the nut itself cause many interfering contours to be detected during the edge detection process, resulting in poor positioning accuracy. Summary of the invention

[0004] In order to solve the problems of low positioning accuracy, low positioning rate and susceptibility to environmental influences in the prior art, the purpose of the present invention is to provide a high-altitude nut center positioning method based on machine vision with high accuracy, strong environmental adaptability and fast positioning speed.

[0005] To achieve the above object, the present invention adopts the following technical solution: a method for locating the center of a high-altitude nut based on machine vision, the method comprising the following steps in order:

[0006] (1) obtaining image data of the nut by photographing with a camera, and preprocessing the image data of the nut to obtain a data set, and dividing the data set into a training set and a validation set;

[0007] (2) improving the YOLOv5 network model to obtain an improved YOLOv5 network model, inputting the training set into the improved YOLOv5 network model for training, and obtaining a trained YOLOv5 network model; inputting the nut image to be detected into the trained YOLOv5 network model for recognition, obtaining a recognition image of the nut, and cropping the recognition image of the nut to obtain a cropped nut image;

[0008] (3) Use the Zero-DCE brightness enhancement algorithm to enhance the brightness of the cropped nut image to obtain a brightness enhanced nut image;

[0009] (4) Processing the brightness enhanced nut image using the RCF edge detection algorithm to obtain the nut image after edge detection;

[0010] (5) Perform Hough circle detection on the nut image after edge detection, determine the inner circle radius through camera parameters and shooting distance, and obtain the center coordinate C1 of the inner circle of the nut;

[0011] (6) Perform Hough line detection on the nut image after edge detection, determine the center point of the nut through hexagonal angle constraint and vertex constraint, and obtain the nut center coordinate C2;

[0012] (7) Take a weighted average of the obtained nut inner circle center coordinate C1 and nut center coordinate C2 to obtain the final nut center coordinate C.

[0013] In step (1), the preprocessing refers to randomly cropping the image data of the nut, randomly flipping to expand the data set, and annotating the data with labelimg.

[0014] The step (2) comprises the following steps:

[0015] (2a) The YOLOv5 network model is improved to obtain an improved YOLOv5 network model: the YOLOv5 network model includes an input end, a backbone network, a neck network and a head network. An efficient multi-scale attention mechanism is added to the backbone network to fuse multi-scale information and reduce interference from other information; a bidirectional feature pyramid network is added to the neck network to enhance the ability to fuse features of different scales and increase detection accuracy; the loss function of the YOLOv5 network model is replaced with a SIoU structured loss function to improve the model convergence speed;

[0016] (2b) Use the training data set to train the improved YOLOv5 network model until the loss function reaches convergence, update the parameters, use the sgd optimizer during training, set the momentum of the sgd optimizer to 0.937, set weight_decay to 5e-4, and train for 200 rounds in total; momentum is momentum gradient descent, and weight_decay is weight decay;

[0017] (2c) The image of the nut to be detected is input into the trained YOLO v5 model for prediction, and the recognition image of the nut is obtained and cropped to obtain the cropped nut image.

[0018] The step (3) comprises the following steps:

[0019] (3a) The Zero-DCE brightness enhancement algorithm uses the DCE network model to estimate a set of best-fit light enhancement curves, namely the L curve. The DCE network model continuously iterates the L curve until the loss function is minimized. The image enhancement second-order curve is expressed as:

[0020] L(I(x);α)=I(x)+αI(x)(1-I(x))(1),

[0021] Where: x is the pixel coordinate; L(I(x); α) is the enhanced image, α∈[-1,1] is the trainable curve parameter, and each pixel is normalized to [0,1]; I(x) is the input cropped nut image;

[0022] (3b) By iterating formula (1), we get the high-order curve:

[0023] L n (x) = L n-1 (x)+α n L n-1 (x)(1-L n-1 (x))(2),

[0024] Where n is the number of iterations, and when n is 1, it degenerates to the second order;

[0025] Replace α in the formula with matrix A to obtain the pixel curve:

[0026] L n (x) = L n-1 (x)+A n L n-1 (x)(1-L n-1 (x))(3),

[0027] In formula (3), L n (x) is the picture of the nut after brightness enhancement.

[0028] The step (5) comprises the following steps:

[0029] (5a) Convert the nut image after edge detection into a grayscale image and remove noise using Gaussian filtering to obtain a grayscale image after noise removal;

[0030] (5b) Use the Canny operator to extract the edge of the grayscale image after removing the noise, and obtain an image containing only the outline of the nut;

[0031] (5c) According to the object imaging ratio of industrial cameras and the pixel size of industrial cameras, we can obtain:

[0032]

[0033] Where: ω is the width of the camera's field of view; d is the inner diameter of the nut; Ω is the pixel width of the camera's CMOS sensor; D is the pixel width of the inner diameter of the nut;

[0034] (5d) is obtained by transforming Equation (4):

[0035]

[0036] According to the selected nut specification and Equation (5), the pixel width of the inner diameter of the nut is between [D - i, D + i], where i is the allowable deviation value of the diameter, which is set to 10 pixel values;

[0037] (5e) Use the Hough circle detection method to perform Hough circle detection on the image containing only the nut contour to obtain the inner circle of the nut, determine the center of the circle from the inner circle, and thus obtain the center coordinates C1 of the inner circle of the nut.

[0038] The said step (6) includes the following steps:

[0039] (6a) After performing Hough line detection, multiple lines are obtained, and some redundant lines are removed through mathematical constraints; according to the cropped nut picture, the width and height of the cropped image are obtained, and the center coordinates of the image are calculated and denoted as point O;

[0040] (6b) With the camera height unchanged, the radius of the inner circle of the nut is obtained and denoted as R; the distance between each line and point O is calculated and denoted as L; when L < R + j, the line is deleted, where j is the error value, which is set to 5 pixel values;

[0041] (6c) A hexagonal nut is a regular hexagon in mathematics, and there is an angular constraint relationship between the sides of the hexagon. The included angle between any two sides satisfies one of 0°, 60°, and 120°; calculate the angle between each line and other lines. When the included angle satisfies one of 0°, 60°, and 120°, a vote is obtained. Let the number of lines with the most votes be a. If a is greater than 6, then the top a lines with the most votes are taken as the correct candidate sides; if a is less than or equal to 6, then the top six lines with the most votes are taken as the correct candidate sides;

[0042] (6d) Each side on the correct candidate side intersects with all other sides, and their intersection points are taken as candidate vertices. There is a distance constraint between the candidate vertices. Let the side length of the hexagon be J, and the distance between any two vertices should satisfy J, One of 2J; when the candidate vertex satisfies the distance constraint and the intersection point is in the cropped nut image and outside the inner circle of the nut, it gets a vote; the correct candidate vertex satisfies the distance constraint, so the correct candidate vertex gets more total votes, and the wrong candidate vertex gets less total votes; as with the candidate edge screening, considering that there are similar intersection points, the number of intersection points with the most votes needs to be calculated and recorded as m. If m is greater than 6, the first m candidate vertices with the most votes are taken as the correct candidate vertices; if m is less than or equal to 6, the first six candidate vertices with the most votes are taken as the correct candidate vertices;

[0043] (6e) Three non-parallel points determine a circle, that is, a center of the circle is obtained; the centers of multiple circles are calculated by taking three different correct candidate vertices through permutations and combinations, and the average value of these center coordinates is used to obtain the nut center coordinate C2.

[0044] It can be seen from the above technical scheme that the beneficial effects of the present invention are as follows: first, the present invention uses an improved YOLOv5 network model to initially locate the nut and crop it to obtain a cropped nut image, which greatly reduces the interference of the image around the nut on the positioning, screens useful information, and removes useless information; second, the Zero-DCE brightness enhancement algorithm adopted by the present invention adaptively enhances the brightness of the low-brightness image according to the brightness of the nut image, improves the brightness of the image, and increases the dark details; third, the RCF edge detection algorithm adopted by the present invention greatly improves the contours of the six edges and inner circle of the nut to be detected, and reduces the contours of other images, greatly reduces the influence of nut wear and stains on subsequent Hough circle-line detection, and improves the accuracy of nut center positioning; fourth, the present invention locates the processed nut image based on the Hough circle detection and Hough line detection algorithms, obtains the center coordinates C1 of the nut inner circle by Hough circle detection, obtains the center coordinates C2 of the nut by Hough line detection, and obtains the final nut center coordinates C after weighted averaging, thereby improving robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of the method of the present invention;

[0046] Figure 2 This is a diagram of nut identification results in the present invention;

[0047] Figure 3 This is a comparison diagram of edge detection in the present invention;

[0048] Figure 4 Schematic diagram of the distance constraints on the sides of the hexagon in the present invention. DETAILED DESCRIPTION

[0049] like Figure 1As shown, a method for locating the center of a high-altitude nut based on machine vision includes the following steps in sequence:

[0050] (1) obtaining image data of the nut by photographing with a camera, and preprocessing the image data of the nut to obtain a data set, and dividing the data set into a training set and a validation set;

[0051] (2) improving the YOLOv5 network model to obtain an improved YOLOv5 network model, inputting the training set into the improved YOLOv5 network model for training, and obtaining a trained YOLOv5 network model; inputting the nut image to be detected into the trained YOLOv5 network model for recognition, obtaining a recognition image of the nut, and cropping the recognition image of the nut to obtain a cropped nut image;

[0052] (3) Use the Zero-DCE brightness enhancement algorithm to enhance the brightness of the cropped nut image to obtain a brightness enhanced nut image;

[0053] (4) Processing the brightness enhanced nut image using the RCF edge detection algorithm to obtain the nut image after edge detection;

[0054] (5) Perform Hough circle detection on the nut image after edge detection, determine the inner circle radius through camera parameters and shooting distance, and obtain the center coordinate C1 of the inner circle of the nut;

[0055] (6) Perform Hough line detection on the nut image after edge detection, determine the center point of the nut through hexagonal angle constraint and vertex constraint, and obtain the nut center coordinate C2;

[0056] (7) Take a weighted average of the obtained nut inner circle center coordinate C1 and nut center coordinate C2 to obtain the final nut center coordinate C.

[0057] In step (1), the preprocessing refers to randomly cropping the image data of the nut, randomly flipping to expand the data set, and annotating the data with labelimg to obtain the data set, and dividing the data set into a training set and a validation set in a ratio of 8:2.

[0058] The step (2) comprises the following steps:

[0059] (2a) The YOLOv5 network model is improved to obtain an improved YOLOv5 network model: the YOLOv5 network model includes an input end, a backbone network, a neck network and a head network. An efficient multi-scale attention mechanism is added to the backbone network to fuse multi-scale information and reduce interference from other information; a bidirectional feature pyramid network is added to the neck network to enhance the ability to fuse features of different scales and increase detection accuracy; the loss function of the YOLOv5 network model is replaced with a SIoU structured loss function to improve the model convergence speed;

[0060] (2b) Use the training data set to train the improved YOLOv5 network model until the loss function reaches convergence, update the parameters, use the sgd optimizer during training, set the momentum of the sgd optimizer to 0.937, set weight_decay to 5e-4, and train for 200 rounds in total; momentum is momentum gradient descent, and weight_decay is weight decay;

[0061] (2c) The image of the nut to be detected is input into the trained YOLO v5 model for prediction, and the recognition image of the nut is obtained and cropped to obtain the cropped nut image.

[0062] The step (3) comprises the following steps:

[0063] (3a) The Zero-DCE brightness enhancement algorithm uses the DCE network model, i.e., the deep curve estimation network model, to estimate a set of best-fit light enhancement curves, i.e., the L curves. The DCE network model continuously iterates the L curves until the loss function is minimized. The image enhancement second-order curve is expressed as:

[0064] L(I(x);α)=I(x)+αI(x)(1-I(x)) (1),

[0065] Where: I(x) is the cropped nut image of the input; x is the pixel coordinate; L(I(x); α) is the enhanced image, α∈[-1,1] is the trainable curve parameter, and each pixel is normalized to [0,1]; applying the light enhancement curve to the input RGB channels can better maintain the inherent color and avoid overfitting. In addition, the light enhancement curve can also increase or decrease the dynamic range of the input image, which can not only enhance the low-light area, but also avoid overexposure.

[0066] (3b) By iterating formula (1), a high-order curve is obtained, which makes the adjustment more flexible, so that the model can adapt to various low-light conditions:

[0067] L n (x) = L n-1 (x)+α n Ln-1 (x)(1-L n-1 (x))(2),

[0068] Where n is the number of iterations, and when n is 1, it degenerates to the second order;

[0069] The high-order curve does not meet the requirements, because it still uses a unified α for the entire image, and the α of each pixel should be different. Replace α in the formula with matrix A to obtain the pixel curve:

[0070] L n (x) = L n-1 (x)+A n L n-1 (x)(1-L n-1 (x))(3),

[0071] In formula (3), L n (x) is the picture of the nut after brightness enhancement.

[0072] Formula (1) is the second-order image enhancement curve, which is the first iterative processing of the nut image. Formula (2) is obtained by iterating formula (1), which can make the adjustment more flexible, so that the model can adapt to nut images in various low-light conditions. Formula (3) is because α is applied to all pixels, which is a global adjustment, which will lead to over-enhancement / under-enhancement of the local area of ​​the nut. In order to achieve local adjustment, the single parameter α of the high-order curve is changed to a pixel-level parameter to obtain a pixel curve, that is, each pixel of a given input image has a curve corresponding to the best fit α to adjust its dynamic range, so as to prevent the problem of over-enhancement of the brighter area of ​​the nut resulting in overexposure and under-enhancement of the darker area resulting in insufficient information.

[0073] The matrix A has the same dimension as the input image. Assuming that the pixel intensity in the local area is consistent (the adjustment curve is consistent), the adjacent pixels in the output result remain monotonic. The value of each pixel in the enhanced image is within the normalized range [0,1] to avoid information loss due to truncation overflow; the curve is monotonic to ensure the difference between adjacent pixels; the curve is simple in form and differentiable during gradient back propagation. These are the three key factors.

[0074] The Zero-DCE brightness enhancement algorithm takes a low-light image as input and generates a high-order curve as output. The curve is then used as input, pixel by pixel, and the output image is adjusted. The Zero-DCE brightness enhancement algorithm is a purely data-driven method that takes multiple light enhancement factors into account in the design of the non-reference loss function, so it has better robustness, wider image dynamic range adjustment, and lower computational burden.

[0075] The step (5) comprises the following steps:

[0076] (5a) Convert the nut image after edge detection into a grayscale image and remove noise using Gaussian filtering to obtain a denoised grayscale image;

[0077] (5b) Use the canny operator to extract edges from the denoised grayscale image to obtain an image containing only the nut contour;

[0078] (5c) The nut selected for the tower crane connection is a high-strength large hexagon nut for steel structures, with a thread specification of M36 and a certain inner circle radius. From the formula for the field of view range of an industrial camera, when the distance between the camera and the object is determined, the field of view width of the industrial camera can be calculated. According to the imaging ratio relationship of the industrial camera and the pixel size of the industrial camera, we get:

[0079]

[0080] In the formula: ω is the field of view width of the camera; d is the inner circle diameter of the nut; Ω is the pixel width of the camera CMOS sensor; D is the pixel width of the inner circle diameter of the nut;

[0081] (5d) Transformed from formula (4) to get:

[0082]

[0083] According to the selected nut specification and formula (5), the pixel width of the inner circle diameter of the nut is between [D - i, D + i], where i is the allowable deviation value of the diameter, set to 10 pixel values;

[0084] (5e) Use the Hough circle detection method to perform Hough circle detection on the image containing only the nut contour to obtain the inner circle of the nut, determine the center of the circle from the inner circle, and thus obtain the center coordinates C1 of the inner circle of the nut.

[0085] The said step (6) includes the following steps:

[0086] (6a) After performing Hough line detection, multiple lines are obtained, and some redundant lines are removed through mathematical constraints; according to the cropped nut image, the width and height of the cropped image are obtained, and the center point coordinates of the image are calculated, denoted as point O;

[0087] (6b) With the camera height unchanged, obtain the radius of the inner circle of the nut, denoted as R; calculate the distance from each line to point O, denoted as L; when L < R + j, delete the line, where j is the error value, set to 5 pixel values; in this way, the lines inside the inner circle of the nut can be deleted because the correct candidate edges must be outside the inner circle;

[0088] (6c) Mathematically, a hexagonal nut is a regular hexagon. There is an angle constraint relationship between the sides of the hexagon. The angle between any two sides satisfies one of 0°, 60°, and 120°. Calculate the angle between each straight line and other straight lines. When the angle satisfies one of 0°, 60°, and 120°, obtain a vote. Let the number of straight lines with the most votes be a. If a is greater than 6, then the first a straight lines with the most votes are taken as the correct candidate sides. If a is less than or equal to 6, then the first six straight lines with the most votes are taken as the correct candidate sides.

[0089] (6d) Each edge on the correct candidate edge intersects with all other edges, and their intersection points are taken as candidate vertices. There are distance constraints between candidate vertices. Let the side length of the hexagon be J, and the distance between any two vertices should satisfy J. One of 2J; when the candidate vertex satisfies the distance constraint and the intersection point is in the cropped nut image and outside the inner circle of the nut, it gets a vote; the correct candidate vertex satisfies the distance constraint, so the correct candidate vertex gets more total votes, and the wrong candidate vertex gets less total votes; as with the candidate edge screening, considering that there are similar intersection points, the number of intersection points with the most votes needs to be calculated and recorded as m. If m is greater than 6, the first m candidate vertices with the most votes are taken as the correct candidate vertices; if m is less than or equal to 6, the first six candidate vertices with the most votes are taken as the correct candidate vertices;

[0090] (6e) Three non-parallel points determine a circle, that is, a center of the circle is obtained; the centers of multiple circles are calculated by taking three different correct candidate vertices through permutations and combinations, and the average value of these center coordinates is used to obtain the nut center coordinate C2.

[0091] Figure 2 The recognition result is shown. The recognition frame is very accurate and the confidence level is high. Based on this recognition frame, 50 pixels are added to each side to crop the nut in preparation for subsequent positioning.

[0092] Figure 3 This shows that processing by the RCF edge detection algorithm and then by the Canny operator has less noise and less interference than directly using the Canny operator.

[0093] Figure 4 This is a schematic diagram of the distance constraints on the sides of the hexagon in the present invention. There are distance constraints between the vertices of the six corners of the hexagon, that is, the distance between any two vertices satisfies one of AB, AC, and AD. Each candidate vertex is voted to obtain the correct vertex. Taking point A as an example, when point A and any other vertex meet the distance constraint, point A gets one vote. The distance between the correct candidate vertex and all other vertices usually meets the distance constraint, so it gets the votes of all other points.

[0094] In summary, the present invention uses an improved YOLOv5 network model to initially locate the nut and crop it to obtain a cropped nut image, which greatly reduces the interference of the image around the nut on positioning, screens useful information, and removes useless information; the Zero-DCE brightness enhancement algorithm adopted by the present invention adaptively enhances the brightness of the low-brightness image according to the brightness of the nut image, improves the brightness of the image, and increases the dark details; the RCF edge detection algorithm adopted by the present invention greatly improves the contours of the six edges and inner circle of the nut required for detection, and reduces the contours of other images, greatly reduces the influence of nut wear and stains on subsequent Hough circle-line detection, and improves the accuracy of nut center positioning; the present invention locates the processed nut image based on Hough circle detection and Hough line detection algorithms, obtains the center coordinates C1 of the nut inner circle through Hough circle detection, obtains the center coordinates C2 of the nut through Hough line detection, and obtains the final nut center coordinates C after weighted averaging, thereby improving robustness.

Claims

1. A method for locating the center of a high-altitude nut based on machine vision, characterized in that: The method comprises the following steps in the following order: (1) obtaining image data of the nut by photographing with a camera, and preprocessing the image data of the nut to obtain a data set, and dividing the data set into a training set and a validation set; (2) improving the YOLOv5 network model to obtain an improved YOLOv5 network model, inputting the training set into the improved YOLOv5 network model for training, and obtaining a trained YOLOv5 network model; inputting the nut image to be detected into the trained YOLOv5 network model for recognition, obtaining a recognition image of the nut, and cropping the recognition image of the nut to obtain a cropped nut image; (3) Use the Zero-DCE brightness enhancement algorithm to enhance the brightness of the cropped nut image to obtain a brightness enhanced nut image; (4) Processing the brightness enhanced nut image using the RCF edge detection algorithm to obtain the nut image after edge detection; (5) Perform Hough circle detection on the nut image after edge detection, determine the inner circle radius through camera parameters and shooting distance, and obtain the center coordinate C1 of the inner circle of the nut; (6) Perform Hough line detection on the nut image after edge detection, determine the center point of the nut through hexagonal angle constraint and vertex constraint, and obtain the nut center coordinate C2; (7) Take a weighted average of the obtained nut inner circle center coordinate C1 and nut center coordinate C2 to obtain the final nut center coordinate C.

2. The high-altitude nut center positioning method based on machine vision according to claim 1 is characterized in that: In step (1), the preprocessing refers to randomly cropping and randomly flipping the image data of the nut, and labeling the data with labelimg.

3. The high-altitude nut center positioning method based on machine vision according to claim 1 is characterized in that: The step (2) comprises the following steps: (2a) The YOLOv5 network model is improved to obtain an improved YOLOv5 network model: the YOLOv5 network model includes an input end, a backbone network, a neck network and a head network. An efficient multi-scale attention mechanism is added to the backbone network to fuse multi-scale information and reduce interference from other information; a bidirectional feature pyramid network is added to the neck network to enhance the ability to fuse features of different scales and increase detection accuracy; the loss function of the YOLOv5 network model is replaced with a SIoU structured loss function to improve the model convergence speed; (2b) Use the training data set to train the improved YOLOv5 network model until the loss function reaches convergence, update the parameters, use the sgd optimizer during training, set the momentum of the sgd optimizer to 0.937, set weight_decay to 5e-4, and train for 200 rounds in total; momentum is momentum gradient descent, and weight_decay is weight decay; (2c) The nut image to be detected is input into the trained YOLO v5 model for prediction, the recognition image of the nut is obtained, and the recognition image of the nut is cropped to obtain a cropped nut image.

4. The high-altitude nut center positioning method based on machine vision according to claim 1 is characterized in that: The step (3) comprises the following steps: (3a) The Zero-DCE brightness enhancement algorithm uses the DCE network model to estimate a set of best-fitting light enhancement curves, namely L curves. The DCE network model iteratively processes the L curves until the loss function is minimized, obtaining an image enhancement second-order curve expressed as: L(I(x); α) = I(x) + αI(x)(1 - I(x)) (1), where: x is the pixel coordinate; L(I(x); α) is the enhanced image, α ∈ [-1, 1] is the trainable curve parameter, and each pixel is normalized to [0, 1]; I(x) is the input cropped nut image; (3b) By iterating equation (1), a higher-order curve is obtained: L n (x)=L n-1 (x)+α n L n-1 (x)(1-L n-1 (x))(2), where n is the number of iterations, and it degenerates to the second order when n is 1; Replacing α in the equation with matrix A, a pixel curve is obtained: L n (x)=L n-1 (x)+A n L n-1 (x)(1-L n-1 (x))(3), In formula (3), L n (x) is the picture of the nut after brightness enhancement.

5. The high-altitude nut center positioning method based on machine vision according to claim 1 is characterized in that: The step (5) includes the following steps: (5a) Convert the edge-detected nut image to a grayscale image and denoise it using Gaussian filtering to obtain a denoised grayscale image; (5b) Use the canny operator to extract edges from the denoised grayscale image to obtain an image containing only the nut contour; (5c) According to the object imaging ratio relationship of the industrial camera and the pixel size of the industrial camera, it is obtained that: where: ω is the width of the camera's field of view; d is the diameter of the inner circle of the nut; Ω is the pixel width of the camera's CMOS sensor; D is the pixel width of the inner circle diameter of the nut; (5d) Transformed from equation (4): According to the selected nut specification and equation (5), the pixel width of the inner circle diameter of the nut is between [D - i, D + i], where i is the allowable deviation value of the diameter, set to 10 pixel values; (5e) Use the Hough circle detection method to perform Hough circle detection on the image containing only the nut contour to obtain the inner circle of the nut, determine the center of the circle from the inner circle, and thus obtain the center coordinates C1 of the inner circle of the nut.

6. The method for center positioning of high-altitude nuts based on machine vision according to claim 1 is characterized in that: The step (6) includes the following steps: (6a) After performing Hough line detection, multiple lines are obtained, and some redundant lines are removed through mathematical constraints; according to the cropped nut image, the width and height of the cropped image are obtained, and the center point coordinates of the image are calculated, denoted as point O; (6b) With the camera height unchanged, the radius of the inner circle of the nut is obtained, denoted as R; calculate the distance between each line and point O, denoted as L; when L < R + j, delete the line, where j is the error value, set to 5 pixel values; (6c) A hexagonal nut is a regular hexagon in mathematics, and there is an angular constraint relationship between the sides of the hexagon. The included angle between any two sides satisfies one of 0°, 60°, and 120°; calculate the angle between each line and other lines. When the included angle satisfies one of 0°, 60°, and 120°, a vote is obtained. Let the number of lines with the most votes be a. If a is greater than 6, then the top a lines with the most votes are taken as the correct candidate edges; if a is less than or equal to 6, then the top six lines with the most votes are taken as the correct candidate edges; (6d) Each edge on the correct candidate edge intersects with all other edges, and their intersection points are taken as candidate vertices. There are distance constraints between candidate vertices. Let the side length of the hexagon be J. The distance between any two vertices should satisfy J, One of 2J; when the candidate vertex satisfies the distance constraint and the intersection point is in the cropped nut image and outside the inner circle of the nut, it gets one vote; the correct candidate vertex satisfies the distance constraint, so the correct candidate vertex gets more total votes, and the wrong candidate vertex gets less total votes; As with the candidate edge screening, considering the similar intersections, let the number of intersections with the most votes be m. If m is greater than 6, the first m candidate vertices with the most votes are taken as the correct candidate vertices; if m is less than or equal to 6, the first six candidate vertices with the most votes are taken as the correct candidate vertices. (6e) Three non-parallel points determine a circle, that is, a center of the circle is obtained; the centers of multiple circles are calculated by taking three different correct candidate vertices through permutations and combinations, and the average value of these center coordinates is used to obtain the nut center coordinate C2.

Citation Information

Patent Citations

  • Nut positioning detection method, device and system based on machine vision

    CN114387262A

  • Bolt detecting and positioning method based on YOLOv5 and Realsense

    CN115330751A

  • X-ray security check image dangerous goods detection algorithm based on YOLOv7

    CN117274774A

  • Mine low-illumination small target person detection method and device based on vision

    CN117423064A

  • Visual pose identification method for bolts and nuts on large flange

    CN118195986A

Cited By

  • AI-driven omnibearing posture monitoring method and system, medium and program product

    CN119961870A

  • Parking nut geometric full-automatic detection system

    CN122089711A