A hexagonal head bolt missing and loosening image recognition method based on an optimal hexagon rule

Abstract

This invention belongs to the field of deep learning image recognition technology and discloses a method for recognizing missing and loose hexagonal head bolts based on the optimal hexagonal rule. The method includes: developing a keypoint image detection algorithm based on YOLOv12 to accurately locate the coordinates of the six vertices of the head of a standard hexagonal head bolt; using the Graham scan method to obtain the circumcircle convex hull of the six vertices; and optimizing the vertex coordinates using the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton optimization algorithm, centered on the centroid of the convex hull, to minimize the sum of errors between the actual and estimated vertex coordinates, generating optimized regular hexagonal vertex coordinates, and calculating the bolt rotation angle. This invention effectively solves the problem of excessive vertex offset in bolt loosening detection, improves detection efficiency and accuracy, and, after training, obtains the optimal model weights, enabling accurate detection of bolt loosening angles and enhancing the reliability and practicality of bolt loosening defect detection.

Description

Technical Field

[0001] This invention belongs to the field of deep learning image recognition and relates to an image recognition method for missing and loose hexagonal head bolts based on the optimal hexagonal method. Background Technology

[0002] Bolts, as key basic components of mechanical connections, are widely used in construction, machinery, and aerospace. However, they are susceptible to loosening and even detachment due to long-term exposure to alternating loads and environmental factors, threatening structural safety. Minor displacements caused by mechanical vibration can wear down the threads, leading to loosening; temperature stress caused by differences in material properties can result in preload loss; and instantaneous impact loads exceeding interfacial friction can directly cause slippage. In engineering practice, the detection and monitoring of bolt conditions must balance quantitative accuracy and operational efficiency. Existing detection methods have significant limitations: traditional visual scribing and tapping methods, while intuitive and convenient, rely on human experience, cannot identify minor loosenings, and are easily limited by space and viewing angle; torque wrenches, while capable of applying torque quantitatively, can have axial force measurement errors of up to 50% due to thread friction and other factors, making it difficult to meet the requirements for high precision and rapid monitoring. Among contact sensor-based monitoring methods, piezoelectric ultrasonic and electromechanical impedance methods can achieve real-time quantitative assessment of preload, but they have high requirements for sensor deployment, coupling status, and excitation signal stability, limiting their application in multi-bolt groups or harsh working conditions. Image recognition-based computer vision methods can achieve non-contact, automated detection, but current technologies mainly focus on identifying missing or broken bolts. Accurate judgment of the loosening angle heavily relies on the clarity and integrity of the marking lines and the robustness of the algorithm. Furthermore, the markings themselves are prone to wear and detachment, making them difficult to adapt to complex lighting and background interference. Existing methods still face challenges in achieving rapid, accurate, and robust perception of the loosening status of large-scale bolt groups, necessitating the development of a monitoring technology that balances high precision, strong adaptability, and high efficiency. Summary of the Invention

[0003] The purpose of this invention is to address the problem that while existing vision-based bolt loosening detection methods have high accuracy under ideal conditions, they suffer from poor robustness and high maintenance costs in practical engineering due to the susceptibility of markers to environmental corrosion, detachment, and wear. Consequently, these methods are unable to meet the high-precision non-destructive testing requirements for bolt loosening defects in complex environments. This invention provides a method for image recognition of missing and loose hexagonal head bolts based on the optimal hexagonal rule.

[0004] To achieve the above objectives, the present invention employs the following technical solution:

[0005] A keypoint image detection algorithm based on YOLOv12 was developed to accurately locate the coordinates of the six vertices of the head of a standard hexagonal head bolt. The Graham scan method was used to obtain the circumcircle convex hull of the six vertices. Using the centroid of the convex hull as the center, the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton optimization algorithm was used to optimize the vertex coordinates, minimizing the sum of errors between the actual and estimated vertex coordinates, generating optimized regular hexagonal vertex coordinates, and realizing the calculation of the bolt rotation angle. This invention effectively solves the problem of excessive vertex offset in bolt loosening detection, improves detection efficiency and accuracy, and, after training, obtains the optimal model weights, which can accurately detect the bolt loosening angle, enhancing the reliability and practicality of bolt loosening defect detection.

[0006] Compared with the prior art, the present invention has the following beneficial effects:

[0007] This invention constructs a bolt loosening identification algorithm based on the optimal hexagonal rule and YOLOv12 keypoint detection. It analyzes the bolt head vertex positioning deviation in complex environments to clarify the correlation between perspective distortion and angle measurement error. Perspective transformation is used to automatically correct the image detected by the model and establish a geometrically optimized model of the bolt's hexagonal contour. The circumcircle convex hull, centroid, and rotation angle between the vertex and the bolt are calculated and input for loosening state determination. This invention effectively solves the problems of low quantization accuracy and poor environmental robustness caused by easy marker detachment, image distortion, and perspective changes in traditional visual methods. It improves the accuracy and stability of bolt loosening angle measurement. Experimental verification shows that it can achieve high-precision identification of loosening angles (error range -1.62° to 2°, maximum relative error 5.5%), avoiding reliance on manual markers and subjective misjudgment, and enhancing the reliability and practicality of bolt loosening state detection. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating the image recognition method for missing and loose hexagonal head bolts based on the optimal hexagonal rule of the present invention.

[0010] Figure 2 A schematic diagram showing the bolt data used in the experiment;

[0011] Figure 3 This is a schematic diagram showing the changes in the image before and after correction;

[0012] Figure 4 A schematic diagram of data with missing bolts marked for use in the experiment;

[0013] Figure 5 This is a schematic diagram of the YOLOv12 model network structure;

[0014] Figure 6 A schematic diagram of data used in the experiment, showing the bolt loosening indicators;

[0015] Figure 7 A schematic diagram illustrating the optimization of vertex coordinates for the optimal hexagonal rule;

[0016] Figure 8 This is a schematic diagram for angle calculation;

[0017] Figure 9 A schematic diagram of the bounding box loss for the model training and validation sets;

[0018] Figure 10 This is a diagram illustrating the model's precision, recall, and map50 metrics.

[0019] Figure 11 This is a diagram illustrating the model training loss.

[0020] Figure 12 A schematic diagram illustrating structural indicators for model identification;

[0021] Figure 13 This is a schematic diagram of the test results for the mixed loosening test of three-color bolts; Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0025] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper," "lower," "horizontal," or "inner" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of the invention is in use, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, terms such as "first" and "second" are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] Furthermore, the use of the term "horizontal" does not imply that the component must be absolutely horizontal, but rather that it can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0027] In the description of the embodiments of the present invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0028] The present invention will now be described in further detail with reference to the accompanying drawings:

[0029] See Figure 1 This invention discloses a method for image recognition of missing and loose hexagonal head bolts based on the optimal hexagonal rule, comprising:

[0030] S101, Constructing an image recognition method for missing and loose hexagonal head bolts that combines the optimal hexagonal rule with deep learning technology;

[0031] S102, based on camera intrinsic parameters, performs perspective transformation and distortion correction on the original image to eliminate geometric deformation caused by shooting angle and lens effects, and obtains high-quality, distortion-free bolt images to build a standard dataset for subsequent analysis;

[0032] The acquisition of high-quality, distortion-free bolt image data includes collecting and capturing shooting parameters to construct a perspective transformation matrix and correcting the bolt vertex coordinates image; specifically, the bolt data used in the experiment is as follows: Figure 2As shown, the left image displays the original grayscale image of the bolts; in the right image, bolts 1#, 3#, 4#, 7#, 8#, 9#, 15#, and 16# are marked in blue, bolts 5#, 11#, 12#, and 13# are marked in red, and the remaining bolts remain gray. The coordinates of the bolt center points are calculated by identifying the vertices of the hexagonal bolts. These center point coordinates are then sorted, and the top-left, bottom-left, top-right, and bottom-right bolt center points are selected as reference points. Perspective transformation is then performed on the image based on these four reference points. Let the plane coordinates of the distorted image be (u′, v′). ′ The standard plane coordinates are (u, v). The distorted image view plane can be transformed into the standard nodal view plane using formula (8). T is the image perspective transformation matrix, which is obtained from the coordinates of the four selected reference points and the four theoretical standard points. Figure 3 Images before and after distortion correction are shown. The left image is the original image with distortion (bolts 1#, 3#, 4#, 7#, 8#, 9#, 15#, and 16# are blue, 6# and 11# are red, and the rest are gray); the right image is the image after perspective transformation correction, with the bolt numbers and colors matching those in the left image. Figure 1 As can be seen, the distortion has been effectively corrected.

[0033] S103, based on the acquired standard dataset, an improved YOLOv12 target detection algorithm is used to establish a high-precision deep learning model 1 to locate and identify hexagonal head bolts in the input image;

[0034] The constructed standard dataset uses labelimg image annotation software to annotate the collected bolt data. The basic annotation process is as follows: (1) Using a smartphone or consumer-grade SLR camera, or a lightweight drone, a relatively clear partial image of the bolt component's front elevation is collected close to the component; (2) The collected images are annotated according to the missing parts. The bolt missing part label is "hole", such as Figure 4 As shown, Figure 4 The bolt in the upper left corner is marked as a missing bolt, and the rest are normal cases with no missing bolts; (3) The dataset is expanded by random rotation, scaling, translation, adding noise and blurring; (4) The images are divided into training set, test set and validation set according to the ratio of 8:1:1. A total of 640 images were collected in the dataset. After the images were expanded by methods such as rotation, translation and contrast modification, there were a total of 5760 images. 342 images were missing from the dataset.

[0035] The training set image data was then labeled and input into the YOLOv12 object detection model for training. The training rounds were 100, the batch size of the training samples was 16, the initial learning rate was set to 0.01, and the learning rate decay factor was 0.1, to train a high-precision missing detection model.

[0036] The core improvements to the YOLOv12 algorithm include the Area Attention mechanism and the Residual Efficient Layer Aggregation Network (R-ELAN). The Area Attention mechanism divides the feature map into multiple regions and uses a reshape operation to reduce computational complexity, significantly improving efficiency. The Residual Efficient Layer Aggregation Network (R-ELAN) improves upon the efficient layer aggregation network structure by introducing residual connections from input to output and incorporating a scaling factor, effectively stabilizing the training process and optimizing gradient flow. Simultaneously, its redesigned feature aggregation method significantly reduces computational cost and memory usage, thus maintaining efficient feature fusion capabilities. The specific model architecture is as follows: Figure 5 As shown;

[0037] Based on the acquired standard dataset, S104 uses an improved YOLOv12 object detection algorithm to establish a high-precision deep learning model 2, which identifies the coordinates of the six vertices of the hexagonal head bolt in the input image, providing the initial geometric information for state judgment.

[0038] The constructed standard dataset uses labelimg image annotation software to annotate the collected bolt data. The basic annotation process is as follows: (1) Use a smartphone or consumer-grade SLR camera, or a lightweight drone, to get close to the component and collect relatively clear partial images of the bolt component's front elevation; (2) Annotate the collected images according to the bolt vertices. Figure 6 As shown; (3) The dataset was expanded by random rotation, scaling, translation, adding noise and blurring; (4) The images were divided into training set, test set and validation set in an 8:1:1 ratio. A total of 640 images were collected in the loosening dataset. After the images were expanded by methods such as rotation, translation and contrast modification, there were a total of 5760 images. 342 images were missing from the dataset.

[0039] S105 uses the optimal hexagonal rule to reconstruct the positional distribution image of the hexagonal bolt cross-section based on the initial geometric information, and identifies and locates the corner points and mass points of the hexagonal bolts, specifically:

[0040] Vertex coordinate reference point identification:

[0041] Step 1: Identify the coordinates of the six vertices of the hexagonal head bolt and obtain the vertex coordinate point set P. i ={x i y i (i = 1, 2, 3, ..., 6). Select the point with the smallest ordinate among the six vertices as the starting point for numbering the six vertices of the bolt, and denote it as P1 = {x1, y1} (if there are two points with the same ordinate, take the point with the smallest abscissa). Calculate the polar angle θ of the remaining points relative to P1 according to the polar angle sorting method. iEuclidean distance d i ,

[0042] θ i =atan2(y i -y1,x i -x1)

[0043]

[0044] According to θ i Sort in ascending order (if the polar angles are the same, sort by d). i (arranged in ascending order), resulting in the sorted set of coordinate points P. i = {P1, P2, ..., P6}. Construct a convex hull based on the sorted point set, initialize a stack stack = [P1, P2, P3], and traverse the remaining points P. k (k>3) For the two points P at the top of the stack i P j (P j (Using the top of the stack), calculate the cross product to determine the direction.

[0045] cross(P i P j P k ′)=(x j -x i )(y k ′-y i )-(y j -y i (x) k ′-x i )

[0046] If cross > 0, then point P k Located in vector The current stack vertex P is in the counter-clockwise direction. j Keep, and keep P k Push it onto the stack. If cross ≤ 0, then point P... k Located in vector The stack vertices are either clockwise or collinear. At this point, the stack vertex P... j If a vertex is not a convex hull vertex, pop it from the stack and backtrack to check its predecessor. Finally, all vertices in the stack form the convex hull vertices in counter-clockwise order.

[0047] Step 2: Calculation of the centroid of the bolt vertex:

[0048] Generate the vertices of a regular hexagon centered on the centroid of the convex hull. Based on the calculated centroid coordinates,

[0049]

[0050] Where j = 1, 2, ..., 6, r is the radius of the regular hexagon, and r is the minimum distance from the vertex of the convex hull to the centroid.

[0051] Step 3: Vertex coordinate optimization

[0052] By minimizing the distance difference between the actual and estimated vertices, the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton optimization algorithm is used to adjust the vertex coordinates, minimizing the sum of the differences between the actual and estimated values ​​of the six vertices. The resulting coordinates are the optimized vertex coordinates. The objective function is...

[0053]

[0054] Among them, (x i y i (x) represents the actual vertex coordinates. j ′,y j ′) is the estimated vertex coordinates, and θ is the rotation angle.

[0055] Step 4: Optimization steps of the BFGS quasi-Newton optimization algorithm:

[0056] (1) Take the initial point θ0∈R n The initial symmetric positive definite matrix is ​​B0, with precision ε > 0. Let the iteration counter k = 0.

[0057] (2) Calculate the gradient of the current objective function. If the gradient norm ||g k If ||≤ε, then the algorithm is considered to have converged, and computation is stopped. The optimal solution is θ. * =θ k Otherwise, calculate the search direction d. k :

[0058] d k =-B k -1 g k

[0059] (3) Update the step size λ according to the linear search mechanism based on the Wolfe condition. k ;

[0060] (4) Let θ k+1 =θ k +λ k *d k Calculate g k+1 If ||g k+1 ||≤ε, the optimal solution is θ * =θ k+1 Otherwise, proceed to the next step;

[0061] (5) Calculate the variable increment and gradient difference, and calculate B using the correction formula of the BFGS algorithm. k+1 :

[0062]

[0063] Let k = k + 1, then proceed to step 2 until the gradient norm ||g k ||≤ε.

[0064] The final result is as follows Figure 7 As shown

[0065] S106, calculate the rotation angle of the bolt relative to the initial state, compare the rotation angle with a preset threshold, thereby realizing the automatic identification of the bolt loosening state.

[0066] The angle calculation is specifically as follows:

[0067] Using a vertex coordinate reference point identification algorithm, the point P1 with the smallest ordinate among the six vertices of the bolt is found and used as a reference point. The slope of the line segment connecting P1 and the centroid is then calculated.

[0068]

[0069] The angle corresponding to the slope is calculated as follows:

[0070]

[0071] The bolt loosening angle can be obtained by interpolating the angles between the line connecting point P1 and the centroid and the y-axis twice.

[0072] Δθ=θ-θ′

[0073] In the formula, θ represents the angle between the line connecting the current vertex P1 of the hexagonal bolt and its centroid and the y-axis, θ′ represents the angle between the line connecting the vertex P1′ of the hexagonal bolt and its centroid and the y-axis in the previous recognition image, and Δθ represents the bolt loosening angle. A schematic diagram of the angle calculation is shown below. Figure 8 As shown

[0074] To eliminate the vertex identification error in the actual detection process, a loosening threshold θ is set. threshold Perform quantitative judgment on bolt loosening. When |Δθ|>θ threshold The bolt is then determined to be loose. Experiments show that when the bolt loosening angle is between 10° and 15°, the preload loss amplitude exhibits low dispersion during this stage, transitioning from a tight to a loose state. Therefore, the maximum loosening angle threshold θ in the image recognition process presented in this paper... threshold Set to 10°.

[0075] Example:

[0076] This invention discloses an image recognition method for missing and loose hexagonal bolts based on the optimal hexagonal rule.

[0077] A 16-hole bolted gusset plate was selected, using high-strength bolts of HS10.9S grade, with dimensions of M30*80. The original color of the bolts was grayish-brown. Considering the possible colors of bolts in actual engineering projects, some bolts were painted blue and red to verify the robustness of the algorithm.

[0078] The Dell G15 was used as the model training platform, with a 12th Gen Intel(R) Core(TM) i7-12700H CPU (14 cores, 20 threads) and an NVIDIA GeForce RTX 3060 Laptop GPU with 8GB of VRAM. The YOLOv12 deep learning framework and OpenCV image processing library were used. Training images were taken with an iPhone 14 Pro (48MP).

[0079] Before the experiment, a bolt dataset was first constructed. The process for establishing the bolt loosening dataset is as follows: (1) Using a smartphone or consumer-grade SLR camera, or a lightweight drone, a relatively clear partial image of the front elevation of the bolt component was collected from close proximity to the component; (2) The collected images were labeled according to the missing and loosening datasets. The bolt missing label was "hole", and the bolt loosening dataset was labeled with the bolt vertices, such as Figure 7 and Figure 8 As shown; (3) The dataset was expanded by random rotation, scaling, translation, adding noise and blurring; (4) The images were divided into training set, test set and validation set in an 8:1:1 ratio. A total of 640 images were collected in the loosening dataset. After the images were expanded by methods such as rotation, translation and contrast modification, there were a total of 5760 images. 342 images were missing from the dataset.

[0080] The model was trained for 100 epochs, with a batch size of 16 training samples, an initial learning rate of 0.01, and a learning rate decay factor of 0.1. The loosening and missing detection models were trained separately.

[0081] In the model validation section, in addition to the collected dataset, 100 additional bolt images were taken to verify the model's generalization ability. The bolts were rotated by a certain angle and then photographed again. The images taken before and after were input into the model to detect the loosening angle. The identified rotation angle was compared with the actual rotation angle to verify the reliability of the proposed method.

[0082] First, the bolt missing detection model is trained on the dataset. The model output is as follows. Figure 9 and Figure 10The diagram shows the bounding box losses for the training and validation sets. It can be seen that the training set bounding box loss steadily decreased from an initial 0.7 to around 0.48, with a smooth curve and no drastic oscillations, indicating a stable optimization process. The validation set bounding box loss also steadily decreased, with a difference of <0.01 from the training loss, indicating that the model did not overfit. Precision and recall remained stable above 0.9, showing an ideal upward trend. mAP50 reached above 0.9, indicating that the model has high prediction accuracy under different conditions and can effectively identify targets.

[0083] To test the effectiveness of the bolt missing detection and recognition, three bolts were randomly removed from the 16-hole bolt connection node plate for testing. The results are as follows: Figure 11 This indicates that bolt holes and bolts can be identified.

[0084] The bolt loosening detection model dataset was then trained, and the model training results are as follows: Figure 12 As shown, the loss curves all exhibit a healthy convergence trend, indicating that the training process is effective and the model has successfully learned patterns from the data, effectively completing tasks such as target localization, target category identification, and keypoint localization. Specifically, the keypoint loss on the training set decreases sharply from 3.9, eventually stabilizing between 0.01 and 0.02, while the keypoint loss on the validation set decreases from 2.6, eventually stabilizing between 0.04 and 0.05. The difference between the training and validation losses remains very small, indicating that the model performs well not only on the training set but also has ideal generalization ability. Experiments were conducted using a node plate connected by 16 bolts, and the results are as follows... Figure 12 The precision and recall of bounding box detection (B) training set were 0.99 and 1, respectively, with an mAP50 of 0.995. The precision and recall (mAP50) of pose estimation (P) were both 0.996, indicating that the model has extremely high localization accuracy. The loss curves of both the training and validation sets of keypoints showed a good downward trend, and the difference between training and validation losses remained within a reasonable range, indicating that the model did not exhibit overfitting. The keypoint mAP50-95 (P) was 0.99, which also demonstrated excellent adaptability at multiple scales.

[0085] To further verify the robustness of the algorithm, 16 mixed images of three-color bolt heads (gray, blue, and red) were used for detection. Bolts 1 to 11 were randomly rotated between 20° and 45°. The detection results are as follows. Figure 13 As shown, the results indicate that bolts 1-11 all exhibited significant rotation. The rotation angle identified by the model was close to the actual rotation angle, with an error range between -1.62° and 2°. The largest rotation angle identification error occurred in bolt 9, with an error of 5.5%. Bolts 12-16 did not exhibit significant rotation. The above identification results can accurately determine bolt loosening.

[0086] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for image recognition of missing and loose hexagonal head bolts based on the optimal hexagonal rule, characterized in that, include: A method for recognizing missing and loose hexagonal head bolts by combining the optimal hexagonal rule with deep learning technology is constructed. Based on camera intrinsic parameters, perspective transformation and distortion correction are performed on the original image to eliminate geometric deformation caused by shooting angle and lens effects, thereby obtaining high-quality, distortion-free bolt images and constructing a standard dataset for subsequent analysis. Based on the obtained standard dataset, an improved YOLOv12 object detection algorithm was used to establish a high-precision deep learning model 1, which was used to locate and identify hexagonal head bolts in the input image. Based on the obtained standard dataset, an improved YOLOv12 object detection algorithm was used to establish a high-precision deep learning model 2. The model was used to locate and identify the coordinates of the six vertices of the hexagonal head bolt in the input image, providing the initial geometric information for state judgment. The optimal hexagonal rule is applied to the initial geometric information to reconstruct the position distribution image of the hexagonal bolt cross-section, and the corner points and mass points of the hexagonal bolt are identified and located. The rotation angle of the bolt relative to its initial state is calculated, and this rotation angle is compared with a preset threshold to automatically determine the loose state of the bolt.

2. The image recognition method for missing and loose hexagonal head bolts based on the optimal hexagonal rule according to claim 1, characterized in that, The process of acquiring health data of the bolt structure to be tested includes image capture and distortion correction; the experimental images were captured using an iPhone 14 Pro with a resolution of 48 megapixels; the acquired raw image data was processed based on perspective transformation to obtain a corrected standard image dataset.

3. The image recognition method for missing and loose hexagonal head bolts based on the optimal hexagonal rule according to claim 2, characterized in that, Based on the constructed standard dataset, a high-precision deep learning model 1 is established using the improved YOLOv12 object detection algorithm, specifically as follows: Phase 1: Automatic identification of missing bolts, which consists of three steps: Step 1: Acquire images of bolt connection nodes and divide the images into training, testing, and validation sets in an 8:1:1 ratio; Step 2: Label the training set image data and input it into the YOLOv12 object detection model for training; Step 3: Input the test set into the trained bolt missing detection model for identification.

4. The image recognition method for missing and loose hexagonal head bolts based on the optimal hexagonal rule according to claim 3, characterized in that, Based on the constructed standard dataset, a high-precision deep learning model 2 is established using the improved YOLOv12 object detection algorithm, specifically as follows: The second stage: bolt rotation angle positioning, which also consists of three steps: Step 1: Acquire images of bolt connection nodes, apply perspective transformation algorithm for image correction, and divide the training set, test set, and validation set into an 8:1:1 ratio; Step 2: Label the training set image data and input it into the YOLOv12 object detection model for training; Step 3: Input the test set into the trained bolt missing detection model for identification.

5. The image recognition method for missing and loose hexagonal head bolts based on the optimal hexagonal rule according to claim 4, characterized in that, The optimal hexagonal rule is applied to the output corner recognition image to reconstruct the position distribution image of the hexagonal bolt cross-section, and the corner points and mass points of the hexagonal bolt are identified and located, specifically as follows: Vertex coordinate reference point identification: Step 1: Identify the coordinates of the six vertices of the hexagonal head bolt and obtain the vertex coordinate point set P. i ={x i y i (i = 1, 2, 3, ..., 6). Select the point with the smallest ordinate among the six vertices as the starting point for numbering the six vertices of the bolt, and denote it as P1 = {x1, y1} (if there are two points with the same ordinate, take the point with the smallest abscissa). Calculate the polar angle θ of the remaining points relative to P1 according to the polar angle sorting method. i Euclidean distance d i , i i =atan2(y i -y1,x i -x1) According to θ i Sort in ascending order (if the polar angles are the same, sort by d). i (arranged in ascending order), resulting in the sorted set of coordinate points P. i = {P1, P2, ..., P6}. Construct a convex hull based on the sorted point set, initialize a stack stack = [P1, P2, P3], and traverse the remaining points P. k (k>3) For the two points P at the top of the stack i P j (P j (Using the top of the stack), calculate the cross product to determine the direction. cross(P i ,P j ,P k ′)=(x j -x i )(y k ′-y i )-(y j -y i )(x k ′-x i ) If cross > 0, then point P k Located in vector The current stack vertex P is in the counter-clockwise direction. j Keep, and keep P k Push it onto the stack. If cross ≤ 0, then point P... k Located in vector The stack vertices are either clockwise or collinear. At this point, the stack vertex P... j If a vertex is not a convex hull vertex, pop it from the stack and backtrack to check its predecessor. Finally, all vertices in the stack form the convex hull vertices in counter-clockwise order. Step 2: Calculation of the centroid of the bolt vertex: Generate the vertices of a regular hexagon centered on the centroid of the convex hull. Based on the calculated centroid coordinates, Where j = 1, 2, ..., 6, r is the radius of the regular hexagon, and r is the minimum distance from the vertex of the convex hull to the centroid. Step 3: Vertex coordinate optimization By minimizing the distance difference between the actual and estimated vertices, the Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton optimization algorithm is used to adjust the vertex coordinates, minimizing the sum of the differences between the actual and estimated values ​​of the six vertices. The resulting coordinates are the optimized vertex coordinates. The objective function is... Among them, (x i y i (x) represents the actual vertex coordinates. j ′,y j ′) is the estimated vertex coordinates, and θ is the rotation angle. Step 4: Optimization steps of the BFGS quasi-Newton optimization algorithm: (1) Take the initial point θ0∈R n The initial symmetric positive definite matrix is ​​B0, with precision ε > 0. Let the iteration counter k = 0. (2) Calculate the gradient of the current objective function. If the gradient norm ||g k If ||≤ε, then the algorithm is considered to have converged, and computation is stopped. The optimal solution is θ. * =θ k Otherwise, calculate the search direction d. k :d k =-B k -1 g k (3) Update the step size λ according to the linear search mechanism based on the Wolfe condition. k ; (4) Let θ k+1 =θ k +λ k *d k Calculate g k+1 If ||g k+1 ||≤ε, the optimal solution is θ * =θ k+1 Otherwise, proceed to the next step; (5) Calculate the variable increment and gradient difference, and calculate B using the correction formula of the BFGS algorithm. k+1 : (6) Let k = k + 1, go to step 2, until the gradient norm ||g| ... k ||≤ε.

6. The image recognition method for missing and loose hexagonal head bolts based on the optimal hexagonal rule according to claim 5, characterized in that, The rotation angle of the bolt relative to its initial state is calculated based on the optimized corner coordinates. This rotation angle is then compared with a preset threshold to automatically determine the bolt's loosening state. Specifically: Using a vertex coordinate reference point identification algorithm, the point P1 with the smallest ordinate among the six vertices of the bolt is found and used as a reference point. The slope of the line segment connecting P1 and the centroid is then calculated. The angle corresponding to the slope is calculated as follows: The bolt loosening angle can be obtained by interpolating the angles between the line connecting point P1 and the centroid and the y-axis twice. Δθ=θ-θ′ In the formula, θ represents the angle between the line connecting the current hexagonal bolt vertex P1 and the centroid and the y-axis, θ′ represents the angle between the line connecting the hexagonal bolt vertex P1′ and the centroid and the y-axis in the previous recognition image, and Δθ represents the bolt loosening angle.

Images