A Deep Learning-Based Intelligent Detection Method and System for Loosening High-Strength Bolts

Abstract

This invention relates to the fields of machine vision and intelligent operation and maintenance technology, and particularly to a high-strength bolt loosening intelligent detection method and system based on deep learning. The method acquires visible light, near-infrared, and polarized light images of the bolt as initial images; fuses the initial images to generate a fused image, and adjusts the image contrast and brightness to obtain a target image; uses a pre-trained target detection model to detect bolts in the target image to generate ROI images; extracts candidate regions for marker lines from the ROI images and segments them to obtain a marker line skeleton image; calculates the geometric features and structural constraints of the skeleton, and fuses them to determine whether it is a target marker line to obtain a verification result; finally, based on the verification result, extracts the current principal direction angle of the marker line, compares it with a reference angle, and combines it with multi-frame detection results for temporal fusion to determine whether the bolt is loose to obtain a loosening status result. This invention improves the accuracy, efficiency, and reliability of bolt loosening detection, and reduces the risks and costs of manual inspection.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and intelligent operation and maintenance technology, and in particular to a high-strength bolt loosening intelligent detection method and system based on deep learning. Background Technology

[0002] In the installation, operation, and maintenance of wind turbine generators, the reliability of high-strength bolt connections directly affects the safety of the entire unit. Currently, to prevent bolt loosening, highly visible marking lines are typically applied to the bolt or nut surface as visual indicators of the initial tightening position. The detection of these marking lines still heavily relies on maintenance personnel climbing the tower for manual visual inspection, tapping, or torque checks, resulting in low efficiency, high false negative rates, significant reliance on subjective experience, and high risks associated with working at heights. In recent years, although image-based automatic detection methods have emerged, they largely depend on single visible light imaging and simple color and contour features. In the actual operating environment of wind turbines, these methods are easily affected by oil stains, fading, high light, obstructions, and complex backgrounds, leading to significantly insufficient robustness. Furthermore, existing methods generally lack adaptability to marking line degradation and fail to achieve rapid, collaborative detection of multiple bolts across the entire unit. They also exhibit significant deficiencies in functional structure modeling, such as continuity across components and radial distribution, making it difficult for the system's practicality, real-time performance, and measurement accuracy to meet the development needs of intelligent, high-precision, and predictive operation and maintenance in the wind power industry. Summary of the Invention

[0003] Therefore, the present invention provides a high-strength bolt loosening intelligent detection method and system based on deep learning to solve the aforementioned problems existing in the prior art.

[0004] To achieve the above objectives, on the one hand, the present invention provides a high-strength bolt loosening intelligent detection method based on deep learning, comprising:

[0005] Step S1: Real-time acquisition of visible light, near-infrared light, and polarized light images of the bolt to obtain an initial image;

[0006] Step S2: The initial image is fused using a fusion algorithm to generate a fused image. The fused image is then adjusted for contrast and brightness using an adaptive illumination invariant transformation algorithm to obtain the target image.

[0007] Step S3: Use a pre-trained target detection model to monitor the bolts in the target image to generate a ROI image of the bolts;

[0008] Step S4: Extract candidate regions for marker lines from the ROI image, and segment the candidate regions using a predefined segmentation network model to obtain a marker line skeleton image.

[0009] Step S5: Calculate the geometric features and structural constraints of the skeleton based on the skeleton image of the marker line to obtain the calculation result. Then, fuse the skeleton image of the marker line and the calculation result to determine whether it is the target marker line and obtain the verification result.

[0010] Step S6: Extract the current main direction angle of the marker line based on the verification result, compare it with the reference angle to obtain the comparison result, and perform time-series fusion with the multi-frame detection results of the same bolt to determine whether the bolt is loose to obtain the loose state result.

[0011] Furthermore, the process of step S3 includes:

[0012] The target detection model is used to detect bolts in the target image to generate an initial detection box;

[0013] The initial detection box is optimized by using a graph attention network to model the spatial relationship between bolts and adjust the position and size of the detection box to obtain the target detection box.

[0014] The frontal view transformation matrix of the bolt is calculated based on the target detection box, and then projected onto the normalized frontal view region to generate the ROI image.

[0015] Furthermore, the process of optimizing the initial detection box using a graph attention network and modeling the spatial relationships between bolts to adjust the position and size of the detection box to obtain the target detection box includes:

[0016] Construct bolt node feature vectors, including detection box coordinates and semantic information;

[0017] Graph attention networks are used to calculate the attention weights between nodes to quantify the spatial relationships between bolts;

[0018] The node feature vector is updated based on the attention weights, and the coordinate offset of the detection box is predicted through a fully connected network.

[0019] The initial detection box position and size are adjusted according to the coordinate offset to obtain the target detection box.

[0020] Furthermore, the process of step S4 includes:

[0021] Construct a feature template for the marker line, defining its shape and size;

[0022] A sliding window is used in the ROI image, and the similarity with the feature template is calculated to obtain the candidate regions for the marker lines;

[0023] The segmentation network model is used to perform pixel-level segmentation of the candidate region of the marker line to extract the connected regions of the marker line and obtain the segmentation mask.

[0024] The segmentation mask is skeletonized to obtain a marker line skeleton image.

[0025] Furthermore, the process of step S5 includes:

[0026] A predefined library of marker line feature templates;

[0027] The aspect ratio, linearity, and width consistency of the marked line skeleton image are extracted to obtain the geometric features;

[0028] The radiality and cross-boundary properties of the marked line skeleton image are extracted to obtain the structural constraints;

[0029] The geometric features and structural constraints are matched with the marker line feature template library to calculate a similarity score;

[0030] A weighted fusion is performed based on the segmentation mask, the geometric features, and the structural constraints to determine a comprehensive similarity score;

[0031] The verification result is obtained by comparing the comprehensive similarity score with a predefined similarity threshold to determine whether it is a valid target marker line.

[0032] Furthermore, the process of determining the comprehensive similarity score by weighted fusion based on the segmentation mask, the geometric features, and the structural constraints includes:

[0033] The segmentation mask is binarized to extract the boundary information of the marker lines;

[0034] Calculate the confidence score of the segmentation mask based on the boundary information;

[0035] The geometric features are normalized and their similarity scores are calculated.

[0036] The structural constraints are quantitatively calculated to obtain similarity scores.

[0037] The confidence score, the similarity score of geometric features, and the similarity score of structural constraints are weighted and summed to obtain the comprehensive similarity score.

[0038] Furthermore, step S6 includes the following process:

[0039] Based on the verification results, a multi-method fusion strategy is used to calculate the current main direction angle to obtain the target angle.

[0040] The relative rotation angle is calculated by comparing the target angle with the preset reference angle;

[0041] Based on the relative rotation angle, a Kalman filter is used to perform temporal fusion of the detection results of multiple frames, and the target fusion angle is obtained by combining multi-view weighted fusion.

[0042] Based on the difference between the target fusion angle and the reference angle, a layered early warning mechanism is used to determine whether the bolt is loose in order to obtain the bolt loosening result.

[0043] Furthermore, the process of extracting and calculating the current principal direction angle using a multi-method fusion strategy to obtain the target angle includes:

[0044] The Radon transform is applied to the skeleton image of the marked line to obtain the line direction;

[0045] The gradient of each pixel in the skeleton image of the marked line is calculated using the structure tensor to obtain the overall dominant direction;

[0046] By performing line fitting on the skeleton pixels using least squares, the angle corresponding to the slope of the line is obtained;

[0047] A consistency check is performed based on the angle corresponding to the line direction, the overall dominant direction, and the line slope. If it is within the tolerance range, the median value is taken as the target angle; otherwise, outliers are discarded, the average of the remaining two is taken as the target angle, and the confidence level is reduced.

[0048] Furthermore, the process of determining whether the bolt is loose based on the difference between the target fusion angle and the reference angle through a layered early warning mechanism to obtain the bolt loosening result includes:

[0049] Pre-set angle thresholds corresponding to slight loosening, significant loosening, and normal state;

[0050] Calculate the difference between the target fusion angle and the reference angle;

[0051] If the difference exceeds the slight loosening threshold, a slight loosening warning is triggered; if it exceeds the significant loosening threshold, a significant loosening warning is triggered.

[0052] Record the historical difference sequence of bolts, use an exponential smoothing model to fit the trend of change, and if the rate of change is consistently positive and large, a trend warning will be triggered even if the current difference does not exceed the threshold, prompting attention to the health degradation trend of the bolts.

[0053] On the other hand, the present invention also provides a high-strength bolt loosening intelligent detection system based on deep learning, comprising:

[0054] The image acquisition module is used to acquire visible light, near-infrared light, and polarized light images of the bolt in real time to obtain an initial image;

[0055] An image fusion module, connected to the image acquisition module, is used to fuse the initial image using a fusion algorithm to generate a fused image. The fused image is then adjusted for contrast and brightness using an adaptive illumination invariant transformation algorithm to obtain a target image.

[0056] The target detection module, connected to the image fusion module, is used to monitor the bolts in the target image using a pre-trained target detection model to generate a ROI image of the bolts;

[0057] The region extraction module, connected to the target detection module, is used to extract candidate regions for marker lines from the ROI image, and to segment the candidate regions for marker lines using a predefined segmentation network model to obtain a marker line skeleton image.

[0058] The feature verification module, connected to the region extraction module, is used to calculate the geometric features and structural constraints of the skeleton based on the marker line skeleton image to obtain the calculation result, and to fuse the marker line skeleton image and the calculation result to determine whether it is the target marker line to obtain the verification result.

[0059] The loosening judgment module, connected to the feature verification module, is used to extract the current main direction angle of the marker line based on the verification result, compare it with the reference angle to obtain the comparison result, and perform time-series fusion with the multi-frame detection results of the same bolt to determine whether the bolt is loose to obtain the loosening state result.

[0060] Compared with existing technologies, the advantages of this invention lie in its ability to automatically detect bolt loosening through multimodal image acquisition and fusion, target detection, marker line extraction and verification, and angle analysis and fusion. Multimodal image acquisition and fusion effectively overcomes interference from complex environments, providing high-quality images. Target detection and marker line extraction and verification improve detection accuracy and reliability. Angle analysis and fusion enable precise assessment of the bolt's rotation state. These steps work together to form a complete detection process, effectively improving the automation and accuracy of the detection, and providing strong support for the safe operation and maintenance of equipment.

[0061] In particular, fusing visible light, near-infrared, and polarized light images allows for the comprehensive utilization of the advantages of each modality, yielding more complete and abundant information about the bolts and their marking lines. The fused image contains information from multiple modalities, providing a more robust data foundation for subsequent marking line detection and analysis, thus improving the accuracy and reliability of the detection.

[0062] In particular, by calculating the frontal view transformation matrix and generating the ROI image, the influence of viewpoint differences on detection can be eliminated. This allows subsequent marker line detection and analysis to be performed under a unified standard viewpoint, improving the system's adaptability to images from different viewpoints and enhancing the robustness and versatility of the entire detection process, making it suitable for various practical detection scenarios. The precise position and size information of the target detection box helps subsequent marker line extraction and analysis steps to more efficiently focus on the critical areas of the bolt, avoiding the processing of irrelevant areas, reducing computational load and resource consumption, and improving the overall operating efficiency of the detection system.

[0063] In particular, by constructing feature templates for marker lines, sliding window feature matching, segmentation network model segmentation, and skeletonization, a series of specific and detailed methods can be used to effectively extract candidate regions for marker lines and generate high-quality marker line skeleton images. This provides strong support for subsequent bolt loosening detection and can meet the requirements of intelligent detection of high-strength bolt loosening under complex working conditions.

[0064] In particular, by comprehensively considering the geometric features, structural constraints, and confidence information of the segmentation mask of the marker line, the characteristics of the marker line can be described more comprehensively. Weighted fusion of this information allows for a more accurate determination of whether the marker line conforms to a predefined feature template, improving the accuracy of marker line recognition and reducing false positives and false negatives. The predefined marker line feature template library contains various types of marker line features, adaptable to marker lines of different shapes, sizes, and structures. In actual detection, regardless of whether the marker line is a straight line, curve, V-shape, or other shape, as long as its features have a sufficiently high similarity to a template in the library, it can be accurately identified, improving the versatility and adaptability of the detection system and meeting diverse marker line recognition needs in different scenarios.

[0065] In particular, by extracting the orientation angle of the marker line using three methods—Radon transform, structure tensor, and least squares line fitting—and performing consistency checks and fusion processing, the main orientation angle of the marker line can be accurately determined under different conditions. This method fully utilizes the advantages of various algorithms, complementing and verifying each other, effectively avoiding the problem of inaccurate angle calculations caused by the limitations or errors of a single method, thus improving the accuracy and reliability of the detection results. Using Kalman filtering to perform temporal fusion of multi-frame detection results can remove noise and jitter interference during the detection process, resulting in more stable and accurate rotation angle estimates. Simultaneously, combining multi-view weighted fusion, comprehensively considering the detection results and their confidence levels under different viewpoints, further improves the accuracy and comprehensiveness of rotation angle measurement, providing a reliable basis for judging bolt loosening. Attached Figure Description

[0066] Figure 1A flowchart illustrating the intelligent detection method for high-strength bolt loosening based on deep learning provided by this invention;

[0067] Figure 2 This is a flowchart illustrating step S3 in the deep learning-based intelligent detection method for loosening high-strength bolts provided by the present invention.

[0068] Figure 3 This is a flowchart illustrating step S5 in the deep learning-based intelligent detection method for loosening high-strength bolts provided by the present invention.

[0069] Figure 4 This is a schematic diagram of the structure of the intelligent detection system for high-strength bolt loosening based on deep learning provided by the present invention. Detailed Implementation

[0070] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0071] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0072] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0073] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0074] Please see Figure 1 As shown, this invention provides a deep learning-based intelligent detection method for high-strength bolt loosening, comprising:

[0075] Step S1: Real-time acquisition of visible light, near-infrared light, and polarized light images of the bolt to obtain an initial image;

[0076] Specifically, a high-resolution RGB camera is employed, providing rich color and texture information to clearly capture the marking lines and details on the bolt surface. The camera is equipped with autofocus and exposure functions to adapt to different lighting conditions and shooting distances. Using a camera sensitive to the near-infrared band allows it to penetrate certain oil stains and surface coatings, revealing the outline of the underlying marking lines. A hardware synchronization trigger mechanism ensures that images of all modalities are acquired at the same time. This avoids image mismatch problems caused by time differences, especially when the bolt or camera is moving. By installing a polarizing filter in front of the camera lens, images at different polarization angles are acquired. Subsequently, image processing algorithms are used to calculate the degree of polarization, effectively suppressing high-gloss reflections from the metal surface and improving the visibility of the marking lines. The camera system is arranged 360 degrees around the bolt, enabling it to capture images of the bolt from multiple angles, ensuring that the marking lines are clearly recorded from different viewing angles.

[0077] Step S2: The initial image is fused using a fusion algorithm to generate a fused image. The fused image is then adjusted for contrast and brightness using an adaptive illumination invariant transformation algorithm to obtain the target image.

[0078] Specifically, wavelet transform is applied to the initial image to decompose it into sub-band images of different scales and orientations, including low-frequency sub-band images (approximate images) and high-frequency sub-band images (horizontal, vertical, and diagonal detail images). For example, commonly used wavelet functions such as Haar wavelets or Daubechies wavelets can be used for decomposition. The decomposition scale can be selected based on the image size and features, typically involving 2-3 levels of decomposition. Based on the image fusion objective and the characteristics of different modal images, appropriate fusion rules are selected to fuse the wavelet-decomposed sub-band images. For low-frequency sub-band images, a weighted average method can be used, summing the grayscale values ​​of corresponding pixels in the low-frequency sub-band images of each modality according to certain weights. The weights can be determined based on the importance of each modality in the low-frequency region. For example, in the fusion of visible light and near-infrared images, if the visible light image contains more overall structural information in the low-frequency region, it can be assigned a slightly higher weight. For high-frequency sub-band images, the absolute value maximization method can be used. This method selects the pixel with the largest absolute value in the high-frequency sub-band image corresponding to each modality image to retain more detail information. For example, when fusing visible light and polarized light images, the polarized light image can better suppress high-light reflection in certain details. The corresponding high-frequency sub-band image may have larger absolute values ​​in these areas. The absolute value maximization method can highlight these detailed features, achieving image feature fusion. The fused low-frequency and high-frequency sub-band images are then subjected to inverse wavelet transform to obtain the fused image. The inverse wavelet transform process is the reverse of the wavelet transform. By integrating sub-band images of different scales and orientations, a fused image containing multimodal image features is reconstructed.

[0079] Specifically, based on Retinex theory, Gaussian filtering is used to process the fused image to estimate the illumination components. A Gaussian kernel of appropriate size (e.g., a larger kernel size) is selected, and a convolution operation is performed on the fused image to obtain the illumination component image. Gaussian filtering smooths the image, removes high-frequency details, and retains low-frequency illumination variation information, thus achieving illumination component estimation. Using formulas from Retinex theory, a logarithmic transformation is used to convert the multiplicative relationship of the fused image into an additive relationship, and the reflection component image is calculated. Specifically, a logarithmic operation is performed on the fused image and the estimated illumination component image, and then the two are subtracted to obtain the reflection component image. This image reflects the inherent color and texture attributes of the object, is independent of illumination variations, and provides more stable visual features. For the reflection component image, an adaptive contrast-limited histogram equalization algorithm is applied. The image is divided into several small blocks, and the histogram within each block is equalized to enhance local contrast. Simultaneously, the magnitude of contrast amplification is limited to avoid excessive noise amplification that degrades image quality. The CLAHE algorithm effectively enhances image details, especially in both dark and bright areas, making features like marker lines more clearly visible. It adaptively selects gamma correction parameters based on the image's global average brightness. The average grayscale value of all pixels is calculated. If the average brightness is greater than 160 (image too bright), a gamma value of 0.8 is selected for correction to enhance dark details; if the average brightness is less than 80 (image too dark), a gamma value of 1.2 is selected to improve overall brightness and suppress noise; if the average brightness is between 80 and 160, the gamma value remains at 1.0, and no gamma correction is applied. Gamma correction further adjusts the image's brightness and contrast, optimizing its visual effect and making it clearer and more natural. Since Retinex processing and subsequent contrast and brightness enhancement may cause color distortion, color restoration is necessary. A color restoration factor is used to adjust each color channel of the processed image to restore its true colors. The color restoration factor can be determined based on the original color characteristics of the image and the changes during the processing. By using matrix operations or lookup tables, the grayscale values ​​of the three color channels of the image are adjusted to make the colors of the image more accurate and natural, and finally obtain the target image.

[0080] Specifically, fusing visible light, near-infrared, and polarized light images allows for the comprehensive utilization of the advantages of each modality, yielding more complete and abundant information about the bolts and their marking lines. The fused image contains information from multiple modalities, providing a more robust data foundation for subsequent marking line detection and analysis, thus improving the accuracy and reliability of the detection.

[0081] Step S3: Use a pre-trained target detection model to monitor the bolts in the target image to generate a ROI image of the bolts;

[0082] Specifically, such as Figure 2 As shown, the process of step S3 includes:

[0083] Step S31: Use the target detection model to detect bolts in the target image to generate an initial detection box;

[0084] Specifically, a lightweight deep learning architecture for object detection is chosen, such as MobileNetV3+SSD-Lite or YOLOv5n. The model is pre-trained using a large dataset of labeled bolt images, enabling it to learn feature representations of bolts under different lighting, angles, and background conditions. During training, model parameters, such as learning rate and batch size, are adjusted to optimize performance and improve detection accuracy and speed.

[0085] Specifically, the model extracts features from the target image using a convolutional neural network, obtaining feature maps at different locations and scales within the image. Then, the detection head in the model analyzes the feature maps to predict the location and category probability of the bolt, generating initial detection boxes.

[0086] Step S32: Optimize the initial detection box using a graph attention network, model the spatial relationship between bolts to adjust the position and size of the detection box to obtain the target detection box;

[0087] Specifically, the process of optimizing the initial detection box using a graph attention network and modeling the spatial relationship between bolts to adjust the position and size of the detection box to obtain the target detection box includes:

[0088] Construct bolt node feature vectors, including detection box coordinates and semantic information;

[0089] Specifically, for each initial detection box, its coordinate information is extracted, including parameters such as the center coordinates, width, and height of the detection box. This coordinate information reflects the position and size of the detection box in the image. Combining the class probabilities and feature maps output by the object detection model, semantic information is encoded for the bolts within the detection boxes. A feature extraction network, such as a small convolutional neural network, can be used to extract features from the image region within the detection box, obtaining a fixed-length feature vector containing semantic information such as the bolt's appearance and shape. Alternatively, the feature vector at the corresponding position in the detection box from the feature map output by the object detection model can be directly used as semantic information. The detection box coordinates and semantic information are combined to construct the bolt node feature vector, which serves as the input node feature for the graph attention network.

[0090] Graph attention networks are used to calculate the attention weights between nodes to quantify the spatial relationships between bolts;

[0091] Specifically, all bolt node feature vectors are constructed into a graph structure, where each node represents a bolt, and the connections between nodes represent potential spatial relationships between bolts. When constructing the graph, factors such as distance and angle between bolts can be considered to determine the connection weights between nodes. Alternatively, a fully connected graph can be constructed first, allowing the graph attention network to automatically learn the correlations between nodes. The graph attention network calculates attention weights between nodes through an attention mechanism to quantify the spatial relationships between bolts. Specifically, for each node, an attention function (such as a multilayer perceptron-based attention mechanism or a multi-head self-attention mechanism) is used to calculate the attention score between that node and other nodes. The attention function typically considers factors such as similarity and correlation between node feature vectors, and through operations such as weight matrices and activation functions, maps node features to an attention space, in which the attention weights between nodes are calculated.

[0092] The node feature vector is updated based on the attention weights, and the coordinate offset of the detection box is predicted through a fully connected network.

[0093] Specifically, the calculated attention weights are used to weight and sum the bolt node feature vectors, updating the feature vector of each node. The updated node feature vector incorporates feature information from other associated bolt nodes, more comprehensively reflecting the position and semantic features of the bolts in the overall spatial layout, thus providing a more accurate feature representation for subsequent detection box optimization. The updated node feature vectors are then input into a fully connected network. Based on the learned mapping relationship between features and coordinate offsets, the fully connected network predicts the coordinate offset of each detection box. The structure of the fully connected network can include multiple hidden layers, and nonlinear factors are introduced through activation functions to improve the model's fitting ability. During training, supervised learning is used, with the error between the predicted coordinate offset and the true value used as the loss function to adjust the parameters of the fully connected network, enabling it to accurately predict the coordinate offset.

[0094] The initial detection box position and size are adjusted according to the coordinate offset to obtain the target detection box.

[0095] Specifically, the predicted coordinate offsets are added to the center coordinates of the initial detection box to obtain new center coordinates; at the same time, the width and height of the initial detection box are scaled or translated according to the predicted width and height offsets to obtain the adjusted position and size of the detection box.

[0096] Step S33: Calculate the frontal view transformation matrix of the bolt based on the target detection box, project it onto the normalized frontal view region, and generate the ROI image.

[0097] Specifically, geometric transformation methods such as perspective transformation or affine transformation are used to achieve viewpoint transformation. First, the coordinates of the four corner points of the detection box in the source image are determined. Then, based on the shape and size requirements of the bolt under the frontal view, the coordinates of the four corner points of the target image are set. By solving the transformation equation, the transformation matrix is ​​obtained. This transformation matrix maps the image pixels within the detection box to the corresponding positions in the ROI image. Using the calculated transformation matrix, a projection transformation is performed on the image region within the detection box, transforming it to the frontal view. During the transformation process, methods such as bilinear interpolation are used to resample pixel values ​​to ensure the quality and integrity of the transformed image.

[0098] Specifically, by calculating the frontal view transformation matrix and generating the ROI image, the impact of viewpoint differences on detection can be eliminated. This allows subsequent marker line detection and analysis to be performed under a unified standard viewpoint, improving the system's adaptability to images from different viewpoints and enhancing the robustness and versatility of the entire detection process, making it suitable for various practical detection scenarios. The precise position and size information of the target detection box helps subsequent marker line extraction and analysis steps to more efficiently focus on the critical areas of the bolt, avoiding the processing of irrelevant areas, reducing computational load and resource consumption, and improving the overall operating efficiency of the detection system.

[0099] Step S4: Extract candidate regions for marker lines from the ROI image, and segment the candidate regions using a predefined segmentation network model to obtain a marker line skeleton image.

[0100] Specifically, step S4 includes the following process:

[0101] Construct a feature template for the marker line, defining its shape and size;

[0102] Specifically, a large number of bolt marking line images are analyzed to summarize the shape and size characteristics of the marking lines. For example, marking lines are typically straight lines, curves, or specific symbols, with a certain width range and length ratio. Simultaneously, the variations in marking lines under different lighting, angles, and environmental conditions are considered to ensure that the feature templates can cover various possible marking line shapes. Based on the analysis results, marking line feature templates are designed. Templates of various shapes and sizes can be used, such as straight line templates of different lengths and widths, and curve templates of different curvatures. These templates are stored in a template library for use in subsequent feature matching processes. The template library can be stored as image files, in matrix form, or in other computer-processable formats.

[0103] A sliding window is used in the ROI image, and the similarity with the feature template is calculated to obtain the candidate regions for the marker lines;

[0104] Specifically, a sliding window is set on the ROI image. The size and shape of the window can be adjusted according to the typical size and shape of the marker lines. For example, if the marker lines are mostly thin straight lines, a smaller rectangular window can be set; if the marker lines may be large circular symbols, a circular or square window can be set. The step size of the window can be determined based on the image resolution and marker line density, and is generally small to ensure accurate detection of each part of the marker line. The image region within the sliding window is compared pixel-by-pixel with the marker line feature template to calculate the similarity. Various similarity measurement methods can be used, such as normalized cross-correlation coefficients. For each window position, its similarity value with each template in the template library is calculated, and the window positions with high similarity are recorded as candidate regions for marker lines.

[0105] The segmentation network model is used to perform pixel-level segmentation of the candidate region of the marker line to extract the connected regions of the marker line and obtain the segmentation mask.

[0106] Specifically, a suitable deep learning segmentation network model for marker line segmentation is selected, and the model is trained using a large dataset of images labeled with marker lines. The labeled data can be pixel-level marker line mask images. During training, the model's network structure, hyperparameters (such as learning rate and regularization coefficient), and loss function are adjusted to optimize segmentation performance and improve accuracy and speed. Data augmentation techniques, such as rotation, flipping, and scaling, can be used to expand the training dataset and enhance the model's adaptability to different scenes and variations. The extracted candidate region images of marker lines are input into the trained segmentation network model. The model performs feature extraction and segmentation operations on the image through network structures such as convolutional layers, pooling layers, and upsampling layers, outputting a segmentation mask of the same size as the input image. Each pixel in the segmentation mask is labeled as belonging to either the marker line or the background, thereby achieving pixel-level segmentation of the marker lines and extracting the connected regions of the marker lines.

[0107] The segmentation mask is skeletonized to obtain a marker line skeleton image.

[0108] Specifically, classic skeletonization algorithms, such as the Zhang-Suen algorithm and median transformation, are used to skeletonize the segmentation mask. These algorithms iteratively remove boundary pixels of the target region while preserving the target's topology and connectivity, ultimately obtaining the target's skeleton image. Post-processing is then applied to the skeletonized image, such as removing isolated points, filling holes, and smoothing skeleton lines, to optimize the image quality. Morphological operations (such as erosion, dilation, opening, and closing operations) or curve fitting-based methods can be used to smooth the skeleton lines, removing burrs and irregularities caused by noise or inaccurate segmentation, making the skeleton lines clearer, more continuous, and smoother, and more consistent with the shape characteristics of actual marker lines.

[0109] Specifically, by constructing feature templates for marker lines, sliding window feature matching, segmentation network model segmentation, and skeletonization, a series of detailed methods can be used to effectively extract candidate regions for marker lines and generate high-quality marker line skeleton images. This provides strong support for subsequent bolt loosening detection and meets the requirements for intelligent detection of high-strength bolt loosening under complex working conditions.

[0110] Step S5: Calculate the geometric features and structural constraints of the skeleton based on the skeleton image of the marker line to obtain the calculation result. Then, fuse the skeleton image of the marker line and the calculation result to determine whether it is the target marker line and obtain the verification result.

[0111] Specifically, such as Figure 3 As shown, the process of step S5 includes:

[0112] Step S51: Predefine the marker line feature template library;

[0113] Specifically, a large number of sample images of marker lines of different types, shapes, and sizes are collected, including common marker line types such as straight lines, curves, V-shapes, single lines, and double lines. For each sample image, its geometric features and structural constraints are extracted, and these features are parameterized and stored in a marker line feature template library. At the same time, each template marker line type is labeled to ensure accurate identification during matching.

[0114] Step S52: Extract the aspect ratio, linearity, and width consistency of the marked line skeleton image to obtain the geometric features;

[0115] Specifically, for the marker line skeleton image, its length and width are calculated. The length is obtained by calculating the maximum distance between pixels along the principal axis of the skeleton image, while the width is obtained by calculating the average distance perpendicular to the principal axis. The aspect ratio, the ratio of length to width, reflects the elongation of the marker line. Using the coordinates of the pixels in the skeleton image, a straight line is fitted, and the average distance from each pixel in the skeleton image to this line is calculated. Linearity can be defined as the ratio of 1 to this average distance; the higher the linearity, the closer the marker line is to a straight line. Along the length of the marker line skeleton, its width is measured at regular intervals, and the ratio of the standard deviation to the mean of these width values ​​is calculated. The smaller this ratio, the less variation in the width of the marker line and the better the width consistency.

[0116] Step S53: Extract the radiality and cross-boundary properties of the marked line skeleton image to obtain the structural constraints;

[0117] Specifically, the relative positional relationship between the marker line skeleton and other bolt components or structures is analyzed. For example, the angle between the marker line and the bolt center is calculated, and the pixel distribution at different angles is statistically analyzed. If the marker line has a large and continuous pixel distribution within a certain angle range, it is considered to have a certain radial characteristic, which can be quantified by calculating a radial score. The marker line is also checked to see if it crosses certain specific boundaries or regions of the bolt. For example, by comparing it with a predefined boundary template of the bolt, the ratio of the intersection length of the marker line skeleton with the boundary to the total length of the marker line is calculated. If the ratio exceeds a certain threshold, the marker line is considered to have a boundary-crossing characteristic, which can also be represented by a score.

[0118] Step S54: Match the geometric features and structural constraints with the marker line feature template library to calculate a similarity score;

[0119] Specifically, the extracted geometric features and structural constraints are normalized to construct feature vectors for the marker lines. These feature vectors are then compared to each template in the marker line feature template library, and a similarity score is calculated. Methods such as Euclidean distance and cosine similarity can be used to measure the similarity between feature vectors. A higher similarity score indicates a better match between the marker line and the template.

[0120] Step S55: Perform weighted fusion based on the segmentation mask, the geometric features, and the structural constraints to determine the comprehensive similarity score;

[0121] Specifically, the process of determining the comprehensive similarity score by weighted fusion based on the segmentation mask, the geometric features, and the structural constraints includes:

[0122] The segmentation mask is binarized to extract the boundary information of the marker lines;

[0123] Specifically, the segmentation mask is binarized to convert pixel values ​​to 0 or 1, and the boundary information of the marker lines is extracted.

[0124] Calculate the confidence score of the segmentation mask based on the boundary information;

[0125] Specifically, the confidence score of the segmentation mask is calculated based on the integrity and continuity of the boundary pixels. For example, if the boundary pixels are continuous and closed, the confidence score is high; if there are many breaks or gaps in the boundary, the confidence score is low.

[0126] The geometric features are normalized and their similarity scores are calculated.

[0127] Specifically, after normalizing the geometric features, the aspect ratio, linearity, and width consistency are compared with the corresponding features in the template library, and their similarity scores are calculated respectively. Then, these similarity scores are weighted and summed according to pre-set weights (such as weights determined by experience or experiments) to obtain the comprehensive similarity score of the geometric features.

[0128] The structural constraints are quantitatively calculated to obtain similarity scores.

[0129] Specifically, structural constraints are quantified and converted into numerical similarity scores. Similarly, the radial and cross-boundary similarity scores are weighted and summed according to set weights to obtain a comprehensive similarity score for the structural constraints.

[0130] The confidence score, the similarity score of geometric features, and the similarity score of structural constraints are weighted and summed to obtain the comprehensive similarity score.

[0131] Specifically, the confidence score, the comprehensive similarity score of geometric features, and the comprehensive similarity score of structural constraints are multiplied by their respective weighting coefficients (the weighting coefficients can be set according to the importance of each feature to the identification of the marker line, such as through experiments or expert experience). Then, these three weighted scores are added together to obtain the final comprehensive similarity score.

[0132] Step S56: Compare the comprehensive similarity score with a predefined similarity threshold to determine whether it is a valid target marker line, and obtain the verification result.

[0133] Specifically, based on actual detection needs and a large amount of experimental data, one or more similarity thresholds are pre-set. For example, a primary threshold (such as 0.7) can be set to distinguish between valid target marker lines and non-target marker lines. Furthermore, multiple thresholds can be set according to the importance of the marker lines and the complexity of the detection scenario (such as a slight match threshold of 0.5, a moderate match threshold of 0.7, and a high match threshold of 0.9) to achieve multi-level warnings and more detailed classification. The threshold setting can be determined through statistical analysis of the comprehensive similarity scores of a large number of marker line samples. The distribution of the comprehensive similarity scores of these samples is calculated, the score distribution range of valid target marker lines and non-target marker lines is analyzed, and the score value that can effectively distinguish between the two is selected as the threshold.

[0134] Specifically, by comprehensively considering the geometric features, structural constraints, and confidence information of the segmentation mask of the marker line, the characteristics of the marker line can be described more comprehensively. Weighted fusion of this information allows for a more accurate determination of whether the marker line conforms to a predefined feature template, improving the accuracy of marker line recognition and reducing false positives and false negatives. The predefined marker line feature template library contains various types of marker line features, adaptable to marker lines of different shapes, sizes, and structures. In actual detection, regardless of whether the marker line is a straight line, curve, V-shape, or other shape, as long as its features have a sufficiently high similarity to a template in the library, it can be accurately identified, improving the versatility and adaptability of the detection system and meeting diverse marker line recognition needs in different scenarios.

[0135] Step S6: Extract the current main direction angle of the marker line based on the verification result, compare it with the reference angle to obtain the comparison result, and perform time-series fusion with the multi-frame detection results of the same bolt to determine whether the bolt is loose to obtain the loose state result.

[0136] Specifically, step S6 includes the following process:

[0137] Based on the verification results, a multi-method fusion strategy is used to calculate the current main direction angle to obtain the target angle.

[0138] Specifically, the process of extracting and calculating the current principal direction angle using a multi-method fusion strategy to obtain the target angle includes:

[0139] The Radon transform is applied to the skeleton image of the marked line to obtain the line direction;

[0140] Specifically, straight lines in an image are detected by calculating the projection of the image at different angles. In the Radon transform, the transform angle is typically scanned from 0 degrees to 180 degrees in a certain step (e.g., 1 degree). For the projection data obtained at each angle, the angle corresponding to the peak point is found, and this angle is a candidate value for the line direction of the marker line.

[0141] The gradient of each pixel in the skeleton image of the marked line is calculated using the structure tensor to obtain the overall dominant direction;

[0142] Specifically, the gradient magnitude and direction are calculated for each pixel in the marker line skeleton image. The gradient can be calculated using operators such as Sobel, Scharr, or Gaussian gradient operators. The gradient magnitude reflects the intensity of the gray-level change at that pixel, while the gradient direction indicates the direction of the fastest gray-level change. A structure tensor is constructed based on the pixel gradient information. The structure tensor is a 2×2 matrix used to describe the directionality and consistency of local regions in the image. By performing eigenvalue decomposition on the structure tensor, the overall dominant direction of the image is obtained, which serves as another candidate value for the marker line's direction.

[0143] By performing line fitting on the skeleton pixels using least squares, the angle corresponding to the slope of the line is obtained;

[0144] Specifically, the coordinates of all non-zero pixels in the marker line skeleton image are collected; these coordinates represent the position of the marker line in the image. A line is fitted to these pixel coordinates using the least squares method to determine the slope of the fitted line. Based on the relationship between slope and angle (angle equals arctangent slope), the angle between the fitted line and the horizontal axis is calculated, serving as the third candidate value for the marker line's direction.

[0145] A consistency check is performed based on the angle corresponding to the line direction, the overall dominant direction, and the line slope. If it is within the tolerance range, the median value is taken as the target angle; otherwise, outliers are discarded, the average of the remaining two is taken as the target angle, and the confidence level is reduced.

[0146] Specifically, based on actual testing requirements and allowable error, a pre-defined angle tolerance range (e.g., ±2 degrees) is used to determine the consistency among the candidate angle values ​​obtained by the three methods. The line direction obtained from the Radon transform, the overall dominant direction calculated from the structure tensor, and the line slope angle obtained from least-squares line fitting are compared pairwise. If all three angles are within each other's tolerance range, they are considered consistent, and the median of these three angles is taken as the final current dominant direction angle of the marker line (target angle). If one or two angles exceed the tolerance range, these outliers are discarded. If two angles remain, their average is taken as the target angle, and the confidence level of this measurement is reduced by one level to indicate a decrease in the reliability of the angle calculation result. If none of the three angles are within each other's tolerance range, the measurement result is unreliable and requires re-testing.

[0147] The relative rotation angle is calculated by comparing the target angle with the preset reference angle;

[0148] Specifically, the angle of the marked line on the bolt in its initial tightened state is pre-recorded as a reference angle. This reference angle can be obtained by measuring it with precise measuring tools during bolt installation or maintenance, or by high-precision image detection in the initial state, and stored in the detection system's database. The extracted current principal direction angle (target angle) is subtracted from the reference angle to obtain the relative rotation angle. The relative rotation angle reflects the degree of rotation of the bolt relative to its initial state, and its sign indicates the direction of rotation (clockwise or counterclockwise).

[0149] Based on the relative rotation angle, a Kalman filter is used to perform temporal fusion of the detection results of multiple frames, and the target fusion angle is obtained by combining multi-view weighted fusion.

[0150] Specifically, a Kalman filter is initialized based on the bolt's motion and measurement models. Parameters are set for the filter, including state variables (such as rotation angle and angular velocity), observation variables (currently measured relative rotation angle), state transition matrix, observation matrix, process noise covariance matrix, and observation noise covariance matrix. These parameters can be estimated and adjusted based on the bolt's actual motion characteristics and the accuracy of the measuring equipment. The relative rotation angles of the same bolt detected at multiple consecutive moments are input into the Kalman filter. The filter predicts the state at the next moment based on the state transition model and updates the predicted state based on the current observation value to obtain the optimal estimated rotation angle. Kalman filtering smooths noise and jitter in multi-frame detection results, improving the accuracy and stability of rotation angle estimation. For the rotation angle results of the same bolt detected from multiple perspectives, weights are calculated based on the observation confidence of each perspective and the angle between the perspective and the bolt axis. Detection results with high observation confidence and small angles between the perspective and the bolt axis (closer to the frontal view) have higher weights. The rotation angle results of each perspective are multiplied by their corresponding weights and then summed to obtain the target fused angle.

[0151] Based on the difference between the target fusion angle and the reference angle, a layered early warning mechanism is used to determine whether the bolt is loose in order to obtain the bolt loosening result.

[0152] Specifically, the process of determining whether the bolt is loose based on the difference between the target fusion angle and the reference angle through a layered early warning mechanism to obtain the bolt loosening result includes:

[0153] Pre-set angle thresholds corresponding to slight loosening, significant loosening, and normal state;

[0154] Specifically, based on the actual usage requirements and historical data of the bolts, pre-set thresholds for slight loosening (e.g., 5 degrees), significant loosening (e.g., 10 degrees), and normal condition (e.g., 3 degrees). These thresholds can be determined by the mechanical characteristics of the bolts, the usage environment, and the experience of experts in the field, and can be optimized and adjusted according to actual testing results.

[0155] Calculate the difference between the target fusion angle and the reference angle;

[0156] If the difference exceeds the slight loosening threshold, a slight loosening warning is triggered; if it exceeds the significant loosening threshold, a significant loosening warning is triggered.

[0157] Specifically, the difference between the target fusion angle and the reference angle is calculated. If the difference exceeds the slight loosening threshold but does not reach the significant loosening threshold, a slight loosening warning is triggered, indicating that the bolt is slightly loose and requires observation and further monitoring. If the difference exceeds the significant loosening threshold, a significant loosening warning is triggered, indicating that the bolt is significantly loose and requires timely maintenance and tightening.

[0158] Record the historical difference sequence of bolts, use an exponential smoothing model to fit the trend of change, and if the rate of change is consistently positive and large, a trend warning will be triggered even if the current difference does not exceed the threshold, prompting attention to the health degradation trend of the bolts.

[0159] Specifically, the historical difference sequence of bolts is recorded, and the sequence is fitted using an exponential smoothing model to analyze its trend. If the rate of change of the difference is consistently positive and large (e.g., the rate of change exceeds a certain set threshold), a trend warning is triggered even if the current difference does not exceed the slight loosening threshold, prompting operators to pay attention to the health degradation trend of the bolts and take preventive maintenance measures in advance to avoid potential failures and accidents caused by bolt loosening.

[0160] Specifically, the direction angle of the marker line is extracted using three methods: Radon transform, structure tensor, and least squares line fitting. Consistency checks and fusion processing are then performed to accurately determine the principal direction angle of the marker line under different conditions. This method fully utilizes the advantages of various algorithms, complementing and validating each other, effectively avoiding inaccurate angle calculations caused by the limitations or errors of a single method, thus improving the accuracy and reliability of the detection results. Using Kalman filtering to perform temporal fusion of multi-frame detection results removes noise and jitter interference during the detection process, resulting in more stable and accurate rotation angle estimates. Simultaneously, combining multi-view weighted fusion comprehensively considers the detection results and their confidence levels under different viewpoints, further improving the accuracy and comprehensiveness of rotation angle measurement and providing a reliable basis for judging bolt loosening.

[0161] Specifically, such as Figure 4 As shown, the present invention also provides a high-strength bolt loosening intelligent detection system based on deep learning, comprising:

[0162] Image acquisition module 10 is used to acquire visible light, near-infrared light and polarized light images of the bolt in real time to obtain an initial image;

[0163] The image fusion module 20 is connected to the image acquisition module 10 and is used to fuse the initial image using a fusion algorithm to generate a fused image. The fused image adjusts the image contrast and brightness through an adaptive illumination invariant transformation algorithm to obtain the target image.

[0164] The target detection module 30 is connected to the image fusion module 20 and is used to use a pre-trained target detection model to monitor the bolts in the target image to generate the ROI image of the bolts;

[0165] The region extraction module 40 is connected to the target detection module 30 and is used to extract candidate regions for marker lines from the ROI image and to segment the candidate regions for marker lines using a predefined segmentation network model to obtain a marker line skeleton image.

[0166] The feature verification module 50, connected to the region extraction module 40, is used to calculate the geometric features and structural constraints of the skeleton based on the marker line skeleton image to obtain the calculation result, and to fuse the marker line skeleton image and the calculation result to determine whether it is a target marker line to obtain the verification result.

[0167] The loosening judgment module 60 is connected to the feature verification module 50. It is used to extract the current main direction angle of the marker line according to the verification result, compare it with the reference angle to obtain the comparison result, and combine it with the multi-frame detection results of the same bolt to perform time-series fusion to determine whether the bolt is loose to obtain the loosening state result.

[0168] Specifically, the high-strength bolt loosening intelligent detection system based on deep learning provided by the present invention can execute the high-strength bolt loosening intelligent detection method based on deep learning in the embodiments of the present invention, and can achieve the same technical effect, which will not be repeated here.

[0169] Specifically, this invention achieves automated detection of bolt loosening through steps such as multimodal image acquisition and fusion, target detection, marker line extraction and verification, and angle analysis and fusion. Multimodal image acquisition and fusion effectively overcomes interference from complex environments, providing high-quality images. Target detection and marker line extraction and verification improve detection accuracy and reliability. Angle analysis and fusion enables precise assessment of the bolt's rotation state. These steps work together to form a complete detection process, effectively improving the automation and accuracy of the detection, and providing strong support for the safe operation and maintenance of equipment.

[0170] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0171] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A deep learning-based intelligent detection method for loosening high-strength bolts, characterized in that, include: Step S1: Real-time acquisition of visible light, near-infrared light, and polarized light images of the bolt to obtain an initial image; Step S2: The initial image is fused using a fusion algorithm to generate a fused image. The fused image is then adjusted for contrast and brightness using an adaptive illumination invariant transformation algorithm to obtain the target image. Step S3: Use a pre-trained target detection model to detect bolts in the target image to generate a bolt ROI image; Step S4: Extract candidate regions for marker lines from the ROI image, and segment the candidate regions using a predefined segmentation network model to obtain a marker line skeleton image. The process of step S4 includes: Construct a feature template for the marker line, defining its shape and size; A sliding window is used in the ROI image, and the similarity with the feature template is calculated to obtain the candidate regions for the marker lines; The segmentation network model is used to perform pixel-level segmentation of the candidate region of the marker line to extract the connected regions of the marker line and obtain the segmentation mask. The segmentation mask is skeletonized to obtain a marker line skeleton image; Step S5: Calculate the geometric features and structural constraints of the skeleton based on the skeleton image of the marker line to obtain the calculation result. Then, fuse the skeleton image of the marker line and the calculation result to determine whether it is the target marker line and obtain the verification result. The process of step S5 includes: A predefined library of marker line feature templates; The aspect ratio, linearity, and width consistency of the marked line skeleton image are extracted to obtain the geometric features; The radiality and cross-boundary properties of the marked line skeleton image are extracted to obtain the structural constraints; The geometric features and structural constraints are matched with the marker line feature template library to calculate a similarity score; A weighted fusion is performed based on the segmentation mask, the geometric features, and the structural constraints to determine a comprehensive similarity score; The process of determining the comprehensive similarity score by weighted fusion based on the segmentation mask, the geometric features, and the structural constraints includes: The segmentation mask is binarized to extract the boundary information of the marker lines; Calculate the confidence score of the segmentation mask based on the boundary information; The geometric features are normalized and their similarity scores are calculated. The structural constraints are quantitatively calculated to obtain similarity scores. The confidence score, the similarity score of geometric features, and the similarity score of structural constraints are weighted and summed to obtain the comprehensive similarity score. The verification result is obtained by comparing the comprehensive similarity score with a predefined similarity threshold to determine whether it is a valid target marker line. Step S6: Extract the current main direction angle of the marker line based on the verification result, compare it with the reference angle to obtain the comparison result, and perform time-series fusion with the multi-frame detection results of the same bolt to determine whether the bolt is loose to obtain the loose state result.

2. The intelligent detection method for high-strength bolt loosening based on deep learning according to claim 1, characterized in that, The process of step S3 includes: The target detection model is used to detect bolts in the target image to generate an initial detection box; The initial detection box is optimized by using a graph attention network to model the spatial relationship between bolts and adjust the position and size of the detection box to obtain the target detection box. The frontal view transformation matrix of the bolt is calculated based on the target detection box, and then projected onto the normalized frontal view region to generate the ROI image.

3. The intelligent detection method for high-strength bolt loosening based on deep learning according to claim 2, characterized in that, The process of optimizing the initial detection box using a graph attention network and modeling the spatial relationship between bolts to adjust the position and size of the detection box to obtain the target detection box includes: Construct bolt node feature vectors, including detection box coordinates and semantic information; Graph attention networks are used to calculate the attention weights between nodes to quantify the spatial relationships between bolts; The node feature vector is updated based on the attention weights, and the coordinate offset of the detection box is predicted through a fully connected network. The initial detection box position and size are adjusted according to the coordinate offset to obtain the target detection box.

4. The intelligent detection method for high-strength bolt loosening based on deep learning according to claim 3, characterized in that, The process of step S6 includes: Based on the verification results, a multi-method fusion strategy is used to calculate the current main direction angle to obtain the target angle. The relative rotation angle is calculated by comparing the target angle with the preset reference angle; Based on the relative rotation angle, a Kalman filter is used to perform temporal fusion of the detection results of multiple frames, and the target fusion angle is obtained by combining multi-view weighted fusion. Based on the difference between the target fusion angle and the reference angle, a layered early warning mechanism is used to determine whether the bolt is loose in order to obtain the loosening status result.

5. The intelligent detection method for high-strength bolt loosening based on deep learning according to claim 4, characterized in that, The process of extracting and calculating the current main direction angle using a multi-method fusion strategy to obtain the target angle includes: The Radon transform is applied to the skeleton image of the marked line to obtain the line direction; The gradient of each pixel in the skeleton image of the marked line is calculated using the structure tensor to obtain the overall dominant direction; By performing line fitting on the skeleton pixels using least squares, the angle corresponding to the slope of the line is obtained; A consistency check is performed based on the angle corresponding to the line direction, the overall dominant direction, and the line slope. If it is within the tolerance range, the median value is taken as the target angle; otherwise, outliers are discarded, the average of the remaining two is taken as the target angle, and the confidence level is reduced.

6. The intelligent detection method for high-strength bolt loosening based on deep learning according to claim 5, characterized in that, The process of determining whether a bolt is loose based on the difference between the target fusion angle and the reference angle using a layered early warning mechanism to obtain the bolt loosening result includes: Pre-set angle thresholds corresponding to slight loosening, significant loosening, and normal state; Calculate the difference between the target fusion angle and the reference angle; If the difference exceeds the slight loosening threshold, a slight loosening warning is triggered; if it exceeds the significant loosening threshold, a significant loosening warning is triggered. Record the historical difference sequence of bolts, use an exponential smoothing model to fit the trend of change, and if the rate of change is consistently positive and large, a trend warning will be triggered even if the current difference does not exceed the threshold, prompting attention to the health degradation trend of the bolts.

7. A deep learning-based intelligent detection system for high-strength bolt loosening, used to implement the deep learning-based intelligent detection method for high-strength bolt loosening as described in any one of claims 1-6, characterized in that, include: The image acquisition module is used to acquire visible light, near-infrared light, and polarized light images of the bolt in real time to obtain an initial image; An image fusion module, connected to the image acquisition module, is used to fuse the initial image using a fusion algorithm to generate a fused image. The fused image is then adjusted for contrast and brightness using an adaptive illumination invariant transformation algorithm to obtain a target image. The target detection module, connected to the image fusion module, is used to detect bolts in the target image using a pre-trained target detection model to generate a bolt ROI image; The region extraction module, connected to the target detection module, is used to extract candidate regions for marker lines from the ROI image, and to segment the candidate regions for marker lines using a predefined segmentation network model to obtain a marker line skeleton image. The feature verification module, connected to the region extraction module, is used to calculate the geometric features and structural constraints of the skeleton based on the marker line skeleton image to obtain the calculation result, and to fuse the marker line skeleton image and the calculation result to determine whether it is the target marker line to obtain the verification result. The loosening judgment module, connected to the feature verification module, is used to extract the current main direction angle of the marker line based on the verification result, compare it with the reference angle to obtain the comparison result, and perform time-series fusion with the multi-frame detection results of the same bolt to determine whether the bolt is loose to obtain the loosening state result.

Images