Visual monitoring and early warning method and system for looseness of high-strength bolts of steel truss bridge

Abstract

The invention relates to the technical field of bolt looseness monitoring, in particular to a visual monitoring and early warning method and system for looseness of a high-strength bolt of a steel truss bridge, and the method comprises the steps: obtaining a bolt gray image and a reference gray image of the high-strength bolt on the steel truss bridge; strong edge points and weak edge points are obtained, and the strong edge points are clustered; calculating the linearity of each strong edge point, and screening and rejecting the strong edge points in the cluster; determining a first evaluation value and a second evaluation value of each weak edge point, adjusting a high threshold in a dual-threshold edge detection algorithm, re-dividing the weak edge points and strong edge points in a cluster, and extracting an edge line in the bolt grayscale image; acquiring a matched edge line of each edge line in the bolt grayscale image; and obtaining a loosening coefficient of the bolt gray level image, and monitoring and early warning the loosening state of the high-strength bolt. According to the invention, the monitoring accuracy of the loosening state of the high-strength bolt is improved.

Description

Technical Field

[0001] This application relates to the field of bolt loosening monitoring technology, specifically to a visual monitoring and early warning method and system for loosening high-strength bolts in steel truss bridges. Background Technology

[0002] As a core component of modern transportation infrastructure, the structural health of steel truss bridges directly affects public safety and operational efficiency. High-strength bolts are key components in truss node connections, playing a crucial role in connecting main load-bearing components and transmitting enormous shear or tensile forces. Due to long-term exposure to alternating loads, wind and rain erosion, and vibration, they are highly susceptible to preload decay or even loosening and detachment, leading to significant safety risks such as structural connection failure.

[0003] Traditional methods primarily rely on manual tapping or torque-angle methods for detection. Manual tapping is subjective, inefficient, difficult to cover hidden areas, highly dependent on inspector experience, unable to achieve real-time online monitoring, and carries the risk of working at heights. Torque-angle methods, which involve re-tightening the bolt and measuring torque or rotation angle to determine looseness, severely disrupt traffic and risk damaging the bolt or coating, affecting its lifespan. While computer vision technology can monitor the loosening of high-strength bolts in real time, edge detection and extraction of bolt edges in images are hampered by unavoidable corrosion, rust, oil, and coating deterioration, resulting in numerous interfering textures. This makes dual-threshold edge detection highly dependent on manually set global thresholds. Thresholds that are too high can lead to broken edges, while thresholds that are too low introduce significant noise, making it difficult to adapt to complex and changing real-world scenarios. This results in inaccurate bolt edge extraction, impacting the detection accuracy of high-strength bolt loosening. Summary of the Invention

[0004] To address the aforementioned technical problems, a visual monitoring and early warning method and system for loosening high-strength bolts in steel truss bridges is provided to solve the existing issues.

[0005] The solution to the technical problem addressed in this application is to provide a visual monitoring and early warning method and system for loosening high-strength bolts in steel truss bridges, comprising the following steps: In a first aspect, embodiments of this application provide a visual monitoring and early warning method for loosening high-strength bolts in steel truss bridges, the method comprising the following steps: Obtain the grayscale images of the high-strength bolts on the steel truss bridge and the reference grayscale image; Based on the gradient intensity of pixels in the grayscale image of the bolt, strong edge points and weak edge points are obtained, and strong edge points are clustered. Analyze the linear distribution of each strong edge point and other strong edge points within the cluster, calculate the linearity of each strong edge point, and filter and remove strong edge points within the cluster. Based on the symmetry of the bolt in the bolt grayscale image, all feature points symmetrical to each weak edge point are selected. The consistency of the gradient direction and the difference of grayscale values ​​between each weak edge point and its corresponding feature point are analyzed. Combined with the linearity when the feature point is a strong edge point, the first evaluation value of each weak edge point is determined. The second evaluation value of each weak edge point is calculated by considering the distance and linearity between each weak edge point and the strong edge points in the neighboring clusters. Combining the gradient strength of the weak edge points with the first evaluation value, the high threshold in the dual-threshold edge detection algorithm is adjusted, and the weak edge points and the strong edge points in the clusters are re-divided. Based on the newly divided strong edge points, the edge lines in the bolt grayscale image are extracted. The edge lines in the bolt grayscale image are matched with the reference grayscale image to obtain the matching edge lines of each edge line in the bolt grayscale image. The bolt shadow area is extracted from the bolt grayscale image and the reference grayscale image respectively. By analyzing the difference in the area of ​​the bolt shadow area between the bolt grayscale image and the reference grayscale image, as well as the angle between the edge line and its matching edge line in the bolt grayscale image, the loosening coefficient of the bolt grayscale image is obtained. This allows for the monitoring and early warning of the loosening status of high-strength bolts.

[0006] Preferably, obtaining strong edge points and weak edge points includes: setting a preset initial high threshold and a preset initial low threshold, wherein the initial high threshold is a preset multiple of the initial low threshold; and recording pixels in the bolt grayscale image with gradient intensity greater than or equal to the initial high threshold as strong edge points, and recording pixels in the bolt grayscale image with gradient intensity greater than or equal to the initial low threshold and less than the initial high threshold as weak edge points.

[0007] Preferably, the step of calculating the linearity of each strong edge point and filtering and removing strong edge points within the cluster includes: Select the strong edge point within each cluster that has the smallest distance to each strong edge point, and denote it as the neighboring point; denote the straight line connecting each strong edge point and its neighboring point as the characteristic line; calculate the sum of the distances from all strong edge points within each cluster to the characteristic line, and denote it as the distribution distance; The linearity is the ratio of the gradient intensity at each strong edge point to the distribution distance; Calculate the average linearity of all strong edge points within each cluster, and remove strong edge points within the cluster whose linearity is less than the average.

[0008] Preferably, the selection of all feature points symmetrical to each weak edge point includes: Calculate the spatial distance between the cluster center of any cluster and the cluster centers of all other clusters, and denote any cluster and the cluster with the largest spatial distance to it as a pair of parallel clusters; Calculate the distance between any two strong edge points within each pair of parallel clusters, denoted as the relative distance, and obtain the midpoint coordinates between the two strong edge points corresponding to the maximum relative distance; use the pixel point corresponding to the mean of the midpoint coordinates of all pairs of parallel clusters as the bolt center point; Construct a rectangle with each weak edge point as a vertex and the center point of the bolt as the center point of the rectangle; take the vertices within the rectangle other than the vertices corresponding to each weak edge point as all feature points corresponding to each weak edge point.

[0009] Preferably, determining the first evaluation value for each weak edge point includes: If each feature point belongs to a strong edge point, the discriminant coefficient of each feature point is the linearity of the strong edge point to which the feature point belongs; If each feature point is not a strong edge point, the discriminant coefficient of each feature point is calculated as follows: Calculate the cosine of the angle between the gradient direction of each weak edge point and the gradient direction of each feature point, and denote it as the direction consistency. Denote the difference in gray values ​​between each weak edge point and each feature point as the relative difference. Use the ratio of the direction consistency to the relative difference as the discrimination coefficient of each feature point. The first evaluation value is the sum of the discriminant coefficients of all feature points corresponding to each weak edge point.

[0010] Preferably, the calculation of the second evaluation value for each weak edge point includes: The cluster to which the strongest edge point closest to each weak edge point belongs is denoted as the neighbor cluster; Calculate the interval distance between each weak edge point and each strong edge point in its neighboring cluster; calculate the sum of the ratios of the linearity of all strong edge points in the neighboring cluster corresponding to each weak edge point to the interval distance. The second evaluation value is the ratio of the sum to the minimum interval distance.

[0011] Preferred, adjusted high threshold The calculation process is as follows: ,in, The preset initial high threshold, For the first Gradient strength at weak edge points For the first The product of the first and second evaluation values ​​of each weak edge point This represents the number of all weak edge points in the bolt grayscale image. This is the normalization function.

[0012] Preferably, the step of obtaining the matching edge lines of each edge line in the grayscale image of the bolt includes: Line detection is performed to extract edge lines from all newly defined strong edge points in the bolt grayscale image and from the bolt region in the reference grayscale image. The sum of the LBP values ​​of all pixels between the midpoint of each edge line in the bolt grayscale image and the center point of the bolt is recorded as the first sum. The sum of the LBP values ​​of all pixels between the midpoint of each edge line in the reference grayscale image and the center point of the bolt region in the reference grayscale image is recorded as the second sum. The reciprocal of the difference between the first sum and the second sum is calculated as the matching degree between each edge line in the bolt grayscale image and each edge line in the reference grayscale image. The edge line with the highest matching degree in the reference grayscale image is selected as the matching edge line for each edge line in the bolt grayscale image.

[0013] Preferably, the process of obtaining the loosening coefficient from the bolt grayscale image to monitor and issue early warnings for the loosening state of high-strength bolts includes: Image segmentation is performed on the bolt grayscale image and the reference grayscale image respectively, and the bolt shadow region in the bolt grayscale image and the reference grayscale image is extracted respectively; the relative change rate of the bolt shadow region area between the bolt grayscale image and the reference grayscale image is calculated as the loosening factor of the bolt grayscale image; Calculate the mean of the angles between all edge lines and their matching edge lines in the bolt grayscale image, and use this as the loosening angle of the bolt grayscale image; The sum of the sine of the loosening angle and the loosening factor is used as the loosening coefficient of the bolt grayscale image; If the loosening coefficient is greater than the preset threshold, the high-strength bolt will become loose; otherwise, the high-strength bolt will not become loose.

[0014] Secondly, this application also provides a visual monitoring and early warning system for loose high-strength bolts in steel truss bridges, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any of the above-described visual monitoring and early warning methods for loose high-strength bolts in steel truss bridges.

[0015] This application has at least the following beneficial effects: This application classifies pixels in a bolt grayscale image by gradient intensity, identifying strong and weak edge points. Its advantage lies in utilizing the dual-threshold mechanism of the Canny operator for initial edge point screening. Strong edge points are clustered, leveraging prior knowledge of bolt shape and classifying them based on gradient direction and spatial distance. The linearity of each strong edge point is calculated, and strong edge points within clusters are filtered and removed. This approach considers the linear distribution of each strong edge point with other strong edge points in the same cluster, assessing the significance of linear distribution characteristics and thus identifying and removing strong edge points caused by surface defects on the bolt. Interference points such as rust and coating deterioration are identified. All feature points symmetrical to each weak edge point are selected, and the first evaluation value of each weak edge point is determined. This utilizes the central and axial symmetry characteristics of the bolt to find symmetrical feature points for each weak edge point. By determining whether a feature point belongs to a strong edge point and the similarity of the gradient direction grayscale between the feature point and the weak edge point, a preliminary assessment of whether the weak edge point is located on the true bolt edge line is made. The second evaluation value of each weak edge point is calculated. This further determines the probability that the weak edge point is a point on the true bolt edge line by analyzing the distance distribution between each weak edge point and the strong edge points in the neighboring clusters. The high threshold in the dual-threshold edge detection algorithm is further refined. The algorithm re-classifies weak edge points and strong edge points within clusters. Based on the newly defined strong edge points, edge lines in the bolt grayscale image are extracted. This approach dynamically adjusts the high threshold in the dual-threshold edge detection algorithm by considering the possibility that weak edge points lie on the actual bolt edge lines. This allows weak edge points with insufficient gradient strength but geometric and correlation features resembling bolt edges to be classified as new strong edge points, overcoming edge breakage or noise interference problems caused by traditional methods. This significantly improves the detection accuracy and continuity of bolt edge pixels. Furthermore, matching edge lines are obtained for each edge line in the bolt grayscale image. This approach considers the texture from the edge lines to the center of the bolt region between the bolt grayscale image and the baseline grayscale image. To assess the differences in features, the edge lines in the bolt grayscale image are matched with those in the reference grayscale image to evaluate the rotation of the edge lines in the bolt grayscale image. This yields the loosening coefficient of the bolt grayscale image, enabling monitoring and early warning of the loosening status of high-strength bolts. The beneficial effect lies in assessing the bolt's loosening status by examining the angle between the matched edge lines in the bolt grayscale image and the reference grayscale image. Furthermore, the difference in the area of ​​the bolt's shadow region between the bolt grayscale image and the reference grayscale image further evaluates the loosening status, preventing situations where the bolt rotates a full circle without changing the angle. This multi-dimensional monitoring of bolt loosening status improves the accuracy of monitoring the loosening status of high-strength bolts. Attached Figure Description

[0016] The following is a detailed description of a visual monitoring and early warning method for loosening high-strength bolts in steel truss bridges, in conjunction with the accompanying drawings.

[0017] Figure 1 A flowchart illustrating the steps of a visual monitoring and early warning method for loosening high-strength bolts in a steel truss bridge, as provided in this application embodiment; Figure 2 A flowchart illustrating the steps of the method for obtaining the loosening coefficient provided in this application embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description, in conjunction with the accompanying drawings and implementation examples, provides a visual monitoring and early warning method and system for loosening high-strength bolts in steel truss bridges. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0020] Please see Figure 1 The diagram illustrates a flowchart of a visual monitoring and early warning method for loosening high-strength bolts in a steel truss bridge, according to an embodiment of this application. The method includes the following steps: Step 1: Obtain the grayscale images of the high-strength bolts on the steel truss bridge and the reference grayscale image.

[0021] Steel truss bridges are crucial engineering structures in transportation. Bolts are one of the essential components connecting steel structural members. In most steel truss bridges, components are interconnected through bolted joints. However, due to the prolonged exposure of bolts to the natural environment, they are frequently subjected to impact loads and repeated loads from vehicles and wind, as well as numerous continuous vibrations. This can lead to a decrease in the working stability of high-strength bolt connections, making them prone to loosening and falling off to varying degrees. Loosening of high-strength bolts results in insufficient preload, causing bolted connections to fail and reducing the bridge's load-bearing capacity. Therefore, to ensure the structural performance of steel truss bridges, it is necessary to monitor the loosening of high-strength bolts.

[0022] Based on the above analysis, a high-resolution, protected industrial camera is deployed on the steel truss bridge, along with a supplementary light. The supplementary light consists of a 30W infrared LED array and a polarizing filter kit. The supplementary light and the industrial camera are fixedly mounted on the truss structure at a distance of 300-500mm from the bolt group. The supplementary light and the industrial camera maintain a 30° lateral angle to ensure optimal lighting effect. The height and angle of the industrial camera are adjusted using a universal joint so that the camera is perpendicularly aligned with the flange face of the bolt. In this embodiment, the high-strength bolts are common hexagonal shapes. Therefore, bolt images are acquired every night, and the bolt images are processed into grayscale images to obtain bolt grayscale images.

[0023] In this embodiment, a weighted average grayscale method is used to perform grayscale processing on the bolt image. The weighted average grayscale method is a well-known technique and will not be described in detail here.

[0024] It should be noted that, since the traffic flow on the steel truss bridge is relatively low at night, bolt images were collected at 3 a.m. to reduce the impact of vehicle traffic on the bridge.

[0025] Secondly, a bolt image with the bolt not loosened is manually selected as the reference image, and the reference image is converted to grayscale to obtain the reference grayscale image. The reference grayscale image is then manually annotated to obtain the bolt area in the reference grayscale image.

[0026] In this embodiment, the Labelme annotation tool is used for annotation. The use of the Labelme annotation tool is a well-known technology and will not be described in detail here.

[0027] At this point, we have obtained the bolt grayscale image and the bolt region in the reference grayscale image.

[0028] Step 2: Based on the gradient intensity of pixels in the bolt grayscale image, obtain strong edge points and weak edge points, and cluster the strong edge points; analyze the linear distribution of each strong edge point and other strong edge points within the cluster, calculate the linearity of each strong edge point, and filter and remove the strong edge points within the cluster.

[0029] When analyzing whether a bolt is loose in a grayscale image, the Canny edge detection algorithm is typically used to obtain edge pixels to extract the bolt's edge contour. Changes in this contour are then analyzed to detect the bolt's looseness. Traditional Canny edge detection algorithms primarily calculate the gradient of pixels in the image and use a dual-threshold method to detect edges. The dual thresholds are a high threshold and a low threshold. The high threshold detects strong edges, while the low threshold detects weak edges. If the high threshold is set too high, the detected edges will be very sparse and discontinuous, potentially ignoring many true edge points and resulting in incomplete edge lines that fail to accurately reflect the bolt's actual shape. If the low threshold is set too low, the Canny edge detection algorithm may misdetect noise points as edges, generating a large number of false edge points, thus reducing detection accuracy and efficiency.

[0030] Calculate the gradient intensity of each pixel in the grayscale image of the bolt; In this embodiment, the Canny edge detection algorithm is used to calculate the gradient intensity of each pixel in the grayscale image of the bolt. The Canny edge detection algorithm is a well-known technology and will not be described in detail here.

[0031] Set a preset initial high threshold and a preset initial low threshold, wherein the initial high threshold is a preset multiple of the initial low threshold; It should be noted that the high threshold is generally set to the 90th or 95th percentile of the gradient intensity in the image, and the low threshold is generally set to a certain proportion of the high threshold, usually between 1:2 and 1:3. In this embodiment, the preset initial high threshold is denoted as […]. The preset initial low threshold is That is, the initial high threshold is three times the initial low threshold, where the initial high threshold is... The value is 180. As for other implementation methods, the implementer can set it according to the actual situation.

[0032] In the grayscale image of a bolt, pixels with gradient strength greater than or equal to the initial high threshold are denoted as strong edge points, and pixels with gradient strength greater than or equal to the initial low threshold and less than the initial high threshold are denoted as weak edge points. It should be noted that pixels with gradient strength less than the initial low threshold are information points and do not belong to pixels on the edge.

[0033] Because the high-strength bolts on the steel truss bridge are hexagonal, corresponding to six edge lines of equal length, and each edge line is adjacent to two edge lines and parallel to one edge line; secondly, the gradient directions of pixels on the same edge line should be the same or opposite, and the spatial distance should be relatively small. Therefore, all strong edge points are classified as follows: Calculate the number of pixels contained between any two strong edge points; Calculate the cosine of the angle between the gradient directions of any two strong edge points; It should be noted that the calculation of gradient direction is a well-known technique and will not be elaborated here.

[0034] The ratio between the quantity and the absolute value of the cosine value is used as the metric distance between any two strong edge points; It should be noted that, in order to avoid the denominator being 0 when calculating the ratio, a preset value greater than 0 is added to the denominator. In this embodiment, the preset value greater than 0 is 1. In other implementation methods, the implementer can set it according to the actual situation.

[0035] It should be noted that the smaller the number, the closer the two strong edge points are; the larger the absolute value of the cosine value, the more parallel the gradient directions of the two strong edge points are; and the smaller the metric distance, the greater the possibility that the two strong edge points are on the same edge line.

[0036] Based on the metric distance, all strong edge points are clustered to obtain multiple clusters; In this embodiment, the k-means clustering algorithm is used to cluster all strong edge points. Since the high-strength bolt is a regular hexagon, the number of clusters is 6. Therefore, each cluster corresponds to an edge line. The k-means clustering algorithm is a well-known technology and will not be described in detail here. As other implementation methods, implementers can use other methods of existing technology, such as the DBSCAN clustering algorithm. This embodiment does not impose any special restrictions on this.

[0037] Secondly, due to the long-term use of high-strength bolts on steel truss bridges, their surfaces are prone to defects such as corrosion, rust, coating deterioration, and dirt buildup. These defects can also cause strong color changes, which can be misdetected as strong edge points by the Canny operator. In addition, the surface of high-strength bolts has certain texture information, which can lead to some interference points being mixed in with the detected strong edge points, making it impossible to accurately reflect the edge line of the bolt. Therefore, it is necessary to distinguish strong edge points within clusters.

[0038] Based on the above analysis, the distribution of strong edge points along the edge lines of a bolt is usually linear, while the interference points caused by defects on the bolt surface are usually clustered. Therefore, we need to analyze the linear distribution of different edge points within the local area of ​​each strong edge point and calculate the linearity, specifically: Select the strong edge point within each cluster that has the smallest distance to any strong edge point, and denote it as the neighboring point; The straight line connecting any strong edge point to its neighboring points is denoted as the characteristic line. Calculate the sum of the distances from all strong edge points within each cluster to the feature line, and denote it as the distribution distance; It should be noted that the distance from a strong edge point to a feature line is calculated using the distance from a point to a line. The calculation of the distance from a strong edge point to a feature line is a well-known technique and will not be elaborated here.

[0039] The ratio of the gradient intensity of any strong edge point to the distribution distance is taken as the linearity of any strong edge point. It should be noted that the greater the gradient intensity, the more drastic the color change of the strong edge point, and the more significant its edge characteristics. The greater the distribution distance, the farther the strong edge points in the cluster deviate from this feature line, the more scattered their distribution, and the less linear the distribution characteristics of any strong edge point. The smaller the distribution distance, the more closely the strong edge points in the cluster are distributed near the feature line. The greater the linearity, the more linear the distribution characteristics of any strong edge point, and the more likely it is to be a true edge point in the bolt.

[0040] Furthermore, based on linearity, strong edge points within the clusters are filtered out to remove interference points on the bolt surface caused by defects. Specifically: Calculate the average linearity of all strong edge points within each cluster, and remove strong edge points within the cluster whose linearity is less than the average. It should be noted that in subsequent calculations, strong edge points within a cluster do not include interference points represented by strong edge points with linearity less than the average value.

[0041] At this point, the filtered strong edge points are obtained.

[0042] Step 3: Based on the symmetry of the bolt in the bolt grayscale image, select all feature points symmetrical to each weak edge point, analyze the consistency of the gradient direction and the difference in grayscale value between each weak edge point and its corresponding feature point, and combine the linearity when the feature point is a strong edge point to determine the first evaluation value of each weak edge point.

[0043] Secondly, due to improper setting of the high threshold parameter in the Canny detection algorithm, strong edge points often cannot completely represent edge line information, resulting in a small number of detected strong edge points and missing many real edge points. In other words, the detected edge pixels are discontinuous and have a break phenomenon. When using strong edge points directly for Hough line detection, it may not be possible to detect straight lines or the detected straight lines may deviate from the actual edge line of the bolt, thus affecting the accuracy and efficiency of subsequent bolt loosening detection.

[0044] Therefore, by analyzing whether weak edge points in the bolt grayscale image conform to the linear distribution of strong edge points, we can determine whether weak edge points are true edge points. Specifically: Calculate the spatial distance between the cluster center of any cluster and the cluster centers of all other clusters, and denote any cluster and the cluster with the largest spatial distance to it as a pair of parallel clusters; It should be noted that spatial distance is measured by counting the number of pixels contained between two cluster centers; secondly, the largest spatial distance between the cluster centers of two clusters indicates that the two corresponding edge lines of the two clusters are farthest apart. Therefore, the distance between two relatively parallel sides in a regular hexagon is the largest. Thus, the two clusters are a pair of parallel clusters because the spatial distance between them is the largest. Since the high-strength bolt is a regular hexagon, there are three pairs of parallel clusters, which means there are three pairs of parallel edge lines.

[0045] Calculate the distance between any two strong edge points within each pair of parallel clusters, denoted as the relative distance, and obtain the coordinates of the midpoint between the two strong edge points corresponding to the maximum relative distance. The pixel point corresponding to the mean of the midpoint coordinates of all pairs of parallel clusters is taken as the bolt center point; It should be noted that in the regular hexagon represented by the high-strength bolt, the distance between the two points on the diagonal is the largest. Therefore, in a pair of parallel clusters, the two strong edge points corresponding to the largest relative distance represent the two points on the diagonal. Ideally, the bolt center is the intersection of the three diagonals, that is, the midpoints of the three pairs of parallel clusters are at the same position. However, due to the interference of defects, the midpoints of the three pairs of parallel clusters are inconsistent. Therefore, the average coordinates of the midpoints of the three pairs of parallel clusters are taken, and the corresponding pixel is taken as the bolt center point.

[0046] Construct a rectangle with any weak edge point as the vertex and the center point of the bolt as the center point of the rectangle; denote the remaining vertices within the rectangle, excluding the vertex corresponding to any weak edge point, as feature points. If each feature point is a strong edge point, the discriminant coefficient of each feature point is the linearity of the corresponding strong edge point. If each feature point is not a strong edge point, the calculation process of the discriminant coefficient of each feature point is as follows: Calculate the cosine of the angle between the gradient direction of each feature point and the gradient direction of any weak edge point, and denote it as the direction consistency. The difference in grayscale value between each feature point and any weak edge point is denoted as the relative difference; In this embodiment, the absolute value of the difference between the gray values ​​of each feature point and any weak edge point is denoted as the relative difference.

[0047] The ratio of the directional consistency to the relative difference is used as the discrimination coefficient for each feature point; It should be noted that, in order to avoid the denominator being 0 when calculating the ratio, a preset value greater than 0 is added to the denominator. In this embodiment, the preset value greater than 0 is 1. In other implementation methods, the implementer can set it according to the actual situation.

[0048] The sum of the discrimination coefficients of all feature points corresponding to any weak edge point is used as the first evaluation value of any weak edge point. It should be noted that if each feature point belongs to a strong edge point, the greater its linearity and the larger its discriminant coefficient, the greater the likelihood that the feature point is a real edge point. Since bolts have symmetry, the weak edge point is more likely to be an edge point on the actual edge line. If each feature point does not belong to a strong edge point, the more consistent the gradient direction of the weak edge point and the feature point are, and the smaller the gray level difference, i.e., the larger the discriminant coefficient, the greater the likelihood that the weak edge point and the feature point belong to the same edge line. Therefore, the weak edge point may be part of the actual edge line. The larger the obtained first evaluation value, the more likely the weak edge point is located on the actual edge line.

[0049] Thus, the first evaluation value of each weak edge point is obtained.

[0050] Step 4: Calculate the second evaluation value of each weak edge point by using the distance and linearity between each weak edge point and the strong edge points in the neighboring clusters. Combine the gradient intensity of the weak edge points and the first evaluation value to adjust the high threshold in the dual-threshold edge detection algorithm, and re-divide the weak edge points and the strong edge points in the clusters. Based on the newly divided strong edge points, extract the edge lines in the bolt grayscale image.

[0051] Furthermore, since the bolt surface may contain certain texture information, weak edge points corresponding to the bolt surface texture may be misclassified as strong edge points. Therefore, the correlation between weak edge points and different strong edge points within the clusters to which neighboring strong edge points belong is analyzed, and a second evaluation value is calculated, specifically: The cluster to which the strongest edge point closest to any of the weak edge points belongs is denoted as the neighboring cluster; Calculate the distance between each strong edge point in the neighboring cluster and any weak edge point; It should be noted that the interval distance is measured by the number of pixels contained between each strong edge point and any weak edge point.

[0052] Calculate the sum of the ratios of the linearity of all strong edge points within the neighboring cluster to the interval distance; use the ratio of the sum to the minimum interval distance as the second evaluation value of any weak edge point; It should be noted that the greater the linearity, the more the strong edge point conforms to the edge characteristics of the bolt; the smaller the interval distance, the closer the weak edge point is to the strong edge point; the larger the sum, the closer the distribution of the weak edge point is to the strong edge point in the neighboring cluster, reflecting the greater the possibility that the weak edge point is an edge point on the actual edge line; and the larger the second evaluation value, the more likely the weak edge point is to be part of the actual edge line.

[0053] Secondly, based on the first and second evaluation values, the high threshold of the Canny edge detection algorithm is adjusted, specifically as follows: The product of the first evaluation value and the second evaluation value is used as the evaluation coefficient for each weak edge point; in, The adjusted high threshold, The preset initial high threshold, For the first Gradient strength at weak edge points For the first The product of the first and second evaluation values ​​of each weak edge point is the evaluation coefficient. This represents the number of all weak edge points in the bolt grayscale image. For the normalization function, in this embodiment, the maximum and minimum value normalization method is used for normalization processing. The maximum and minimum value normalization method is a well-known technique and will not be described in detail here. As other implementation methods, implementers can use other methods of the prior art, such as the sigmoid function, etc. This embodiment does not impose any special restrictions on this.

[0054] It should be noted that, The smaller the value, the closer the gradient strength of the weak edge point is to H, indicating a more significant color change at the weak edge point and a closer similarity to the gradient strength of a strong edge point. Conversely, the larger the evaluation coefficient, the more likely the weak edge point is to be located on the actual edge line, and therefore, the more likely it should be classified as a strong edge point. The larger the threshold, the lower the initial high threshold should be, thereby classifying weak edge points into strong edge points.

[0055] Based on the adjusted high threshold, all weak edge points and all strong edge points are re-divided to obtain newly divided strong edge points. Hough line detection is performed on the newly divided strong edge points to extract edge lines. The region formed by all edge lines extracted from the bolt grayscale image is taken as the bolt region. In this embodiment, the Hough line detection algorithm is used to detect all new strong edge points, resulting in six corresponding edge lines. The area formed by the six edge lines is the bolt area. The Hough line detection algorithm is a well-known technology and will not be described in detail here.

[0056] It should be noted that when reclassifying weak and strong edge points, strong edge points with linearity less than the average value that were previously removed are not included.

[0057] This completes the acquisition of the edge lines of the bolt region in the bolt grayscale image.

[0058] Step 5: Match each edge line in the bolt grayscale image with the reference grayscale image to obtain the matching edge lines of each edge line in the bolt grayscale image; extract the bolt shadow area in the bolt grayscale image and the reference grayscale image respectively; obtain the loosening coefficient of the bolt grayscale image by the difference in the area of ​​the bolt shadow area between the bolt grayscale image and the reference grayscale image, and the angle between the edge line in the bolt grayscale image and its matching edge line; monitor and warn of the loosening status of high-strength bolts.

[0059] Furthermore, by analyzing the matching between each edge line in the bolt grayscale image and the edge lines within the bolt region in the reference grayscale image, the matching degree is calculated, specifically as follows: Hough line detection is performed on the bolt region in the baseline grayscale image to extract the edge lines; The sum of the LBP values ​​of all pixels between the midpoint of each edge line and the center point of the bolt in the grayscale image of the bolt is recorded as the first sum value. The sum of the LBP values ​​of all pixels between the midpoint of each edge line in the reference grayscale image and the center point of the bolt is recorded as the second sum value. In this embodiment, the calculation of LBP values ​​(Local binary patterns) is a well-known technique and will not be described in detail here.

[0060] Calculate the reciprocal of the difference between the first sum and the second sum, and use it as the matching degree between each edge line in the bolt grayscale image and each edge line in the reference grayscale image; In this embodiment, the reciprocal of the absolute value of the difference between the first sum and the second sum is calculated as the matching degree between each edge line in the bolt grayscale image and each edge line in the reference grayscale image; It should be noted that the LBP value reflects the local texture information of the pixel. The smaller the difference between the first sum and the second sum, that is, the greater the matching degree, the more similar the texture information between the edge line and the center point of the bolt between the bolt grayscale image and the reference grayscale image, and the more likely the two edge lines are the same.

[0061] The edge line with the highest matching degree in the reference grayscale image is selected as the matching edge line for each edge line in the bolt grayscale image; Calculate the mean of the angles between all edge lines and their matching edge lines in the bolt grayscale image, and use this as the loosening angle of the bolt grayscale image; It should be noted that if each edge line is parallel to its matching edge line, i.e., the loosening angle is 0, the bolt may not be loose in the bolt grayscale image. However, if each edge line has an angle with its matching edge line, i.e., the loosening angle is greater than 0, the bolt appears loose in the bolt grayscale image.

[0062] Secondly, if the bolt loosens, and the loosening angle is one or more rotations (i.e., the bolt rotates one or more times when it loosens), the calculated loosening angle is 0. However, the bolt is actually loose, so the loosening angle cannot accurately detect the bolt loosening. Because there is a 30° lateral angle between the industrial camera and the supplementary lighting during image acquisition, the image will contain a shadow area of ​​the bolt due to the lighting effect. This shadow area is adjacent to the bolt area. If the bolt loosens, the area of ​​the shadow area will change. Therefore, we analyze the difference between the area of ​​the shadow area in the bolt grayscale image and the area of ​​the shadow area in the reference grayscale image to calculate the loosening factor, specifically: The bolt grayscale image and the reference grayscale image are segmented separately, and the bolt shadow region in the bolt grayscale image and the bolt shadow region in the reference grayscale image are extracted respectively. In this embodiment, the Otsu threshold segmentation algorithm is used to segment the image, dividing the bolt grayscale image into two regions. The smaller region is designated as the bolt shadow region. The Otsu threshold segmentation algorithm is a well-known technique and will not be described in detail here.

[0063] The relative rate of change between the area of ​​the bolt shadow region in the bolt grayscale image and the area of ​​the bolt shadow region in the reference grayscale image is calculated and used as the loosening factor of the bolt grayscale image. It should be noted that the area is measured by counting the number of pixels within the bolt's shadow region. Secondly, the calculation of the relative rate of change is a well-known technique. The calculation process for the relative rate of change is as follows: ,in, This represents the area of ​​the bolt's shadow region in the bolt's grayscale image. This represents the area of ​​the bolt shadow region in the baseline grayscale image.

[0064] It should be noted that the larger the loosening factor, the greater the difference between the area of ​​the bolt shadow region in the bolt grayscale image and the area of ​​the bolt shadow region in the reference grayscale image, reflecting the greater possibility that the bolt is loose in the bolt grayscale image.

[0065] Furthermore, based on the loosening angle and loosening factor, a loosening coefficient is calculated to assess whether the bolt has become loose, specifically: The sum of the sine of the loosening angle and the loosening factor is used as the loosening coefficient of the bolt grayscale image; It should be noted that a larger loosening coefficient indicates a greater likelihood of bolt loosening. The flowchart illustrating the steps of the method for obtaining the loosening coefficient provided in this application embodiment is as follows: Figure 2 As shown.

[0066] If the loosening coefficient is greater than the preset threshold, the high-strength bolt will become loose and an early warning will be issued; otherwise, the high-strength bolt will not become loose.

[0067] In this embodiment, the process of obtaining the preset threshold is as follows: the average value of the loosening coefficient of all bolt grayscale images collected in the historical period is used as the preset threshold.

[0068] Based on the same inventive concept as the above method, this application embodiment also provides a visual monitoring and early warning system for loose high-strength bolts of steel truss bridges, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described methods for visual monitoring and early warning of loose high-strength bolts of steel truss bridges.

[0069] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0070] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this application, without departing from the content of the technical solution of this application, shall fall within the protection scope of the technical solution of this application.

Claims

1. A high-strength bolt loosening visual monitoring and early warning method for a steel truss bridge, characterized in that, The method comprises the following steps: Obtaining a bolt gray image and a reference gray image of a high-strength bolt on a steel truss bridge; Based on the gradient intensity of the pixel points in the bolt gray image, strong edge points and weak edge points are obtained, and the strong edge points are clustered; The linear distribution of each strong edge point in the cluster and the remaining strong edge points is analyzed, the linearity of each strong edge point is calculated, the strong edge points in the cluster are screened and removed; Based on the symmetry of the bolt in the bolt gray image, all feature points symmetric to each weak edge point are selected, the consistency of the gradient direction between each weak edge point and its corresponding feature point and the difference of the gray values are analyzed, and the first evaluation value of each weak edge point is determined by combining the linearity of the feature point being a strong edge point; The second evaluation value of each weak edge point is calculated by the distance between each weak edge point and the strong edge points in the adjacent cluster and the linearity, the high threshold in the double-threshold edge detection algorithm is adjusted by combining the gradient intensity of the weak edge point and the first evaluation value, the weak edge points and the strong edge points in the cluster are re-divided, and the edge lines in the bolt gray image are extracted based on the newly divided strong edge points; the matching edge lines of each edge line in the bolt gray image are obtained by matching each edge line in the bolt gray image with the reference gray image; The bolt shadow areas in the bolt gray image and the reference gray image are extracted respectively, the looseness coefficient of the bolt gray image is obtained by the difference of the areas of the bolt shadow areas between the bolt gray image and the reference gray image and the angle between the edge lines in the bolt gray image and the matching edge lines, and the looseness state of the high-strength bolt is monitored and warned.

2. The visual monitoring and early warning method for loosening high-strength bolts in steel truss bridges as described in claim 1, characterized in that, The strong edge points and the weak edge points are obtained by setting a preset initial high threshold and a preset initial low threshold, wherein the initial high threshold is a preset multiple of the initial low threshold; the pixel points in the bolt gray image with a gradient intensity greater than or equal to the initial high threshold are recorded as strong edge points, and the pixel points in the bolt gray image with a gradient intensity greater than or equal to the initial low threshold and less than the initial high threshold are recorded as weak edge points.

3. The visual monitoring and early warning method for looseness of high-strength bolts of a steel truss bridge according to claim 1, characterized in that, The linearity of each strong edge point is calculated, and the strong edge points in the cluster are screened and removed, which comprises: Selecting the strong edge point with the smallest distance to each strong edge point in each cluster as a neighboring point; connecting a straight line between each strong edge point and the neighboring point as a feature straight line; calculating the sum of the distances of all strong edge points in each cluster to the feature straight line as a distribution distance; The linearity is the ratio of the gradient intensity of each strong edge point to the distribution distance; The average value of the linearity of all strong edge points in each cluster is calculated, and the strong edge points with a linearity less than the average value in the cluster are removed.

4. The visual monitoring and early warning method for looseness of high-strength bolts of a steel truss bridge according to claim 1, characterized in that, The selection of all feature points symmetric to each weak edge point comprises: Calculating the spatial distance between the cluster center of any cluster and the cluster center of the remaining clusters, and recording the any cluster and the cluster with the largest spatial distance as a pair of parallel clusters; Calculate the distance between any two strong edge points in each pair of parallel clusters, denoted as relative distance, and obtain the midpoint coordinates between the two strong edge points corresponding to the maximum relative distance; take the pixel point corresponding to the mean of the midpoint coordinates of all pairs of parallel clusters as the bolt center point; Take each weak edge point as a vertex and the bolt center point as the center point of the rectangle, and construct a rectangular frame; take the remaining vertices in the rectangular frame except the vertices corresponding to each weak edge point as all feature points corresponding to each weak edge point.

5. The visual monitoring and early warning method for looseness of high-strength bolts of a steel truss bridge according to claim 1, characterized in that, The determination of the first evaluation value of each weak edge point comprises: If each feature point belongs to a strong edge point, the discriminant coefficient of each feature point is the linearity of the strong edge point to which the feature point belongs; If each feature point is not a strong edge point, the calculation process of the discriminant coefficient of each feature point is as follows: Calculate the cosine value of the included angle between the gradient direction of each weak edge point and the gradient direction of each feature point thereof, denoted as direction consistency; take the difference between the gray values between each weak edge point and each feature point thereof as relative difference; take the ratio of the direction consistency and the relative difference as the discriminant coefficient of each feature point; The first evaluation value is the sum of the discriminant coefficients of all feature points corresponding to each weak edge point.

6. The visual monitoring and early warning method for looseness of high-strength bolts of a steel truss bridge according to claim 1, characterized in that, The calculation of the second evaluation value of each weak edge point comprises: Take the cluster to which the strong edge point closest to each weak edge point belongs as the adjacent cluster; Calculate the interval distance between each weak edge point and each strong edge point in the adjacent cluster thereof; calculate the cumulative sum of the ratio of the linearity of all strong edge points in the adjacent cluster corresponding to each weak edge point to the interval distance; The second evaluation value is the ratio of the cumulative sum to the minimum interval distance.

7. The visual monitoring and early warning method for loosening high-strength bolts in steel truss bridges as described in claim 2, characterized in that, Adjusted high threshold The calculation process is as follows: ,in, The preset initial high threshold, For the first Gradient strength at weak edge points For the first The product of the first and second evaluation values ​​of each weak edge point This represents the number of all weak edge points in the bolt grayscale image. This is the normalization function.

8. The visual monitoring and early warning method for looseness of high-strength bolts of a steel truss bridge according to claim 4, characterized in that, The acquisition of the matching edge line of each edge line in the bolt gray image comprises: Perform straight line detection on all newly divided strong edge points in the bolt gray image and the bolt region in the reference gray image respectively to extract edge lines; Take the sum of the LBP values of all pixel points between the midpoint of each edge line in the bolt gray image and the bolt center point as the first sum value; take the sum of the LBP values of all pixel points between the midpoint of each edge line in the reference gray image and the center point of the bolt region in the reference gray image as the second sum value; calculate the inverse of the difference between the first sum value and the second sum value as the matching degree between each edge line in the bolt gray image and each edge line in the reference gray image; Select the edge line with the maximum matching degree in the reference gray image as the matching edge line of each edge line in the bolt gray image.

9. The visual monitoring and early warning method for looseness of high-strength bolts of a steel truss bridge according to claim 1, characterized in that, The obtaining of the loosening coefficient of the bolt gray image, the monitoring and early warning of the loosening state of the high-strength bolt, comprises: Perform image segmentation on the bolt gray image and the reference gray image respectively to extract the bolt shadow regions in the bolt gray image and the reference gray image respectively; calculate the relative change rate of the area of the bolt shadow region between the bolt gray image and the reference gray image as the loosening factor of the bolt gray image; Calculate the mean of the included angles between all edge lines in the bolt gray image and their matching edge lines as the loosening angle of the bolt gray image; Take the sum of the sine value of the loosening angle and the loosening factor as the loosening coefficient of the bolt gray image; If the looseness coefficient is greater than a preset threshold, the high-strength bolt is loose, otherwise, the high-strength bolt is not loose.

10. A steel truss bridge high-strength bolt loosening visual monitoring and early warning system, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that, The processor implements the steps of the steel truss bridge high-strength bolt looseness visual monitoring and early warning method according to any one of claims 1-9 when executing the computer program.

Images