A communication tower bolt loosening detection method and system

Abstract

The application discloses a communication tower bolt loosening detection method and system, and belongs to the technical field of communication tower bolt loosening detection.The application solves the problem of low detection precision of the existing bolt loosening detection method.The application firstly positions the bolt position in a to-be-detected image, then cuts out a bolt sub-image from the to-be-detected image according to the positioning result, then detects the shooting angle of each bolt sub-image, and designs a specific loosening detection method for bolts with different shooting angles, so that the detection deviation caused by the same detection method for bolt images with different shooting angles in the same image can be avoided, and the bolt loosening detection precision is improved.The application can be applied to the communication tower bolt loosening detection.

Description

Technical Field

[0001] This invention belongs to the field of communication tower bolt loosening detection technology, specifically relating to a method and system for detecting communication tower bolt loosening. Background Technology

[0002] In recent years, the safety monitoring of iron towers has received increasing attention. The good operating status of iron towers is directly related to the safety of people or things around the tower, the maintenance costs of the tower builders or users, the information security of the area covered by the tower, and even the safety of economic and social development.

[0003] There are many types of iron towers currently under construction or in use in my country. Communication towers are an important type, specifically including wireless communication towers, angle steel communication towers, and single-tube mobile towers. As crucial infrastructure supporting communication equipment, communication towers must withstand their own weight, the weight of the equipment, and loads from natural environments such as wind, rain, and snow. Bolts are key components connecting the various parts of the tower. If bolts loosen, the installation position of the communication equipment may shift or sway, affecting the pointing accuracy of the antenna and the normal operation of the equipment. This can lead to weakened, interrupted, or degraded communication signals, impacting the stability and reliability of communication services. For mobile communication networks, unstable signals can cause poor call quality and slow data transmission rates, reducing the user experience. More importantly, loose bolts can also compromise the integrity and stability of the tower structure, reducing its load-bearing capacity and potentially causing serious safety accidents such as tower tilting or collapse, posing a significant threat to the safety of surrounding people and property. For example, in strong winds, loose bolts cannot effectively secure tower components, potentially leading to structural imbalance and ultimately tower collapse.

[0004] Therefore, regular bolt loosening inspections of communication towers are necessary. These inspections allow for the timely detection and resolution of loose bolts, preventing more serious structural damage and equipment malfunctions. If loose bolts are not detected and repaired promptly, they may accelerate wear on adjacent components, further damaging the tower structure and communication equipment, increasing future maintenance and replacement costs. Early detection and tightening of loose bolts can effectively extend the service life of communication towers and equipment, reducing overall maintenance costs.

[0005] Traditional bolt loosening detection methods mainly rely on manual inspection, which is wasteful of manpower and resources and has low efficiency. With the development of deep learning technology, researchers have begun to apply it to the automated detection of bolt loosening. However, existing bolt loosening detection methods do not consider the impact of image shooting angle on detection accuracy. Therefore, the detection accuracy of existing bolt loosening detection methods is still low. Proposing a new bolt loosening detection method is an urgent problem to be solved. Summary of the Invention

[0006] The purpose of this invention is to solve the problem of low detection accuracy of existing bolt loosening detection methods, and to propose a method and system for detecting bolt loosening in communication towers.

[0007] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0008] According to one aspect of the present invention, a method for detecting loose bolts on a communication tower is provided, the method specifically comprising the following steps:

[0009] Step 1: Input the image to be detected into the bolt target detection network, use the bolt target detection network to output the bolt position in the image to be detected, and extract the image of the bolt position from the image to be detected;

[0010] Step 2: Use the images extracted in Step 1 as input to the bolt angle recognition model. The output of the bolt angle recognition model is either a frontal view or a side view of the bolt.

[0011] Step 3: Perform bolt loosening detection on bolt images taken from a frontal view and bolt images taken from a side view.

[0012] According to another aspect of the present invention, a communication tower bolt loosening detection system is provided, the system comprising a target image acquisition module, a bolt target detection module, a bolt angle recognition module, and a bolt loosening detection module, wherein:

[0013] The image acquisition module is used to acquire the image to be detected;

[0014] The bolt target detection module is used to detect the position of bolts in the image to be detected and to extract the image of the bolt position from the image to be detected.

[0015] The bolt target detection module is implemented based on a bolt target detection network, and the working process of the bolt target detection network is as follows:

[0016] In the bolt target detection network, the image to be detected first passes through the first convolutional layer, and then the output of the first convolutional layer is used as the input of the first ReLU activation function layer;

[0017] The output of the first ReLU activation function layer is used as the input of the second convolutional layer, and then the output of the second convolutional layer is used as the input of the second ReLU activation function layer.

[0018] The output of the second ReLU activation function layer is used as the input of the first max pooling layer, the output of the first max pooling layer is used as the input of the third convolutional layer, and the output of the third convolutional layer is used as the input of the third ReLU activation function layer.

[0019] The output of the third ReLU activation function layer is used as the input of the fourth convolutional layer, and then the output of the fourth convolutional layer is used as the input of the fourth ReLU activation function layer.

[0020] The output of the fourth ReLU activation function layer is used as the input of the second max pooling layer, and the output of the second max pooling layer is used as the input of the fifth convolutional layer.

[0021] The output of the fifth convolutional layer is used as the input of the fifth ReLU activation function layer, and then the output of the fifth ReLU activation function layer is used as the input of the sixth convolutional layer.

[0022] The output of the sixth convolutional layer is used as the input of the sixth ReLU activation function layer, and then the output of the sixth ReLU activation function layer is used as the input of the seventh convolutional layer.

[0023] The output of the seventh convolutional layer is used as the input of the seventh ReLU activation function layer, and then the output of the seventh ReLU activation function layer is used as the input of the eighth convolutional layer.

[0024] The output of the eighth convolutional layer is used as the input of the eighth ReLU activation function layer, and then the output of the eighth ReLU activation function layer is used as the input of the ninth convolutional layer.

[0025] The output of the ninth convolutional layer is used as the input of the tenth convolutional layer, the output of the tenth convolutional layer is used as the input of the eleventh convolutional layer, and the output of the eleventh convolutional layer is used as the input of the third max pooling layer.

[0026] The output of the third max pooling layer is used as the input of the twelfth convolutional layer, and the output of the twelfth convolutional layer is used as the input of the thirteenth convolutional layer.

[0027] The output of the thirteenth convolutional layer is used as the input of the fourteenth convolutional layer, and then the output of the fourteenth convolutional layer is used as the input of the fourth max pooling layer.

[0028] The output of the fourth max pooling layer is used as the input of the fifteenth convolutional layer, and then the output of the fifteenth convolutional layer is used as the input of the sixteenth convolutional layer.

[0029] The output of the sixteenth convolutional layer is used as the input of the seventeenth convolutional layer, and then the output of the seventeenth convolutional layer is used as the input of the ninth ReLU activation function layer.

[0030] The output of the fourteenth convolutional layer is passed through the first downsampling unit, and then the output of the first downsampling unit is added to the output of the eleventh convolutional layer to obtain the sum a;

[0031] Add a to the output of the sixteenth convolutional layer to get the sum b, and then add b to the output of the ninth ReLU activation function layer to get the sum c.

[0032] The c is passed through the second downsampling unit, and then the output of the second downsampling unit is added to b to obtain the sum d;

[0033] d is passed through the third downsampling unit, and then the output of the third downsampling unit is added to a to obtain the sum e;

[0034] The output of the eleventh convolutional layer is passed through the fourth downsampling unit, and then the output of the fourth downsampling unit is added together to obtain the sum f.

[0035] The output of the eleventh convolutional layer is passed through the first deconvolutional layer, and then the output of the first deconvolutional layer is used as the input of the eighteenth convolutional layer.

[0036] The output of the 18th convolutional layer is added to the output of the 14th convolutional layer to obtain the sum g;

[0037] The g is passed through the second deconvolution layer, and the output of the second deconvolution layer is used as the input of the nineteenth convolution layer. The output of the nineteenth convolution layer is added to the output of the sixteenth convolution layer to obtain the sum h.

[0038] h is passed through the third deconvolution layer, and then the output of the third deconvolution layer is added to the output of the ninth ReLU activation function layer to obtain the sum i.

[0039] Pass i through the fifth downsampling unit, and then add the output of the fifth downsampling unit to h to obtain the sum j;

[0040] Pass j through the sixth downsampling unit, and then add the output of the sixth downsampling unit to g to obtain the sum k;

[0041] Pass k through the seventh downsampling unit, and then add the output of the eleventh convolutional layer to the output of the seventh downsampling unit to obtain the summation result m;

[0042] The bolt positions in the image to be detected are obtained based on f and m;

[0043] The bolt angle recognition module is used to detect the shooting angle of the image at the location of the bolt.

[0044] The bolt angle recognition module is implemented based on a bolt angle recognition model, and the working process of the bolt angle recognition model is as follows:

[0045] In the bolt angle recognition model, the input image first passes through the twentieth convolutional layer, and then the output of the twentieth convolutional layer is used as the input of the first convolutional block.

[0046] The output of the first convolutional block is used as the input of the tenth ReLU activation function layer, and then the output of the tenth ReLU activation function layer is used as the input of the second convolutional block.

[0047] The output of the second convolutional block is used as the input of the eleventh ReLU activation function layer, and then the output of the eleventh ReLU activation function layer is used as the input of the third convolutional block.

[0048] The output of the third convolutional block is used as the input of the twelfth ReLU activation function layer, and then the output of the twelfth ReLU activation function layer is used as the input of the fourth convolutional block;

[0049] The output of the fourth convolutional block is used as the input of the thirteenth ReLU activation function layer, and then the output of the thirteenth ReLU activation function layer is used as the input of the average pooling layer.

[0050] The output of the average pooling layer is used as the input of the fully connected layer, and the classification result is output through the fully connected layer.

[0051] The bolt loosening detection module is used to detect bolt loosening based on the detection results from the shooting angle;

[0052] (1) The specific process for detecting bolt loosening in bolt images taken from a frontal view is as follows:

[0053] Step 1: Perform Gaussian smoothing on the bolt image to obtain the Gaussian smoothed image;

[0054] Step 2: Calculate the gray-level gradient magnitude of each pixel in the image after Gaussian smoothing, and then perform non-maximum suppression on the gray-level gradient magnitude of the pixels to obtain the image after non-maximum suppression.

[0055] The method for calculating the gray-scale gradient magnitude is as follows:

[0056] M[i,j]=|P x [i,j]|+|P y [i,j]|

[0057] Where M[i,j] represents the grayscale gradient magnitude of pixel (i,j);

[0058] P x[i,j] represents the approximate value of the first-order partial derivative of pixel (i,j) in the x-direction;

[0059] P y [i,j] represents the approximate value of the first-order partial derivative of pixel (i,j) in the y-direction;

[0060] |P x [i,j]| represents P x The absolute value of [i,j];

[0061] |P y [i,j]| represents P y The absolute value of [i,j];

[0062] P x [i,j]=(I[i,j+1]-I[i,j]+I[i+1,j+1]-I[i+1,j]) / 2

[0063] P y [i,j]=(I[i,j]-I[i+1,j]+I[i,j+1]-I[i+1,j+1]) / 2

[0064] Where I[i,j+1] represents the gray value of pixel (i,j+1) in the image after Gaussian smoothing;

[0065] I[i,j] represents the gray value of pixel (i,j) in the image after Gaussian smoothing;

[0066] I[i+1,j+1] represents the gray value of pixel (i+1,j+1) in the image after Gaussian smoothing.

[0067] I[i+1,j] represents the gray value of pixel (i+1,j) in the image after Gaussian smoothing.

[0068] Step 3: Use a dual threshold detection method to detect edges in the image after non-maximum suppression, and obtain the bolt loosening detection result based on the edge detection result;

[0069] The dual-threshold detection method is specifically as follows:

[0070] Step 31: Set two thresholds α1 and α2, where α1 is greater than α2;

[0071] Step 32: Pixels with gray-level gradient magnitudes greater than threshold α1 in the image after non-maximum suppression are designated as strong edge pixels, and pixels with gray-level gradient magnitudes between α1 and α2 in the image after non-maximum suppression are designated as weak edge pixels.

[0072] Step 33: For any weak edge pixel, determine whether there is a strong edge pixel in the 4×4 neighborhood of the pixel;

[0073] If there is a strong edge pixel in the 4×4 neighborhood of the pixel, then adjust the pixel to be a strong edge pixel;

[0074] If there are no strong edge pixels in the 4×4 neighborhood of a pixel, then no processing is required for that pixel.

[0075] Similarly, after traversing each weak edge pixel, all strong edge pixels are obtained, and the straight edge of the nut is determined based on the strong edge pixels.

[0076] Step 34: Calculate the rotation angle β of the nut based on each straight edge of the nut;

[0077]

[0078] Where A represents the number of straight edges of the nut;

[0079] β a This represents the angle between the edge of the a-th straight line of the nut and the positive x-axis.

[0080] rem(·) represents the calculation of β a The remainder of / 60;

[0081] Step 35: Calculate the bolt loosening angle β * :

[0082] β * =|β-β r |

[0083] Where, β r This indicates the reference angle when the bolt is not loose.

[0084] |·| represents taking the absolute value;

[0085] If the bolt is loosened at an angle β * If the value exceeds the set threshold, the bolt is considered loose.

[0086] Otherwise, the bolts are not loose;

[0087] (2) The specific process for detecting bolt loosening in bolt images taken from a side view is as follows:

[0088] Calculate the ratio of the nut thickness to the length of the bolt above the nut. If the ratio is greater than a set threshold, the bolt is loose; otherwise, the bolt is not loose.

[0089] The beneficial effects of this invention are:

[0090] This invention first locates the bolt positions in the image to be detected, then extracts sub-images of each bolt from the image based on the location results, and then performs shooting angle detection on each sub-image of the bolt. Specific loosening detection methods are designed for bolts with different shooting angles, which can avoid detection deviation caused by using the same detection method for bolt images with different shooting angles in the same image, and improve the accuracy of bolt loosening detection. Attached Figure Description

[0091] Figure 1 This is a flowchart of a method for detecting loose bolts on communication towers according to the present invention;

[0092] Figure 2 This is a structural block diagram of a communication tower bolt loosening detection system according to the present invention. Detailed Implementation

[0093] Specific implementation method one: Combining Figure 1 This embodiment describes a method for detecting loose bolts on communication towers, which specifically includes the following steps:

[0094] Step 1: Input the image to be detected into the bolt target detection network, use the bolt target detection network to output the bolt position in the image to be detected, and extract the image of the bolt position from the image to be detected;

[0095] Step 2: Use the images extracted in Step 1 as input to the bolt angle recognition model. The output of the bolt angle recognition model is either a frontal view or a side view of the bolt.

[0096] Step 3: Perform bolt loosening detection on bolt images taken from a frontal view and bolt images taken from a side view.

[0097] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the working process of the bolt target detection network is as follows:

[0098] In the bolt target detection network, the image to be detected first passes through the first convolutional layer, and then the output of the first convolutional layer is used as the input of the first ReLU activation function layer;

[0099] The output of the first ReLU activation function layer is used as the input of the second convolutional layer, and then the output of the second convolutional layer is used as the input of the second ReLU activation function layer.

[0100] The output of the second ReLU activation function layer is used as the input of the first max pooling layer, the output of the first max pooling layer is used as the input of the third convolutional layer, and the output of the third convolutional layer is used as the input of the third ReLU activation function layer.

[0101] The output of the third ReLU activation function layer is used as the input of the fourth convolutional layer, and then the output of the fourth convolutional layer is used as the input of the fourth ReLU activation function layer.

[0102] The output of the fourth ReLU activation function layer is used as the input of the second max pooling layer, and the output of the second max pooling layer is used as the input of the fifth convolutional layer.

[0103] The output of the fifth convolutional layer is used as the input of the fifth ReLU activation function layer, and then the output of the fifth ReLU activation function layer is used as the input of the sixth convolutional layer.

[0104] The output of the sixth convolutional layer is used as the input of the sixth ReLU activation function layer, and then the output of the sixth ReLU activation function layer is used as the input of the seventh convolutional layer.

[0105] The output of the seventh convolutional layer is used as the input of the seventh ReLU activation function layer, and then the output of the seventh ReLU activation function layer is used as the input of the eighth convolutional layer.

[0106] The output of the eighth convolutional layer is used as the input of the eighth ReLU activation function layer, and then the output of the eighth ReLU activation function layer is used as the input of the ninth convolutional layer.

[0107] The output of the ninth convolutional layer is used as the input of the tenth convolutional layer, the output of the tenth convolutional layer is used as the input of the eleventh convolutional layer, and the output of the eleventh convolutional layer is used as the input of the third max pooling layer.

[0108] The output of the third max pooling layer is used as the input of the twelfth convolutional layer, and the output of the twelfth convolutional layer is used as the input of the thirteenth convolutional layer.

[0109] The output of the thirteenth convolutional layer is used as the input of the fourteenth convolutional layer, and then the output of the fourteenth convolutional layer is used as the input of the fourth max pooling layer.

[0110] The output of the fourth max pooling layer is used as the input of the fifteenth convolutional layer, and then the output of the fifteenth convolutional layer is used as the input of the sixteenth convolutional layer.

[0111] The output of the sixteenth convolutional layer is used as the input of the seventeenth convolutional layer, and then the output of the seventeenth convolutional layer is used as the input of the ninth ReLU activation function layer.

[0112] The output of the fourteenth convolutional layer is passed through the first downsampling unit, and then the output of the first downsampling unit is added to the output of the eleventh convolutional layer to obtain the sum a;

[0113] Add a to the output of the sixteenth convolutional layer to get the sum b, and then add b to the output of the ninth ReLU activation function layer to get the sum c.

[0114] The c is passed through the second downsampling unit, and then the output of the second downsampling unit is added to b to obtain the sum d;

[0115] d is passed through the third downsampling unit, and then the output of the third downsampling unit is added to a to obtain the sum e;

[0116] The output of the eleventh convolutional layer is passed through the fourth downsampling unit, and then the output of the fourth downsampling unit is added together to obtain the sum f.

[0117] The output of the eleventh convolutional layer is passed through the first deconvolutional layer, and then the output of the first deconvolutional layer is used as the input of the eighteenth convolutional layer.

[0118] The output of the 18th convolutional layer is added to the output of the 14th convolutional layer to obtain the sum g;

[0119] The g is passed through the second deconvolution layer, and the output of the second deconvolution layer is used as the input of the nineteenth convolution layer. The output of the nineteenth convolution layer is added to the output of the sixteenth convolution layer to obtain the sum h.

[0120] h is passed through the third deconvolution layer, and then the output of the third deconvolution layer is added to the output of the ninth ReLU activation function layer to obtain the sum i.

[0121] Pass i through the fifth downsampling unit, and then add the output of the fifth downsampling unit to h to obtain the sum j;

[0122] Pass j through the sixth downsampling unit, and then add the output of the sixth downsampling unit to g to obtain the sum k;

[0123] Pass k through the seventh downsampling unit, and then add the output of the eleventh convolutional layer to the output of the seventh downsampling unit to obtain the summation result m;

[0124] The bolt positions in the image to be detected are obtained based on f and m.

[0125] The other steps and parameters are the same as in Specific Implementation Method 1.

[0126] In this invention, the summation result m represents the target recognition result in the output image to be detected, and the summation result f represents the target localization result in the output image to be detected. By detecting the target position, the bolt position in the image to be detected can be determined, extracted, and then further detected.

[0127] Specific Implementation Method Three: This implementation method differs from Specific Implementation Method One or Two in that the working process of the bolt angle recognition model is as follows:

[0128] In the bolt angle recognition model, the input image first passes through the twentieth convolutional layer, and then the output of the twentieth convolutional layer is used as the input of the first convolutional block.

[0129] The output of the first convolutional block is used as the input of the tenth ReLU activation function layer, and then the output of the tenth ReLU activation function layer is used as the input of the second convolutional block.

[0130] The output of the second convolutional block is used as the input of the eleventh ReLU activation function layer, and then the output of the eleventh ReLU activation function layer is used as the input of the third convolutional block.

[0131] The output of the third convolutional block is used as the input of the twelfth ReLU activation function layer, and then the output of the twelfth ReLU activation function layer is used as the input of the fourth convolutional block;

[0132] The output of the fourth convolutional block is used as the input of the thirteenth ReLU activation function layer, and then the output of the thirteenth ReLU activation function layer is used as the input of the average pooling layer.

[0133] The output of the average pooling layer is used as the input of the fully connected layer, and the classification result is output through the fully connected layer.

[0134] Other steps and parameters are the same as in specific implementation method one or two.

[0135] This invention classifies bolt images by shooting angle, processing bolt images taken from the front view and bolt images taken from the side view separately, and designing different detection methods for images taken from different directions. This solves the problem that traditional methods use a uniform detection method for images taken from different directions, resulting in low accuracy in bolt loosening detection.

[0136] Specific Implementation Method Four: This implementation method differs from Specific Implementation Methods One to Three in that the first convolutional block contains 3 convolutional units, the second convolutional block contains 4 convolutional units, the third convolutional block contains 6 convolutional units, and the fourth convolutional block contains 3 convolutional units.

[0137] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0138] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the working process of the convolutional unit is as follows:

[0139] Within a convolutional unit, the input of the convolutional unit is passed sequentially through the 21st convolutional layer, the 22nd convolutional layer, the ACON activation function layer, and the 23rd convolutional layer. The output of the 23rd convolutional layer is then added to the input of the convolutional unit, and the result is used as the output of the convolutional unit.

[0140] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0141] In this invention, the working process of each convolutional unit is the same.

[0142] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that the specific process of detecting bolt loosening from bolt images taken at a frontal view angle is as follows:

[0143] Step 1: Perform Gaussian smoothing on the bolt image to obtain the Gaussian smoothed image;

[0144] Step 2: Calculate the gray-level gradient magnitude of each pixel in the image after Gaussian smoothing, and then perform non-maximum suppression on the gray-level gradient magnitude of the pixels to obtain the image after non-maximum suppression.

[0145] Step 3: Use a dual threshold detection method to detect edges in the image after non-maximum suppression, and obtain the bolt loosening detection result based on the edge detection result;

[0146] The dual-threshold detection method is specifically as follows:

[0147] Step 31: Set two thresholds α1 and α2, where α1 is greater than α2;

[0148] Step 32: Pixels with gray-level gradient magnitudes greater than threshold α1 in the image after non-maximum suppression are designated as strong edge pixels, and pixels with gray-level gradient magnitudes between α1 and α2 in the image after non-maximum suppression are designated as weak edge pixels.

[0149] Step 33: For any weak edge pixel, determine whether there is a strong edge pixel in the 4×4 neighborhood of the pixel;

[0150] If there is a strong edge pixel in the 4×4 neighborhood of the pixel, then adjust the pixel to be a strong edge pixel;

[0151] If there are no strong edge pixels in the 4×4 neighborhood of a pixel, then no processing is required for that pixel.

[0152] Similarly, after traversing each weak edge pixel, all strong edge pixels are obtained, and the straight edge of the nut is determined based on the strong edge pixels.

[0153] Step 34: Calculate the rotation angle β of the nut based on each straight edge of the nut;

[0154]

[0155] Where A represents the number of straight edges of the nut;

[0156] βa This represents the angle between the edge of the a-th straight line of the nut and the positive x-axis.

[0157] rem(·) represents the calculation of β a The remainder of / 60;

[0158] Step 35: Calculate the bolt loosening angle β * :

[0159] β * =|β-β r |

[0160] Where, β r This indicates the reference angle when the bolt is not loose.

[0161] |·| represents taking the absolute value;

[0162] If the bolt is loosened at an angle β * If the value exceeds the set threshold, the bolt is considered loose.

[0163] Otherwise, the bolts are not loose.

[0164] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0165] It should be noted that the x-axis direction is the width direction of the image, and the positive x-axis direction is the direction from the left side of the image to the right side of the image.

[0166] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that the calculation of the grayscale gradient magnitude of each pixel in the image after Gaussian smoothing is specifically as follows:

[0167] M[i,j]=|P x [i,j]|+|P y [i,j]|

[0168] Where M[i,j] represents the grayscale gradient magnitude of pixel (i,j);

[0169] P x [i,j] represents the approximate value of the first-order partial derivative of pixel (i,j) in the x-direction;

[0170] P y [i,j] represents the approximate value of the first-order partial derivative of pixel (i,j) in the y-direction;

[0171] |P x [i,j]| represents P x The absolute value of [i,j];

[0172] |P y [i,j]| represents P yThe absolute value of [i,j];

[0173] P x [i,j]=(I[i,j+1]-I[i,j]+I[i+1,j+1]-I[i+1,j]) / 2

[0174] P y [i,j]=(I[i,j]-I[i+1,j]+I[i,j+1]-I[i+1,j+1]) / 2

[0175] Where I[i,j+1] represents the gray value of pixel (i,j+1) in the image after Gaussian smoothing;

[0176] I[i,j] represents the gray value of pixel (i,j) in the image after Gaussian smoothing;

[0177] I[i+1,j+1] represents the gray value of pixel (i+1,j+1) in the image after Gaussian smoothing.

[0178] I[i+1,j] represents the gray value of pixel (i+1,j) in the image after Gaussian smoothing.

[0179] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0180] This invention improves the accuracy of edge detection by designing an edge detection method in an image, thereby improving the accuracy of subsequent bolt loosening detection.

[0181] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that the specific process of detecting bolt loosening from a bolt image taken from a side view is as follows:

[0182] Calculate the ratio of the nut thickness to the length of the bolt above the nut. If the ratio is greater than a set threshold, the bolt is loose; otherwise, the bolt is not loose.

[0183] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0184] The ratio of the nut thickness to the length of the thread above the nut when the bolt is not loose is used as a standard threshold. When the bolt becomes loose, the length of the thread above the nut shortens, thus increasing the calculated ratio. Therefore, when the calculated ratio is greater than the set threshold, it indicates that the bolt is loose.

[0185] Specific Implementation Method Nine: Combining Figure 2This embodiment describes a communication tower bolt loosening detection system. The system includes a target image acquisition module, a bolt target detection module, a bolt angle recognition module, and a bolt loosening detection module, wherein:

[0186] The image acquisition module is used to acquire the image to be detected;

[0187] The bolt target detection module is used to detect the position of bolts in the image to be detected and to extract the image of the bolt position from the image to be detected.

[0188] The bolt target detection module is implemented based on a bolt target detection network, and the working process of the bolt target detection network is as follows:

[0189] In the bolt target detection network, the image to be detected first passes through the first convolutional layer, and then the output of the first convolutional layer is used as the input of the first ReLU activation function layer;

[0190] The output of the first ReLU activation function layer is used as the input of the second convolutional layer, and then the output of the second convolutional layer is used as the input of the second ReLU activation function layer.

[0191] The output of the second ReLU activation function layer is used as the input of the first max pooling layer, the output of the first max pooling layer is used as the input of the third convolutional layer, and the output of the third convolutional layer is used as the input of the third ReLU activation function layer.

[0192] The output of the third ReLU activation function layer is used as the input of the fourth convolutional layer, and then the output of the fourth convolutional layer is used as the input of the fourth ReLU activation function layer.

[0193] The output of the fourth ReLU activation function layer is used as the input of the second max pooling layer, and the output of the second max pooling layer is used as the input of the fifth convolutional layer.

[0194] The output of the fifth convolutional layer is used as the input of the fifth ReLU activation function layer, and then the output of the fifth ReLU activation function layer is used as the input of the sixth convolutional layer.

[0195] The output of the sixth convolutional layer is used as the input of the sixth ReLU activation function layer, and then the output of the sixth ReLU activation function layer is used as the input of the seventh convolutional layer.

[0196] The output of the seventh convolutional layer is used as the input of the seventh ReLU activation function layer, and then the output of the seventh ReLU activation function layer is used as the input of the eighth convolutional layer.

[0197] The output of the eighth convolutional layer is used as the input of the eighth ReLU activation function layer, and then the output of the eighth ReLU activation function layer is used as the input of the ninth convolutional layer.

[0198] The output of the ninth convolutional layer is used as the input of the tenth convolutional layer, the output of the tenth convolutional layer is used as the input of the eleventh convolutional layer, and the output of the eleventh convolutional layer is used as the input of the third max pooling layer.

[0199] The output of the third max pooling layer is used as the input of the twelfth convolutional layer, and the output of the twelfth convolutional layer is used as the input of the thirteenth convolutional layer.

[0200] The output of the thirteenth convolutional layer is used as the input of the fourteenth convolutional layer, and then the output of the fourteenth convolutional layer is used as the input of the fourth max pooling layer.

[0201] The output of the fourth max pooling layer is used as the input of the fifteenth convolutional layer, and then the output of the fifteenth convolutional layer is used as the input of the sixteenth convolutional layer.

[0202] The output of the sixteenth convolutional layer is used as the input of the seventeenth convolutional layer, and then the output of the seventeenth convolutional layer is used as the input of the ninth ReLU activation function layer.

[0203] The output of the fourteenth convolutional layer is passed through the first downsampling unit, and then the output of the first downsampling unit is added to the output of the eleventh convolutional layer to obtain the sum a;

[0204] Add a to the output of the sixteenth convolutional layer to get the sum b, and then add b to the output of the ninth ReLU activation function layer to get the sum c.

[0205] The c is passed through the second downsampling unit, and then the output of the second downsampling unit is added to b to obtain the sum d;

[0206] d is passed through the third downsampling unit, and then the output of the third downsampling unit is added to a to obtain the sum e;

[0207] The output of the eleventh convolutional layer is passed through the fourth downsampling unit, and then the output of the fourth downsampling unit is added together to obtain the sum f.

[0208] The output of the eleventh convolutional layer is passed through the first deconvolutional layer, and then the output of the first deconvolutional layer is used as the input of the eighteenth convolutional layer.

[0209] The output of the 18th convolutional layer is added to the output of the 14th convolutional layer to obtain the sum g;

[0210] The g is passed through the second deconvolution layer, and the output of the second deconvolution layer is used as the input of the nineteenth convolution layer. The output of the nineteenth convolution layer is added to the output of the sixteenth convolution layer to obtain the sum h.

[0211] h is passed through the third deconvolution layer, and then the output of the third deconvolution layer is added to the output of the ninth ReLU activation function layer to obtain the sum i.

[0212] Pass i through the fifth downsampling unit, and then add the output of the fifth downsampling unit to h to obtain the sum j;

[0213] Pass j through the sixth downsampling unit, and then add the output of the sixth downsampling unit to g to obtain the sum k;

[0214] Pass k through the seventh downsampling unit, and then add the output of the eleventh convolutional layer to the output of the seventh downsampling unit to obtain the summation result m;

[0215] The bolt positions in the image to be detected are obtained based on f and m;

[0216] The bolt angle recognition module is used to detect the shooting angle of the image at the location of the bolt.

[0217] The bolt angle recognition module is implemented based on a bolt angle recognition model, and the working process of the bolt angle recognition model is as follows:

[0218] In the bolt angle recognition model, the input image first passes through the twentieth convolutional layer, and then the output of the twentieth convolutional layer is used as the input of the first convolutional block.

[0219] The output of the first convolutional block is used as the input of the tenth ReLU activation function layer, and then the output of the tenth ReLU activation function layer is used as the input of the second convolutional block.

[0220] The output of the second convolutional block is used as the input of the eleventh ReLU activation function layer, and then the output of the eleventh ReLU activation function layer is used as the input of the third convolutional block.

[0221] The output of the third convolutional block is used as the input of the twelfth ReLU activation function layer, and then the output of the twelfth ReLU activation function layer is used as the input of the fourth convolutional block;

[0222] The output of the fourth convolutional block is used as the input of the thirteenth ReLU activation function layer, and then the output of the thirteenth ReLU activation function layer is used as the input of the average pooling layer.

[0223] The output of the average pooling layer is used as the input of the fully connected layer, and the classification result is output through the fully connected layer.

[0224] The bolt loosening detection module is used to detect bolt loosening based on the detection results from the shooting angle;

[0225] (1) The specific process for detecting bolt loosening in bolt images taken from a frontal view is as follows:

[0226] Step 1: Perform Gaussian smoothing on the bolt image to obtain the Gaussian smoothed image;

[0227] Step 2: Calculate the gray-level gradient magnitude of each pixel in the image after Gaussian smoothing, and then perform non-maximum suppression on the gray-level gradient magnitude of the pixels to obtain the image after non-maximum suppression.

[0228] The method for calculating the gray-scale gradient magnitude is as follows:

[0229] M[i,j]=|P x [i,j]|+|P y [i,j]|

[0230] Where M[i,j] represents the grayscale gradient magnitude of pixel (i,j);

[0231] P x [i,j] represents the approximate value of the first-order partial derivative of pixel (i,j) in the x-direction;

[0232] P y [i,j] represents the approximate value of the first-order partial derivative of pixel (i,j) in the y-direction;

[0233] |P x [i,j]| represents P x The absolute value of [i,j];

[0234] |P y [i,j]| represents P y The absolute value of [i,j];

[0235] P x [i,j]=(I[i,j+1]-I[i,j]+I[i+1,j+1]-I[i+1,j]) / 2

[0236] P y [i,j]=(I[i,j]-I[i+1,j]+I[i,j+1]-I[i+1,j+1]) / 2

[0237] Where I[i,j+1] represents the gray value of pixel (i,j+1) in the image after Gaussian smoothing;

[0238] I[i,j] represents the gray value of pixel (i,j) in the image after Gaussian smoothing;

[0239] I[i+1,j+1] represents the gray value of pixel (i+1,j+1) in the image after Gaussian smoothing.

[0240] I[i+1,j] represents the gray value of pixel (i+1,j) in the image after Gaussian smoothing.

[0241] Step 3: Use a dual threshold detection method to detect edges in the image after non-maximum suppression, and obtain the bolt loosening detection result based on the edge detection result;

[0242] The dual-threshold detection method is specifically as follows:

[0243] Step 31: Set two thresholds α1 and α2, where α1 is greater than α2;

[0244] Step 32: Pixels with gray-level gradient magnitudes greater than threshold α1 in the image after non-maximum suppression are designated as strong edge pixels, and pixels with gray-level gradient magnitudes between α1 and α2 in the image after non-maximum suppression are designated as weak edge pixels.

[0245] Step 33: For any weak edge pixel, determine whether there is a strong edge pixel in the 4×4 neighborhood of the pixel;

[0246] If there is a strong edge pixel in the 4×4 neighborhood of the pixel, then adjust the pixel to be a strong edge pixel;

[0247] If there are no strong edge pixels in the 4×4 neighborhood of a pixel, then no processing is required for that pixel.

[0248] Similarly, after traversing each weak edge pixel, all strong edge pixels are obtained, and the straight edge of the nut is determined based on the strong edge pixels.

[0249] Step 34: Calculate the rotation angle β of the nut based on each straight edge of the nut;

[0250]

[0251] Where A represents the number of straight edges of the nut;

[0252] β a This represents the angle between the edge of the a-th straight line of the nut and the positive x-axis.

[0253] rem(·) represents the calculation of β a The remainder of / 60;

[0254] Step 35: Calculate the bolt loosening angle β * :

[0255] β * =|β-β r |

[0256] Where, β r This indicates the reference angle when the bolt is not loose.

[0257] |·| represents taking the absolute value;

[0258] If the bolt is loosened at an angle β * If the value exceeds the set threshold, the bolt is considered loose.

[0259] Otherwise, the bolts are not loose;

[0260] (2) The specific process for detecting bolt loosening in bolt images taken from a side view is as follows:

[0261] Calculate the ratio of the nut thickness to the length of the bolt above the nut. If the ratio is greater than a set threshold, the bolt is loose; otherwise, the bolt is not loose.

[0262] Specific Implementation Method 10: This implementation method differs from Specific Implementation Method 9 in that the first convolutional block includes 3 convolutional units, the second convolutional block includes 4 convolutional units, the third convolutional block includes 6 convolutional units, and the fourth convolutional block includes 3 convolutional units.

[0263] The working process of the convolutional unit is as follows:

[0264] Within a convolutional unit, the input of the convolutional unit is passed sequentially through the 21st convolutional layer, the 22nd convolutional layer, the ACON activation function layer, and the 23rd convolutional layer. The output of the 23rd convolutional layer is then added to the input of the convolutional unit, and the result is used as the output of the convolutional unit.

[0265] The other steps and parameters are the same as in Specific Implementation Method Nine.

[0266] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for detecting loose bolts on communication towers, characterized in that, The method specifically includes the following steps: Step 1: Input the image to be detected into the bolt target detection network, use the bolt target detection network to output the bolt position in the image to be detected, and extract the image of the bolt position from the image to be detected; The working process of the bolt target detection network is as follows: In the bolt target detection network, the image to be detected first passes through the first convolutional layer, and then the output of the first convolutional layer is used as the input of the first ReLU activation function layer; The output of the first ReLU activation function layer is used as the input of the second convolutional layer, and then the output of the second convolutional layer is used as the input of the second ReLU activation function layer. The output of the second ReLU activation function layer is used as the input of the first max pooling layer, the output of the first max pooling layer is used as the input of the third convolutional layer, and the output of the third convolutional layer is used as the input of the third ReLU activation function layer. The output of the third ReLU activation function layer is used as the input of the fourth convolutional layer, and then the output of the fourth convolutional layer is used as the input of the fourth ReLU activation function layer. The output of the fourth ReLU activation function layer is used as the input of the second max pooling layer, and the output of the second max pooling layer is used as the input of the fifth convolutional layer. The output of the fifth convolutional layer is used as the input of the fifth ReLU activation function layer, and then the output of the fifth ReLU activation function layer is used as the input of the sixth convolutional layer. The output of the sixth convolutional layer is used as the input of the sixth ReLU activation function layer, and then the output of the sixth ReLU activation function layer is used as the input of the seventh convolutional layer. The output of the seventh convolutional layer is used as the input of the seventh ReLU activation function layer, and then the output of the seventh ReLU activation function layer is used as the input of the eighth convolutional layer. The output of the eighth convolutional layer is used as the input of the eighth ReLU activation function layer, and then the output of the eighth ReLU activation function layer is used as the input of the ninth convolutional layer. The output of the ninth convolutional layer is used as the input of the tenth convolutional layer, the output of the tenth convolutional layer is used as the input of the eleventh convolutional layer, and the output of the eleventh convolutional layer is used as the input of the third max pooling layer. The output of the third max pooling layer is used as the input of the twelfth convolutional layer, and the output of the twelfth convolutional layer is used as the input of the thirteenth convolutional layer. The output of the thirteenth convolutional layer is used as the input of the fourteenth convolutional layer, and then the output of the fourteenth convolutional layer is used as the input of the fourth max pooling layer. The output of the fourth max pooling layer is used as the input of the fifteenth convolutional layer, and then the output of the fifteenth convolutional layer is used as the input of the sixteenth convolutional layer. The output of the sixteenth convolutional layer is used as the input of the seventeenth convolutional layer, and then the output of the seventeenth convolutional layer is used as the input of the ninth ReLU activation function layer. The output of the fourteenth convolutional layer is passed through the first downsampling unit, and then the output of the first downsampling unit is added to the output of the eleventh convolutional layer to obtain the sum a; Add a to the output of the sixteenth convolutional layer to get the sum b, and then add b to the output of the ninth ReLU activation function layer to get the sum c. The c is passed through the second downsampling unit, and then the output of the second downsampling unit is added to b to obtain the sum d; d is passed through the third downsampling unit, and then the output of the third downsampling unit is added to a to obtain the sum e; The output of the eleventh convolutional layer is passed through the fourth downsampling unit, and then the output of the fourth downsampling unit is added together to obtain the sum f. The output of the eleventh convolutional layer is passed through the first deconvolutional layer, and then the output of the first deconvolutional layer is used as the input of the eighteenth convolutional layer. The output of the 18th convolutional layer is added to the output of the 14th convolutional layer to obtain the sum g; The g is passed through the second deconvolution layer, and the output of the second deconvolution layer is used as the input of the nineteenth convolution layer. The output of the nineteenth convolution layer is added to the output of the sixteenth convolution layer to obtain the sum h. h is passed through the third deconvolution layer, and then the output of the third deconvolution layer is added to the output of the ninth ReLU activation function layer to obtain the sum i. Pass i through the fifth downsampling unit, and then add the output of the fifth downsampling unit to h to obtain the sum j; Pass j through the sixth downsampling unit, and then add the output of the sixth downsampling unit to g to obtain the sum k; Pass k through the seventh downsampling unit, and then add the output of the eleventh convolutional layer to the output of the seventh downsampling unit to obtain the summation result m; The bolt positions in the image to be detected are obtained based on f and m; Step 2: Use the images extracted in Step 1 as input to the bolt angle recognition model. The output of the bolt angle recognition model is either a frontal view or a side view of the bolt. Step 3: Perform bolt loosening detection on bolt images taken from a frontal view and bolt images taken from a side view.

2. The method for detecting loose bolts on a communication tower according to claim 1, characterized in that, The working process of the bolt angle recognition model is as follows: In the bolt angle recognition model, the input image first passes through the twentieth convolutional layer, and then the output of the twentieth convolutional layer is used as the input of the first convolutional block. The output of the first convolutional block is used as the input of the tenth ReLU activation function layer, and then the output of the tenth ReLU activation function layer is used as the input of the second convolutional block. The output of the second convolutional block is used as the input of the eleventh ReLU activation function layer, and then the output of the eleventh ReLU activation function layer is used as the input of the third convolutional block. The output of the third convolutional block is used as the input of the twelfth ReLU activation function layer, and then the output of the twelfth ReLU activation function layer is used as the input of the fourth convolutional block; The output of the fourth convolutional block is used as the input of the thirteenth ReLU activation function layer, and then the output of the thirteenth ReLU activation function layer is used as the input of the average pooling layer. The output of the average pooling layer is used as the input of the fully connected layer, and the classification result is output through the fully connected layer.

3. The method for detecting loose bolts on a communication tower according to claim 2, characterized in that, The first convolutional block contains 3 convolutional units, the second convolutional block contains 4 convolutional units, the third convolutional block contains 6 convolutional units, and the fourth convolutional block contains 3 convolutional units.

4. The method for detecting loose bolts on a communication tower according to claim 3, characterized in that, The working process of the convolutional unit is as follows: Within a convolutional unit, the input of the convolutional unit is passed sequentially through the 21st convolutional layer, the 22nd convolutional layer, the ACON activation function layer, and the 23rd convolutional layer. The output of the 23rd convolutional layer is then added to the input of the convolutional unit, and the result is used as the output of the convolutional unit.

5. The method for detecting loose bolts on a communication tower according to claim 4, characterized in that, The specific process for detecting bolt loosening from bolt images taken at a frontal angle is as follows: Step 1: Perform Gaussian smoothing on the bolt image to obtain the Gaussian smoothed image; Step 2: Calculate the gray-level gradient magnitude of each pixel in the image after Gaussian smoothing, and then perform non-maximum suppression on the gray-level gradient magnitude of the pixels to obtain the image after non-maximum suppression. Step 3: Use a dual threshold detection method to detect edges in the image after non-maximum suppression, and obtain the bolt loosening detection result based on the edge detection result; The dual-threshold detection method is specifically as follows: Step 31: Set dual thresholds and ,and Greater than ; Step 32: After non-maximum suppression, ensure that the gray-level gradient magnitude in the image is greater than the threshold. The pixels are taken as strong edge pixels, and the gray-level gradient magnitude in the image after non-maximum suppression is in the range of strong edge pixels. and The pixels between them are considered weak edge pixels; Step 33: For any weak edge pixel, determine whether there is a strong edge pixel in the 4×4 neighborhood of the pixel; If there is a strong edge pixel in the 4×4 neighborhood of the pixel, then adjust the pixel to be a strong edge pixel; If there are no strong edge pixels in the 4×4 neighborhood of a pixel, then no processing is required for that pixel. Similarly, after traversing each weak edge pixel, all strong edge pixels are obtained, and the straight edge of the nut is determined based on the strong edge pixels. Step 34: Calculate the rotation angle of the nut based on each straight edge of the nut. ; in, Indicates the number of straight edges on the nut; Indicates the first of the nuts The edge of a straight line and The angle between the positive axis and the axis; Indicates calculation The remainder; Step 35: Calculate the bolt loosening angle : in, This indicates the reference angle when the bolt is not loose. Indicates taking the absolute value; If the bolt is loose at an angle If the value exceeds the set threshold, the bolt is considered loose. Otherwise, the bolts are not loose.

6. The method for detecting loose bolts on a communication tower according to claim 5, characterized in that, The calculation of the gray-level gradient magnitude of each pixel in the image after Gaussian smoothing is specifically as follows: in, Represents pixels The grayscale gradient magnitude; Represents pixels exist Approximate value of the first-order partial derivative in the direction; Represents pixels exist Approximate value of the first-order partial derivative in the direction; express The absolute value; express The absolute value; in, Represents the pixels in the image after Gaussian smoothing. grayscale value; Represents the pixels in the image after Gaussian smoothing. grayscale value; Represents the pixels in the image after Gaussian smoothing. grayscale value; Represents the pixels in the image after Gaussian smoothing. The grayscale value.

7. The method for detecting loose bolts on a communication tower according to claim 6, characterized in that, The specific process for detecting bolt loosening from bolt images taken from a side view is as follows: Calculate the ratio of the nut thickness to the length of the bolt above the nut. If the ratio is greater than a set threshold, the bolt is loose; otherwise, the bolt is not loose.

8. A communication tower bolt loosening detection system, characterized in that, The system includes a module for acquiring the image to be detected, a module for detecting bolt targets, a module for recognizing bolt angles, and a module for detecting bolt loosening, wherein: The image acquisition module is used to acquire the image to be detected; The bolt target detection module is used to detect the position of bolts in the image to be detected and to extract the image of the bolt position from the image to be detected. The bolt target detection module is implemented based on a bolt target detection network, and the working process of the bolt target detection network is as follows: In the bolt target detection network, the image to be detected first passes through the first convolutional layer, and then the output of the first convolutional layer is used as the input of the first ReLU activation function layer; The output of the first ReLU activation function layer is used as the input of the second convolutional layer, and then the output of the second convolutional layer is used as the input of the second ReLU activation function layer. The output of the second ReLU activation function layer is used as the input of the first max pooling layer, the output of the first max pooling layer is used as the input of the third convolutional layer, and the output of the third convolutional layer is used as the input of the third ReLU activation function layer. The output of the third ReLU activation function layer is used as the input of the fourth convolutional layer, and then the output of the fourth convolutional layer is used as the input of the fourth ReLU activation function layer. The output of the fourth ReLU activation function layer is used as the input of the second max pooling layer, and the output of the second max pooling layer is used as the input of the fifth convolutional layer. The output of the fifth convolutional layer is used as the input of the fifth ReLU activation function layer, and then the output of the fifth ReLU activation function layer is used as the input of the sixth convolutional layer. The output of the sixth convolutional layer is used as the input of the sixth ReLU activation function layer, and then the output of the sixth ReLU activation function layer is used as the input of the seventh convolutional layer. The output of the seventh convolutional layer is used as the input of the seventh ReLU activation function layer, and then the output of the seventh ReLU activation function layer is used as the input of the eighth convolutional layer. The output of the eighth convolutional layer is used as the input of the eighth ReLU activation function layer, and then the output of the eighth ReLU activation function layer is used as the input of the ninth convolutional layer. The output of the ninth convolutional layer is used as the input of the tenth convolutional layer, the output of the tenth convolutional layer is used as the input of the eleventh convolutional layer, and the output of the eleventh convolutional layer is used as the input of the third max pooling layer. The output of the third max pooling layer is used as the input of the twelfth convolutional layer, and the output of the twelfth convolutional layer is used as the input of the thirteenth convolutional layer. The output of the thirteenth convolutional layer is used as the input of the fourteenth convolutional layer, and then the output of the fourteenth convolutional layer is used as the input of the fourth max pooling layer. The output of the fourth max pooling layer is used as the input of the fifteenth convolutional layer, and then the output of the fifteenth convolutional layer is used as the input of the sixteenth convolutional layer. The output of the sixteenth convolutional layer is used as the input of the seventeenth convolutional layer, and then the output of the seventeenth convolutional layer is used as the input of the ninth ReLU activation function layer. The output of the fourteenth convolutional layer is passed through the first downsampling unit, and then the output of the first downsampling unit is added to the output of the eleventh convolutional layer to obtain the sum a; Add a to the output of the sixteenth convolutional layer to get the sum b, and then add b to the output of the ninth ReLU activation function layer to get the sum c. The c is passed through the second downsampling unit, and then the output of the second downsampling unit is added to b to obtain the sum d; d is passed through the third downsampling unit, and then the output of the third downsampling unit is added to a to obtain the sum e; The output of the eleventh convolutional layer is passed through the fourth downsampling unit, and then the output of the fourth downsampling unit is added together to obtain the sum f. The output of the eleventh convolutional layer is passed through the first deconvolutional layer, and then the output of the first deconvolutional layer is used as the input of the eighteenth convolutional layer. The output of the 18th convolutional layer is added to the output of the 14th convolutional layer to obtain the sum g; The g is passed through the second deconvolution layer, and the output of the second deconvolution layer is used as the input of the nineteenth convolution layer. The output of the nineteenth convolution layer is added to the output of the sixteenth convolution layer to obtain the sum h. h is passed through the third deconvolution layer, and then the output of the third deconvolution layer is added to the output of the ninth ReLU activation function layer to obtain the sum i. Pass i through the fifth downsampling unit, and then add the output of the fifth downsampling unit to h to obtain the sum j; Pass j through the sixth downsampling unit, and then add the output of the sixth downsampling unit to g to obtain the sum k; Pass k through the seventh downsampling unit, and then add the output of the eleventh convolutional layer to the output of the seventh downsampling unit to obtain the summation result m; The bolt positions in the image to be detected are obtained based on f and m; The bolt angle recognition module is used to detect the shooting angle of the image at the location of the bolt. The bolt angle recognition module is implemented based on a bolt angle recognition model, and the working process of the bolt angle recognition model is as follows: In the bolt angle recognition model, the input image first passes through the twentieth convolutional layer, and then the output of the twentieth convolutional layer is used as the input of the first convolutional block. The output of the first convolutional block is used as the input of the tenth ReLU activation function layer, and then the output of the tenth ReLU activation function layer is used as the input of the second convolutional block. The output of the second convolutional block is used as the input of the eleventh ReLU activation function layer, and then the output of the eleventh ReLU activation function layer is used as the input of the third convolutional block. The output of the third convolutional block is used as the input of the twelfth ReLU activation function layer, and then the output of the twelfth ReLU activation function layer is used as the input of the fourth convolutional block; The output of the fourth convolutional block is used as the input of the thirteenth ReLU activation function layer, and then the output of the thirteenth ReLU activation function layer is used as the input of the average pooling layer. The output of the average pooling layer is used as the input of the fully connected layer, and the classification result is output through the fully connected layer. The bolt loosening detection module is used to detect bolt loosening based on the detection results from the shooting angle; (1) The specific process for detecting bolt loosening in bolt images taken from a frontal view is as follows: Step 1: Perform Gaussian smoothing on the bolt image to obtain the Gaussian smoothed image; Step 2: Calculate the gray-level gradient magnitude of each pixel in the image after Gaussian smoothing, and then perform non-maximum suppression on the gray-level gradient magnitude of the pixels to obtain the image after non-maximum suppression. The method for calculating the gray-scale gradient magnitude is as follows: in, Represents pixels The grayscale gradient magnitude; Represents pixels exist Approximate value of the first-order partial derivative in the direction; Represents pixels exist Approximate value of the first-order partial derivative in the direction; express The absolute value; express The absolute value; in, Represents the pixels in the image after Gaussian smoothing. grayscale value; Represents the pixels in the image after Gaussian smoothing. grayscale value; Represents the pixels in the image after Gaussian smoothing. grayscale value; Represents the pixels in the image after Gaussian smoothing. grayscale value; Step 3: Use a dual threshold detection method to detect edges in the image after non-maximum suppression, and obtain the bolt loosening detection result based on the edge detection result; The dual-threshold detection method is specifically as follows: Step 31: Set dual thresholds and ,and Greater than ; Step 32: After non-maximum suppression, ensure that the gray-level gradient magnitude in the image is greater than the threshold. The pixels are taken as strong edge pixels, and the gray-level gradient magnitude in the image after non-maximum suppression is in the range of strong edge pixels. and The pixels between them are considered weak edge pixels; Step 33: For any weak edge pixel, determine whether there is a strong edge pixel in the 4×4 neighborhood of the pixel; If there is a strong edge pixel in the 4×4 neighborhood of the pixel, then adjust the pixel to be a strong edge pixel; If there are no strong edge pixels in the 4×4 neighborhood of a pixel, then no processing is required for that pixel. Similarly, after traversing each weak edge pixel, all strong edge pixels are obtained, and the straight edge of the nut is determined based on the strong edge pixels. Step 34: Calculate the rotation angle of the nut based on each straight edge of the nut. ; in, Indicates the number of straight edges on the nut; Indicates the first of the nuts The edge of a straight line and The angle between the positive axis and the axis; Indicates calculation The remainder; Step 35: Calculate the bolt loosening angle : in, This indicates the reference angle when the bolt is not loose. Indicates taking the absolute value; If the bolt is loose at an angle If the value exceeds the set threshold, the bolt is considered loose. Otherwise, the bolts are not loose; (2) The specific process for detecting bolt loosening in bolt images taken from a side view is as follows: Calculate the ratio of the nut thickness to the length of the bolt above the nut. If the ratio is greater than a set threshold, the bolt is loose; otherwise, the bolt is not loose.

9. The communication tower bolt loosening detection system according to claim 8, characterized in that, The first convolutional block contains 3 convolutional units, the second convolutional block contains 4 convolutional units, the third convolutional block contains 6 convolutional units, and the fourth convolutional block contains 3 convolutional units. The working process of the convolutional unit is as follows: Within a convolutional unit, the input of the convolutional unit is passed sequentially through the 21st convolutional layer, the 22nd convolutional layer, the ACON activation function layer, and the 23rd convolutional layer. The output of the 23rd convolutional layer is then added to the input of the convolutional unit, and the result is used as the output of the convolutional unit.

Images