Tower cylinder bolt abnormity identification method and device

Abstract

The embodiment of the invention provides a tower cylinder bolt abnormity identification method and device. The method comprises the steps that an image acquisition device arranged on a tower cylinder inspection robot is used for acquiring a front image and a side image of each tower cylinder bolt; performing preprocessing operation on the front image and the side image to obtain a front image block corresponding to the front image and a side image block corresponding to the side image; performing abnormal region identification on the front image block and thread feature identification on the side image block respectively, and outputting identification results including an abnormal type, a quantization parameter and confidence respectively; performing grading processing on the recognition result according to the confidence coefficient in the recognition result, and outputting a judgment result; and the judgment result, the bolt ID and the inspection time are stored in an associated mode, and alarm information is output according to the judgment result and pushed to a manager terminal device. According to the invention, the safety and the detection efficiency of tower bolt inspection can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method and apparatus for identifying abnormal tower bolts. Background Technology

[0002] Wind turbine tower bolts, as core components connecting tower segments, are constantly exposed to complex outdoor environments (such as high salt spray, strong winds, and sudden temperature changes), making them prone to rust, cracks, deformation, and other cosmetic abnormalities. Failure to detect and address these issues promptly can lead to serious safety accidents such as bolt breakage and tower tilting. However, current maintenance methods primarily rely on manual climbing and inspection. Yet, manual inspection has the following drawbacks: 1. Low efficiency: A single wind turbine tower contains thousands of bolts, and manual inspection takes 2-3 days per unit, which cannot meet the operation and maintenance needs of large-scale wind farms; 2. Poor accuracy: Microcracks smaller than 0.2mm are difficult to detect with the naked eye, and minor abnormalities such as a deviation of 1-2 threads are easily missed; 3. Low safety: Climbing operations carry the risk of falling from heights, especially in inclement weather; 4. Data fragmentation: Manually recorded abnormal data lacks a unified format, making it difficult to form a full lifecycle archive and trace the development trend of abnormalities.

[0003] Therefore, how to solve the above problems is an urgent issue that needs to be addressed. Summary of the Invention

[0004] This application provides a method and apparatus for identifying abnormal tower bolts, aiming to improve the above-mentioned problems.

[0005] In a first aspect, this application provides a method for identifying abnormal tower bolts. The method includes: using an image acquisition device mounted on a tower inspection robot to acquire a frontal image and a side image of each tower bolt, wherein the frontal image focuses on the bolt head and the connection between the bolt and the tower, and the side image focuses on the threaded portion of the bolt. Preprocessing operations are performed on the front image and the side image respectively to obtain a front image block corresponding to the front image and a side image block corresponding to the side image; Anomaly region recognition is performed on the frontal image block and serration feature recognition is performed on the side image block, respectively, and recognition results including anomaly type, quantization parameters, and confidence level are output respectively; The identification results are graded based on the confidence level in the identification results, and a judgment result is output. The judgment result is associated with the bolt ID and inspection time and stored. An alarm message is output and pushed to the management personnel terminal device based on the judgment result.

[0006] In one possible embodiment, preprocessing operations are performed on the frontal image and the side image respectively to obtain a frontal image patch corresponding to the frontal image and a side image patch corresponding to the side image, including: A Gaussian filtering algorithm with a kernel size of 3×3 is used to filter environmental noise and sensor noise in the frontal image and the side image respectively, to obtain the filtered initial frontal image and initial side image. A histogram equalization algorithm is used to adjust the gray value distribution of the initial frontal image and the initial side image, so that the average gray value of the processed middle frontal image and the middle side image is stabilized in the range of 120-150. Based on the lightweight YOLOv8-nano model, the bolt core region is located in the middle front image and the middle side image. The middle front image and the middle side image are then cropped into 800×800 pixel front image blocks and side image blocks, and the tower background area is removed.

[0007] In one possible embodiment, anomaly region identification is performed on the frontal image patch and serration feature identification is performed on the side image patch, respectively, and identification results including anomaly type, quantization parameters, and confidence level are output, including: A 400×400 pixel head focus sub-image is cropped from the front image block with the center of the bolt head as the origin; A lightweight YOLOv8 model is used to locate four types of abnormal regions in the head focusing sub-image: rust, cracks, deformation, and missing parts. Candidate boxes containing location coordinates, preliminary anomaly type, and initial confidence level are output. The abnormal regions corresponding to the candidate boxes are cropped into independent sub-images and input into a ResNet50 residual network containing an SE attention module. The weights of key feature channels such as rust texture and crack edges are strengthened through a squeezing-excitation mechanism to correct the anomaly type determination results and optimize the confidence. Calculate quantization parameters for each corrected anomaly type; Based on the corrected anomaly type, the quantization parameters, and the optimized confidence level, a structured bolt head anomaly identification result is generated. The bolt head anomaly identification result includes the anomaly type, quantization parameters, confidence level, and associated frontal image block identifier. Perform tooth feature recognition on the side image block and output tooth recognition results including anomaly type, quantization parameters, and confidence level.

[0008] In one possible embodiment, calculating the quantization parameters for each corrected anomaly type includes: If the anomaly in the independent sub-image is corrosion, the independent sub-image is converted from RGB format to HSV format to generate a corrosion binary mask image; The percentage of rusted area is determined by comparing the number of rusted pixels in the rusted binary mask image with the total number of pixels in the sub-image. If the anomaly in the independent subgraph is a crack, the Zhang-Suen skeleton extraction algorithm is used to obtain a crack skeleton line with a width of 1 pixel in the independent subgraph. The pixel length of the crack skeleton line is calculated using the arcLength function; The crack length is obtained based on the pixel length of the crack skeleton line and the image scale. If the anomaly in the independent subgraph is a deformation, extract the actual head contour from the independent subgraph; The actual outline of the head is compared with the preset standard outline of the same specification bolt, and the outline similarity is calculated. The shape deviation rate is determined based on the contour similarity. If the anomaly in the independent sub-image is a missing pixel, the number of pixels in the missing region of the independent sub-image is counted and then compared with the total number of pixels in the head-focused sub-image. The percentage of the missing area is calculated based on the number of pixels in the missing region and the total number of pixels.

[0009] In one possible embodiment, performing serration feature recognition on the side image patch and outputting a serration recognition result including anomaly type, quantization parameters, and confidence level includes: The Canny edge detection algorithm was used to extract the edge lines of the threaded parts. Effective wire contour lines are selected from the wire edge lines based on the periodic characteristics of the wires. The effective thread contour lines are clustered in parallel using the Hough transform algorithm to determine the pixel coordinates corresponding to the upper boundary of the thread being 10mm below the bolt head and the lower boundary of the thread being the pixel coordinates corresponding to the last continuous thread contour line. The effective thread area is then cropped. Ten sampling lines are uniformly arranged within the effective tooth area along the tooth axis, wherein the sampling line spacing = effective tooth area height ÷ 10; The number of thread peaks for each sampling line is counted using a peak detection algorithm. The average number of peaks for 10 sampling lines is calculated and rounded to obtain the number of protruding threads. If the number of peaks on a certain sampling line deviates from the average value by more than a threshold, then the thread deformation in that area is determined and the corresponding sampling line coordinates are marked. An anomaly type is generated based on the number of protruding threads and the determination result of thread deformation; Determine the corresponding quantization parameters based on the anomaly type; Confidence level is calculated based on the consistency of the number of peaks; A structured fiber identification result is generated based on the anomaly type, the quantization parameter, and the confidence level.

[0010] In one possible embodiment, the identification result includes the bolt head anomaly identification result and the thread identification result. The identification result is then graded based on the confidence level within the identification result, and a judgment result is output, including: If the confidence level of the bolt head anomaly identification result is ≥0.8 and the anomaly type matches the quantification parameters, then the bolt head anomaly identification result is used as the local inference result of the head; if the confidence level is 0.5-0.8, the head focus subgraph and the bolt head anomaly identification result are uploaded to the cloud for secondary inference; if the confidence level is <0.5, it is marked as a head that needs manual review. If the confidence level of the fiber identification result is ≥0.8 and the anomaly type matches the quantization parameters, the fiber identification result is used as the local fiber inference result; if the confidence level of the fiber identification result is 0.5-0.8, the valid fiber region and the fiber identification result are uploaded to the cloud for secondary inference; if the confidence level of the fiber identification result is <0.5, it is marked as a fiber requiring manual review. Based on the local inference results of the head, the local inference results of the thread, the secondary inference results in the cloud, and the manual review results, a determination result of the bolt appearance is generated.

[0011] In one possible embodiment, the cloud-based secondary inference: The cloud retrieves historical thread detection data of the bolts from a preset database. Compare the current number of protruding threads of the bolt with the difference between the historical number of protruding threads in the historical thread detection data; If the difference in change is ≥2 teeth, then add the "risk of loosening of the filament" label to the determination result.

[0012] In one possible embodiment, outputting alarm information based on the determination result and pushing it to the management personnel terminal device includes: If the anomaly type and corresponding quantification parameter in the judgment result match the preset level 1 alarm level, a structured high-risk alarm information is generated. The high-risk alarm information includes bolt ID, inspection time, anomaly type, quantification parameter, emergency handling suggestion, and URL of associated image. If the anomaly type and corresponding quantification parameter in the judgment result match the preset level 2 alarm level, a structured medium-risk alarm information is generated. The medium-risk alarm information includes bolt ID, inspection time, anomaly type, quantification parameter, planned processing suggestion, and URL of associated image. If the anomaly type and corresponding quantification parameters in the judgment result match the preset three-level alarm level, a structured low-risk alarm information is generated. The low-risk alarm information includes bolt ID, inspection time, anomaly type, quantification parameters, and routine handling suggestions.

[0013] In one possible embodiment, The preset level 1 alarm level satisfies: The degree of corrosion is greater than 30%, the length of the crack is greater than 0.5 mm, the shape deviation rate of the deformation is greater than 5%, the bolt is completely missing, and the number of threads is greater than 2. The preset level 2 alarm level satisfies: The degree of corrosion is >10% and <30%, the length of the crack is <0.5mm, the shape deviation rate of the deformation is ≤5%, the bolt head is damaged, and the number of threads is 1-2 threads off. The preset three-level alarm levels satisfy the following: The degree of corrosion is less than 10%, the number of threads is off by 1 thread and there is no deformation.

[0014] Secondly, this application also provides a tower bolt abnormality identification device, the device comprising: The image acquisition unit is used to acquire front and side images of each tower bolt using an image acquisition device mounted on the tower inspection robot. The front image focuses on the bolt head and the connection between the bolt and the tower, while the side image focuses on the threaded portion of the bolt. The preprocessing unit is used to perform preprocessing operations on the front image and the side image respectively to obtain a front image block corresponding to the front image and a side image block corresponding to the side image; The recognition unit is used to perform abnormal region recognition on the frontal image block and to perform tooth feature recognition on the side image block, and output recognition results including abnormality type, quantization parameters and confidence level respectively. The determination unit is used to classify the identification result according to the confidence level in the identification result and output the determination result. An abnormal alarm unit is used to associate and store the judgment result with the bolt ID and inspection time, and output alarm information based on the judgment result and push it to the management personnel terminal device.

[0015] The tower bolt anomaly identification method and device provided in this application, through a full-process design of image acquisition, preprocessing, multi-dimensional recognition, hierarchical reasoning, and hierarchical alarm, simultaneously covers the appearance inspection of the bolt head and threads, outputting structured results containing anomaly type, quantitative parameters, and confidence levels, and linking the bolt ID with the inspection time to form a closed loop. This overcomes the limitations of traditional single-image detection, simultaneously detecting anomalies in the bolt head (rust, cracks, etc.) and threads (quantity, deformation), avoiding the risk of missed detections of "normal head but failed threads," and improving detection coverage. Simultaneously, it ensures consistency in bolt detection results for different models and under different environments, effectively solving the problem of fragmented data from manual inspections. Furthermore, by linking the detection results with the bolt ID and inspection time, it provides a foundation for subsequent full lifecycle archive construction and trend analysis, effectively shortening maintenance response time. Moreover, robotic inspection effectively avoids manual high-altitude climbing, improving inspection safety. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the structure of an electronic device provided in the first embodiment of this application; Figure 2 A flowchart illustrating a method for identifying abnormal tower bolts provided in the second embodiment of this application; Figure 3 This is a schematic diagram of the functional modules of a tower bolt abnormality identification device provided in the third embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] First Embodiment Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. In this application, it can be... Figure 1The schematic diagram shown illustrates an example of an electronic device 100 used to implement the tower bolt anomaly identification method and apparatus according to embodiments of this application.

[0020] like Figure 1 The diagram shows the structure of an electronic device 100. The electronic device 100 includes one or more processors 102, one or more storage devices 104, and input devices 106. These components are interconnected via a bus system and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 1 The components and structure of the electronic device 100 shown are merely exemplary and not limiting; the electronic device may have, as needed. Figure 1 The components shown may also have Figure 1 Other components and structures not shown.

[0021] The processor 102 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

[0022] The storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the program instructions to implement the functions described in the embodiments of this application below and / or other desired functions. Various application programs and various data may also be stored in the computer-readable storage medium, such as various data used and / or generated by the application programs.

[0023] The input device 106 may be a device used by a user to input commands, and may include one or more of the following: keyboard, mouse, microphone, and touch screen.

[0024] Second embodiment: Reference Figure 2 The flowchart shown illustrates a method for identifying abnormal tower bolts. This method specifically includes the following steps: Step S201: Use the image acquisition device installed on the tower inspection robot to acquire front and side images of each tower bolt.

[0025] The front image focuses on the bolt head and the connection between the bolt and the tower, while the side image focuses on the threaded portion of the bolt.

[0026] In one implementation, the image acquisition device is a 4K high-definition camera. This 4K high-definition camera includes a 360° rotatable camera module (viewing angle deviation ≤5°) and an image compression unit, with a camera resolution of 4096×2160. For a single bolt, the camera captures one image each from the front and side within 0.5 seconds. During shooting, a laser ranging module ensures the distance between the lens and the bolt remains stable at 30±5cm (to avoid image blur). The image compression unit uses the JPEG algorithm with a compression quality of 70%, reducing the size of a single image from 20MB to 3-5MB. After compression, the image is transmitted to the local control system via a 5G module with a transmission delay ≤0.3 seconds.

[0027] Understandably, by designing the image acquisition device in the above manner, it can ensure image quality (i.e., 4K resolution combined with laser ranging (stable object distance) ensures that details of the bolt head (such as 0.2mm microcracks) and thread texture are clearly distinguishable in the image, improving image clarity by 300% and providing high-quality data for subsequent feature extraction); secondly, it can improve acquisition efficiency (acquisition speed of 0.5 seconds per bolt, reducing the inspection time of a single fan (approximately 3000 bolts) from 2 days to 2 hours, improving efficiency by 12 times); finally, it can also reduce transmission costs (JPEG compression (70% quality) reduces the size of a single image from 20MB to 3-5MB, 5G transmission latency ≤0.3 seconds, avoiding data congestion, and reducing cloud storage costs).

[0028] As one implementation, before step S201, the method further includes: constructing a sample library.

[0029] The specific process of constructing this sample library is as follows: S01: Sample collection, collecting front and side images of tower bolts with service life of 1-15 years, in coastal high salt spray environment and northern low temperature environment, covering M16-M30 specifications; S02: Virtual sample generation. Virtual head samples with light / medium / heavy corrosion are generated through texture synthesis algorithm, virtual crack samples with a length of 0.2-2mm are generated through line generation algorithm, and virtual thread samples with 1-5 teeth are generated through parametric modeling. S03: Sample annotation. Use the LabelImg annotation tool to annotate the anomaly type, anomaly area coordinates, quantitative parameter reference values ​​(such as the reference range of rust area ratio), bolt specifications, and environmental parameters of the sample. S04: Sample balancing and training. Adjust the ratio of normal samples to abnormal samples in the sample library to 1:1.2, with a total size of no less than 100,000 images. These images are used to train the lightweight YOLOv8 model from step S203, the SE-ResNet50 model, and the contour extraction algorithm proposed below. The model training accuracy is ≥95%.

[0030] Step S202: Perform preprocessing operations on the front image and the side image respectively to obtain a front image block corresponding to the front image and a side image block corresponding to the side image.

[0031] As one implementation method, step S202 includes: using a Gaussian filtering algorithm with a filter kernel size of 3×3 to filter environmental noise and sensor noise in the front image and the side image respectively, to obtain the filtered initial front image and initial side image; using a histogram equalization algorithm to adjust the gray value distribution of the initial front image and the initial side image, so that the average gray value of the processed intermediate front image and intermediate side image is stabilized in the range of 120-150; locating the bolt core region of the intermediate front image and the intermediate side image based on the lightweight YOLOv8-nano model, cropping the intermediate front image and the intermediate side image into 800×800 pixel front image blocks and side image blocks, and removing the tower background region.

[0032] Among them, the front image block focuses on the bolt head, such as the head area accounting for ≥80%. The side image block focuses on the bolt threads, such as the thread area accounting for ≥80%.

[0033] Understandably, employing a 3×3 Gaussian filter to effectively filter out interference such as metal dust and sensor noise can significantly improve the image signal-to-noise ratio and avoid abnormal misjudgments caused by noise. Furthermore, histogram equalization unifies the image grayscale values ​​under different lighting conditions (such as strong midday light and weak cloudy light), reducing detection bias caused by uneven lighting. Finally, YOLOv8-nano cropping (head / tooth ratio ≥ 80%) removes redundant background information (such as the tower surface), reducing the computational load of subsequent feature extraction by 60% and increasing inference speed by 2 times.

[0034] Step S203: Perform abnormal region recognition on the frontal image block and tooth feature recognition on the side image block respectively, and output the recognition results including abnormality type, quantization parameters and confidence level respectively.

[0035] As one implementation method, step S203 includes: cropping a 400×400 pixel head-focused sub-image from the frontal image block with the bolt head center as the origin; using a lightweight YOLOv8 model to locate four types of abnormal regions (rust, crack, deformation, and missing parts) in the head-focused sub-image, and outputting candidate boxes containing position coordinates, preliminary abnormality types, and initial confidence scores; cropping the abnormal regions corresponding to the candidate boxes into independent sub-images, inputting them into a ResNet50 residual network containing an SE attention module, strengthening the weights of key feature channels such as rust texture and crack edges through a squeezing-excitation mechanism, correcting the abnormality type determination results, and optimizing the confidence scores; calculating quantization parameters for each corrected abnormality type; generating a structured bolt head abnormality recognition result based on the corrected abnormality type, the quantization parameters, and the optimized confidence scores, wherein the bolt head abnormality recognition result includes the abnormality type, quantization parameters, confidence scores, and associated frontal image block identifiers; performing thread feature recognition on the side image block, and outputting thread recognition results containing the abnormality type, quantization parameters, and confidence scores.

[0036] Optionally, the YOLOv8 model adds a 20×20 pixel small target detection head (adapted to crack detection) and configures a slender anchor frame with a ratio of 1:5 or 1:8 (adapted to crack morphology).

[0037] Understandably, by inputting a 400×400 head-focused sub-image into a lightweight YOLOv8 model for processing, then using SE-ResNet50 for feature enhancement, and finally outputting a structured recognition structure, accurate identification of bolt head anomalies can be effectively achieved.

[0038] Optionally, the step of calculating quantization parameters for each corrected anomaly type includes: if the anomaly in the independent sub-image is corrosion, converting the independent sub-image from RGB format to HSV format to generate a corrosion binary mask image; counting the number of corrosion pixels in the corrosion binary mask image and the total number of pixels in the sub-image to determine the corrosion area ratio; if the anomaly in the independent sub-image is a crack, using a preset skeleton extraction algorithm (such as the Zhang-Suen skeleton extraction algorithm) to obtain a 1-pixel-width crack skeleton line in the independent sub-image; calculating the pixel length of the crack skeleton line using the arcLength function; obtaining the crack length based on the pixel length of the crack skeleton line and the image scale; if the anomaly in the independent sub-image is deformation, extracting the actual head contour in the independent sub-image (e.g., using Canny edge detection with dual thresholds of 80-160); comparing the actual head contour with a preset standard contour of the same specification bolt (e.g., the standard contour of an M20 hexagonal head bolt) to calculate the contour similarity (e.g., using matchShapes). (function); determine the shape deviation rate based on the contour similarity; if the anomaly in the independent sub-image is missing, count the number of pixels in the missing region in the independent sub-image and the total number of pixels in the head-focused sub-image; calculate the missing area ratio based on the number of pixels in the missing region and the total number of pixels.

[0039] Optionally, the HSV format is: set the rust color threshold: H (hue) = 0-20°, S (saturation) = 50%-80%, V (lightness) = 40%-70%.

[0040] Optionally, the percentage of rusted area = (number of rusted pixels ÷ total number of pixels in the sub-image) × 100%.

[0041] Optionally, the image scale is 1 pixel, i.e., the image scale is 0.01 mm. Therefore, the actual crack length = the pixel length of the crack skeleton line × 0.01 mm.

[0042] Optionally, the shape deviation rate = (1 - contour similarity) × 100%.

[0043] Optionally, the percentage of missing area = (number of pixels in the missing region ÷ total number of pixels in the head focused sub-image) × 100%.

[0044] It is understandable that the above methods can effectively improve the accuracy of quantification parameters, thereby improving the accuracy of anomaly identification.

[0045] Optionally, the step of performing tooth feature recognition on the side image block and outputting a tooth recognition result including anomaly type, quantization parameters, and confidence level includes: extracting tooth edge lines using the Canny edge detection algorithm; filtering effective tooth contour lines from the tooth edge lines based on tooth periodicity features; performing parallel clustering on the effective tooth contour lines using the Hough transform algorithm to determine the upper boundary of the tooth as the pixel coordinates corresponding to 10mm below the bolt head and the lower boundary of the tooth as the pixel coordinates corresponding to the last continuous tooth contour line, and cropping to obtain the effective tooth region; uniformly setting 10 sampling lines along the tooth axis direction within the effective tooth region, wherein the sampling line spacing = effective tooth region height ÷ 10; counting the number of tooth peaks on each sampling line using a peak detection algorithm, and calculating 10 The average number of peaks on each sampling line is rounded to obtain the number of protruding teeth. If the number of peaks on a certain sampling line deviates from the average value by more than a threshold (e.g., 2), the teeth in that area are determined to be deformed, and the coordinates of the corresponding sampling line are marked. Anomaly types are generated based on the number of protruding teeth and the determination results of tooth deformation. The corresponding quantization parameters are determined based on the anomaly types. The confidence level is calculated based on the consistency of the number of peaks (confidence level = number of sampling lines with deviation ≤ 1 ÷ 10). A structured tooth recognition result is generated based on the anomaly type, the quantization parameters, and the confidence level.

[0046] Optionally, the upper boundary of the thread is 10mm below the bolt head, calculated at 1 pixel = 0.0625mm. The lower boundary is the outline of the last continuous thread.

[0047] Step S204: The identification result is graded according to the confidence level in the identification result, and the judgment result is output.

[0048] The identification results include the bolt head anomaly identification results and the thread identification results.

[0049] As one implementation method, step S204 includes: if the confidence level of the bolt head anomaly identification result is ≥0.8 and the anomaly type matches the quantization parameters, then the bolt head anomaly identification result is used as the head local inference result; if the confidence level is 0.5-0.8, the head focused sub-image and the bolt head anomaly identification result are uploaded to the cloud for secondary inference; if the confidence level is <0.5, it is marked as a head requiring manual review; if the confidence level of the thread identification result is ≥0.8 and the anomaly type matches the quantization parameters, then the thread identification result is used as the thread local inference result; if the confidence level of the thread identification result is 0.5-0.8, the effective thread area and the thread identification result are uploaded to the cloud for secondary inference; if the confidence level of the thread identification result is <0.5, it is marked as a thread requiring manual review; based on the head local inference result, the thread local inference result, the cloud secondary inference result, and the manual review result, a bolt appearance judgment result is generated.

[0050] Understandably, the above-mentioned hierarchical processing can ensure a balance between inference efficiency and accuracy. Furthermore, manual review of low confidence levels (<0.5) can solve the problem of judging extremely complex anomalies (such as multiple types of superposition) and effectively improve the overall judgment accuracy.

[0051] Optionally, the cloud-based secondary reasoning includes: calling historical thread detection data of the bolt in a preset database in the cloud; comparing the change difference between the current number of protruding threads of the bolt and the historical number of protruding threads in the historical thread detection data; if the change difference is ≥ 2 threads, then adding a "thread loosening risk" label to the judgment result.

[0052] Step S205: The judgment result is associated with and stored with the bolt ID and inspection time, and an alarm message is output and pushed to the management personnel terminal device according to the judgment result.

[0053] As one implementation method, step S205 includes: if the anomaly type and corresponding quantification parameter in the judgment result match a preset level 1 alarm level, generating structured high-risk alarm information, the high-risk alarm information including bolt ID, inspection time, anomaly type, quantification parameter, emergency handling suggestion, and URL of associated image; if the anomaly type and corresponding quantification parameter in the judgment result match a preset level 2 alarm level, generating structured medium-risk alarm information, the medium-risk alarm information including bolt ID, inspection time, anomaly type, quantification parameter, planned handling suggestion, and URL of associated image; if the anomaly type and corresponding quantification parameter in the judgment result match a preset level 3 alarm level, generating structured low-risk alarm information, the low-risk alarm information including bolt ID, inspection time, anomaly type, quantification parameter, and routine handling suggestion.

[0054] The preset level 1 alarm level satisfies the following: The degree of corrosion is greater than 30%, the length of the crack is greater than 0.5 mm, the shape deviation rate of the deformation is greater than 5%, the bolt is completely missing, and the number of threads is greater than 2. The preset level 2 alarm level satisfies: The degree of corrosion is >10% and <30%, the length of the crack is <0.5mm, the shape deviation rate of the deformation is ≤5%, the bolt head is damaged, and the number of threads is 1-2 threads off. The preset three-level alarm levels satisfy the following: The degree of corrosion is less than 10%, the number of threads is off by 1 thread and there is no deformation.

[0055] Understandably, alarm response grading allows for targeted handling of different levels of situations, improving the timeliness of anomaly handling.

[0056] As one implementation method, the method further includes: detecting whether the alarm information is completed; if not, upgrading the currently unprocessed alarm information according to the completion time corresponding to different alarm levels, and pushing the corresponding alarm information according to the upgraded level.

[0057] For example, a Level 2 alarm will automatically escalate to Level 1 if it is not handled within 48 hours, and a Level 3 alarm will automatically escalate to Level 2 if it is not handled within 72 hours.

[0058] Third embodiment: See Figure 3 The illustrated tower bolt anomaly identification device includes: an image acquisition unit 510, a preprocessing unit 520, an identification unit 530, a judgment unit 540, and an anomaly alarm unit 550. The specific functions of each unit are as follows: The image acquisition unit 510 is used to acquire front and side images of each tower bolt using an image acquisition device mounted on the tower inspection robot. The front image focuses on the bolt head and the connection between the bolt and the tower, and the side image focuses on the threaded portion of the bolt. Preprocessing unit 520 is used to perform preprocessing operations on the front image and the side image respectively to obtain a front image block corresponding to the front image and a side image block corresponding to the side image; The recognition unit 530 is used to perform abnormal region recognition on the frontal image block and to perform tooth feature recognition on the side image block, and output recognition results including abnormality type, quantization parameters and confidence level respectively. The determination unit 540 is used to perform hierarchical processing on the recognition result according to the confidence level in the recognition result and output the determination result; The abnormal alarm unit 550 is used to associate and store the judgment result with the bolt ID and inspection time, and output alarm information according to the judgment result and push it to the management personnel terminal device.

[0059] It should be noted that the specific implementation of each of the above units is described in the above method embodiments, and will not be repeated here.

[0060] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program, which, when run by a processing device, executes the steps of any of the tower bolt anomaly identification methods provided in Embodiment 2 above.

[0061] The computer program product of the tower bolt anomaly identification method and device provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0062] It should be noted that if the above functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0063] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

Claims

1. A method for identifying abnormal tower bolts, characterized in that, The method includes: The front and side images of each tower bolt are acquired using an image acquisition device mounted on the tower inspection robot. The front image focuses on the bolt head and the connection between the bolt and the tower, while the side image focuses on the threaded portion of the bolt. Preprocessing operations are performed on the front image and the side image respectively to obtain a front image block corresponding to the front image and a side image block corresponding to the side image; Anomaly region recognition is performed on the frontal image block and serration feature recognition is performed on the side image block, respectively, and recognition results including anomaly type, quantization parameters, and confidence level are output respectively; The identification results are graded based on the confidence level in the identification results, and a judgment result is output. The judgment result is associated with the bolt ID and inspection time and stored. An alarm message is output and pushed to the management personnel terminal device based on the judgment result.

2. The method according to claim 1, characterized in that, Preprocessing operations are performed on the frontal image and the side image respectively to obtain a frontal image patch corresponding to the frontal image and a side image patch corresponding to the side image, including: A Gaussian filtering algorithm with a kernel size of 3×3 is used to filter environmental noise and sensor noise in the frontal image and the side image respectively, to obtain the filtered initial frontal image and initial side image. A histogram equalization algorithm is used to adjust the gray value distribution of the initial frontal image and the initial side image, so that the average gray value of the processed middle frontal image and the middle side image is stabilized in the range of 120-150. Based on the lightweight YOLOv8-nano model, the bolt core area is located in the middle front image and the middle side image. The middle front image and the middle side image are then cropped into 800×800 pixel front image blocks and side image blocks, and the tower background area is removed.

3. The method according to claim 2, characterized in that, Anomaly region identification is performed on the frontal image patch, and serration feature identification is performed on the side image patch. Identification results, including anomaly type, quantization parameters, and confidence level, are output respectively. A 400×400 pixel head focus sub-image is cropped from the front image block with the center of the bolt head as the origin; A lightweight YOLOv8 model is used to locate four types of abnormal regions in the head focusing sub-image: rust, cracks, deformation, and missing parts. Candidate boxes containing location coordinates, preliminary anomaly type, and initial confidence level are output. The abnormal regions corresponding to the candidate boxes are cropped into independent sub-images and input into a ResNet50 residual network containing an SE attention module. The weights of key feature channels such as rust texture and crack edges are strengthened through a squeezing-excitation mechanism to correct the anomaly type determination results and optimize the confidence. Calculate quantization parameters for each corrected anomaly type; Based on the corrected anomaly type, the quantization parameters, and the optimized confidence level, a structured bolt head anomaly identification result is generated. The bolt head anomaly identification result includes the anomaly type, quantization parameters, confidence level, and associated frontal image block identifier. Perform tooth feature recognition on the side image block and output tooth recognition results including anomaly type, quantization parameters, and confidence level.

4. The method according to claim 3, characterized in that, The calculation of quantization parameters for each corrected anomaly type includes: If the anomaly in the independent sub-image is corrosion, the independent sub-image is converted from RGB format to HSV format to generate a corrosion binary mask image; The percentage of rusted area is determined by comparing the number of rusted pixels in the rusted binary mask image with the total number of pixels in the sub-image. If the anomaly in the independent sub-image is a crack, a preset skeleton extraction algorithm is used to obtain a crack skeleton line with a width of 1 pixel in the independent sub-image. The pixel length of the crack skeleton line is calculated using the arcLength function; The crack length is obtained based on the pixel length of the crack skeleton line and the image scale. If the anomaly in the independent subgraph is a deformation, extract the actual head contour from the independent subgraph; The actual outline of the head is compared with the preset standard outline of the same specification bolt, and the outline similarity is calculated. The shape deviation rate is determined based on the contour similarity. If the anomaly in the independent sub-image is a missing pixel, the number of pixels in the missing region of the independent sub-image is counted and then compared with the total number of pixels in the head-focused sub-image. The percentage of the missing area is calculated based on the number of pixels in the missing region and the total number of pixels.

5. The method according to claim 3, characterized in that, The process of performing serration feature recognition on the side image block outputs a serration recognition result containing anomaly type, quantization parameters, and confidence level, including: The Canny edge detection algorithm is used to extract the edge lines of the threaded parts; Effective wire contour lines are selected from the wire edge lines based on the periodic characteristics of the wires. The effective thread contour lines are clustered in parallel using the Hough transform algorithm to determine the upper boundary of the thread as the pixel coordinates corresponding to 10mm below the bolt head and the lower boundary of the thread as the pixel coordinates corresponding to the last continuous thread contour line, and the effective thread area is obtained by cropping. Ten sampling lines are uniformly arranged within the effective tooth area along the tooth axis, wherein the sampling line spacing = effective tooth area height ÷ 10; The number of thread peaks for each sampling line is counted by the peak detection algorithm. The average number of peaks for 10 sampling lines is calculated and rounded to obtain the number of protruding threads. If the number of peaks on a certain sampling line deviates from the average value by more than a threshold, then the thread deformation in that area is determined and the corresponding sampling line coordinates are marked. An anomaly type is generated based on the number of protruding threads and the determination result of thread deformation; Determine the corresponding quantization parameters based on the anomaly type; Confidence level is calculated based on the consistency of the number of peaks; A structured fiber identification result is generated based on the anomaly type, the quantization parameter, and the confidence level.

6. The method according to claim 5, characterized in that, The identification results include the bolt head anomaly identification results and the thread identification results. The identification results are graded based on the confidence level, and a judgment result is output, including: If the confidence level of the bolt head anomaly identification result is ≥0.8 and the anomaly type matches the quantification parameters, then the bolt head anomaly identification result is used as the local inference result of the head; if the confidence level is 0.5-0.8, the head focus subgraph and the bolt head anomaly identification result are uploaded to the cloud for secondary inference; if the confidence level is <0.5, it is marked as a head that needs manual review. If the confidence level of the fiber identification result is ≥0.8 and the anomaly type matches the quantization parameters, the fiber identification result is used as the local fiber inference result; if the confidence level of the fiber identification result is 0.5-0.8, the valid fiber region and the fiber identification result are uploaded to the cloud for secondary inference; if the confidence level of the fiber identification result is <0.5, it is marked as a fiber requiring manual review. Based on the local inference results of the head, the local inference results of the thread, the secondary inference results in the cloud, and the manual review results, a determination result of the bolt appearance is generated.

7. The method according to claim 6, characterized in that, The cloud-based secondary inference includes: The cloud retrieves historical thread detection data of the bolts from a preset database. Compare the current number of protruding threads of the bolt with the difference between the historical number of protruding threads in the historical thread detection data; If the difference in change is ≥2 teeth, then add the "risk of loosening of the filament" label to the determination result.

8. The method according to any one of claims 1-7, characterized in that, Based on the determination result, an alarm message is output and pushed to the management personnel's terminal device, including: If the anomaly type and corresponding quantification parameter in the judgment result match the preset level 1 alarm level, a structured high-risk alarm information is generated. The high-risk alarm information includes bolt ID, inspection time, anomaly type, quantification parameter, emergency handling suggestion, and URL of associated image. If the anomaly type and corresponding quantification parameter in the judgment result match the preset level 2 alarm level, a structured medium-risk alarm information is generated. The medium-risk alarm information includes bolt ID, inspection time, anomaly type, quantification parameter, planned processing suggestion, and URL of associated image. If the anomaly type and corresponding quantification parameters in the judgment result match the preset three-level alarm level, a structured low-risk alarm information is generated. The low-risk alarm information includes bolt ID, inspection time, anomaly type, quantification parameters, and routine handling suggestions.

9. The method according to claim 8, characterized in that, The preset level 1 alarm level satisfies: The degree of corrosion is greater than 30%, the length of the crack is greater than 0.5 mm, the shape deviation rate of the deformation is greater than 5%, the bolt is completely missing, and the number of threads is greater than 2. The preset level 2 alarm level satisfies: The degree of corrosion is >10% and <30%, the length of the crack is <0.5mm, the shape deviation rate of the deformation is ≤5%, the bolt head is damaged, and the number of threads is 1-2 threads off. The preset three-level alarm levels satisfy the following: The degree of corrosion is less than 10%, the number of threads is off by 1 thread and there is no deformation.

10. A tower bolt anomaly identification device, characterized in that, The device includes: The image acquisition unit is used to acquire front and side images of each tower bolt using an image acquisition device mounted on the tower inspection robot. The front image focuses on the bolt head and the connection between the bolt and the tower, while the side image focuses on the threaded portion of the bolt. The preprocessing unit is used to perform preprocessing operations on the front image and the side image respectively to obtain a front image block corresponding to the front image and a side image block corresponding to the side image; The recognition unit is used to perform abnormal region recognition on the frontal image block and to perform tooth feature recognition on the side image block, and output recognition results including abnormality type, quantization parameters and confidence level respectively. The determination unit is used to classify the identification result according to the confidence level in the identification result and output the determination result. An abnormal alarm unit is used to associate and store the judgment result with the bolt ID and inspection time, and output alarm information based on the judgment result and push it to the management personnel terminal device.

Images