Bolt loosening detection method and system based on unmanned aerial vehicle and multi-modal analysis

Abstract

This invention discloses a bolt loosening detection method and system based on UAVs and multimodal analysis, belonging to the field of structural health monitoring technology. It includes acquiring structural data of towers to construct a three-dimensional virtual mapping and performing simulation to dynamically generate inspection tracks for the UAV; the UAV simultaneously collects high-definition video and sound field data of the bolts based on the tracks, forming a multimodal data stream; calculations are performed by fusing visual and acoustic features to output quantitative diagnostic results; an active verification process is initiated for abnormal or poorly quality results to obtain supplementary data for correction and form a final diagnosis; a detection report is generated based on the final diagnostic data, constructing a full lifecycle health record. This invention adopts a technical solution combining three-dimensional task planning, multimodal fusion diagnosis, active verification and correction, and full lifecycle management, which can improve detection accuracy and efficiency while achieving dynamic assessment and trend prediction of the health status of bolt components.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring technology, and in particular to a bolt loosening detection method and system based on unmanned aerial vehicles and multimodal analysis. Background Technology

[0002] High-voltage transmission towers, bridges, wind turbines, and other large-scale infrastructure are critical nodes in transportation networks, and their structural safety is paramount. These structures are typically composed of numerous steel components connected by high-strength bolts. During long-term service, these bolts can loosen due to factors such as vibration, temperature changes, and corrosion, potentially leading to structural connection failure and major safety accidents. Therefore, regular and accurate inspection of the tightness of critical bolts is an essential step in ensuring the safe operation of large-scale infrastructure.

[0003] Currently, the main methods for detecting loose bolts include manual inspection from towers and inspection using drones. Manual inspection relies on workers climbing to high places to use torque wrenches or visual inspection, which is not only inefficient and labor-intensive but also poses extremely high personal safety risks. Drone inspection, to some extent, replaces manual high-altitude work. It mainly uses visible light cameras to take visual images, which are then interpreted manually or by simple image recognition algorithms to check for obvious defects such as missing bolts or loose bolts. Some existing technologies also attempt to use single sensors, such as thermal imagers or acoustic sensors, for auxiliary judgment.

[0004] However, the aforementioned existing technologies have significant limitations. UAV-based visible light inspection methods struggle to identify minute loosening in its early stages, and their results are highly susceptible to interference from lighting conditions, shooting angle, and surface contaminants, leading to high rates of missed detections and false alarms. Single-sensor detection methods, such as simple acoustic analysis, are also easily affected by ambient background noise, making it difficult to arrive at reliable diagnostic conclusions. Furthermore, existing UAV inspections often employ pre-set fixed routes, lacking refined planning for optimal bolt observation posture, resulting in inconsistent data quality and a lack of mechanisms for online verification of abnormal results, making it difficult to guarantee the reliability of the inspection. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a bolt loosening detection method and system based on unmanned aerial vehicles (UAVs) and multimodal analysis. It employs a technical solution that combines three-dimensional task planning, multimodal fusion diagnosis, proactive verification and correction, and full lifecycle management. This approach can improve detection accuracy and efficiency while enabling dynamic assessment and trend prediction of the health status of bolt components.

[0006] The above objectives can be achieved through the following approach:

[0007] A bolt loosening detection method based on UAVs and multimodal analysis includes acquiring structured data of the tower to be inspected, constructing a virtual mapping in three-dimensional space, performing detection simulation within the virtual mapping, analyzing the spatial distribution and optimal observation posture of key bolt nodes, and comprehensively considering environmental constraints and sensor parameters to dynamically plan and generate a three-dimensional inspection trajectory and sensor action sequence for the UAV. Based on the three-dimensional inspection trajectory and the sensor action sequence, the UAV drives its onboard high-definition camera and acoustic signature acquisition device to perform multi-angle detection on each bolt node, simultaneously acquiring high-definition video streams and spatial sound field data with precise spatiotemporal stamp associations, forming a spatiotemporally continuous multimodal data stream and transmitting it in real time. The method also involves receiving the multimodal data stream and extracting the geometric properties of the bolts. The system extracts contour and texture details, and extracts the frequency domain distribution characteristics of vibration energy. Cross-attention calculation and fusion are then performed to output a quantitative diagnostic result of bolt loosening status. The system receives the quantitative diagnostic result and performs online diagnosis. If the result shows an anomaly, it autonomously generates a secondary detailed inspection command and drives the drone back to the target location, performing zoom, hovering, and orbiting actions to collect supplementary data. Based on the supplementary data, the quantitative diagnostic result is iteratively corrected to form final diagnostic data that is verified online. Based on the final diagnostic data, the system automatically integrates the multimodal data stream to generate a legally valid encrypted inspection report, which is then uploaded to the cloud for archiving, constructing a full lifecycle health record for each bolt component of the tower.

[0008] Optionally, the step of dynamically planning and generating a 3D inspection trajectory and sensor action sequence for the UAV includes: importing the 3D scanned point cloud data of the tower to be inspected, reconstructing its geometric structure, and generating a virtual tower model containing the 3D coordinates of all key bolt nodes; using the bolt nodes in the virtual tower model as targets, performing a comprehensive virtual viewpoint traversal calculation, eliminating invalid viewpoints with self-occlusion of tower components, and selecting and determining the optimal observation posture for data acquisition for each node; acquiring and fusing real-time meteorological data of the UAV's current location, surrounding obstacle information, and the UAV's own dynamics and sensor performance parameters to construct a dynamic constraint region in 3D space; using the global inspection flight time and total energy consumption as optimization objectives, and under the premise of avoiding the constraint region, performing path planning calculation to generate a 3D inspection trajectory, and simultaneously generating a sensor action sequence.

[0009] Optionally, the formation and real-time transmission of a spatiotemporally continuous multimodal data stream includes: the UAV controlling its position and speed based on the three-dimensional inspection trajectory, driving the gimbal according to the sensor action sequence, and adjusting the pointing of the high-definition camera and acoustic signature acquisition device in real time to ensure that the detection axis is precisely aligned with each bolt node when it arrives at each bolt node; using an onboard unified clock source as a reference, acquiring high-definition video images and spatial sound field data, and attaching a spatiotemporal stamp containing three-dimensional coordinates and attitude information, performing multiplexing processing and encapsulating it into a standard format multimodal data stream, and transmitting it continuously in real time to ground analysis equipment through an onboard wireless data link.

[0010] Optionally, the multimodal data stream includes: a high-definition spatiotemporal video sequence and synchronous spatial sound field data, wherein: the high-definition spatiotemporal video sequence is used to provide visual appearance information of the bolt and surrounding components, and to extract geometric contours, relative positions and surface texture details that characterize the fastening state; the synchronous spatial sound field data is used to provide mechanical vibration information of the bolt under environmental excitation, and to extract the energy distribution characteristics in the frequency domain caused by structural micro-motion and wind-induced vibration.

[0011] Optionally, the quantitative diagnostic result of the output bolt loosening state includes: processing the high-definition spatiotemporal video sequence in the multimodal data stream, identifying and segmenting independent bolt target regions in the video frames, and calculating and extracting visual state features; based on the multimodal data stream and the video frames, processing is performed in strict synchronization at spatiotemporal stamps, and the data is converted to the frequency domain through time-frequency analysis to calculate and extract acoustic features; using the visual state features and the acoustic features, the correlation and confidence weight between different modal features are evaluated through cross-attention, and a weighted fusion calculation is performed to output the quantitative diagnostic result.

[0012] Optionally, the method further includes: parsing the three-dimensional attitude information of the UAV when collecting each segment of data from the spatiotemporal stamps of the multimodal data stream; comparing the optimal observation attitude with the three-dimensional attitude information in real time, calculating the spatial deviation, and generating a dynamic quality control factor characterizing the execution accuracy and data quality of the inspection flight.

[0013] Optionally, the final diagnostic data for online verification includes: receiving the quantitative diagnostic results and performing a diagnosis based on the dynamic quality control factor; if the diagnostic results show an anomaly, it is determined that a second detailed investigation needs to be initiated; for the bolt nodes of the second detailed investigation, autonomously planning and generating a detailed investigation trajectory to guide the UAV to accurately return to the target position, and simultaneously generating supplementary data collection instructions; driving the UAV to execute the detailed investigation trajectory and the supplementary data collection instructions, supplementing multimodal data and analyzing it again, and performing confidence-weighted fusion of the analysis results and the quantitative diagnostic results to output the final diagnostic data for online verification.

[0014] Optionally, the step of generating a detailed survey track to guide the UAV to accurately return to the target position and simultaneously generating supplementary data acquisition instructions includes: taking the current three-dimensional position of the UAV as the starting point, the three-dimensional coordinates of the bolt nodes in the secondary detailed survey as the ending point, and taking into account the constraint area in three-dimensional space, performing real-time optimization calculation of the local path to generate a detailed survey track to guide the UAV to safely return to the target position; and based on the bolt node positions in the secondary detailed survey, performing refined detection actions of hovering and shooting around the optimal observation attitude as supplementary data acquisition instructions.

[0015] Optionally, the construction of a full life-cycle health record for each bolt component of the tower includes: creating a unique identification code for each key bolt node of the tower to be inspected, and archiving the final diagnostic data, the encrypted inspection report, and the corresponding spatiotemporal stamp information of each inspection into a time-series database under the identification code; performing longitudinal regression analysis on the historical diagnostic data under the same identification code in the time-series database, calculating and extracting the characteristics of the bolt component loosening state and risk level changing over time, and establishing a health status decline trend curve; and performing extrapolation prediction based on the decline trend curve to construct a full life-cycle health record for each bolt component of the tower.

[0016] Based on the same inventive concept, this invention also provides a bolt loosening detection system based on UAV and multimodal analysis. The system includes: a three-dimensional task planning module, used to acquire structured data of the tower to be inspected, construct a virtual mapping in three-dimensional space, and perform detection simulation within the virtual mapping; analyze the spatial distribution and optimal observation posture of key bolt nodes; comprehensively consider environmental constraints and sensor parameters to dynamically plan and generate a three-dimensional inspection trajectory and sensor action sequence for the UAV; a multimodal data acquisition module, used by the UAV to drive its onboard high-definition camera and acoustic signature acquisition device based on the three-dimensional inspection trajectory and the sensor action sequence, to perform multi-angle detection of each bolt node, simultaneously acquiring high-definition video streams and spatial sound field data with precise spatiotemporal stamp associations, forming a spatiotemporally continuous multimodal data stream and transmitting it in real time; and a multimodal fusion diagnostic module, used to receive... The multimodal data stream extracts the geometric contour and texture details of the bolts, and extracts the frequency domain distribution characteristics of vibration energy. It then performs cross-attention calculation and fusion to output a quantitative diagnostic result of the bolt loosening state. An active verification and correction module receives the quantitative diagnostic result and performs online diagnosis. If the diagnostic result shows an anomaly, it autonomously generates a secondary detailed inspection command and drives the drone back to the target location, performing zoom, hovering, and orbiting actions to collect supplementary data. Based on the supplementary data, it iteratively corrects the quantitative diagnostic result to form the final online verified diagnostic data. A full lifecycle management module automatically integrates the multimodal data stream based on the final diagnostic data, generates a legally valid encrypted inspection report, and uploads the encrypted inspection report to the cloud for archiving, constructing a full lifecycle health record for each bolt component of the tower.

[0017] Compared with the prior art, the present invention has the following advantages: 1. A closed-loop design combining multimodal data fusion and active verification correction improves the accuracy and reliability of bolt loosening detection. This method uses cross-attention fusion to combine visual features that provide static appearance information with acoustic features that reflect dynamic response, enabling complementary assessment of bolt condition. This overcomes the limitations of single sensors, which are susceptible to interference from environmental factors such as lighting and noise. Furthermore, by conducting secondary detailed examination and iterative correction of anomaly diagnosis results, the stability and reliability of the final diagnostic conclusion are ensured. 2. It achieves a high degree of automation and intelligence throughout the entire inspection process, improving operational efficiency and safety. Through 3D virtual simulation and dynamic path planning, it can autonomously generate optimized inspection trajectories and sensor action sequences, avoiding the complexity and blind spots of manual planning and reducing unnecessary flight and data acquisition time. The entire inspection, diagnosis, and verification process requires no manual intervention, reducing reliance on operators' professional skills and mitigating the risks of manual high-altitude operations. 3. By constructing a full lifecycle health record, the detection process is elevated from a single static assessment to a long-term, dynamic predictive management level. It not only provides the current bolt tightness but also establishes a health status degradation trend curve through longitudinal regression analysis of historical data, thereby predicting future performance degradation of components. This forward-looking analytical capability provides data support for developing scientific preventative maintenance strategies, helping to optimize resource allocation and prevent problems before they occur.

[0018] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the bolt loosening detection method based on UAV and multimodal analysis according to an embodiment of the present invention.

[0021] Figure 2 This is a comparison chart of the multimodal diagnostic results of an embodiment of the present invention.

[0022] Figure 3 This is a diagram illustrating the evolution of bolt lifecycle risk composition according to an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the bolt loosening detection system based on UAV and multimodal analysis according to an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Reference Figure 1One embodiment of the present invention proposes a bolt loosening detection method based on UAV and multimodal analysis. It adopts a technical solution that combines three-dimensional task planning, multimodal fusion diagnosis, active verification and correction and full life cycle management, which can improve the accuracy and efficiency of detection while realizing dynamic assessment and trend prediction of the health status of bolt components.

[0026] The method described in this embodiment specifically includes: The structured data of the tower to be inspected is acquired, a virtual mapping in three-dimensional space is constructed, and detection simulation is performed in the virtual mapping. The spatial distribution and optimal observation attitude of key bolt nodes are analyzed. Environmental constraints and sensor parameters are comprehensively considered to dynamically plan and generate three-dimensional inspection tracks and sensor action sequences for UAVs. Based on the three-dimensional inspection trajectory and the sensor action sequence, the UAV drives the onboard high-definition camera and voiceprint acquisition device to perform multi-angle detection on each bolt node, and simultaneously collects high-definition video streams and spatial sound field data with precise time and space stamps, forming a spatiotemporally continuous multimodal data stream and transmitting it in real time. The system receives the multimodal data stream, extracts the geometric contour and texture details of the bolt, extracts the distribution features of vibration energy in the frequency domain, performs cross-attention calculation and fusion, and outputs the quantitative diagnostic results of the bolt loosening state. The system receives the quantitative diagnostic results and performs online diagnosis. If the diagnostic results show an anomaly, it autonomously generates a secondary detailed investigation command and drives the UAV back to the target position. It performs zoom, hovering, and orbiting actions to collect supplementary data. Based on the supplementary data, it iteratively corrects the quantitative diagnostic results to form the final diagnostic data for online verification. Based on the final diagnostic data, the multimodal data stream is automatically integrated to generate a legally valid encrypted test report, which is then uploaded to the cloud for archiving, thus constructing a full lifecycle health record for each bolt component of the tower.

[0027] By adopting a technical solution that combines three-dimensional task planning, multimodal fusion diagnosis, active verification and correction, and full life cycle management, it is possible to improve the accuracy and efficiency of detection while realizing dynamic assessment and trend prediction of the health status of bolt components.

[0028] Optionally, the dynamic planning and generation of 3D inspection tracks and sensor action sequences for the UAV includes: Import the 3D scan point cloud data of the tower to be inspected, perform geometric structure reconstruction, and generate a virtual tower model containing the 3D coordinates of all key bolt nodes; Specifically, a drone equipped with a LiDAR scanner or a high-definition camera is used to scan the tower under inspection by circling it or performing oblique photogrammetry to acquire high-density 3D point cloud data. After receiving this 3D point cloud data, geometric reconstruction is performed to generate a virtual tower model that is highly consistent with the physical entity in size and structure. Subsequently, 3D target detection is used on this model to automatically identify and mark the positions of all key bolt nodes, outputting their precise 3D coordinates in a unified coordinate system.

[0029] Using the bolt nodes in the virtual tower model as targets, a full-range virtual viewpoint traversal calculation is performed to eliminate invalid viewpoints that are self-occluded by tower components, and the optimal observation posture for data acquisition is selected and determined for each node.

[0030] Specifically, for each critical bolt node in the virtual tower model, a series of discrete candidate observation viewpoints are generated on a virtual sphere with a predetermined detection distance as the center. For each candidate observation viewpoint, the sensor line of sight is simulated through ray casting calculation; that is, a virtual ray is emitted from the candidate viewpoint to the bolt node. If the ray path intersects with any other part of the virtual tower model, the viewpoint is considered to be occluded, invalid, and discarded. Among all the remaining valid viewpoints, multiple factors such as observation angle, expected imaging resolution, and lighting simulation effect are comprehensively considered and scored. The viewpoint with the highest score is determined as the optimal observation attitude for that node. This attitude information includes the UAV's three-dimensional spatial position and the sensor's orientation.

[0031] By acquiring and fusing real-time meteorological data of the drone's current location, information on surrounding obstacles, and the drone's own dynamics and sensor performance parameters, a dynamic constraint region is constructed in three-dimensional space.

[0032] Specifically, the system obtains real-time meteorological data, such as wind speed and direction, from authoritative meteorological services regarding the drone's operating area via a network interface. It also retrieves pre-set digital elevation maps that include surrounding fixed obstacles such as tall buildings and trees. Furthermore, the drone itself provides dynamic and performance parameters, including its maximum flight speed, minimum turning radius, and remaining range at its current battery level. All this information is fused and mapped into a three-dimensional space. For example, a safety radius envelope is generated around obstacles, and areas with excessively high wind speeds are marked as high-cost areas. Together, these constitute a dynamically updated set of flight constraint areas used for path planning.

[0033] With the optimization objectives of global inspection flight time and total energy consumption, and under the premise of avoiding the constrained area, a three-dimensional inspection trajectory is calculated by path planning, and a sensor action sequence is generated simultaneously.

[0034] Specifically, path planning treats the optimal observation attitude of all key bolt nodes as mandatory waypoints. The algorithm optimizes under a joint cost-driven framework, which integrates total flight time and total energy consumption. For example, the importance of these two metrics can be adjusted by setting weighting coefficients; time weight can be increased when the task is urgent, while energy consumption weight can be increased when the inspection distance is long and battery life is the primary constraint. During the solution process, the algorithm completely avoids the flight constraint region constructed in the previous step, ultimately calculating a smooth, collision-free 3D inspection track connecting all waypoints. Simultaneously, an action sequence generator matches specific sensor actions for each waypoint on this track, such as adjusting gimbal pitch angle, setting camera zoom parameters, and enabling or disabling acoustic signature acquisition. This set of instructions forms a sensor action sequence strictly synchronized with the 3D inspection track.

[0035] Optionally, the process of forming a spatiotemporally continuous multimodal data stream and transmitting it in real time includes: The UAV controls its position and speed based on the three-dimensional inspection trajectory, drives the gimbal according to the sensor action sequence, and adjusts the direction of the high-definition camera and acoustic fingerprint acquisition device in real time to ensure that the detection axis is accurately aligned with each bolt node when it reaches the node. Specifically, the flight controller integrated on the UAV receives the 3D inspection track generated in the previous stage and decomposes it into a series of commands containing the desired position, velocity, and attitude. This flight controller integrates real-time feedback from the onboard inertial measurement unit and GPS receiver, compares it with the desired values ​​in the commands, and adjusts the output of each motor through closed-loop control to control the UAV's position and velocity, ensuring it strictly follows the 3D inspection track. Simultaneously, it analyzes the sensor action sequence, sending the defined gimbal attitude angle, camera focal length, and other commands to the gimbal's drive mechanism to control the pointing of the high-definition camera and acoustic signature acquisition device. This ensures that when the UAV reaches the position corresponding to the optimal observation attitude, the sensor's detection axis is aligned with the target bolt node.

[0036] Based on the airborne unified clock source, high-definition video images and spatial sound field data are collected, and spatiotemporal stamps containing three-dimensional coordinates and attitude information are added. The data are then multiplexed and encapsulated into a standard format multimodal data stream, which is continuously transmitted to ground analysis equipment in real time via the airborne wireless data link.

[0037] Specifically, the data acquisition and processing process uses an onboard unified high-precision clock source as the time reference to ensure absolute time accuracy. When the high-definition camera captures each frame of video image and the acoustic signature acquisition device records each audio data segment, a timestamp is immediately obtained from the clock source, and the UAV's three-dimensional coordinates and attitude information at that moment are simultaneously obtained from the flight controller. This information is then combined into a complete spatiotemporal stamp. Subsequently, the high-definition video images and spatial sound field data with accompanying spatiotemporal stamps are multiplexed. This process can employ techniques such as time-division multiplexing to encapsulate the visual and acoustic data and their corresponding spatiotemporal stamps together into data packets conforming to standard streaming media protocol specifications, forming a spatiotemporally continuous multimodal data stream. Finally, this encapsulated multimodal data stream is continuously transmitted to ground analysis equipment via an onboard wireless data link, such as 5G mobile communication technology or dedicated microwave, in real-time broadcast mode.

[0038] Optionally, the multimodal data stream includes: a high-definition spatiotemporal video sequence and synchronized spatial sound field data, wherein: The high-definition spatiotemporal video sequence is used to provide visual appearance information of the bolt and surrounding components, and to extract geometric contours, relative positions and surface texture details that characterize the fastening state.

[0039] Specifically, this high-definition spatiotemporal video sequence was continuously captured during flight by a high-definition camera mounted on a drone, providing detailed visual appearance information of the bolt and its connecting components. From this video sequence, various visual features characterizing the bolt's tightness were extracted. For example, edge detection and contour matching allowed for precise calculation of the geometric contours of the bolt head, nut, and washer, which were then compared with digital templates of standard parts to identify deformation. Feature point detection and tracking determined the precise relative position of the nut to the bolt, quantifying its withdrawal distance or rotation angle. Texture analysis identified surface details such as rust and metal powder caused by fretting friction, all of which serve as visual evidence of bolt looseness.

[0040] The synchronous spatial sound field data is used to provide mechanical vibration information of bolts under environmental excitation and to extract the energy distribution characteristics in the frequency domain caused by structural fretting and wind-induced vibration.

[0041] Specifically, this synchronized spatial sound field data is collected by an acoustic signature acquisition device that works in sync with a high-definition camera to capture the weak mechanical vibration signals of bolts under environmental excitations such as wind-induced vibration. Since the tightness of a bolt directly affects the local stiffness and damping characteristics of its connection, a loose bolt will exhibit a significantly different vibration response compared to its tightened state. For effective analysis, the acquired time-domain acoustic signal is converted to the frequency domain using a fast Fourier transform time-frequency analysis method. This process reveals the inherent frequency composition of the signal. Based on this, the energy distribution characteristics in the frequency domain can be further calculated and extracted, such as looking for abnormal low-frequency resonance peaks or whether the vibration energy within a specific frequency band exceeds the normal range. These frequency domain characteristics are strong acoustic evidence for judging bolt looseness.

[0042] Optionally, the quantitative diagnostic results of the output bolt loosening status include: The high-definition spatiotemporal video sequence in the multimodal data stream is processed to identify and segment independent bolt target regions in the video frames, and to calculate and extract visual state features. Specifically, the system receives high-definition spatiotemporal video sequences from a multimodal data stream and processes the video frames. Independent bolt target regions are automatically identified and precisely segmented within the video frames, thus separating them from the complex background environment. Next, within the segmented bolt target regions, visual state features are calculated and extracted. These features may include the nut's withdrawal height, the number of exposed thread turns, the gap width between the washer and the contact surface, and the presence of texture anomalies such as rust or metal powder in the bolt connection area. These calculated features are combined into a high-dimensional vector as the visual state feature.

[0043] Based on the multimodal data stream and the video frame, the processing is strictly synchronized on the spatiotemporal stamp, and the data is converted to the frequency domain through time-frequency analysis to calculate and extract acoustic features.

[0044] Specifically, for each video frame being analyzed, the corresponding time segment is located in the synchronized spatial sound field data based on its associated spatiotemporal stamp. Then, time-frequency analysis techniques, such as short-time Fourier transform or wavelet transform, are applied to this acoustic signal to transform it from the time domain to the frequency domain, generating a spectrum that reflects the changes in the signal's frequency components over time. Based on this, a series of acoustic features are calculated and extracted from this spectrum, such as the energy concentration within a specific frequency band, the peak value and offset of the principal resonant frequency, and the harmonic structure characteristics of the spectrum. These acoustic features are also organized into a high-dimensional vector.

[0045] Using the visual state features and the acoustic features, the correlation and confidence weights between different modal features are evaluated through cross-attention, and a weighted fusion calculation is performed to output a quantitative diagnostic result.

[0046] Specifically, visual state feature vectors and acoustic feature vectors are used to dynamically assess the interrelationship and relative importance between the two modalities. In this scenario, instead of simply concatenating or weighting the two features in a fixed ratio, the visual features are directed to "focus" on the most relevant parts of the acoustic features, and vice versa, thus dynamically calculating the confidence weight of each modality in the current diagnosis. For example, when the visual state features show obvious rust, the mechanism increases the attention weight to the high-frequency components of the acoustic features representing micro-friction. Finally, the dynamically weighted features are deeply fused, and a numerical value or risk level score representing the probability of loosening is output as the final quantitative diagnostic result.

[0047] Optionally, the method further includes: The three-dimensional attitude information of the UAV when collecting each segment of data is parsed from the spatiotemporal stamps of the multimodal data stream; Specifically, it receives multimodal data streams transmitted from the UAV in real time. For each data packet in the data stream, it parses the attached spatiotemporal stamp and extracts the actual three-dimensional attitude information of the UAV when it collected that segment of data. This information is usually calculated by fusing the onboard GPS and inertial measurement equipment and includes the UAV's real-time attitude angles.

[0048] The optimal observation attitude is compared with the three-dimensional attitude information in real time to calculate the spatial deviation and generate a dynamic quality control factor that characterizes the accuracy of the inspection flight and the data quality.

[0049] Specifically, two sets of data are compared synchronously in real time: one set is the optimal observation attitude previously determined for the bolt node during the planning phase, and the other set is the actual 3D attitude information of the UAV parsed from the spatiotemporal stamp at this moment. Attitude deviation is quantified by calculating the angular distance between the two attitude sets, for example, by applying quaternion arithmetic to obtain the unique minimum rotation angle. The calculated deviation value is the spatial deviation. Finally, the spatial deviation is converted into a normalized dynamic quality control factor. A smaller spatial deviation is mapped to a factor value close to 1, indicating high flight execution accuracy and reliable data quality; conversely, a larger spatial deviation is mapped to a factor value close to 0, indicating that data quality may be degraded due to flight disturbances.

[0050] Optionally, the final diagnostic data forming the online verification includes: Receive the quantitative diagnostic results and perform a diagnosis based on the dynamic quality control factors. If the diagnostic results show an abnormality, it is determined that a second detailed investigation needs to be initiated. Specifically, the system receives the quantitative diagnostic results from the previous stage and simultaneously acquires the corresponding dynamic quality control factor. A decision-making logic determines whether to initiate a secondary detailed investigation. This logic includes two parallel conditions: first, whether the risk level indicated by the quantitative diagnostic result is higher than a predetermined alarm threshold; second, whether the value of the dynamic quality control factor is lower than a predetermined data reliability threshold. When at least one of the two conditions is met, i.e., the diagnostic result itself is abnormal, the decision function determines that a secondary detailed investigation process needs to be initiated. The alarm threshold here is set according to the safety operation and maintenance specifications for power equipment; the data reliability threshold can be determined through calibration experiments to establish the relationship between the dynamic quality control factor value and the final diagnostic accuracy.

[0051] For the bolt nodes identified in the secondary detailed investigation, the system autonomously plans and generates a detailed investigation trajectory to guide the UAV back to the target location precisely, and simultaneously generates supplementary data collection instructions. Specifically, once a secondary detailed investigation is initiated, a local, real-time path optimization calculation is performed for the specific bolt node. This calculation starts from the UAV's current real-time 3D position and ends at the known precise 3D coordinates of the target bolt node, again taking into account the dynamically constrained region already constructed in 3D space, thereby generating a detailed investigation track that guides the UAV safely and efficiently back to the target position. Simultaneously, a set of supplementary data acquisition instructions is created. This instruction set specifies the refined detection actions that the UAV needs to perform after arriving at the target position, such as increasing the optical zoom to capture clearer surface texture details or extending the hovering time to collect more stable spatial sound field data.

[0052] The drone is driven to execute the detailed flight path and the supplementary data collection instructions, supplement multimodal data and analyze it again, and the analysis results are fused with the quantitative diagnostic results by confidence weighting to output the final diagnostic data for online verification.

[0053] Specifically, the UAV strictly executes the detailed flight path investigation and supplementary data collection instructions to obtain a new, higher-quality supplementary multimodal data set. This supplementary multimodal data undergoes the same analysis process to generate supplementary analysis results. Then, this supplementary analysis result is fused with the original quantitative diagnostic results using a confidence-weighted method to output the final diagnostic data for online verification. This fusion calculation can be expressed by the following formula: .

[0054] in, The final diagnostic data is the final quantitative result characterizing the state of bolt loosening after online verification and correction. This is the original quantitative diagnostic result; To supplement the analysis results; The weight of the original diagnostic result is determined by the fact that it is positively correlated with the dynamic quality control factor corresponding to the original data. The higher the flight execution accuracy, the larger the value of the dynamic quality control factor, and the larger the weight. To supplement the weighting of the analysis results, its value is typically set to a fixed value because it originates from refined data collection under controlled conditions, ensuring high reliability. Through this weighted fusion method directly linked to data collection quality, quantitative correction of the original diagnostic results is achieved, ultimately resulting in more reliable online-verified final diagnostic data, such as... Figure 2 As shown, the comparison between visual diagnostic results, acoustic diagnostic results, and the final multimodal fusion result is presented, and alarm thresholds are marked, demonstrating the role of multimodal fusion in improving diagnostic confidence.

[0055] Optionally, the process of generating a detailed flight path that guides the UAV to accurately return to the target location, and simultaneously generating supplementary data collection instructions, includes: Starting from the current three-dimensional position of the UAV and ending at the three-dimensional coordinates of the bolt nodes obtained from the secondary detailed investigation, and taking into account the constraint area in three-dimensional space, a local path optimization calculation is performed in real time to generate a detailed investigation track that guides the UAV to safely return to the target position. Specifically, after determining that a secondary detailed inspection of a certain bolt node is necessary, the current 3D position reported in real time by the UAV's onboard navigation equipment is used as the starting point, and the precise 3D coordinates of the bolt node are retrieved as the ending point. Simultaneously, the dynamic constraint region in the current 3D space is revisited and updated to ensure that the planned path can avoid the latest adverse weather conditions or temporary obstacles. Based on this, a safe and smooth trajectory connecting the starting and ending points is quickly calculated; this trajectory is the detailed inspection track that guides the UAV safely back to the target position.

[0056] Based on the bolt node positions obtained from the secondary detailed investigation, a refined detection action involving hovering and shooting is performed around the optimal observation posture as a supplementary data acquisition command.

[0057] Specifically, supplementary data acquisition instructions are planned synchronously. This planning is based on the bolt node location requiring secondary detailed investigation and closely follows the optimal observation attitude determined for that node during the initial planning. A series of refined detection actions are generated, which are encoded into a time-series instruction. For example, the instruction may first guide the UAV to perform precise hovering and shooting at the optimal observation attitude point to obtain shake-free video and low-noise sound field data; subsequently, it may include a zoom instruction to drive the high-definition camera to perform optical zoom to capture minute details such as bolt threads and gasket gaps at maximum resolution; in addition, it may include a circling flight instruction centered on the bolt node, driving the UAV to perform small-range arc flight to collect data from multiple compensated angles to eliminate potential reflection or occlusion problems from a single observation. This series of action instructions together constitutes the supplementary data acquisition instruction.

[0058] Optionally, the full life-cycle health records of each bolt component of the tower construction include: A unique identification code is created for each key bolt node of the tower to be inspected, and the final diagnostic data, the encrypted inspection report and the corresponding spatiotemporal stamp information of each inspection are archived into the time-series database under the identification code; Specifically, the process begins by creating a unique identification code for each critical bolt node of the tower to be inspected. This identification code can be composed of information such as the tower's geographic location code, structural location number, and bolt serial number, ensuring its unique traceability throughout its entire lifecycle. After each inspection task is completed, the final diagnostic data of the bolt node, verified online, along with the automatically generated encrypted inspection report and the precise spatiotemporal stamp information corresponding to the data collection, are archived as a complete data record in a dedicated time-series database. This time-series database uses this identification code as the primary key to efficiently store and retrieve the status information of the bolt throughout its history in chronological order.

[0059] A longitudinal regression analysis is performed on the historical diagnostic data under the same identity code in the time series database to calculate and extract the characteristics of the changes in the loosening state and risk level of bolt components over time, and to establish a decline trend curve of the health status.

[0060] Specifically, after a time-series database accumulates multiple historical diagnostic data under the same identification code, the repeated observation data of the same object at different time points are analyzed. Using the quantitative diagnostic results of the bolt component as the dependent variable and the detection time as the independent variable, a function is fitted to the historical data points to calculate and extract the quantitative characteristics of the bolt's health status changing over time, such as the rate of degradation. Through this analysis, a mathematical model can be established to describe the decline trend curve of the bolt's health status, such as... Figure 3As shown, the archive not only records the overall risk trend, but also further decomposes the risk into multiple dimensions such as visual morphology, structural vibration and acquisition quality, revealing the dynamic evolution of its health status.

[0061] Extrapolation predictions are made based on the aforementioned degradation trend curve to construct a full life-cycle health record for each bolt component of the tower.

[0062] Specifically, extrapolation calculations are performed based on the established health status decline trend curve. By substituting future time values ​​into the mathematical model of this curve, the health status of the bolt at any future time can be predicted. The most important application of this function is its ability to predict when the risk level of the bolt will reach the maintenance or alarm threshold. This threshold is set according to the operation and maintenance safety regulations of power facilities. Finally, the unique identification code, all historical data records of each inspection, the established decline trend curve model, and the future status prediction generated based on this model are integrated and structured for storage, thereby constructing a dynamically updated and comprehensive full lifecycle health record for each bolt component of the tower.

[0063] Based on the same inventive concept, this invention also provides a bolt loosening detection system based on unmanned aerial vehicles and multimodal analysis, such as... Figure 4 As shown, the system includes: The 3D task planning module is used to acquire structured data of the tower to be inspected, construct a virtual mapping in 3D space, and perform detection simulation in the virtual mapping. It analyzes the spatial distribution and optimal observation attitude of key bolt nodes, and comprehensively considers environmental constraints and sensor parameters to dynamically plan and generate 3D inspection tracks and sensor action sequences for UAVs. The multimodal data acquisition module is used by the UAV to drive the onboard high-definition camera and voiceprint acquisition device to perform multi-angle detection on each bolt node based on the three-dimensional inspection trajectory and the sensor action sequence, and simultaneously acquire high-definition video streams and spatial sound field data with precise time and space stamp association, forming a spatiotemporally continuous multimodal data stream and transmitting it in real time. The multimodal fusion diagnostic module is used to receive the multimodal data stream, extract the geometric contour and texture details of the bolt, extract the distribution features of vibration energy in the frequency domain, perform cross-attention calculation and fusion, and output the quantitative diagnostic results of the bolt loosening state. The active verification and correction module is used to receive the quantitative diagnostic results and perform online diagnosis. If the diagnostic results show an anomaly, it autonomously generates a secondary detailed investigation command and drives the UAV to return to the target position, performs zoom, hovering and orbiting actions to collect supplementary data, and iteratively corrects the quantitative diagnostic results based on the supplementary data to form the final diagnostic data for online verification. The full lifecycle management module is used to automatically integrate the multimodal data stream based on the final diagnostic data, generate a legally valid encrypted test report, and upload the encrypted test report to the cloud for archiving, thereby constructing a full lifecycle health record for each bolt component of the tower.

[0064] To verify the feasibility and effectiveness of this invention in practical applications, it was applied to an automated inspection of the bolt condition of one of the towers on a 220 kV high-voltage transmission line in a mountainous area. The area experiences variable wind conditions and has a complex tower structure, making traditional manual inspection methods insufficient for a comprehensive, regular, and accurate assessment of the numerous bolts' condition. This embodiment aims to achieve efficient, accurate, and intelligent detection and full lifecycle management of the bolt loosening status of this tower using the method disclosed in this invention.

[0065] First, for a pole numbered "G-201" to be inspected, a high-precision 3D point cloud data of the pole was obtained by using a lidar device mounted on a drone. Next, using this point cloud data, a virtual pole model was generated using a Poisson surface reconstruction algorithm, and the 3D coordinates of all 200 key bolt nodes were determined on the model through 3D object detection. Then, a virtual viewpoint traversal calculation was performed for each bolt node, eliminating invalid viewpoints caused by self-occlusion due to pole crossarms or insulator strings, and selecting an optimal observation attitude for each node. Simultaneously, the real-time wind speed of the inspection area was obtained from a network meteorological service (7 m / s), and combined with known obstacle information such as surrounding trees from a digital map, a dynamic constraint area was constructed in 3D space. Finally, with the global inspection flight time and total energy consumption as joint optimization objectives, a 3D inspection trajectory visiting all optimal observation attitude points was planned while avoiding the constructed constraint area, and a sensor action sequence including gimbal pitch and camera zoom was generated simultaneously.

[0066] An industrial-grade drone, equipped with corresponding sensing devices, flies strictly according to the 3D inspection flight path and sensor action sequence generated in the previous step. The onboard flight controller precisely controls the drone's position and speed, while the gimbal's drive mechanism ensures that the detection axes of the high-definition camera and the acoustic signature acquisition device are always precisely aligned with the target bolt node. The entire acquisition process uses an onboard unified clock source synchronized with the second pulse signal of the Global Positioning System as a reference. Each frame of high-definition video and each segment of spatial sound field data is appended with a high-precision spatiotemporal stamp containing accurate 3D coordinates and attitude information. The encapsulated spatiotemporally continuous multimodal data stream is then transmitted in real time to ground analysis equipment via a fifth-generation mobile communication wireless data link.

[0067] Ground-based analysis equipment receives and processes this multimodal data stream. Taking a bolt node numbered "B-32" on tower G-201 as an example, visual features such as "nut withdrawal height of 1.2 mm" and "3 exposed threads" are extracted from the high-definition spatiotemporal video sequence using image processing algorithms. Simultaneously, based on the spatiotemporal stamp, synchronized spatial sound field data is located, and acoustic features such as "abnormal resonance peak near 110 Hz" are extracted through time-frequency analysis. Both visual and acoustic features point to a loose state, therefore, they are assigned high confidence weights. The final quantitative diagnostic result for this bolt is 0.88, indicating a high degree of suspicion of looseness.

[0068] The predetermined optimal observation attitude of node B-32 was extracted from the sensor action sequence, and the actual 3D attitude at the time of data acquisition was parsed from the spatiotemporal stamp of the real-time data stream. Comparison revealed a significant spatial deviation between the actual and target positions due to a sudden crosswind. The calculated dynamic quality control factor for this acquisition was 0.62, lower than the preset data confidence threshold of 0.7. Since the initial quantitative diagnosis was abnormal and the dynamic quality control factor indicated questionable data quality, a secondary detailed investigation process was automatically triggered. Starting from the UAV's current position and ending at the B-32 node coordinates, a return detailed investigation trajectory was quickly planned, and supplementary data acquisition commands including actions such as "hover for 20 seconds" and "optical zoom to 12x" were generated. After executing these commands, the UAV acquired higher-quality supplementary multimodal data, and the supplementary analysis result was 0.91. Finally, the original and supplementary results were weighted and fused with confidence levels, outputting a final online diagnostic data of 0.90, confirming that the bolt was loose. For other bolts, such as the bolt numbered "C-05", the dynamic quality control factor is 0.97 and the initial quantitative diagnostic result is 0.16. Therefore, there is no need to start a second detailed inspection, and the final diagnostic data is 0.16.

[0069] After the inspection was completed, a unique identification code was created for each bolt on tower G-201. The final diagnostic data, encrypted inspection report, and related spatiotemporal stamp information for node B-32 during this inspection were archived in a time-series database under its identification code. Further historical diagnostic data from the bolt's two previous inspections were retrieved, and longitudinal regression analysis was initiated to establish a decline trend curve for its health status. Extrapolation based on this curve model predicted that the bolt's condition would reach a severe risk threshold within approximately three months. A proactive maintenance work order, including recommended maintenance timelines and a chain of historical evidence, was automatically generated, achieving a shift from passive discovery to proactive prediction.

[0070] It should be noted that the functional division and information interaction between the various modules described above are logical, but in terms of physical implementation, they can be integrated on the same software platform or deployed in a distributed manner. The connections between them represent data flow and control flow, aiming to collaboratively achieve the objectives of this invention. The above descriptions are merely exemplary embodiments of this invention and should not be construed as limiting the scope of protection of this invention.

Claims

1. A bolt loosening detection method based on UAV and multimodal analysis, characterized in that, The method includes: The structured data of the tower to be inspected is acquired, a virtual mapping in three-dimensional space is constructed, and detection simulation is performed in the virtual mapping. The spatial distribution and optimal observation attitude of key bolt nodes are analyzed. Environmental constraints and sensor parameters are comprehensively considered to dynamically plan and generate three-dimensional inspection tracks and sensor action sequences for UAVs. Based on the three-dimensional inspection trajectory and the sensor action sequence, the UAV drives the onboard high-definition camera and voiceprint acquisition device to perform multi-angle detection on each bolt node, and simultaneously collects high-definition video streams and spatial sound field data with precise time and space stamps, forming a spatiotemporally continuous multimodal data stream and transmitting it in real time. The system receives the multimodal data stream, extracts the geometric contour and texture details of the bolt, extracts the distribution features of vibration energy in the frequency domain, performs cross-attention calculation and fusion, and outputs the quantitative diagnostic results of the bolt loosening state. The system receives the quantitative diagnostic results and performs online diagnosis. If the diagnostic results show an anomaly, it autonomously generates a secondary detailed investigation command and drives the UAV back to the target position. It performs zoom, hovering, and orbiting actions to collect supplementary data. Based on the supplementary data, it iteratively corrects the quantitative diagnostic results to form the final diagnostic data for online verification. Based on the final diagnostic data, the multimodal data stream is automatically integrated to generate a legally valid encrypted test report, which is then uploaded to the cloud for archiving, thus constructing a full lifecycle health record for each bolt component of the tower.

2. The bolt loosening detection method based on UAV and multimodal analysis according to claim 1, characterized in that, The process of dynamically planning and generating 3D inspection tracks and sensor action sequences for UAVs includes: Import the 3D scan point cloud data of the tower to be inspected, perform geometric structure reconstruction, and generate a virtual tower model containing the 3D coordinates of all key bolt nodes; Using the bolt nodes in the virtual tower model as targets, a full-range virtual viewpoint traversal calculation is performed to eliminate invalid viewpoints that are self-occluded by tower components, and the optimal observation posture for data acquisition is selected and determined for each node. By acquiring and fusing real-time meteorological data of the drone's current location, information on surrounding obstacles, and the drone's own dynamics and sensor performance parameters, a dynamic constraint region is constructed in three-dimensional space. With the optimization objectives of global inspection flight time and total energy consumption, and under the premise of avoiding the constrained area, a three-dimensional inspection trajectory is calculated by path planning, and a sensor action sequence is generated simultaneously.

3. The bolt loosening detection method based on UAV and multimodal analysis according to claim 1, characterized in that, The process of forming and transmitting a spatiotemporally continuous multimodal data stream in real time includes: The UAV controls its position and speed based on the three-dimensional inspection trajectory, drives the gimbal according to the sensor action sequence, and adjusts the direction of the high-definition camera and acoustic fingerprint acquisition device in real time to ensure that the detection axis is accurately aligned with each bolt node when it reaches the node. Based on the airborne unified clock source, high-definition video images and spatial sound field data are collected, and spatiotemporal stamps containing three-dimensional coordinates and attitude information are added. The data are then multiplexed and encapsulated into a standard format multimodal data stream, which is continuously transmitted to ground analysis equipment in real time via the airborne wireless data link.

4. The bolt loosening detection method based on UAV and multimodal analysis according to claim 3, characterized in that, The multimodal data stream includes: high-definition spatiotemporal video sequences and synchronized spatial sound field data, wherein: The high-definition spatiotemporal video sequence is used to provide visual appearance information of the bolt and surrounding components, and to extract geometric contours, relative positions and surface texture details that characterize the fastening state. The synchronous spatial sound field data is used to provide mechanical vibration information of bolts under environmental excitation and to extract the energy distribution characteristics in the frequency domain caused by structural micro-motion and wind-induced vibration.

5. The bolt loosening detection method based on UAV and multimodal analysis according to claim 1, characterized in that, The quantitative diagnostic results of the output bolt loosening status include: The high-definition spatiotemporal video sequence in the multimodal data stream is processed to identify and segment independent bolt target regions in the video frames, and to calculate and extract visual state features. Based on the multimodal data stream and the video frame, the processing is strictly synchronized in terms of time and space stamps, and the data is converted to the frequency domain through time-frequency analysis to calculate and extract acoustic features. Using the visual state features and the acoustic features, the correlation and confidence weights between different modal features are evaluated through cross-attention, and a weighted fusion calculation is performed to output a quantitative diagnostic result.

6. The bolt loosening detection method based on UAV and multimodal analysis according to claim 2, characterized in that, The method further includes: The three-dimensional attitude information of the UAV when collecting each segment of data is parsed from the spatiotemporal stamps of the multimodal data stream; The optimal observation attitude is compared with the three-dimensional attitude information in real time to calculate the spatial deviation and generate a dynamic quality control factor that characterizes the accuracy of the inspection flight and the data quality.

7. The bolt loosening detection method based on UAV and multimodal analysis according to claim 6, characterized in that, The final diagnostic data used to generate online verification includes: Receive the quantitative diagnostic results and perform a diagnosis based on the dynamic quality control factors. If the diagnostic results show an abnormality, it is determined that a second detailed investigation needs to be initiated. For the bolt nodes identified in the secondary detailed investigation, the system autonomously plans and generates a detailed investigation trajectory to guide the UAV back to the target location precisely, and simultaneously generates supplementary data collection instructions. The drone is driven to execute the detailed flight path and the supplementary data collection instructions, supplement multimodal data and analyze it again, and the analysis results are fused with the quantitative diagnostic results by confidence weighting to output the final diagnostic data for online verification.

8. The bolt loosening detection method based on UAV and multimodal analysis according to claim 7, characterized in that, The process of generating a detailed flight path that guides the UAV to accurately return to the target location, and simultaneously generating supplementary data collection instructions, includes: Starting from the current three-dimensional position of the UAV and ending at the three-dimensional coordinates of the bolt nodes obtained from the secondary detailed investigation, and taking into account the constraint area in three-dimensional space, a local path optimization calculation is performed in real time to generate a detailed investigation track that guides the UAV to safely return to the target position. Based on the bolt node positions obtained from the secondary detailed investigation, a refined detection action involving hovering and shooting is performed around the optimal observation posture as a supplementary data acquisition command.

9. The bolt loosening detection method based on UAV and multimodal analysis according to claim 1, characterized in that, The full life-cycle health records of each bolt component of the tower construction include: A unique identification code is created for each key bolt node of the tower to be inspected, and the final diagnostic data, the encrypted inspection report and the corresponding spatiotemporal stamp information of each inspection are archived into the time-series database under the identification code; A longitudinal regression analysis is performed on the historical diagnostic data under the same identity code in the time series database to calculate and extract the characteristics of the changes in the loosening state and risk level of bolt components over time, and to establish a decline trend curve of the health status. Extrapolation predictions are made based on the aforementioned degradation trend curve to construct a full life-cycle health record for each bolt component of the tower.

10. A bolt loosening detection system based on UAV and multimodal analysis, applied to the bolt loosening detection method based on UAV and multimodal analysis as described in any one of claims 1-9, characterized in that, The system includes: The 3D task planning module is used to acquire structured data of the tower to be inspected, construct a virtual mapping in 3D space, and perform detection simulation in the virtual mapping. It analyzes the spatial distribution and optimal observation attitude of key bolt nodes, and comprehensively considers environmental constraints and sensor parameters to dynamically plan and generate 3D inspection tracks and sensor action sequences for UAVs. The multimodal data acquisition module is used by the UAV to drive the onboard high-definition camera and voiceprint acquisition device to perform multi-angle detection on each bolt node based on the three-dimensional inspection trajectory and the sensor action sequence, and simultaneously acquire high-definition video streams and spatial sound field data with precise time and space stamp association, forming a spatiotemporally continuous multimodal data stream and transmitting it in real time. The multimodal fusion diagnostic module is used to receive the multimodal data stream, extract the geometric contour and texture details of the bolt, extract the distribution features of vibration energy in the frequency domain, perform cross-attention calculation and fusion, and output the quantitative diagnostic results of the bolt loosening state. The active verification and correction module is used to receive the quantitative diagnostic results and perform online diagnosis. If the diagnostic results show an anomaly, it autonomously generates a secondary detailed investigation command and drives the UAV to return to the target position, performs zoom, hovering and orbiting actions to collect supplementary data, and iteratively corrects the quantitative diagnostic results based on the supplementary data to form the final diagnostic data for online verification. The full lifecycle management module is used to automatically integrate the multimodal data stream based on the final diagnostic data, generate a legally valid encrypted test report, and upload the encrypted test report to the cloud for archiving, thereby constructing a full lifecycle health record for each bolt component of the tower.

Images