Multi-machine collaborative closed-loop intelligent inspection method and device for tower clusters
By constructing a digital twin foundation for tower clusters and multi-UAV collaborative task allocation, combined with multimodal data processing and full lifecycle trend prediction, the problems of low inspection efficiency, insufficient accuracy, and passive operation and maintenance in large-scale tower clusters have been solved, achieving efficient and accurate fully closed-loop intelligent inspection.
Patent Information
- Application Number
- CN202611051638.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-15
- Publication Date
- 2026-11-10
AI Technical Summary
Existing drone inspection methods suffer from several problems in large-scale tower clusters, including low efficiency of single-machine serial operation, rough flight paths leading to missed inspections, insufficient accuracy of single-modal recognition, data silos preventing model evolution, and passive operation and maintenance failing to predict degradation trends. These issues make it impossible to meet the intelligent operation and maintenance needs of large-scale clusters.
By constructing a digital twin foundation for the tower base station cluster, we can achieve collaborative task allocation and refined route planning for multiple drones, integrate multimodal data anomaly identification and predictive maintenance driven by full lifecycle data, and form a fully closed-loop intelligent inspection system, including multi-target optimized allocation, multimodal data processing and full lifecycle trend prediction.
It significantly improves the inspection efficiency, anomaly identification accuracy, and predictive maintenance capabilities of large-scale tower clusters, overcoming the shortcomings of traditional solutions such as low efficiency, insufficient accuracy, data silos, and passive operation and maintenance. It realizes the transformation from single-machine serial to multi-machine collaboration, from static route planning to dynamic refined planning, from single-modal identification to multi-modal fusion hierarchical diagnosis, and from data silos to federated collaborative evolution.
Smart Images

Figure CN123067445A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of communication technology, and in particular to a multi-machine collaborative closed-loop intelligent inspection method and device for tower clusters. Background Technology
[0002] Currently, the scale of communication tower base stations is growing exponentially. Traditional manual inspection methods are inefficient, costly, and involve high-altitude operations, making them unable to meet the maintenance needs of large-scale clusters. Although drone inspection has been gradually adopted as an alternative, existing methods still have significant limitations.
[0003] Single-drone point-to-point inspections lack global task scheduling and multi-drone collaboration capabilities. When facing clusters of tens of thousands of towers, a single point of failure can lead to mission interruption, and efficiency drops exponentially as the scale expands. Fixed-route inspections rely on manually preset paths, which cannot generate refined routes based on the spatial distribution of tower components. This results in frequent missed inspections of critical parts and cannot be dynamically adjusted according to weather conditions or task priorities. Single-modal visual recognition relies solely on visible light images, without integrating multi-dimensional data such as infrared, point clouds, and acoustic signatures. Furthermore, it lacks dedicated recognition models for different components, resulting in insufficient accuracy and generalization capabilities. Detection results are merely recorded. Local storage on edge gateways leads to data silos, preventing the sharing and optimization of models across different regions and tower types, and hindering continuous improvement in generalization capabilities.
[0004] Therefore, there is an urgent need for an intelligent tower cluster inspection method that deeply integrates multi-source data, has multi-machine collaborative scheduling and global optimization, achieves component-level fine coverage and multi-modal hierarchical identification, constructs a closed-loop handling process, and realizes predictive maintenance based on full life cycle data. Summary of the Invention
[0005] This invention provides a multi-machine collaborative closed-loop intelligent inspection method and system for tower clusters. This method can significantly improve the inspection efficiency, anomaly identification accuracy, automation level of operation and maintenance, and predictive maintenance capability of large-scale tower clusters.
[0006] Firstly, a multi-machine collaborative closed-loop intelligent inspection method for tower clusters is provided, including: Construct a digital twin foundation for the tower base station cluster and acquire the status information of the inspection drone; The inspection task is extracted from the inspection task pool, and the inspection drone is assigned a task based on the priority of the inspection task and the status information of the inspection drone. The corresponding inspection route is generated according to the task assignment result and the digital twin base. The system controls the inspection drone to perform inspections along the inspection route, receives multimodal data of the tower base station cluster collected by the inspection drone, performs anomaly identification on the multimodal data, and adjusts the inspection route according to the anomaly identification results. Based on the full lifecycle inspection data, the degradation trend value of the components in the tower base station cluster is predicted. When the degradation trend value of the components reaches the warning threshold, a preventive inspection task is generated and placed in the inspection task pool.
[0007] In this way, by constructing a digital twin foundation for the tower base station cluster, integrating multi-source heterogeneous data to achieve multi-UAV collaborative task allocation and refined route planning, and combining multi-modal data anomaly identification and full life-cycle data-driven predictive maintenance, a fully closed-loop intelligent inspection system is formed. This significantly improves the inspection efficiency, anomaly identification accuracy, and predictive maintenance capabilities of large-scale tower clusters, overcoming the shortcomings of existing technologies such as low efficiency of single-unit serial operation, rough route leading to missed detections, insufficient single-modal identification accuracy, data silos preventing model evolution, and passive operation and maintenance failing to predict degradation trends.
[0008] In some possible implementations, the digital twin infrastructure for constructing the tower base station cluster includes: Collect all element data of each tower in the tower base station cluster. The all element data includes building information model drawings, as-built data, spatial coordinates, tower type, component list and spatial location, surrounding geographic information and elevation data. Based on the building information model drawings and lidar point cloud data, geometric registration and texture rendering are performed to generate a digital twin corresponding to the physical tower. The digital twin includes the geometric parameters, attribute information and spatial location of each component of each tower. By integrating the digital twins of each tower, geographic information, and no-fly zone data, the digital twin foundation of the tower base station cluster is obtained.
[0009] In this way, by collecting full-element data of the tower and integrating building information model and lidar point cloud to generate a digital twin, precise component-level spatial coordinates and geometric parameter benchmarks are provided for subsequent refined route planning. This enables the route to cover every core component of each base station, fundamentally avoiding the problem of missed detection of key parts caused by the lack of accurate spatial data in traditional solutions.
[0010] In some possible implementations, the task allocation of the inspection drone based on the priority of the inspection task and the status information of the inspection drone includes: The execution order of each inspection task is determined according to its priority, with inspection drones being assigned to high-priority tasks first. Based on the status information of each inspection drone, select available drones that meet the endurance and space constraints; With the optimization goals of minimizing total inspection time, minimizing total flight mileage, and balancing the load of each UAV, each inspection task is assigned to the corresponding available UAV, generating a list of target base stations, inspection sequence, take-off and landing hangars, and battery swapping plans for each available UAV.
[0011] In this way, by introducing a task priority-driven multi-objective optimization allocation mechanism, while ensuring that high-priority emergency tasks are responded to as soon as possible, the system also takes into account the overall inspection efficiency, energy consumption economy, and load balance among drones. This overcomes the shortcomings of traditional solutions, such as no priority scheduling and uneven resource utilization, and achieves the optimal allocation of limited drone resources in a large-scale cluster.
[0012] In some possible implementations, generating the corresponding inspection route based on the task allocation result and the digital twin infrastructure includes: Based on the task allocation results, the list of target base stations to be responsible for each inspection drone and the inspection order of each base station are determined, and a global flight segment route is generated, which connects the take-off and landing hangar with each target base station. Extract the spatial location, geometric parameters, and component type of each tower and component contained in the current target base station from the digital twin base; Obtain the corresponding inspection parameters according to the component type. The inspection parameters include shooting distance, sampling interval, shooting angle, sensor type and safety distance. Based on the spatial position of each component, and combined with the corresponding shooting distance and shooting angle, calculate the three-dimensional spatial coordinates, nose orientation and lens pitch angle of each waypoint; Based on the geometric parameters and sampling interval of each component, the sampling density and sampling step size of each waypoint are determined, and a waypoint sequence covering the entire inspection area of the component is generated. Each waypoint includes three-dimensional coordinates, flight attitude, lens focal length, exposure parameters and dwell time. Path optimization is performed on each waypoint sequence to generate an inspection route for the current target base station.
[0013] In this way, by using precise spatial data at the component level provided by the digital twin foundation, inspection parameters are automatically matched according to different component types and a waypoint sequence covering the entire area to be inspected is generated. This achieves a qualitative leap from manually preset routes to data-driven, component-level refined route planning. While ensuring coverage integrity, the inspection time per base station is minimized through waypoint path optimization, significantly improving inspection efficiency and data collection quality.
[0014] In some possible implementations, the step of determining the list of target base stations to be handled by each inspection drone and the inspection order of each base station based on the task allocation result, and generating a global flight path, includes: Based on the task allocation results, determine the list of target base stations that each inspection drone is responsible for. Obtain the spatial coordinates of each target base station, the location of the take-off and landing hangar of each inspection drone, and the boundary of the no-fly zone; With the optimization objectives of minimizing total flight distance and total power consumption, a path optimization algorithm is used to determine the inspection sequence of each target base station, and a global flight segment route is planned from the take-off and landing hangar, passing through each target base station in sequence, and finally returning to the take-off and landing hangar. Acquire real-time meteorological data and the current remaining power of each of the inspection drones, and use an energy consumption prediction model to calculate the estimated power consumption of the global flight segment route; When the estimated power consumption exceeds the preset safety threshold of the current remaining power, a battery swapping plan is inserted into the global flight segment route. The battery swapping plan includes matching the nearest available hangar and a battery swapping time window.
[0015] In this way, by taking into account meteorological conditions, real-time power and energy consumption forecasts in the overall flight segment planning, and automatically inserting a dynamic battery swapping plan when the power is insufficient, the shortcomings of fixed routes and uncontrollable energy consumption in the traditional solution are overcome, ensuring the feasibility and safety of long-range, multi-base station inspection tasks, and significantly improving the fleet endurance management capability and adaptability to extreme environments.
[0016] In some possible implementations, the step of performing anomaly identification on the multimodal data and adjusting the inspection route based on the anomaly identification results includes: The edge gateway invokes a locally deployed lightweight anomaly detection model to perform real-time preliminary detection on the multimodal data and outputs anomaly detection results, which include anomaly type, anomaly location, and confidence level. The anomaly identification results are classified according to the confidence level; When the anomaly identification result is a Level 1 anomaly, the current inspection task is suspended, the anomaly identification result is reported to the cloud server, and the emergency response procedure is triggered. When the anomaly identification result is a level 2 or level 3 anomaly, the anomaly identification result is reported to the cloud server for re-examination and confirmation, and the inspection drone is controlled to continue to perform the current inspection task. When a route adjustment instruction is received from the cloud server, the inspection route is updated according to the route adjustment instruction.
[0017] In this way, the two-level collaborative architecture of real-time initial inspection by a lightweight model at the edge and re-inspection and confirmation by a high-precision model in the cloud ensures both real-time recognition and accuracy. Based on the anomaly confidence level, a differentiated processing strategy is triggered. For major anomalies, the inspection is suspended and emergency measures are taken, while for general anomalies, the inspection continues and is re-inspected simultaneously. This overcomes the shortcomings of traditional solutions that lack a hierarchical mechanism and achieves refined control of inspection tasks and efficient response to anomalies.
[0018] In some possible implementations, the step of predicting the degradation trend value of components in the tower base station cluster based on full lifecycle inspection data, and generating a preventive inspection task when the component degradation trend value reaches a warning threshold, includes: Historical inspection data of each component is extracted from the full life cycle database. The historical inspection data includes the degree of corrosion, tilt angle, deformation, temperature value and records of abnormal occurrences. Arrange the historical inspection data in chronological order to construct a time-series dataset for each component; The time series dataset is input into a pre-trained degradation trend prediction model, which outputs the degradation trend prediction values of each component within a preset future time period. When the predicted value of the degradation trend reaches the preset warning threshold, a warning report is generated for the component. Preventive inspection tasks are generated based on the early warning report, and these preventive inspection tasks are then added to the inspection task pool.
[0019] In this way, by using full lifecycle time-series inspection data to drive the degradation trend prediction model, preventive inspection tasks are generated in advance and automatically added to the task pool when component degradation reaches the warning threshold. This achieves a fundamental shift from passive post-event alarms to proactive pre-event prediction, enabling maintenance personnel to intervene and handle the situation before a failure occurs, significantly reducing the risk of business interruption and maintenance costs caused by sudden failures.
[0020] Secondly, a multi-machine collaborative fully closed-loop intelligent inspection device for tower clusters is provided, including: The first processing module is used to build the digital twin foundation of the tower base station cluster and obtain the status information of the inspection drone. The second processing module is used to extract inspection tasks from the inspection task pool, allocate tasks to the inspection drone based on the priority of the inspection tasks and the status information of the inspection drone, and generate corresponding inspection routes according to the task allocation results and the digital twin base. The third processing module is used to control the inspection drone to perform inspections along the inspection route, receive multimodal data of the tower base station cluster collected by the inspection drone, identify anomalies in the multimodal data, and adjust the inspection route according to the anomaly identification results. The fourth processing module is used to predict the degradation trend value of components in the tower base station cluster based on the full life cycle inspection data. When the degradation trend value of the component reaches the warning threshold, a preventive inspection task is generated and placed into the inspection task pool.
[0021] Thirdly, an electronic device is provided, comprising: one or more processors; one or more memories; and one or more programs, wherein the one or more programs are stored in the one or more memories, and the one or more programs include instructions that, when executed by the one or more processors, cause the configured device to perform the method as described in the first aspect.
[0022] Fourthly, a computer storage medium is provided, the computer storage medium storing instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect or the second aspect.
[0023] Fifthly, a computer program product is provided, the computer program product storing instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect or the second aspect.
[0024] The beneficial effects of the second to fifth aspects can be referred to the introduction of the beneficial effects of the first aspect above, and will not be repeated here. Attached Figure Description
[0025] Figure 1 This is a flowchart illustrating a multi-machine collaborative fully closed-loop intelligent inspection method for tower clusters provided in an embodiment of the present invention. Figure 2 This is a flowchart illustrating a multi-machine collaborative fully closed-loop intelligent inspection method for tower clusters provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of a multi-machine collaborative closed-loop intelligent inspection device for tower clusters that can be used to implement the method of the present invention. Figure 4 This is a schematic diagram of an electronic device provided by the present invention. Detailed Implementation
[0026] The solutions provided by the embodiments of the present invention will now be described with reference to the accompanying drawings. In the embodiments of the present invention, "multiple" refers to two or more objects, and "various kinds" refers to two or more types. Terms such as "first," "second," etc., are only used to distinguish similar objects and are not necessarily used to describe a specific order or number of objects.
[0027] Currently, the scale of communication tower base stations is growing exponentially. Traditional manual inspection methods suffer from low efficiency, high cost, high risks associated with high-altitude operations, and uncontrollable inspection quality, and are gradually being replaced by drone inspection methods. However, existing drone inspection methods still rely on single-unit operations and static flight path planning, making it difficult to adapt to the intelligent operation and maintenance needs of large-scale tower clusters, and have the following significant limitations.
[0028] The point-to-point inspection solution based on a single UAV and an edge gateway enables real-time analysis of inspection data and dynamic adjustment of flight paths between individual base stations. However, the multi-UAV collaboration of this type of solution is essentially a single-UAV serial operation, lacking global task scheduling and multi-UAV collaborative capabilities. When dealing with clusters of tens of thousands of towers, the inspection efficiency is extremely low. Real-world testing shows that when the number of towers increases from 500 to 5000, the total inspection time for the single UAV solution increases more than tenfold, and a single UAV failure directly leads to task interruption, resulting in zero fault tolerance.
[0029] Fixed inspection schemes based on preset flight paths rely on manually pre-set inspection routes and shooting parameters. They lack the ability to generate detailed flight paths for core tower components such as the tower base, tower body, antenna, feeder, grounding system, and equipment room, making it prone to missing critical areas. Furthermore, the flight paths cannot be dynamically adjusted according to real-time weather conditions such as strong winds, heavy fog, low light, or task priorities, resulting in poor reliability under extreme conditions. Field tests show that under wind conditions of force 6 or higher, the effective image acquisition rate of fixed flight path schemes decreases by more than 40%.
[0030] Anomaly detection schemes based on single-modal visual recognition rely solely on visible light images, failing to integrate multimodal data such as infrared thermal imaging, LiDAR point clouds, and acoustic signatures. Furthermore, they lack dedicated recognition models for different tower components, resulting in insufficient recognition accuracy and generalization capabilities. Additionally, anomaly detection results are merely recorded, lacking a complete closed-loop process for anomaly classification, maintenance assignment, and handling verification, thus failing to achieve end-to-end management from detection to handling.
[0031] Data silos based on local storage at edge gateways mean that inspection data is processed and stored only locally at a single base station edge gateway. This prevents recognition models from sharing optimization results across different regions and tower types, hindering continuous improvement in model generalization capabilities. Furthermore, uploading raw data to the cloud poses privacy and security risks, making collaborative evolution of the entire network's models difficult to achieve.
[0032] The passive operation and maintenance model based on post-event alarms can only detect and alarm after an anomaly occurs. It cannot predict the deterioration trend of components based on the full life cycle inspection data, and cannot upgrade from passive fault handling to proactive preventive operation and maintenance, resulting in high operation and maintenance costs.
[0033] To address the core technical problems in existing technologies, such as poor cluster adaptability, coarse flight path planning, insufficient identification accuracy, severe data silos, and passive operation and maintenance modes, this invention proposes a tower cluster inspection method and system based on multi-machine collaboration and fully closed-loop intelligent inspection for large-scale tower base station clusters. By constructing a cluster-level digital twin foundation and inspection resource pool, global resource visualization and management are achieved; a multi-objective optimization model is used to achieve multi-UAV collaborative task allocation, generating two levels of refined inspection flight paths; multi-modal data such as visible light, infrared, point cloud, and acoustic signatures are integrated, and a two-level hierarchical anomaly identification architecture from edge to cloud is adopted, with federated learning enabling collaborative self-evolution of the entire network model; a fully closed-loop operation and maintenance process is constructed; and a component degradation trend prediction model is built based on full lifecycle data to achieve predictive maintenance. This method achieves a fundamental transformation from single-machine serial to multi-machine collaboration, from static flight paths to dynamic refined planning, from single-modal identification to multi-modal fusion hierarchical diagnosis, from data silos to federated collaborative evolution, and from passive handling to proactive prediction, significantly improving the efficiency, accuracy, security, and intelligence level of tower cluster inspection.
[0034] The following describes a multi-machine collaborative closed-loop intelligent inspection method for tower clusters, as illustrated in an embodiment of the present invention.
[0035] Figure 1 This is a flowchart illustrating a multi-machine collaborative closed-loop intelligent inspection method for tower clusters, as shown below. Figure 1 As shown, it specifically includes: In step S1, the digital twin foundation of the tower cluster and the inspection resource pool are constructed.
[0036] In this embodiment of the invention, firstly, through the data acquisition and base construction layer, all element data of all tower base stations in the target cluster and all element resource data of inspection are collected. After standardization processing, a cluster-level high-precision digital twin base is constructed, and four types of core resources, namely drones, intelligent hangars, edge gateways, and maintenance personnel, are pooled for management.
[0037] Specifically, the full collection of basic data includes: tower design BIM drawings, as-built data, global navigation satellite system coordinates, tower type such as angle steel tower, single-tube tower, three-tube tower or landscape tower, full component list including antenna model and location and azimuth, feeder route, tower base parameters, grounding body layout, equipment list of equipment room, as well as geographic information and elevation data within a 100-meter radius and no-fly zone boundaries.
[0038] Simultaneously, comprehensive resource data for inspection is collected, including drone parameters such as model, endurance, payload configuration, sensor parameters, maximum flight speed, and flight altitude; intelligent hangar data such as location, number of battery swapping stations, swapping time, storage, and communication capabilities; edge gateway data such as deployment location, computing power specifications, storage capacity, and uplink bandwidth; and operation and maintenance resource data such as the area of responsibility of operation and maintenance personnel or teams, skill tags, real-time location, and tool configuration. All collected data undergoes standardized processing, unifying data format, coordinate system, and timestamps to form a standardized basic database.
[0039] Then, based on Building Information Modeling (BIM) combined with LiDAR point cloud and texture mapping technology, a centimeter-level high-precision digital twin is constructed for each tower: using BIM as a priori framework, geometric registration is completed by combining initial inspection LiDAR point cloud data, and detailed rendering is achieved through multi-view texture projection, generating a digital twin that maps to the physical tower. The twin contains the geometric parameters, attribute information, and spatial location of all tower components. By integrating all tower twins, geographic information, inspection resources, and no-fly zone data, a cluster-level digital twin base is constructed, realizing the digital mapping of all elements of the physical world, supporting real-time status updates, 3D visualization, spatial calculation, and simulation pre-playback.
[0040] In terms of resource pooling management, four core resource categories—drones, hangars, edge gateways, and maintenance personnel—are managed in a pooled manner. Each resource is assigned a unique identifier, and a resource status ledger is established. Through 5G or 4G mobile communication networks, resource status data is synchronized in real time: drones synchronize remaining battery power, location, flight status, and sensor health; hangars synchronize battery swapping idle status and battery charging progress; edge gateways synchronize computing power utilization, bandwidth utilization, and storage capacity; and maintenance personnel synchronize online status, task execution progress, and real-time location. All resource status data is synchronized in real time to the digital twin platform, enabling visualized control and unified scheduling of global resources.
[0041] In step S2, global requirements analysis and dynamic priority classification of inspection tasks are performed.
[0042] In this embodiment of the invention, the task scheduling and priority management module supports standardized access for four types of task requirements, and performs structured parsing and dynamic priority classification on all accessed tasks to generate a global task scheduling pool.
[0043] Specifically, the task requirements include: routine inspection requirements, which are routine inspection tasks of all base stations generated according to preset cycles such as monthly, quarterly or annually, with a default inspection of all core components; special inspection requirements, which are targeted inspection tasks for specific scenarios, such as emergency hazard investigation after typhoons or earthquakes, acceptance of new base stations, special verification of antenna azimuth angle, and special testing of grounding systems; alarm-triggered inspection requirements, which are emergency inspection tasks triggered by third-party systems, including environmental monitoring alarms, tower tilt sensor alarms, fire alarms, public-reported hazards, and operator network fault alarms; and preventive inspection requirements, which are component deterioration verification tasks triggered by predictive maintenance modules.
[0044] Then, all access task requirements are structured and analyzed to extract core task features: inspection area, target base station list, inspection component range, accuracy requirements, latest completion deadline, and task source. A five-level task priority classification rule is preset, dynamically classifying tasks based on urgency, impact scope, and risk level, as shown in Table 1 below: Table 1 Task Priority Classification Rules Based on hierarchical rules, all tasks are automatically assigned priorities, unique task identifiers are generated, and the tasks are entered into the global task scheduling pool of the digital twin platform. The task status is updated in real time, including pending assignment, in progress, completed, and overdue.
[0045] In step S3, global task allocation and refined inspection route planning for multi-UAV collaboration are carried out.
[0046] This step is divided into two sub-processes: multi-objective optimization of multi-UAV collaborative task allocation and two-level refined inspection route planning.
[0047] First, in sub-step S31, multi-target optimization of multi-UAV collaborative task allocation is performed.
[0048] First, tasks are extracted from the global task scheduling pool in descending order of priority, with priority given to resource allocation for P0 and P1 level tasks. A list of available resources in the current resource pool is extracted, filtering out resources that do not meet the constraints: drones operating in areas with wind speeds exceeding 12 meters per second, drones with less than 20% remaining battery power, drones without corresponding payloads, fully loaded hangars, and edge gateways with computing power occupancy exceeding 90%. For each task, corresponding inspection resource requirements are matched; for example, P0 level emergency tasks are prioritized for matching the nearest drone, infrared detection tasks are matched for matching drones equipped with infrared thermal imaging cameras, and high-precision tower structure inspection tasks are matched for matching drones equipped with lidar.
[0049] Then, a multi-objective optimization model is constructed with the core objectives of ensuring optimal task completion, maximizing inspection efficiency, minimizing resource consumption, and achieving the most balanced load. The optimization objective function consists of four parts: The priority guarantee objective is to minimize the priority loss function, where is the task priority weight and is the task deadline fulfillment rate, ensuring that high-priority tasks are completed 100% on time. The efficiency objective is to minimize the total time taken for all UAVs to complete their assigned tasks, achieving the shortest total inspection time. The energy consumption objective is to minimize the total flight distance taken for all UAVs to complete their assigned tasks, achieving the lowest total energy consumption. The load balancing objective is to minimize the variance of the task load rate of all UAVs, achieving resource load balancing.
[0050] Specifically, optimize the objective function: Priority protection objectives: ,in For the task i Priority weights (P0 level weight 100, P1 level weight 50, P2 level weight 10, P3 level weight 5, P4 level weight 1). For the task i The time-bound completion rate (1 for on-time completion, 0 for overdue) ensures that high-priority tasks are completed 100% on time; Efficiency goals: ,in For drones j The total time required to complete the assigned tasks is minimized, resulting in the shortest total inspection time. Energy consumption target: ,in For drones j Complete the assigned mission with the lowest possible total flight mileage and energy consumption; Load balancing objectives: ,in For drones j Task load rate, To achieve resource load balancing based on the average load rate of all drones; Constraints: Battery life constraints: Drones j The total estimated power consumption for completing the assigned tasks is ≤ 80% of the current remaining power (with a 20% safety redundancy reserved). Time constraint: The completion time of all tasks ≤ the latest completion time of the tasks; Spatial constraints: The flight path of the drone must not enter the no-fly zone, and a safe distance of ≥5m must be maintained from obstacles; Meteorological constraints: Real-time wind speed in the drone flight area ≤ 12m / s, visibility ≥ 50m, and no moderate or above precipitation; Hanging constraints: The take-off, landing, and battery swapping plans for drones must be matched with the available battery swapping capacity and idle time in the hangar.
[0051] An improved genetic-particle swarm optimization hybrid algorithm is used to solve the above multi-objective optimization model. The details of the algorithm improvement are as follows: A priority weight factor is added to the traditional particle swarm optimization algorithm to amplify the constraints on high-priority tasks, ensuring that the allocation of high-priority tasks is prioritized during iteration; a dynamic inertia weight is designed: inertia weight ,in This represents the current iteration number. The maximum number of iterations, , The algorithm balances global search and local optimization capabilities to avoid getting trapped in local optima. It introduces crossover and mutation operators from genetic algorithms to cross and mutate particles during the iteration process, expanding the solution space and improving the efficiency of finding the global optimal solution. The iteration count reaches a maximum of 500 iterations, or the optimal solution remains unchanged for 50 consecutive iterations. After the solution is obtained, the optimal task allocation scheme is output, including: a list of target base stations to be responsible for each drone, inspection order, take-off and landing hangar, battery swapping plan, and task completion time limit. At the same time, the allocation scheme is synchronized to the digital twin base, corresponding drones, edge gateways, and maintenance personnel.
[0052] In sub-step S32, two-level refined inspection route planning is carried out.
[0053] For each drone's assigned task, a global flight segment route is first generated. Based on the list of base stations assigned to the drone, a global flight segment route is generated from the takeoff / landing hangar to the first base station, the flight segments between base stations, the last base station, and back to the hangar. Combining real-time meteorological data, including current wind speed and direction, forecast wind speed and direction for the next hour, visibility, precipitation, and the drone's real-time status, an Elman neural network power consumption prediction model is used to calculate the estimated power consumption for each flight segment. A traveling salesman problem algorithm is employed to optimize the base station inspection sequence, ensuring the shortest total flight distance and lowest total power consumption for the global flight segment, while avoiding no-fly zones, tall obstacles, and areas with severe weather. If the estimated power consumption exceeds 80% of the drone's remaining battery power, the hangar battery swapping plan is automatically inserted, matching the nearest available hangar and updating the global flight segment route.
[0054] Then, a refined inspection route for a single base station is generated. The tower twin data of the current base station is obtained from the digital twin base, extracting the tower type, height, spatial coordinates, geometric parameters, and inspection requirements of all core components. Based on the inspection task requirements, the core components to be inspected are determined, and inspection parameter standards are preset for each component. See Table 2 below: Table 2 Preset Inspection Parameter Standards Based on the inspection parameters of each component, inspection waypoints are generated for each component. Each waypoint includes the 3D coordinates of the Global Navigation Satellite System, flight altitude, nose direction, lens pitch angle, lens focal length, exposure parameters, and dwell time. A traveling salesman problem algorithm is used to optimize the paths of all waypoints, generating the shortest continuous inspection route. Dynamic safety obstacle avoidance rules are also set: the minimum distance between the route and the tower components and surrounding obstacles is greater than or equal to two meters, and the flight altitude is more than three meters above the highest surrounding obstacle. A digital twin pre-play is performed on the generated refined route to simulate the UAV's flight process, verifying the existence of collision risks and whether the waypoints cover all components to be inspected. After successful verification, the route is preloaded into the UAV flight control system and simultaneously synchronized to the edge gateway and digital twin base station of the corresponding base station.
[0055] In step S4, multimodal inspection data from edge and cloud collaboration is collected, transmitted, and hierarchically identified in real time.
[0056] This step is divided into three sub-processes: multimodal data acquisition and differentiated transmission, two-level hierarchical anomaly identification, and collaborative self-evolution of the whole network model based on federated learning.
[0057] In sub-step S41, multimodal data acquisition and differentiated transmission are performed.
[0058] The drone flies along a precisely planned route, and upon reaching each waypoint, it triggers its onboard multimodal sensors to synchronously collect data, including: Visible light data: Images / videos captured by high-definition cameras with resolutions of 20 megapixels or higher, covering the appearance details of all components; Infrared thermal imaging data: Infrared images acquired by an uncooled infrared camera, with a temperature measurement range of -20℃ to 150℃ and a temperature measurement accuracy of ±2℃. It is used to detect abnormalities such as equipment overheating, loose joints, and water ingress. LiDAR point cloud data: 3D point cloud data collected by 16 or more LiDAR lines, with a point cloud density of ≥100 points / ㎡, used to detect structural anomalies such as tower tilt, deformation, missing parts, and tower base settlement; Voiceprint data: The voiceprints of equipment operation collected by a high-sensitivity microphone are used to detect abnormal operation of computer room air conditioners, UPS, and fans; Environmental sensor data: Real-time environmental data collected by temperature, humidity, air pressure, wind speed, and wind direction sensors; The UAV's onboard edge computing unit performs real-time preprocessing on the collected raw data: Visible light data: distortion correction, dehazing, contrast enhancement, and region of interest (ROI) extraction. Infrared data: non-uniformity correction, temperature calibration, and defect removal. Point cloud data: denoising, point cloud registration, invalid point filtering, and octree compression. Voiceprint data: denoising, frame segmentation, and feature extraction.
[0059] The UAV's onboard edge computing unit performs real-time preprocessing on the collected raw data: visible light data undergoes distortion correction, defogging, contrast enhancement, and region of interest extraction; infrared data undergoes non-uniformity correction, temperature calibration, and bad pixel removal; point cloud data undergoes denoising, point cloud registration, invalid point filtering, and octree compression; and voiceprint data undergoes denoising, framing, and feature extraction.
[0060] Then, for different types of data, we design differentiated compression and transmission strategies to solve the problems of high bandwidth consumption and transmission lag in existing technologies: Visible light video / images: A spatiotemporal feature compression model is adopted, and differentiated compression is performed based on the ROI region. Lossless compression is used for the ROI region of the tower components, and a high compression ratio of 100:1 is used for the background region. Compared with general compression algorithms, bandwidth usage is reduced by more than 60%, while retaining all the details of key components. LiDAR point cloud data: An octree-based geometric compression algorithm is used to remove duplicate and invalid points, achieving a compression ratio of up to 50:1. Infrared, acoustic, and environmental sensor data: Lightweight lossless compression algorithm is used, with a compression ratio of up to 10:1; The drone transmits pre-processed and compressed multimodal data to the edge gateway of the current inspection base station in real time via a 5G / 4G wireless communication network. The transmission process uses the national cryptographic SM4 algorithm for end-to-end encryption to ensure data security.
[0061] In sub-step S42, two-level hierarchical anomaly identification is performed.
[0062] It adopts a two-level recognition architecture of lightweight multimodal initial detection at the edge and high-precision model re-detection in the cloud.
[0063] The first level is the lightweight multimodal anomaly initial detection at the edge. After receiving the data, the edge gateway decrypts and decompresses the data, and calls the locally deployed lightweight multimodal anomaly recognition model to perform real-time initial detection. The single-frame data recognition latency is less than or equal to 200 milliseconds.
[0064] The lightweight model employs an architecture of multimodal feature extraction, cross-modal attention fusion, and component-specific sub-model classification. The multimodal feature extraction backbone network uses a lightweight YOLOv8n network, optimized for edge ARM architecture computing power, to extract features from visible light, infrared, point cloud, and voiceprint data, outputting feature maps for each modality. The cross-modal attention fusion module uses a multi-head cross-modal attention mechanism to spatially and temporally align and fuse features from different modalities, capturing the correlation features between different modalities.
[0065] The component-specific identification sub-model is designed for the six core components of the tower. Each sub-model is specifically optimized for the anomaly type of the corresponding component, including: The tower structure anomaly sub-model, based on point cloud and visible light features, identifies anomalies such as tower tilting, bending, missing or deformed parts, severe corrosion, and loose bolts, and outputs the location of the anomaly, tilt angle, and amount of deformation.
[0066] The antenna and feeder anomaly sub-model, based on visible light and infrared characteristics, identifies anomalies such as antenna azimuth or elevation angle deviation, feeder damage, loose connectors, water ingress, aging, and antenna mast deformation, and outputs angle deviation values and anomaly locations.
[0067] The computer room and environmental anomaly sub-model, based on visible light, infrared and acoustic features, identifies anomalies such as damaged or unauthorized entry into the computer room, air conditioning failure, equipment overheating, uninterruptible power supply anomalies, and battery bulging, and outputs the location and temperature value of the anomaly.
[0068] The Environmental and Safety Anomaly Sub-model, based on visible light, infrared, and point cloud features, identifies anomalies around the tower base such as illegal construction or building work, fallen trees, open flames or smoke, water accumulation or settlement at the tower base, and outputs the location and risk level of the anomaly. The Grounding System Anomaly Sub-model, based on visible light and infrared features, identifies anomalies such as exposed, corroded, or broken grounding electrodes, and detached grounding flat iron. The Ancillary Facilities Anomaly Sub-model identifies anomalies such as damaged or missing ladders, guardrails, and warning signs.
[0069] Each sub-model outputs recognition results, including anomaly type, anomaly 3D coordinates, confidence level, anomaly screenshot or point cloud fragment, and anomaly description. Based on the preset confidence threshold and anomaly classification rules in Table 3, the initial detection results are automatically classified and corresponding actions are triggered. Table 3 Confidence Thresholds and Anomaly Classification Rules The second level involves high-precision cloud-based model re-inspection and anomaly confirmation. After receiving level 2 and 3 anomaly data synchronized from the edge gateway, the cloud-based high-precision multimodal anomaly recognition model is invoked for re-inspection. This high-precision cloud model employs a larger parameter set of YOLOv8x combined with the SwinTransformer architecture, trained using millions of labeled samples from across the network. It also incorporates benchmark data from the China Tower digital twin for secondary verification, comparing the geometric, textural, and temperature differences between the anomaly area and the benchmark model to further confirm the authenticity and severity of the anomaly. Upon completion of the re-inspection, the final anomaly confirmation result is output, the anomaly classification is updated, false positives are removed, and severity assessments and handling suggestions are added. This information is then synchronized to the full lifecycle database of the digital twin platform. The anomaly data confirmed through re-inspection, along with the initial edge inspection results and manually labeled results, serves as positive and negative samples for model training and is incorporated into the model iteration dataset.
[0070] In sub-step S43, the entire network model undergoes collaborative self-evolution based on federated learning.
[0071] The federated learning architecture design adopts a horizontal federated learning architecture with a cloud-based federated learning platform and edge nodes. Each base station's edge gateway is a local training node, and the cloud is a global aggregation node. All raw data is stored locally on the edge gateway, and only encrypted model gradient parameters are uploaded, without uploading the original inspection data.
[0072] Local model training: Every day at midnight, each edge gateway uses the newly added inspection data and anomaly labeling results of the day to fine-tune the local lightweight model and calculate the gradient update parameters of the local model. ,in Number the edge nodes; Gradient-encrypted upload: Each edge gateway uses a homomorphic encryption algorithm to transmit local gradient parameters. Encrypt the data and upload it to the cloud-based federated learning platform, along with the number of training samples for this session. Global gradient secure aggregation: The cloud-based federated learning platform securely aggregates the encrypted gradient parameters uploaded by all edge nodes and calculates the global gradient update parameters. The aggregation formula is: ,in For nodes The number of training samples; Global model update: The cloud updates the parameters of the global model based on the global gradient parameters. At the same time, the accuracy of the global model is verified to ensure that the model accuracy does not decrease; Global model distribution: The cloud will encrypt and distribute the updated global model parameters to all edge gateway nodes; Local model update: Each edge gateway receives global model parameters, updates its local lightweight anomaly detection model, and completes a global collaborative evolution. The preset model iteration cycle is one minor version iteration per day and one major version iteration per month. After each iteration, the model's recognition accuracy, recall rate, and false detection rate are verified. If the model performance degrades, it is automatically rolled back to the previous stable version to ensure the stability of edge recognition. For special scenarios (such as plateaus, coastal areas with high salt fog, and typhoon-prone areas), a scenario-specific personalized model is trained, and model collaborative optimization within the region is achieved through federated learning.
[0073] In step S5, the entire process of handling anomalies in a closed loop and the intelligent dispatching of maintenance tasks are completed.
[0074] In this embodiment of the invention, the anomaly closed-loop handling module realizes full closed-loop management from anomaly identification to task assignment, on-site handling, UAV re-inspection, and result archiving.
[0075] Specifically, after receiving the final confirmation of the anomaly from the cloud, the digital twin platform automatically triggers the corresponding standardized handling procedures based on the anomaly level. Level 1 major anomalies immediately trigger the highest-level audible and visual alarms, simultaneously notifying the corresponding regional operations manager, regional manager, and security management personnel via SMS, application push notifications, and voice calls. At the same time, an emergency response task is automatically generated, including anomaly identifier, anomaly location, anomaly type, severity, on-site risk warnings, handling standards, navigation routes, and completion deadlines. Level 1 anomalies require on-site arrival within two hours and emergency response completion within four hours. Level 2 important anomalies generate routine operations and maintenance tasks, assigned to the corresponding regional operations and maintenance teams. These tasks include anomaly details, handling requirements, and a completion deadline of 24 hours. Level 3 general anomalies generate preventative maintenance tasks, incorporated into the corresponding region's monthly operations and maintenance plan, and handled concurrently during the next routine inspection.
[0076] For intelligent task dispatching, a tag-matching algorithm is used to achieve optimal matching between tasks and maintenance personnel. Core tags for tasks include exception type, difficulty level, location, completion deadline, required skills, and required tools. Tags for maintenance personnel include their area of responsibility, skill qualifications, historical success rate, current task load, real-time location, and tool configuration. Based on tag matching, a suitability score is calculated for each maintenance personnel, prioritizing tasks assigned to the personnel with the highest suitability, closest location, and lowest current load. If the corresponding personnel do not accept the task within ten minutes, the task is automatically escalated to the maintenance team leader, triggering a secondary reminder.
[0077] For end-to-end management of the handling process, maintenance personnel receive tasks through the maintenance application and receive real-time feedback on task status, including "order accepted," "heading to site," "arrived at site," "in progress," and "completed." During the handling process, maintenance personnel need to upload on-site photos, handling videos, and handling records, which are synchronized to the digital twin platform in real time, enabling visualized management and control of the handling process. For Level 1 major anomalies, the system supports video conferencing between on-site maintenance personnel and back-end experts for remote guidance in emergency response.
[0078] Regarding the closed-loop review and archiving of handling results, after maintenance personnel submit a handling completion application, the system automatically triggers a review and inspection task, dispatching drones to conduct a special review of the handled base station, collecting multimodal data of the abnormal area, and verifying whether the abnormality has been completely eliminated through an anomaly identification model. Once the review is passed, the task is automatically closed-loop, and the handling results, review data, and related materials are synchronously stored in the full lifecycle database of the tower base station, updating the status of the digital twin. If the review fails, the system automatically generates a rectification task, reassigns it to the corresponding maintenance personnel, requiring rectification within a specified period until the review is passed.
[0079] In terms of root cause analysis and optimization, historical anomaly data and handling records are statistically analyzed regularly. For high-frequency anomaly types and recurring anomalies, root cause analysis reports are automatically generated to identify the root causes of the anomalies. For example, the root cause of frequent corrosion of iron towers in coastal areas is the high salt spray environment, and the root cause of frequent antenna deviation in a certain area is excessive wind. Targeted optimization and improvement suggestions are put forward, such as replacing anti-corrosion materials and reinforcing antenna supports, to reduce the occurrence rate of similar anomalies from the root cause.
[0080] In step S6, dynamic replanning of the UAV flight path and global closed-loop management of the inspection task are carried out.
[0081] In this embodiment of the invention, the cloud-based task scheduling platform monitors the drone's status and task changes in real time, triggering a global dynamic replanning.
[0082] Specifically, after the edge gateway completes the inspection data identification of the current base station, if there is no major anomaly, it sends a status request to the drone to obtain the drone's real-time status data, including remaining battery power, current location, flight status, sensor health, remaining task list, and progress of completed tasks. The edge gateway synchronizes the drone's real-time status data to the cloud task scheduling platform and the digital twin base. The cloud task scheduling platform monitors the following triggering conditions in real time, and triggers global dynamic replanning when any of the conditions are met: the addition of a P0 or P1 level emergency inspection task; the drone's remaining battery power is lower than the safety redundancy ratio of the original flight path's estimated power consumption; weather conditions in the flight area deteriorate beyond the flight constraints of the original flight path; the drone experiences sensor failure or flight anomalies and cannot continue to execute the original task; the completion deadline of the original task changes or a no-fly zone is temporarily adjusted.
[0083] After triggering replanning, the cloud-based task scheduling platform reacquires the global task status, real-time resource status, and meteorological data, re-runs the multi-objective optimization task allocation model, and generates an updated task allocation scheme and flight path information. If a drone malfunctions, the nearest backup drone is immediately dispatched to take over the remaining tasks, while the malfunctioning drone is directed to return to the nearest hangar for repairs, ensuring uninterrupted mission execution. If only the current drone's flight path needs adjustment, the updated flight path information is directly sent to the drone's flight control system, and the drone synchronously updates its flight plan to continue its inspection mission. The entire replanning process is synchronized in real-time to the digital twin platform, enabling visualized management and control of the global mission.
[0084] In terms of closed-loop management of inspection tasks, the digital twin platform monitors the execution progress, drone status, anomaly identification results, and maintenance handling progress of all inspection tasks in real time, generating a global inspection management dashboard. For tasks nearing their completion deadline, automatic early warnings are triggered, notifying the relevant personnel; for overdue tasks, alarms are automatically escalated and synchronized to higher-level management. After completing all assigned inspection tasks, the drone automatically returns to the designated hangar to perform battery replacement, data clearing, and equipment maintenance. The system automatically updates the task status to completed and generates a complete report for the inspection task, including a list of inspected base stations, flight mileage, flight time, anomaly identification results, handling progress, and relevant attachments, achieving a closed-loop process for the entire inspection task.
[0085] In step S7, predictive maintenance is performed based on the full lifecycle inspection data.
[0086] In this embodiment of the invention, the predictive maintenance module constructs a component degradation trend prediction model based on full lifecycle data, thereby upgrading from passive handling to proactive predictive maintenance.
[0087] Specifically, the first step is to construct a full lifecycle database for telecommunications towers. The system integrates historical inspection data, anomaly handling data, sensor monitoring data, operation and maintenance records, and design and completion documents for all tower base stations, creating a unique full lifecycle file for each tower and enabling full data traceability from construction, network access, operation and maintenance to decommissioning. The database adopts an architecture combining time-series and relational databases. The time-series database stores high-frequency sensor and inspection data, while the relational database stores structured attribute data and operation and maintenance records, supporting millisecond-level data querying and analysis.
[0088] Then, a component degradation trend prediction model was constructed. For the core components of the tower, including the tower structure, antenna, feeder, grounding system, and equipment room, corresponding degradation trend prediction models were built. The data preprocessing step extracted historical inspection data for the corresponding components, including corrosion degree, tilt angle, deformation, temperature value, and anomaly records, and performed data cleaning, normalization, and missing value imputation. Model training used a long short-term memory neural network to construct a time series prediction model, inputting historical degradation data of the components and outputting degradation trend prediction values for the next one, three, and six months. Model validation used historical data for backtesting to ensure the model's prediction accuracy was greater than or equal to 85%, while continuously optimizing the model with new data.
[0089] Regarding predictive maintenance early warning and task triggering, a predictive model is run daily to calculate the future degradation trend of all core tower components. When the predicted degradation level reaches a preset warning threshold, an early warning is automatically triggered, generating a warning report that includes the tower number, component name, predicted degradation level, estimated failure time, risk level, and handling recommendations. For high-risk warnings, a P4-level preventive inspection task is automatically generated and included in the global task scheduling pool. Drones are pre-arranged for specialized inspections and checks to confirm component status and proactively arrange maintenance and handling to prevent failures caused by component degradation.
[0090] Regarding network-wide risk classification and control, based on prediction results and historical anomaly data, the system classifies all tower base stations into three risk levels: high, medium, and low. High-risk towers are subject to focused management, with increased inspection frequency and advance deployment of maintenance resources. The system regularly generates network-wide tower maintenance analysis reports, providing data support for tower renovation, upgrades, and full lifecycle asset management.
[0091] In summary, by constructing a multi-objective optimization model to achieve task allocation and global scheduling for multi-UAV collaboration, the efficiency bottleneck of single-UAV serial operations is overcome. When the number of towers involved in the inspection increases from 500 to 5000, the multi-UAV collaborative scheme, through parallel operation and dynamic task allocation, shows a sublinear increase in total inspection time, while the single-UAV scheme shows a linear increase, resulting in an efficiency improvement of more than 5 times. Simultaneously, the system possesses a fault redundancy mechanism; when a single UAV cannot continue its task due to malfunction or insufficient power, the system automatically schedules a backup UAV to take over the remaining tasks, resulting in a zero task interruption rate.
[0092] Based on the spatial location and geometric parameters of each component provided by the digital twin foundation, dedicated inspection waypoints are generated for core components such as antennas, towers, tower bases, feeders, equipment rooms, and grounding systems. Each waypoint contains precise three-dimensional coordinates, flight attitude, and shooting parameters, ensuring complete coverage of critical areas without omissions. Compared to fixed inspection schemes that rely on manually preset routes, this invention automatically verifies the integrity of route coverage through digital twin pre-simulation, effectively eliminating blind spots, achieving 100% component-level coverage, and reducing the missed inspection rate by more than 90%.
[0093] By integrating four-dimensional multimodal information from visible light images, infrared thermal imaging, LiDAR point clouds, and acoustic signature data, and capturing the correlation features between different modes through a cross-modal attention mechanism, and designing dedicated anomaly recognition sub-models for six types of core components, this invention achieves a leap from general anomaly detection to precise component-level diagnosis. Compared to single-modal solutions relying solely on visible light images, this invention improves the recall rate by more than 30% and reduces the false alarm rate by more than 50% in various anomaly scenarios such as tower tilt, antenna angle deviation, equipment overheating, and feeder damage.
[0094] A two-tiered recognition architecture is employed, consisting of a lightweight edge-based model for real-time initial detection and a high-precision cloud-based model for secondary verification. The edge-based single-frame recognition latency is less than or equal to 200 milliseconds, meeting the real-time requirements of inspections. The cloud-based model performs secondary verification based on large-scale labeled samples and digital twin benchmark data, effectively eliminating false detections at the edge and ensuring accurate anomaly confirmation. This two-tiered collaborative architecture guarantees both real-time response capabilities and recognition accuracy comparable to a centralized, large-model system.
[0095] Based on a federated learning architecture, the network-wide edge gateway models achieve collaborative self-evolution. All raw inspection data is stored locally, with only encrypted model gradient parameters uploaded, ensuring data privacy and security from the outset. Simultaneously, recognition models from different regions and tower types are globally aggregated and share optimization results in the cloud. The model's generalization ability continuously improves with data accumulation, solving the data silo problem of traditional solutions and increasing model iteration efficiency by more than 5 times.
[0096] A fully closed-loop management process has been established, ensuring traceability and controllability throughout the entire process from anomaly detection to resolution. Upon an anomaly trigger, the system automatically generates maintenance tasks and intelligently assigns them to the most suitable maintenance personnel. After resolution, a drone is automatically dispatched for re-inspection and confirmation; the loop is closed only after the re-inspection is passed. Compared to existing technologies that only record anomalies without subsequent tracking, this invention achieves a complete closed loop from anomaly detection to complete elimination, reducing maintenance response time by more than 70%.
[0097] Based on full lifecycle inspection data, a component degradation trend prediction model is constructed, which can predict component degradation trends 1 to 6 months in advance and automatically trigger preventive inspection tasks before failures occur. Compared with the passive operation and maintenance mode of existing technologies that can only achieve after-the-fact alarms, this invention reduces the occurrence rate of unplanned failures of tower base stations by more than 50% and reduces operation and maintenance costs by more than 30%, realizing a fundamental shift from a fault-driven to a data-driven operation and maintenance mode, and significantly improving the stability and availability of communication networks.
[0098] Figure 2 This is a flowchart illustrating a multi-machine collaborative closed-loop intelligent inspection method for tower clusters, as shown below. Figure 2 As shown, it includes: 210: Construct a digital twin foundation for the tower base station cluster and obtain the status information of the inspection drone.
[0099] 220: Extract inspection tasks from the inspection task pool, allocate tasks to the inspection drone based on the priority of the inspection tasks and the status information of the inspection drone, and generate corresponding inspection routes according to the task allocation results and the digital twin base.
[0100] 230: Control the inspection drone to perform inspections along the inspection route, receive multimodal data of the tower base station cluster collected by the inspection drone, identify anomalies in the multimodal data, and adjust the inspection route according to the anomaly identification results.
[0101] 240: Based on the full life cycle inspection data, predict the degradation trend value of the components in the tower base station cluster. When the degradation trend value of the components reaches the warning threshold, generate a preventive inspection task and put it into the inspection task pool.
[0102] In some embodiments, step 210 includes: Collect all-element data of each tower in the tower base station cluster. The all-element data includes building information model drawings, as-built data, spatial coordinates, tower type, component list and spatial location, surrounding geographical information and elevation data.
[0103] Based on the building information model drawings and lidar point cloud data, geometric registration and texture rendering are performed to generate a digital twin corresponding to the physical tower. The digital twin contains the geometric parameters, attribute information and spatial location of each component of each tower.
[0104] By integrating the digital twins of each tower, geographic information, and no-fly zone data, the digital twin foundation of the tower base station cluster is obtained.
[0105] In some embodiments, step 220 includes: The execution order of each inspection task is determined according to its priority, with inspection drones being assigned to high-priority tasks first.
[0106] Based on the status information of each inspection drone, available drones that meet the endurance and space constraints are selected.
[0107] With the optimization goals of minimizing total inspection time, minimizing total flight mileage, and balancing the load of each UAV, each inspection task is assigned to the corresponding available UAV, generating a list of target base stations, inspection sequence, take-off and landing hangars, and battery swapping plans for each available UAV.
[0108] In some embodiments, step 220 includes: Based on the task allocation results, the list of target base stations to be responsible for each inspection drone and the inspection order of each base station are determined, and a global flight segment route is generated, which connects the take-off and landing hangar with each target base station.
[0109] Extract the spatial location, geometric parameters, and component type of each tower and component contained in the current target base station from the digital twin base.
[0110] Obtain the corresponding inspection parameters according to the component type. The inspection parameters include shooting distance, sampling interval, shooting angle, sensor type and safety distance.
[0111] Based on the spatial position of each component, and combined with the corresponding shooting distance and shooting angle, calculate the three-dimensional spatial coordinates, nose orientation, and lens pitch angle of each waypoint.
[0112] Based on the geometric parameters and sampling interval of each component, the sampling density and sampling step size of each waypoint are determined, and a waypoint sequence covering the entire inspection area of the component is generated. Each waypoint includes three-dimensional coordinates, flight attitude, lens focal length, exposure parameters, and dwell time.
[0113] Path optimization is performed on each waypoint sequence to generate an inspection route for the current target base station.
[0114] In some embodiments, step 220 includes: Based on the task allocation results, a list of target base stations to be responsible for each inspection drone is determined.
[0115] Obtain the spatial coordinates of each target base station, the location of the take-off and landing hangar of each inspection drone, and the boundary of the no-fly zone.
[0116] With the optimization objectives of minimizing total flight distance and total power consumption, a path optimization algorithm is used to determine the inspection sequence of each target base station, and a global flight segment route is planned that starts from the take-off and landing hangar, passes through each target base station in sequence, and finally returns to the take-off and landing hangar.
[0117] The system acquires real-time meteorological data and the current remaining power of each of the inspection drones, and uses an energy consumption prediction model to calculate the estimated power consumption of the global flight segment.
[0118] When the estimated power consumption exceeds the preset safety threshold of the current remaining power, a battery swapping plan is inserted into the global flight segment route. The battery swapping plan includes matching the nearest available hangar and a battery swapping time window.
[0119] In some embodiments, step 230 includes: The edge gateway invokes a locally deployed lightweight anomaly detection model to perform real-time preliminary detection on the multimodal data and outputs anomaly detection results, which include anomaly type, anomaly location, and confidence level.
[0120] The anomaly identification results are graded based on the confidence level.
[0121] When the anomaly identification result is a Level 1 anomaly, the current inspection task is suspended, the anomaly identification result is reported to the cloud server, and the emergency handling procedure is triggered.
[0122] When the anomaly identification result is a level 2 or level 3 anomaly, the anomaly identification result is reported to the cloud server for re-examination and confirmation, and the inspection drone is controlled to continue to perform the current inspection task.
[0123] When a route adjustment instruction is received from the cloud server, the inspection route is updated according to the route adjustment instruction.
[0124] In some embodiments, step 240 includes: Historical inspection data of each component is extracted from the full life cycle database. The historical inspection data includes the degree of corrosion, tilt angle, deformation, temperature value and records of abnormal occurrences.
[0125] The historical inspection data is arranged in chronological order to construct a time-series dataset for each component.
[0126] The time series dataset is input into a pre-trained degradation trend prediction model, which outputs the degradation trend prediction values of each component within a preset future time period.
[0127] When the predicted value of the degradation trend reaches the preset warning threshold, a warning report is generated for that component.
[0128] Preventive inspection tasks are generated based on the early warning report, and these preventive inspection tasks are then added to the inspection task pool.
[0129] Figure 3 This is a schematic diagram of a multi-machine collaborative fully closed-loop intelligent inspection device for tower clusters, such as... Figure 3 As shown, it includes: The first processing module is used to build a digital twin foundation for the tower base station cluster and to acquire the status information of the inspection drone.
[0130] The second processing module is used to extract inspection tasks from the inspection task pool, allocate tasks to the inspection drone based on the priority of the inspection tasks and the status information of the inspection drone, and generate corresponding inspection routes according to the task allocation results and the digital twin base.
[0131] The third processing module is used to control the inspection drone to perform inspections along the inspection route, receive multimodal data of the tower base station cluster collected by the inspection drone, identify anomalies in the multimodal data, and adjust the inspection route according to the anomaly identification results.
[0132] The fourth processing module is used to predict the degradation trend value of components in the tower base station cluster based on the full life cycle inspection data. When the degradation trend value of the component reaches the warning threshold, a preventive inspection task is generated and placed into the inspection task pool.
[0133] In some embodiments, the first processing module is specifically used to: collect full-element data of each tower in the tower base station cluster, the full-element data including building information model drawings, as-built data, spatial coordinates, tower type, component list and spatial location, surrounding geographic information and elevation data; perform geometric registration and texture rendering based on the building information model drawings and lidar point cloud data to generate a digital twin corresponding to the physical tower, the digital twin containing the geometric parameters, attribute information and spatial location of each component of each tower; integrate the digital twins of each tower, geographic information and no-fly zone data to obtain the digital twin base of the tower base station cluster.
[0134] In some embodiments, the second processing module is specifically used to: determine the execution order of each inspection task according to the priority of the inspection task, and prioritize the allocation of inspection drones to high-priority tasks; filter available drones that meet the endurance constraints and space constraints according to the status information of each inspection drone; and allocate each inspection task to the corresponding available drone with the optimization objectives of the shortest total inspection time, the lowest total flight mileage, and the balanced load of each drone, and generate a list of target base stations, inspection order, take-off and landing hangars, and battery swapping plans for each available drone.
[0135] In some embodiments, the second processing module is further configured to: determine the list of target base stations to be handled by each inspection drone and the inspection order of each base station according to the task allocation result; generate a global flight route connecting the take-off and landing hangar to each target base station; extract the spatial position, geometric parameters, and component type of each tower and component included in the current target base station from the digital twin base; obtain the corresponding inspection parameters according to the component type, including shooting distance, sampling interval, shooting angle, sensor type, and safety distance; calculate the three-dimensional spatial coordinates, nose orientation, and lens pitch angle of each waypoint based on the spatial position of each component and in combination with the corresponding shooting distance and shooting angle; determine the sampling density and sampling step size of each waypoint according to the geometric parameters and sampling interval of each component, generate a waypoint sequence covering the entire inspection area of the component, each waypoint including three-dimensional coordinates, flight attitude, lens focal length, exposure parameters, and dwell shooting time; optimize the path of each waypoint sequence to generate the inspection route of the current target base station.
[0136] In some embodiments, the second processing module is further configured to: determine the list of target base stations to be handled by each inspection drone based on the task allocation result; obtain the spatial coordinates of each target base station, the location of the take-off and landing hangar of each inspection drone, and the boundary of the no-fly zone; determine the inspection order of each target base station using a path optimization algorithm with the shortest total flight distance and the lowest total power consumption as optimization objectives, and plan a global flight segment route starting from the take-off and landing hangar, passing through each target base station in sequence, and finally returning to the take-off and landing hangar; obtain real-time meteorological data and the current remaining power of each inspection drone, and calculate the estimated power consumption of the global flight segment route using an energy consumption prediction model; when the estimated power consumption exceeds a preset safety threshold of the current remaining power, insert a battery swapping plan into the global flight segment route, the battery swapping plan including matching the nearest available hangar and a battery swapping time window.
[0137] In some embodiments, the third processing module is specifically used for: calling a locally deployed lightweight anomaly identification model through an edge gateway to perform real-time preliminary inspection of the multimodal data, and outputting anomaly identification results, the anomaly identification results including anomaly type, anomaly location, and confidence level; classifying the anomaly identification results according to the confidence level; when the anomaly identification result is a level 1 anomaly, pausing the current inspection task, reporting the anomaly identification result to the cloud server, and triggering an emergency handling procedure; when the anomaly identification result is a level 2 or level 3 anomaly, reporting the anomaly identification result to the cloud server for re-inspection and confirmation, and controlling the inspection drone to continue performing the current inspection task; when receiving a route adjustment instruction returned by the cloud server, updating the inspection route according to the route adjustment instruction.
[0138] In some embodiments, the fourth processing module is specifically used for: extracting historical inspection data of each component from the full lifecycle database, the historical inspection data including corrosion degree, tilt angle, deformation, temperature value and abnormal occurrence records; arranging the historical inspection data in chronological order to construct a time series dataset for each component; inputting the time series dataset into a pre-trained degradation trend prediction model to output a degradation trend prediction value for each component within a preset future time period; generating a warning report for the component when the degradation trend prediction value reaches a preset warning threshold; generating a preventive inspection task based on the warning report, and connecting the preventive inspection task to the inspection task pool.
[0139] In some embodiments, the apparatus further includes: a model training module, configured to acquire historical inspection data and labeled samples from a full lifecycle database, perform feature construction on the historical inspection data to generate a time series dataset, construct a degradation trend prediction model, wherein the degradation trend prediction model is a long short-term memory neural network model, takes the time series dataset as input, outputs the degradation trend prediction value for a preset future time period, optimizes the model parameters through backtesting with historical data, and enables the degradation trend prediction model to learn the mapping relationship between historical time series data and component degradation trends; and uses the trained degradation trend prediction model as a pre-trained degradation trend prediction model.
[0140] In some embodiments, the device further includes: a handling feedback module, configured to push the preventive inspection task to the operation and maintenance platform and receive handling results from operation and maintenance personnel, the handling results including the confirmed degradation type and remediation measures; and to add the handling results as new labeled samples to the full lifecycle database to trigger fine-tuning of the degradation trend prediction model to update the degradation trend prediction model.
[0141] Those skilled in the art will readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the present invention.
[0142] It should be noted that, Figure 3 The division of modules / units is illustrative and represents only one logical functional division; in actual implementation, other division methods are possible. For example, two or more functions can be integrated into a single data acquisition module. The integrated modules described above can be implemented in hardware or as software functional modules.
[0143] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned multi-machine collaborative fully closed-loop intelligent inspection methods for tower clusters. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the multi-machine collaborative fully closed-loop intelligent inspection method for tower clusters shown in any embodiment of the present invention by calling the computer program.
[0144] In one alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The illustrated electronic device 400 includes a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, for example, via a bus 402. Optionally, the electronic device 400 may further include a transceiver 404, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 404 is not limited to one type, and the structure of the electronic device 400 does not constitute a limitation on the embodiments of the present invention.
[0145] Among them, electronic devices can also be terminal devices, which can be any device that can install applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.
[0146] It should be noted that, Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.
[0147] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned multi-machine collaborative closed-loop intelligent inspection methods for tower clusters.
[0148] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.
[0149] It should be noted that the terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of the invention described herein can be implemented in an order other than that shown or described.
[0150] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.
[0151] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A multi-machine collaborative fully closed-loop intelligent inspection method for tower clusters, characterized in that, include: Construct a digital twin foundation for the tower base station cluster and acquire the status information of the inspection drone; The inspection task is extracted from the inspection task pool, and the inspection drone is assigned a task based on the priority of the inspection task and the status information of the inspection drone. The corresponding inspection route is generated according to the task assignment result and the digital twin base. The system controls the inspection drone to perform inspections along the inspection route, receives multimodal data of the tower base station cluster collected by the inspection drone, performs anomaly identification on the multimodal data, and adjusts the inspection route according to the anomaly identification results. Based on the full lifecycle inspection data, the degradation trend value of the components in the tower base station cluster is predicted. When the degradation trend value of the components reaches the warning threshold, a preventive inspection task is generated and placed in the inspection task pool.
2. The method according to claim 1, characterized in that, The digital twin foundation for constructing the tower base station cluster includes: Collect all-element data of each tower in the tower base station cluster. The all-element data includes building information model, as-built data, spatial coordinates, tower type, component list and spatial location, surrounding geographic information and elevation data. Based on the building information model and lidar point cloud data, geometric registration and texture rendering are performed to generate a digital twin corresponding to the physical tower. The digital twin includes the geometric parameters, attribute information and spatial location of each component of each tower. By integrating the digital twins of each tower, geographic information, and no-fly zone data, the digital twin foundation of the tower base station cluster is obtained.
3. The method according to claim 1, characterized in that, The process of allocating tasks to the inspection drone based on the priority of the inspection task and the status information of the inspection drone includes: The inspection drones are assigned to each inspection task according to its priority. Based on the status information of each inspection drone, select available drones that meet the endurance and space constraints; With the optimization goals of minimizing total inspection time, minimizing total flight mileage, and balancing the load of each UAV, each inspection task is assigned to the corresponding available UAV, generating a list of target base stations, inspection sequence, take-off and landing hangars, and battery swapping plans for each available UAV.
4. The method according to claim 1, characterized in that, The step of generating corresponding inspection routes based on task allocation results and the digital twin infrastructure includes: Based on the task allocation results, the list of target base stations to be responsible for each inspection drone and the inspection order of each base station are determined, and a global flight segment route is generated, which connects the take-off and landing hangar with each target base station. Extract the spatial location, geometric parameters, and component type of each tower and component contained in the current target base station from the digital twin base; Obtain the corresponding inspection parameters according to the component type. The inspection parameters include shooting distance, sampling interval, shooting angle, sensor type and safety distance. Based on the spatial position of each component, and combined with the corresponding shooting distance and shooting angle, calculate the three-dimensional spatial coordinates, nose orientation and lens pitch angle of each waypoint; Based on the geometric parameters and sampling interval of each component, the sampling density and sampling step size of each waypoint are determined, and a waypoint sequence covering the entire inspection area of the component is generated. Each waypoint includes three-dimensional coordinates, flight attitude, lens focal length, exposure parameters and dwell time. Path optimization is performed on each waypoint sequence to generate the inspection route for the current target base station.
5. The method according to claim 4, characterized in that, The process of determining the list of target base stations to be handled by each inspection drone and the inspection order of each base station based on the task allocation results, and generating a global flight path, includes: Based on the task allocation results, determine the list of target base stations that each inspection drone is responsible for. Obtain the spatial coordinates of each target base station, the location of the take-off and landing hangar of each inspection drone, and the boundary of the no-fly zone; With the optimization objectives of minimizing total flight distance and total power consumption, a path optimization algorithm is used to determine the inspection sequence of each target base station, and a global flight segment route is planned from the take-off and landing hangar, passing through each target base station in sequence, and finally returning to the take-off and landing hangar. Acquire real-time meteorological data and the current remaining power of each of the inspection drones, and use an energy consumption prediction model to calculate the estimated power consumption of the global flight segment route; When the estimated power consumption exceeds the preset safety threshold of the current remaining power, a battery swapping plan is inserted into the global flight segment route. The battery swapping plan includes matching the nearest available hangar and a battery swapping time window.
6. The method according to claim 1, characterized in that, The step of identifying anomalies in the multimodal data and adjusting the inspection route based on the anomaly identification results includes: The edge gateway invokes a locally deployed lightweight anomaly detection model to perform real-time preliminary detection on the multimodal data and outputs anomaly detection results, which include anomaly type, anomaly location, and confidence level. The anomaly identification results are classified according to the confidence level; When the anomaly identification result is a Level 1 anomaly, the current inspection task is suspended, the anomaly identification result is reported to the cloud server, and the emergency response procedure is triggered. When the anomaly identification result is a level 2 or level 3 anomaly, the anomaly identification result is reported to the cloud server for re-examination and confirmation, and the inspection drone is controlled to continue to perform the current inspection task. When a route adjustment instruction is received from the cloud server, the inspection route is updated according to the route adjustment instruction.
7. The method according to claim 1, characterized in that, The process involves predicting the degradation trend value of components in the tower base station cluster based on full lifecycle inspection data. When the degradation trend value of a component reaches a warning threshold, a preventative inspection task is generated, including: Historical inspection data of each component is extracted from the full life cycle database. The historical inspection data includes the degree of corrosion, tilt angle, deformation, temperature value and records of abnormal occurrences. Arrange the historical inspection data in chronological order to construct a time-series dataset for each component; The time series dataset is input into a pre-trained degradation trend prediction model, which outputs the degradation trend prediction values of each component within a preset future time period. When the predicted value of the degradation trend reaches the preset warning threshold, a warning report is generated for the component. Preventive inspection tasks are generated based on the early warning report, and these preventive inspection tasks are then added to the inspection task pool.
8. A multi-machine collaborative fully closed-loop intelligent inspection device for iron tower clusters, characterized in that, include: The first processing module is used to build the digital twin foundation of the tower base station cluster and obtain the status information of the inspection drone. The second processing module is used to extract inspection tasks from the inspection task pool, allocate tasks to the inspection drone based on the priority of the inspection tasks and the status information of the inspection drone, and generate corresponding inspection routes according to the task allocation results and the digital twin base. The third processing module is used to control the inspection drone to perform inspections along the inspection route, receive multimodal data of the tower base station cluster collected by the inspection drone, identify anomalies in the multimodal data, and adjust the inspection route according to the anomaly identification results. The fourth processing module is used to predict the degradation trend value of components in the tower base station cluster based on the full life cycle inspection data. When the degradation trend value of the component reaches the warning threshold, a preventive inspection task is generated and placed into the inspection task pool.
9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the multi-machine collaborative closed-loop intelligent inspection method for tower clusters as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement the multi-machine collaborative closed-loop intelligent inspection method for tower clusters as described in any one of claims 1 to 7.
