Multi-mode communication base station unmanned aerial vehicle inspection method and system

Abstract

The invention discloses a multi-modal communication base station unmanned aerial vehicle inspection method and system, and relates to the technical field of unmanned aerial vehicle inspection, and the method comprises the steps: constructing a parameterized flight path curve, and generating an executable flight path sequence in combination with speed profile design and observation window triggering occupation; the unmanned aerial vehicle flies according to an executable track sequence, and when entering a window, the unmanned aerial vehicle triggers a visible light camera, a thermal infrared imager and a laser radar in sequence for collection; real-time return or cache supplementary transmission is carried out according to communication segments, archiving is carried out according to batches after uploading to the cloud, and cloud identification is executed; cloud recognition results are classified according to task units and observation windows, a result database and report files are generated and written into a feedback library, and flight logs and cache records are synchronized to an execution library. According to the method, integrated design of route planning and triggering is achieved through multi-source data fusion, and it is guaranteed that the observation pose is accurate and the time sequence is controllable; and edge preprocessing and a segmentation supplementary transmission mechanism are combined, so that the data integrity of the weak coverage section is ensured.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, specifically to a UAV inspection method and system for multimodal communication base stations. Background Technology

[0002] In recent years, the number of communication base stations has grown exponentially with the rapid deployment of 5G and the future 6G, significantly increasing the structural complexity and distribution density of base station facilities. Traditional manual inspection methods face problems such as long cycles, high risks, and discontinuous data in high-altitude, mountainous, and remote areas. Therefore, drone-based inspections are gradually becoming the mainstream approach. With the maturity of drone flight control technology and the improvement of its payload capacity, payloads integrating multimodal sensors such as visible light, infrared, and lidar are becoming a trend. At the same time, research on mission planning, trajectory generation, and edge computing is deepening, especially parametric trajectory modeling combining GIS, DEM, and historical inspection data, which can improve the automation level of inspection tasks to a certain extent. However, most current applications are still concentrated on single-modal image acquisition and simple trajectory planning, lacking systematic research on multi-source information fusion and full-process intelligent processing.

[0003] Existing technologies still have significant shortcomings in practical applications. First, in terms of trajectory planning, most methods are based solely on geometric shortest paths or simple energy consumption models, failing to comprehensively consider the joint optimization of curvature constraints, communication coverage, flight time, and energy consumption. Therefore, it is difficult to generate trajectory sequences that balance flight safety and data integrity. Second, in terms of data acquisition, existing solutions generally rely on single visible light images, resulting in significant deficiencies in target perception under complex environments such as low light, backlight, icing, or high temperatures. The combined use of infrared and point cloud technologies has not yet developed into a mature solution. Third, in terms of data processing, traditional UAV inspections mostly rely on post-event offline analysis, lacking edge computing and cloud collaboration mechanisms. This results in the inability to achieve data caching and delayed backhaul in areas with no or weak signal, and also makes real-time or near-real-time batch identification difficult. Finally, in terms of result classification and feedback mechanisms, existing technologies often remain at the single-report level, lacking a closed-loop optimization framework that feeds back identification results, verification data, and flight logs to mission planning. Therefore, existing technologies cannot achieve a fully integrated inspection process encompassing task planning, multimodal acquisition, edge and cloud collaborative processing, and result feedback, and cannot obtain the systematic and intelligent processing effects of this invention. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is that existing UAV inspection methods have problems such as lack of multi-parameter joint modeling in trajectory planning, insufficient multimodal data acquisition to cover complex environments, lack of edge computing and cloud collaboration mechanisms, and how to achieve closed-loop optimization and result feedback of the entire inspection process.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a UAV inspection method for multimodal communication base stations, comprising collecting base station, line, DEM, and historical data; generating a task point set and allocating collection radii; constructing parameterized trajectory curves; and generating an executable trajectory sequence by combining velocity profile design and observation window triggering. The UAV flies according to the executable trajectory sequence, with gimbal and sensor extrinsic parameters calibrated. Upon entering the window, the visible light camera, infrared thermal imager, and lidar are sequentially triggered to collect data. Each modal data is indexed by batch number and timestamp and written to the cache. The batched modal data is preprocessed in the onboard edge unit, and real-time transmission or cached retransmission is performed according to communication segments. After being uploaded to the cloud, the data is archived by batch and cloud recognition is executed. The cloud recognition results are categorized by task unit and observation window, generating a result database and report file, which are written to the feedback database. Flight logs and cache records are synchronized to the execution database.

[0007] As a preferred embodiment of the UAV inspection method for multimodal communication base stations described in this invention, the step of generating a task point set and allocating a sampling radius includes establishing a multi-layered surrounding observation circle around the base station object based on the camera focal length and desired resolution, sampling at fixed intervals on the broken line of the line object to generate multi-layered, multi-angle candidate observation poses, and forming the final task point set after being eliminated by no-fly zones and obstacle zones.

[0008] As a preferred embodiment of the UAV inspection method for multimodal communication base stations described in this invention, the construction of parameterized trajectory curves includes generating trajectories under control point set constraints using B-splines or spiral models, and outputting trajectory curves within a preset threshold range where the curvature, climb angle, and elevation changes satisfy the preset threshold range.

[0009] As a preferred embodiment of the UAV inspection method for multimodal communication base stations described in this invention, the velocity profile design includes generating a piecewise velocity function based on the trajectory curvature, slope, and observation window dwell time, constraining the speed, acceleration, and jump of the UAV to be within the allowable range of flight control, and embedding sensor triggering placeholder parameters at the observation window.

[0010] As a preferred embodiment of the UAV inspection method for multimodal communication base stations described in this invention, the modal data is written to the cache with batch number and timestamp index, including batch organization when writing to the cache. The batch file contains task unit ID, track segment number, batch number, timestamp, attitude information and sensor status field, and is stored using a circular buffer management strategy.

[0011] As a preferred embodiment of the UAV inspection method for multimodal communication base stations described in this invention, the preprocessing of the airborne edge unit includes performing ambiguity discrimination and duplicate frame merging on the visible light image, performing dark field offset correction on the infrared thermal image data, performing denoising and downsampling on the laser point cloud data, and generating a preprocessing result with a batch number.

[0012] As a preferred embodiment of the UAV inspection method for multimodal communication base stations described in this invention, the generation of the result database and report file, and the writing of the feedback database, includes writing the manually reviewed identification results into the feedback database; at the same time, the flight log, speed deviation and cache records are written into the execution database for the correction of the task point density and communication segmentation strategy of the next task.

[0013] Another objective of this invention is to provide a multimodal communication base station UAV inspection system, which can generate an executable trajectory sequence by constructing parameterized trajectory curves, combining velocity profile design and observation window triggering, thus solving the problem of lack of multi-parameter joint modeling in current UAV inspection methods for trajectory planning.

[0014] As a preferred embodiment of the multimodal communication base station UAV inspection system described in this invention, the system includes: a trajectory planning module, a multimodal acquisition module, and a data processing module. The trajectory planning module is used to collect base station, line, DEM, and historical data, generate a task point set and allocate acquisition radii, construct parameterized trajectory curves, and combine velocity profiles with observation window triggering to form an executable trajectory sequence. The multimodal acquisition module is used for the UAV to fly according to the planned trajectory, calibrate the gimbal and sensor extrinsic parameters, and sequentially trigger visible light, infrared, and lidar acquisition when entering the window, and write each modal data into the cache with batch number and timestamp index. The data processing module is used for batch data to enter the airborne edge unit for preprocessing, and then perform real-time backhaul or cache retransmission according to communication segments. After uploading to the cloud, the data is archived and multimodal recognition is performed. Finally, a result database and report file are generated and written into the feedback library and execution library.

[0015] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement a method for unmanned aerial vehicle (UAV) inspection of a multimodal communication base station.

[0016] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a method for inspecting multimodal communication base stations using unmanned aerial vehicles (UAVs).

[0017] The beneficial effects of this invention are as follows: The multimodal communication base station UAV inspection method provided by this invention introduces multi-source fusion of base stations, lines, DEMs, and historical data during the task planning stage, establishes parameterized flight path curves, and combines the binding relationship between velocity profiles and trigger windows. This achieves an integrated design of acquisition pose, flight constraints, and sensor triggering, thereby ensuring accurate spatial positioning and temporal synchronization during the inspection process. In the multimodal acquisition stage, by executing flight paths through the UAV and combining external parameter calibration and attitude control, the synchronous acquisition of visible light, infrared, and lidar data can be completed sequentially within a specified window. Data is organized into a unified data structure using batch numbers and timestamp indexes, ensuring high consistency of different modalities in the spatiotemporal dimensions. In the edge processing and transmission stage, the onboard unit performs preprocessing on the raw data, such as fuzzy removal, duplicate merging, and point cloud downsampling. Real-time backhaul or buffered retransmission is performed according to communication segments, ensuring data integrity and transmission stability in weak or no-signal areas. This enables batch archiving and centralized identification in the cloud. In the result generation and feedback stage, the identification results are categorized into task units to form a result database and standardized reports. Manual verification and annotation, along with flight logs, are fed back to the training set and execution library, constructing a closed-loop optimization mechanism for task planning, data acquisition, processing, and feedback. Thus, this invention achieves traceability in data acquisition and processing, integrity in cross-modal fusion, stability in edge and cloud collaboration, and iterative optimization of task parameters at every stage of the entire chain, ultimately forming a systematic, structured, and evolvable UAV inspection method for communication base stations. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 The first embodiment of the present invention provides an overall flowchart of a UAV inspection method for a multimodal communication base station. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0021] Example 1, referring to Figure 1 As an embodiment of the present invention, a method for UAV inspection of multimodal communication base stations is provided, comprising: S1: Collect base station, line, DEM and historical data, generate task point set and allocate collection radius, construct parameterized track curve, combine velocity profile design and observation window trigger placeholder, and generate executable track sequence.

[0022] Furthermore, in the implementation of this invention, it is first necessary to establish a task space and data base. The backend management platform will uniformly load multi-source data related to inspection, including the spatial location and geometric information of communication base stations, towers, poles, and optical cable lines, and combine them with GIS vector base maps and DEM digital elevation models to form a three-dimensional operating environment. At the same time, the boundary information of no-fly zones, height-restricted zones, and key protection zones will be imported to generate compliant flight paths subsequently. Historical inspection records will also serve as an important reference, including past task trajectories, camera poses, defect candidate points, and verification annotations, to provide empirical prior information in new flight path planning. Through the fusion and benchmark alignment of the above data, an operating reference surface and task bounding box are established to provide a stable spatial reference for flight path generation.

[0023] After the data space is constructed, the inspection objects need to be described in a graph structure. Base stations, towers, and equipment rooms are defined as first-level nodes, optical cable lines are represented as second-level structures in a polygonal form, each equipment component (such as antennas, feeders, and hardware) is abstracted as a third-level auxiliary node, and the actual observation pose is mapped as a fourth-level observation point. Each node is accompanied by attribute information, including spatial location, geometric dimensions, allowable observation angle interval, minimum safety distance, area control label, and desired image scale. Through this four-level organizational structure, an object graph representation can be formed as follows:

[0024] in, For a set of nodes, For the set of connecting edges, This is an object diagram.

[0025] Based on the object graph, each communication base station and its adjacent line segments can be divided into independent task units, thereby forming a batch inspection task queue.

[0026] Next, a task point set needs to be generated and its acquisition radius determined. For base station objects, multiple layers of surrounding observation circles are generated at the tower center. The radii of these observation circles are calculated based on the camera focal length, sensor size, and desired ground resolution. Simultaneously, multiple acquisition heights are set layer by layer on the DEM, taking into account the tower height, to cover the tower base, middle, and top. For line objects, sampling is performed along the fiber optic cable fold at fixed intervals, and observation poses are generated on both sides in the normal direction of each sampling point to ensure that the line structure is recorded from multiple angles. For areas with significant terrain undulations, additional height correction values ​​are added to ensure the relative height stability of the acquisition points and the operational reference plane. All candidate poses are aggregated into the task point set. After no-fly zone removal and obstacle buffering correction, a valid task point set is obtained, with each point accompanied by a timestamp, batch number, recommended speed value, and sensor trigger mode information.

[0027] It should be noted that after the task point set is determined, a parametric trajectory curve needs to be constructed and a velocity profile designed. First, a feasible connected graph is established using the task points as endpoints, and initial edge weights based on Euclidean distance and terrain cost are calculated. The initial trajectory is often a polygonal path; to meet the aircraft's smoothness and curvature constraints, B-splines or spirals are used for curvature processing to ensure the path has C² continuity and avoid flight instability caused by sharp turns. For base station observation tasks, a closed loop curve needs to be generated around the tower center and spliced ​​with the main trajectory under continuous conditions. Regarding velocity design, segmented velocity profiles are generated based on curvature distribution, slope, and acquisition point windows to ensure that the speed, acceleration, and jump during flight do not exceed the kinematic limits of the flight control platform. In each acquisition interval, trigger positions for camera shutter, infrared integration time, and laser point cloud sampling density are reserved in the trajectory to achieve precise triggering of subsequent acquisition stages.

[0028] Furthermore, after the set of mission points is determined, mathematical modeling is needed to connect the discrete observation points into a continuous and feasible flight path, and to assign a reasonable velocity profile to the UAV. This process mainly includes path curve modeling, curvature and continuity constraints, velocity segmentation design, and the placement of observation trigger points.

[0029] Set of task points Construct a feasible connected graph with endpoints. , where the set of nodes and the set of edges Includes all connections that satisfy safety and feasibility constraints. Edge weights are determined by both geometric distance and terrain cost.

[0030] in, Indicates task point and The edge weight between the two edges represents the total cost of that connected edge. This represents the terrain cost weighting coefficient, used to adjust the degree of influence of terrain slope on the overall cost. The elevation function corresponding to a Digital Elevation Model (DEM) takes spatial location coordinates as input and outputs the terrain elevation value at that location. Indicates the slope of the terrain.

[0031] The initial polyline path needs to be smoothed into a parametric curve, which can be modeled using B-splines as follows:

[0032]

[0033] in, Indicated by arc length parameter The function representing the track position. B-spline basis functions, express, Indicates the curve at Curvature of position, This indicates the maximum curvature constraint that the drone can withstand.

[0034] After generating the parametric curves, segmented velocity profiles need to be designed. The velocity function is:

[0035] in, Indicates the position of the arc length parameter Corresponding flight speed, This represents the acceleration along the flight path. The lateral acceleration caused by curvature is represented by the following constraints: .

[0036] In each observation window Internally, space needs to be reserved for sensor triggering:

[0037] in, Indicates the first The track interval corresponding to each observation window This indicates the minimum dwell time required to complete data collection for this window. Indicates the first The starting arc length of each observation window corresponds to the position of the UAV's flight path when it enters the window and begins data collection. Indicates the first The terminating arc length of each observation window corresponds to the position of the UAV's flight path when it leaves the window and ends the data collection.

[0038] S2: The UAV flies according to the executable flight path sequence, the gimbal and sensor extrinsic parameters are calibrated, and when it enters the window, the visible light camera, infrared thermal imager and lidar are triggered in sequence to collect data. The data of each mode is written to the cache with batch number and timestamp index.

[0039] Furthermore, after the trajectory sequence and velocity profile are generated, the airborne payload is calibrated and assembled using a unified coordinate system. The optical axes of the unmanned aerial vehicle system, gimbal system, and each sensor are denoted as follows: , , Obtained through offline extrinsic parameter solving and And write the extrinsic parameter table of the mission session before takeoff; camera intrinsic parameter matrix The infrared response curve and lidar scanning geometry parameters were calibrated on the ground and then loaded into the mission. The gimbal employs three-axis steady-state pointing control. The gimbal attitude commands are jointly generated by the track tangency, target pointing, and observation window field of view constraints. The command set is timestamped and written into the "attitude-trigger script". This payload configuration involves a combination of a visible light camera (4K resolution), an infrared thermal imager (temperature measurement accuracy ±2℃), and a lidar (scanning radius 50m). The parameters are derived from the established equipment list and project constraints. Time synchronization during mission execution is achieved jointly by the airborne GNSS / BeiDou PPS pulse and the local clock. All sensors receive a one-time time synchronization at mission start and write timestamps triggered by frame arrival interruptions during data acquisition. The time reference is expressed in UNIX epoch nanoseconds, and time drift is corrected through periodic timestamp comparisons. Before entering a window, the flight controller buffers "gimbal preset attitude, camera exposure mode, infrared integration position, and laser line frequency." Upon window arrival, the trigger sequence is executed according to the script, and the "trigger-attitude-position" triplet is recorded. The triplets are archived with the same batch number for subsequent processing to perform cross-modal alignment based on the batch. The visible light camera's shooting strategy prioritizes ground resolution (GSD) and target coverage ratio. Exposure modes are switched based on metering window statistics before entering the shooting window. The product of the rolling shutter readout time and flight speed does not exceed the permissible displacement threshold for a single frame. Lens focal length, distortion coefficient, and image stabilization status are written into the EXIF ​​extended domain of each frame. The infrared thermal imager acquires frames within the shooting window at a set integration time. The integration time is not less than the lower limit of the window dwell time and does not exceed the thermal imager's noise equivalent temperature difference requirement. Thermal image data includes a non-uniformity correction (NUC) sequence number and a blackbody reference label for subsequent temperature field reconstruction. The lidar is set to a line frequency or point frequency within the shooting window, and the point cloud coordinates are... → → → Projecting the transformation chain onto the task geographic coordinate system Furthermore, track tangential and normal information are added during projection to record the scanning phase; the lidar outside the window maintains a low duty cycle to reduce invalid sampling. The above three acquisition channels execute uninterrupted trigger scripts according to the preset obstacle avoidance response time (≤0.5s) and window priority in obstacle avoidance environments and complex terrain. The trigger scripts are fixed in the flight control and payload control channels during the mission generation stage.

[0040] It should be noted that data is written to high-speed solid-state storage on the airborne side using a "batch-modality-frame number" index. Visible light images are saved with lossless or visually lossless compression, infrared frames are saved as radiometrically calibrated 16-bit grayscale matrices or manufacturer RAW streams, and laser point clouds are written in block-based LAS / PCD (or compatible with custom binary blocks). Each data block has a 128-byte header containing the mission unit ID, track segment ID, batch number, observation window number, attitude quaternion, latitude and longitude coordinates, velocity scalar, and sensor status bits. A ring buffer and write amplification control are used during the writing process, with the block size adapting between 64–256MB to meet continuous write rates and power-off recovery requirements. The ring buffer's water level threshold and clearing strategy are set by the mission policy file before takeoff and are consistent with the communication segment configuration to enable buffering and retransmission mechanisms in areas without signal.

[0041] To improve stability and consistency during the acquisition phase, a payload self-check process is performed before the mission begins. This includes lens cleanliness checks, gimbal zero-centering, infrared thermal imager thermal balance timing, and lidar spin detection. After the self-check passes, a "zero-frame" metadata is saved on the ground. The zero-frame contains all payload external parameters, current temperature and humidity, battery temperature, and RMS values ​​of the aircraft vibration, serving as the operational context for each frame during the mission. If temperature drift exceeding a set threshold is detected during the mission, a "calibration window" is inserted without altering the flight path to perform infrared NUC thermal reference imaging and camera exposure baseline sampling. The insertion point of the calibration window is located on a flat segment outside non-critical observation windows. The insertion action is executed by the payload control script at the window switching boundary. Insertion records and batch numbers are consecutively numbered and saved for subsequent processing identification and rejection.

[0042] Quality control is recorded locally at the acquisition end using lightweight indicators without triggering reprocessing. Visible light frames record the Laplacian variance and grayscale histogram quantiles as placeholder indicators for sharpness and exposure. Infrared frames record the temperature difference statistics of line mean and field mean. Laser point cloud records the effective point ratio and scan sector coverage for each block. If a placeholder indicator falls below a threshold, a "retake suggestion" flag with a reason code is recorded at the end of the window, instead of a direct replacement decision at the acquisition end. All placeholder indicators and reason codes are appended to an index file in the batch directory as a CSV file. The index file is written to disk and verified when the task is closed.

[0043] Furthermore, the task data is organized according to a six-level directory structure: "Task Session—Task Unit (TU)—Track Segment—Batch—Modality—Frame". File names use a fixed prefix combined with a timestamp suffix, such as TU12_SEG03_BAT05_RGB_20250929T031523.456Z, TU12_SEG03_BAT05_IR_20250929T031523.789Z, and TU12_SEG03_BAT05_LDR_20250929T031524.012Z. The timestamp accuracy is no less than milliseconds and is aligned with PPS. Three-modal files from the same batch are grouped together in the index table with the same BATCH_ID to facilitate subsequent edge preprocessing and cloud-based deep analysis. This batch-based acquisition and batch-numbered data organization method is consistent with the project's process of "triggering acquisition by velocity profile and generating data packets with batch numbers".

[0044] When the return to home is triggered by either the end of the mission or the reaching of the power threshold, the acquisition end completes the shutdown in the order of "first terminate the trigger, then freeze the attitude, and then shut down the payload". Subsequently, the mission tail record is written. The tail record contains a hash digest of the list of files written, the size of the unwritten disk cache, the power failure recovery point and a summary of statistical placeholder indicators. The tail record is used for file integrity verification and segmentation preparation on the receiving side.

[0045] S3: Batch-processed data of each modality enters the airborne edge unit for preprocessing, and is transmitted back in real time or cached and supplemented according to communication segments. After being uploaded to the cloud, it is archived in batches and cloud recognition is performed.

[0046] Furthermore, after multimodal acquisition is completed and batch-numbered data is generated, all data first enters the onboard edge computing unit for real-time preprocessing. The edge computing unit is equipped with an ARM multi-core processor and a GPU module, and the operating system uses a real-time kernel. The data input channels include camera stream, infrared stream, and laser point cloud stream. When each frame or batch of data enters the processing queue, it first undergoes data integrity verification, which includes frame header hash, batch number continuity, and timestamp monotonicity. Subsequently, a preprocessing algorithm chain is executed, which includes blurred image removal, duplicate frame merging, infrared dark field bias correction, and point cloud denoising and downsampling. The output of this processing chain is cached in local circular storage in the form of a "batch number-modality-preprocessing result" triple, along with a pointer index of the original data, for subsequent uplink or delayed transmission.

[0047] During data caching and preprocessing, edge units need to schedule communication segments according to the task policy file. If the track segment is within the real-time backhaul interval, the data is streamed in batches via the 5G module; in areas with no or weak signal, only batch metadata is recorded and the complete data is stored in the cache, waiting for the next signal to recover before retransmission. The transmission protocol adopts a TCP / UDP hybrid mode, with image data using UDP by default and defect candidate metadata and point cloud blocks using TCP by default, to ensure reliability and sequential consistency. The maximum cache level is set by the task configuration file; when the threshold is exceeded, a cyclic overwrite is triggered, with the overwrite strategy prioritizing the discarding of non-critical window data marked as low priority.

[0048] It should be noted that after data is transmitted to the cloud, it enters a centralized processing platform. The cloud platform is configured with distributed file storage and a parallel computing framework. Preprocessed data is first archived by the scheduler according to batch number and task unit ID, and an index is generated in the distributed database. The cloud processing pipeline performs fusion analysis on the multimodal data: visible light images are processed by a convolutional neural network to extract feature vectors; infrared frames are processed by a temperature correction module to generate a temperature field matrix; and laser point clouds are processed by a 3D registration algorithm to construct a local geometric model. The features of each modality are concatenated in the feature fusion layer. The fused feature tensors are input into the classifier and regressor in batches, outputting structured recognition results, including defect category labels, coordinate locations, and risk levels. The entire processing pipeline uses batch numbers as indexes to ensure the temporal and spatial correspondence of the multimodal data.

[0049] The edge processing results retain the track point ID and observation window number, and establish a mapping with the original task object map. The result file contains fields such as defect category, confidence level, spatial coordinates, and batch number, and generates a result table corresponding to the task unit. The result table is stored in both JSON and CSV formats, which facilitates interface calls with the backend system and manual review and retraining. After data processing is complete, the batch cache of the edge unit will be released according to the receipt to ensure that the limited storage space can be reused in subsequent track segments.

[0050] S4: Cloud-based identification results are categorized by task unit and observation window, generating a results database and report file, which are written to the feedback database. Flight logs and cache records are synchronized to the execution database.

[0051] Furthermore, after the cloud-based fusion processing generates the identification results, the system needs to classify and store the results according to task units, track segments, and observation window numbers. First, the scheduler uses the batch number as the primary key to map the identification outputs to nodes in the task object graph, ensuring that each defect result is bound to a specific tower component, line section, or equipment part. The classified results are written to a unified results database. The results table fields include TASK_UNIT_ID, SEGMENT_ID, WINDOW_ID, BATCH_ID, DEFECT_TYPE, LOCATION_COORD, RISK_LEVEL, and CONFIDENCE. Redundant copies of the results files are generated when they are written to disk and stored in a distributed file system to ensure consistency for subsequent multi-party calls.

[0052] After the results are generated, the system triggers the report compilation process. The report generator extracts archived information from the results database and automatically generates a mission report by combining track curves, observation window parameters, and thumbnails of multimodal data. The report consists of two parts: a structured data table and visual attachments. The data table contains the defect type, geographic coordinates, and corresponding batch number, while the visual attachments include selected image frames, thermal matrix slices, and local point cloud models. Metadata tags are embedded in the generated report before storage. Tag fields include timestamps, equipment numbers, and track version numbers for easy traceability and retrieval. Report files are grouped according to mission units and linked to the historical mission database to ensure consistent organization of data across different periods.

[0053] After the results generation phase is completed, a manual review and feedback mechanism is required. Reviewers retrieve reports through the operations and maintenance management platform interface and manually annotate and revise the recognition results. The review results are written to a "feedback database," which shares the same storage format as the original training set, ensuring it can be directly read by the deep learning framework. Feedback data periodically enters the model update process, merging with existing samples in the transfer learning pipeline, and updating the model weights after data augmentation and batch training. The updated model, after successful validation, is written to the model repository and deployed to the drone platform via the OTA mechanism of edge nodes, ensuring the gradual improvement of the recognizer's performance in future tasks.

[0054] It should be noted that closed-loop optimization also requires handling corrections at the task planning level. Flight logs, velocity profile deviations, and buffer water level records generated during task execution are written to the "execution database." This execution data is used by the scheduler and cost function optimizer after the task is completed, forming the basis for correcting planning parameters. For example, if a certain flight path segment has a large amount of redundant shooting or buffer congestion, the task point density, velocity configuration, or communication segmentation strategy for that segment will be adjusted in the next planning iteration. The execution database and the feedback database are independent of each other, but both are associated with the same task object graph through task unit IDs, thus forming a data closed loop covering "planning—acquisition—identification—feedback."

[0055] Finally, the entire closed-loop optimization process forms a unified version control system. Task reports, review data, flight logs, and update models are all indexed by version numbers, which are generated by combining timestamps and task unit IDs. Each iteration generates a new branch in the repository; old versions are not overwritten but marked as historical snapshots. This ensures that all identification results and planning parameters are traceable, avoiding issues that cannot be reproduced after updates, and also provides a data foundation for subsequent large-scale statistical analysis and cross-task comparisons.

[0056] Example 2, one embodiment of the present invention, provides a method for UAV inspection of multimodal communication base stations. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0057] First, the experiment selected a suburban "three stations and one line" scenario as the verification object, including TU01 (micro base station at the foot of a mountain + 30m tower), TU02 (macro base station on a plain + 45m tower), and TU03 (cable section crossing hills, including two corner towers). Before the task started, the backend management platform loaded the communication asset layer, optical cable line, GIS vector base map, and DEM digital elevation model, overlaid the boundaries of local no-fly zones and height-restricted zones, and unified them to the same coordinates and elevation benchmark. Camera poses, defect candidate points, and verification annotations from the past two years were retrieved from the historical inspection database and used as prior information for task point generation. Then, the inspection objects were organized using a "four-layer object diagram": stations / towers are the first-level nodes, line polygons are the second-level structures, antennas, feeders, hardware, etc. are the third-level auxiliary nodes, and observation poses are the fourth-level nodes. The system generates multi-layered surrounding observation circles for the cores of TU01 and TU02 towers according to the camera focal length and desired ground resolution, and samples along the line skeleton of TU03 at fixed intervals. At the same time, it generates cross-view poses on both sides of the normal. It applies height correction to the points on undulating terrain, completes candidate pose merging and obstacle buffer clipping, and obtains a set of legal task points and their trigger parameter occupancy.

[0058] The trajectory construction phase connects mission points to form an initial polyline path, with closed loop segments configured in the tower center region and smooth transition segments at sharp bends. The overall approach uses a continuous curve scheme to meet flight stability constraints. The flight speed profile provides segmented speeds based on curvature distribution, longitudinal slope, and observation window dwell requirements. Within each window, trigger positions for camera shutter speed, infrared integration, and laser line frequency are written. Before takeoff, overall payload verification is completed: extrinsic parameters of the airframe, gimbal, and three types of sensors are imported into the mission session table; the visible light camera performs distortion and exposure baseline checks; the infrared thermal imager completes thermal balance timing; and the lidar confirms spin and data frame rate. GNSS / BeiDou PPS pulses are synchronized with the local clock once, generating "zero-frame" metadata as the runtime context.

[0059] During the execution phase, the UAV flies according to the executable flight path sequence. Before entering the observation window, the flight controller loads the "gimbal preset attitude, camera exposure mode, infrared integration placeholder, and laser line frequency" into the cache. When the window is triggered, the trigger sequence of visible light, infrared, and lidar is output according to the script, and the "trigger-attitude-position-timestamp-batch number" information is recorded and written to high-speed NVMe storage in batches. Visible light images are saved in a visually lossless manner, infrared frames are saved in 16-bit grayscale after radiometric calibration, and point clouds are written in blocks using LAS / PCD. Each data block includes header information such as task unit ID, flight path ID, batch number, window number, attitude quaternion, latitude, longitude, altitude, and velocity scalars. In weak coverage areas, a caching strategy is activated, and a retransmission marker is added; in covered areas, streaming uploads are performed through the 5G backhaul gateway. At the end of the mission, the shutdown sequence of "first terminate triggering, then freeze attitude, then shut down payload" is triggered and written to the tail record, which includes file list hash, unwritten cache, and power-off recovery point. The entire implementation process was strictly carried out in accordance with the policy documents and security boundaries to ensure data integrity and traceability.

[0060] Table 1 Experimental Data

[0061] As can be seen from the two structural indicators of legal point retention rate and track length, the scheme of this invention achieves a retention rate of 91.5% to 93.2% in TU01 / TU02 / TU03, which is significantly higher than the 83.9% to 84.7% of the comparative scheme. At the same time, the track length is shortened by about 1.9 to 2.1 km, indicating that under the same operational boundary, the task point organization and path connectivity strategy of this scheme can more effectively avoid falling into invalid points in the no-fly / obstacle buffer zone, and complete path compression while maintaining coverage integrity. Further comparing the proportion of curvature outside the threshold, this scheme controls it at 2.1% to 2.4% in all three task units, while the comparative scheme is 6.5% to 7.1%, showing that the smoothness control in the tower center circling section and the sharp curve transition section is more sufficient, reducing the proportion of high curvature track sections, which has a direct supporting significance for maintaining flight stability and triggering sequence consistency.

[0062] From the perspective of acquisition consistency, the observation window satisfaction rate and batch integrity rate of this scheme reached 96.9%–97.8% and 97.9%–98.5% respectively, both significantly better than the comparative scheme (approximately 86.9%–91.4%). This indicates that the gimbal pre-setting, triggering placement, and velocity profile linkage mechanism before the window arrives can effectively guarantee field of view coverage and dwell time, enabling cross-modal frames to have higher temporal / pose consistency in the batch dimension, reducing the processing cost of subsequent alignment and reconstruction. In terms of edge robustness, the frame loss rate and packet loss rate in weak coverage sections of this scheme are controlled at 0.7%–0.8% and 0.9%–1.1% respectively, significantly lower than the 2.8%–3.1% and 4.7%–5.2% of the comparative scheme. This shows that the batch caching and communication segmentation strategy can maintain data integrity in weak coverage sections, and the edge preprocessing chain significantly reduces the uplink proportion of unqualified frames.

[0063] From the perspective of recognition performance, the cloud-based recognition accuracy of this solution ranges from 95.7% to 96.4%, while the comparative solution ranges from 87.9% to 88.6%. Since recognition accuracy is directly affected by data quality, spatiotemporal consistency, and modal coverage, this difference can be attributed to the integrated spatiotemporal-trigger-pose binding implemented in the task planning phase of this solution, and the coordinated configuration of window dwell, shutter / integration, and line frequency during the acquisition phase. This provides the cloud with cleaner and more discriminative samples. Although the total task time is only shortened by 3 to 6 minutes, sufficient data density is maintained while shortening the flight path and improving the window fulfillment rate, thus balancing efficiency and quality.

[0064] In summary, this embodiment demonstrates the universal improvements of the present invention compared to single-modal + manual baseline planning in three typical scenarios (mountain base stations, plain macro base stations, and hilly lines): at the path level, it is reflected in better feasible point retention and curvature control; at the acquisition level, it is reflected in higher window satisfaction and batch integrity; at the communication and edge level, it is reflected in lower frame and packet loss; and at the identification level, it is reflected in higher accuracy. The key is that this solution uses batches as a link to connect the data link of planning—acquisition—preprocessing—identification—archiving—feedback, forming a stable and traceable closed loop. Therefore, the present invention demonstrates creativity and novelty in terms of engineering feasibility, data consistency, and identification end usability. It provides a systematic improvement path for existing technologies in terms of track smoothing, multimodal alignment, weak coverage retransmission, and closed-loop updates, and has the potential for implementation in large-scale communication base station inspection scenarios.

[0065] Example 3, an embodiment of the present invention, provides a multimodal communication base station UAV inspection system, including a trajectory planning module, a multimodal acquisition module, and a data processing module.

[0066] The trajectory planning module is used to collect base station, line, DEM, and historical data, generate a task point set and allocate collection radii, construct parameterized trajectory curves, and combine velocity profiles with observation window triggering to form an executable trajectory sequence. The multimodal acquisition module is used for UAV to fly according to the planned trajectory, calibrate gimbal and sensor extrinsic parameters, and sequentially trigger visible light, infrared, and lidar acquisition when entering the window, and write each modal data into the cache with batch number and timestamp index. The data processing module is used for batch data to enter the airborne edge unit for preprocessing, and then performs real-time backhaul or cache retransmission according to communication segments. After being uploaded to the cloud, it is archived and multimodal recognition is performed. Finally, a result database and report files are generated and written into the feedback library and execution library. If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0068] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0069] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc. It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0070] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for unmanned aerial vehicle (UAV) inspection of multimodal communication base stations, characterized in that, include: Collect base station, line, DEM and historical data, generate task point set and allocate collection radius, construct parameterized track curve, combine velocity profile design and observation window triggering placeholder, and generate executable track sequence; The UAV flies according to the executable flight sequence, the gimbal and sensor extrinsic parameters are calibrated, and when it enters the window, the visible light camera, infrared thermal imager and lidar are triggered in sequence to collect data. The data of each mode is written to the cache with batch number and timestamp index. The batched data of each modality enters the airborne edge unit for preprocessing, and is transmitted back in real time or cached and supplemented according to the communication segments. After being uploaded to the cloud, it is archived in batches and then cloud recognition is performed. The cloud-based identification results are categorized by task unit and observation window, generating a results database and report files, which are written to the feedback database. Flight logs and cached records are synchronized to the execution database.

2. The UAV inspection method for multimodal communication base stations as described in claim 1, characterized in that: The process of generating a task point set and allocating acquisition radii involves establishing a multi-layered surrounding observation circle around base station objects based on the camera focal length and desired resolution, sampling at fixed intervals along the broken lines of line objects to generate multi-layered, multi-angle candidate observation poses, and then removing them from no-fly zones and obstacle zones to form the final task point set.

3. The UAV inspection method for multimodal communication base stations as described in claim 2, characterized in that: The construction of the parameterized trajectory curve includes using a B-spline or spiral model to generate a trajectory under the constraints of a set of control points, and outputting the trajectory curve within a preset threshold range where the curvature, climb angle, and elevation changes satisfy the preset threshold range.

4. The UAV inspection method for multimodal communication base stations as described in claim 3, characterized in that: The velocity profile design includes generating a piecewise velocity function based on the trajectory curvature, slope, and observation window dwell time, constraining the speed, acceleration, and jump of the UAV to be within the allowable range of flight control, and embedding sensor trigger placeholder parameters at the observation window.

5. The UAV inspection method for multimodal communication base stations as described in claim 4, characterized in that: The modal data is written to the cache using batch number and timestamp index, including batch organization during cache writing. The batch file contains task unit ID, track segment number, batch number, timestamp, attitude information and sensor status fields, and is stored using a circular buffer management strategy.

6. The UAV inspection method for multimodal communication base stations as described in claim 5, characterized in that: The preprocessing of the airborne edge unit includes performing ambiguity discrimination and duplicate frame merging on the visible light image, performing dark field offset correction on the infrared thermal image data, performing denoising and downsampling on the laser point cloud data, and generating preprocessing results with batch numbers.

7. The UAV inspection method for multimodal communication base stations as described in claim 6, characterized in that: The generation of the result database and report file, and the writing of the feedback database, includes writing the identification results that have been manually reviewed into the feedback database; Simultaneously, flight logs, speed deviations, and cache records are written to the execution database for adjustments to the task point density and communication segmentation strategy in the next mission.

8. A system employing the UAV inspection method for multimodal communication base stations as described in any one of claims 1 to 7, characterized in that: It includes a trajectory planning module, a multimodal acquisition module, and a data processing module; The trajectory planning module is used to collect base station, line, DEM and historical data, generate task point set and allocate collection radius, construct parameterized trajectory curve, and combine velocity profile and observation window trigger placeholder to form an executable trajectory sequence. The multimodal acquisition module is used for UAV to fly according to the planned trajectory, calibrate the external parameters of the gimbal and sensors, and sequentially trigger visible light, infrared and lidar acquisition when entering the window, and write each modal data into the cache with batch number and timestamp index; The data processing module is used to batch data into the airborne edge unit for preprocessing, and then perform real-time backhaul or cached retransmission according to communication segments. After being uploaded to the cloud, the data is archived and multimodal recognition is performed. Finally, a result database and report file are generated and written into the feedback library and execution library.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the multimodal communication base station UAV inspection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the UAV inspection method for multimodal communication base stations as described in any one of claims 1 to 7.

Images