A path-air-ground cooperative control method and system for unmanned aerial vehicle and robot dog operation

Abstract

This invention provides a method and system for air-ground collaborative path control of UAV and robot dog operations, relating to the field of artificial intelligence technology. The method includes: acquiring three-dimensional terrain data, tower distribution coordinates, and preset inspection tasks for a power transmission line inspection area; fusing obstacle coordinates and potential hazard location information transmitted back from high-altitude UAV detection to obtain fused multi-dimensional spatial data; preprocessing the fused multi-dimensional spatial data to obtain an initial point cloud dataset; and performing grid-based reconstruction and semantic feature mapping based on the spatial adjacency relationships and elevation continuity of the initial point cloud dataset to construct a three-dimensional virtual operation model. This invention improves the intelligence and adaptability of air-ground collaborative inspection of power transmission lines.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for path air-ground cooperative control of drones and robot dogs. Background Technology

[0002] In the operation and maintenance of long-distance power transmission lines in complex mountainous and hilly areas, a collaborative air-ground inspection mode using drones and quadruped robot dogs is adopted. For example, drones take aerial photos and detect obstacles from high altitudes of transmission towers, conductors and surrounding mountains. After identifying potential hazards such as icing and hanging foreign objects, the drones send the obstacle coordinates and inspection points to a unified dispatch platform. The robot dogs, which travel along the ground passage under the line, receive the instructions and conduct close-range inspections of tower foundations, grounding devices and cable trenches. The drones and robot dogs work together to complete automated inspection tasks through data interaction.

[0003] In existing technologies, the control of the UAV's high-altitude inspection path and the robot dog's ground travel path are independent of each other, lacking air-ground spatiotemporal coupling constraints and collaborative optimization mechanisms. In actual operations, it often happens that the robot dog has arrived at the target point but the UAV is still flying around at high altitude, or the UAV has completed the task in the area but the robot dog is unable to follow due to complex terrain obstacles for a long time. This results in misalignment of air-ground operation sequence, low collaborative efficiency, and excessively long overall inspection time, making it difficult to meet the needs of efficient and synchronous packaged operation and maintenance of power transmission lines in complex mountainous areas. Summary of the Invention

[0004] This invention provides a path air-ground collaborative control method and system for drone and robot dog operations, which improves the intelligence and adaptability of air-ground collaborative inspection of power transmission lines.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a path-based air-ground cooperative control method for drone and robot dog operations, the method comprising: The system acquires 3D terrain data, tower distribution coordinates, and preset inspection tasks for the transmission line inspection area. By fusing obstacle coordinates and hazard location information transmitted back from UAV high-altitude detection, it obtains fused multidimensional spatial data. The fused multidimensional spatial data is then preprocessed to obtain an initial point cloud dataset. A three-dimensional virtual operation model is constructed by performing grid-based reconstruction and semantic feature mapping based on the spatial adjacency relationship and elevation continuity of the initial point cloud dataset. Using a three-dimensional virtual operation model as the spatial optimization basis, a step-by-step gradient deduction and path iteration processing mechanism is adopted. Through multi-dimensional passage cost evaluation and temporal beat matching calculation, the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog are obtained. The initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog are parsed into cooperative control commands; the cooperative control commands are sent to the execution terminals of the UAV and the robot dog, and the flight status data of the UAV and the ground pose data of the robot dog are collected; the flight status data of the UAV and the ground pose data of the robot dog are spatiotemporally aligned and motion trend matched with the time window labels in the cooperative control commands to obtain the real-time air-ground cooperative state flow. The actual progress deviation of air-ground operations is obtained by comparing the real-time air-ground cooperative state flow with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog. When the progress deviation exceeds the preset safety threshold, the unfinished section is recalculated to obtain the cooperative control command sequence.

[0006] Furthermore, 3D terrain data, tower distribution coordinates, and preset inspection tasks for the transmission line inspection area are acquired. By fusing obstacle coordinates and hazard location information transmitted back from UAV high-altitude detection, fused multidimensional spatial data is obtained. The fused multidimensional spatial data is then preprocessed to obtain an initial point cloud dataset, including: The system acquires three-dimensional terrain data, tower distribution coordinates, and preset inspection tasks for the transmission line inspection area. It also receives obstacle coordinates and hidden danger location information transmitted back from UAV high-altitude detection. The system integrates the terrain data, coordinates, task list, and transmitted information from multiple sources to obtain fused multi-dimensional spatial data. The fused multidimensional spatial data is then registered using a multi-source coordinate system to unify the spatial reference and eliminate positional deviations between different detection sources, resulting in reference-aligned spatial data. The benchmark-aligned spatial data is subjected to noise filtering and outlier removal to extract effective geographic features and obtain clean spatial feature data. The clean spatial feature data is discretized and segmented into voxels according to a preset spatial resolution to obtain an initial point cloud dataset containing elevation attributes, obstacle distribution attributes, and task location attributes.

[0007] Furthermore, based on the spatial adjacency relationships and elevation continuity of the initial point cloud dataset, a grid-based reconstruction and semantic feature mapping process are performed to construct a 3D virtual operation model, including: Extract the spatial adjacency topology and elevation variation parameters of each discrete point in the initial point cloud dataset, and construct a point cloud feature connectivity network; The point cloud features are connected to a network and then reconstructed into a grid, transforming the discrete point cloud into a continuous terrain surface grid, thus obtaining a gridded terrain base. The gridded terrain base is processed by semantic feature mapping, and each grid unit is classified and labeled as passable area, obstacle-restricted area, tower structure area and inspection target area to obtain the fused classification and labeling result. By integrating classification and labeling results with grid spatial coordinates, a three-dimensional virtual operation model is constructed, which includes terrain undulations, obstacle spatial distribution, and task node locations.

[0008] Furthermore, using a 3D virtual operation model as the spatial optimization basis, and employing a stepped gradient deduction and path iteration processing mechanism, the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog are obtained through multi-dimensional passage cost evaluation and temporal rhythm matching calculation, including: Using spatial nodes in the three-dimensional virtual operation model as the optimization calculation objects, a step-by-step gradient deduction method is used to calculate the multi-dimensional passage cost of each node under the constraints of high-altitude detection field of view coverage requirements and ground obstacle crossing ability, and the node cost distribution matrix is ​​obtained. The node cost distribution matrix is ​​processed by path iteration to dynamically remove redundant path branches that exceed the passage cost threshold, thereby obtaining a set of open-field candidate paths that meet the terrain and safety constraints. By matching the time requirements in the set of candidate air-ground paths with the preset inspection task list, the arrival time of each candidate path node is synchronously calibrated and aligned with the time requirements to obtain the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog that meet the time requirements of air-ground collaborative operation.

[0009] Furthermore, the initial inspection flight path of the UAV and the initial ground movement sequence of the robot dog are parsed into cooperative control commands; these commands are then sent to the execution terminals of both the UAV and the robot dog, and flight status data of the UAV and ground pose data of the robot dog are collected, including: The initial inspection track of the UAV and the initial ground movement sequence of the robot dog are processed by instruction parsing and segmented encoding. The spatiotemporal control parameters of each track node and movement node are extracted to obtain the cooperative control instructions with time window labels. The collaborative control command is sent to the execution terminal of the drone and robot dog through the communication link, triggering the status acquisition function of the drone and robot dog platform to enter the synchronous operation mode; During the execution of collaborative control commands, the flight attitude, speed and altitude information fed back by the UAV execution terminal under the trigger state are read in real time, as well as the pose, joint angle and travel distance information fed back by the robot dog execution terminal. The flight status data of the UAV and the ground pose data of the robot dog are then summarized.

[0010] Furthermore, the flight status data of the UAV, the ground pose data of the robot dog, and the time window labels in the cooperative control commands are spatiotemporally aligned and matched for motion trends to obtain the real-time air-ground cooperative state flow, including: The time window labels in flight status data, ground pose data and cooperative control commands are time-stamped and compensated for, and spatial coordinate system is transformed to eliminate the time difference and coordinate reference difference in data acquisition between UAV and robot dog, and obtain a spatiotemporally aligned data stream. Using the spatiotemporally aligned data stream as the input for trend calculation, short-term motion trend extrapolation and dynamic trajectory matching degree calculation are performed based on the actual pose change rate and the planned trajectory curvature to obtain motion trend matching results. By performing feature fusion and sequence packaging processing on the spatiotemporal aligned data stream and motion trend matching results, a real-time air-ground collaborative state stream is obtained, which shows the relative positional relationship between the UAV and the robot dog and the degree of deviation from the plan execution.

[0011] Furthermore, based on the real-time air-to-ground cooperative state flow and the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog, a trajectory timing comparison is performed to obtain the actual air-to-ground operation progress deviation. When the progress deviation exceeds a preset safety threshold, the unfinished section is recalculated to obtain a cooperative control command sequence, including: The real-time air-ground collaborative status flow is compared with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog by projecting the trajectory space onto the time axis and summarizing the deviation of the actual air-ground operation progress. The actual progress deviation of the air-ground operation is compared with the preset safety threshold. When the progress deviation exceeds the preset safety threshold, the remaining inspection nodes and terrain access constraints not covered by the current drone and robot dog are extracted to obtain the replanning trigger signal for the unfinished section. Based on the replanning trigger signal, taking the current actual pose of the UAV and robot dog as the optimization starting point, and combining terrain accessibility and air-to-ground communication constraints, the access order, dynamic waiting dwell time and terrain following detour strategy of the remaining inspection nodes are solved online iteratively to obtain the cooperative control command sequence to replace the original command.

[0012] Secondly, a path-based air-ground cooperative control system for drone and robot dog operations includes: The acquisition module is used to acquire three-dimensional terrain data, tower distribution coordinates, and preset inspection tasks of the transmission line inspection area. By fusing obstacle coordinates and hidden danger location information transmitted back by UAV high-altitude detection, the fused multi-dimensional spatial data is obtained. The fused multi-dimensional spatial data is preprocessed to obtain the initial point cloud dataset. The building module is used to perform grid-based reconstruction and semantic feature mapping based on the spatial adjacency relationship and elevation continuity of the initial point cloud dataset to build a three-dimensional virtual operation model; The calculation module is used to obtain the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog by using a three-dimensional virtual operation model as the spatial optimization basis, adopting a step-by-step gradient deduction and path iteration processing mechanism, and through multi-dimensional passage cost evaluation and temporal beat matching calculation. The parsing module is used to parse the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog into cooperative control commands; send the cooperative control commands to the execution terminals of the UAV and the robot dog; collect the flight status data of the UAV and the ground pose data of the robot dog; and perform spatiotemporal alignment and motion trend matching processing on the flight status data of the UAV and the ground pose data of the robot dog and the time window labels in the cooperative control commands to obtain the real-time air-ground cooperative state flow. The comparison module is used to compare the trajectory timing with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog based on the real-time air-ground cooperative state flow, and obtain the actual air-ground operation progress deviation; when the progress deviation exceeds the preset safety threshold, the unfinished section is recalculated to obtain the cooperative control command sequence.

[0013] Thirdly, a computing device includes: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0014] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0015] The above-described solution of the present invention has at least the following beneficial effects: This invention overcomes the technical problems of existing technologies, such as the independent path control of UAVs and robot dogs, the lack of air-ground spatiotemporal coupling constraints and collaborative optimization mechanisms, which lead to misaligned operation sequences, low collaborative efficiency, and long overall inspection time, making it difficult to meet the needs of efficient and synchronous operation and maintenance of transmission lines in complex mountainous areas. It achieves the technical effect of realizing collaborative optimization control of UAV and robot dog paths, avoiding misaligned operation sequences, improving collaborative inspection efficiency, shortening inspection cycles, adapting to the intelligent operation and maintenance needs of transmission lines in complex mountainous and hilly areas, and ensuring efficient and synchronous inspection operations. This results in the technical effect of achieving efficient and synchronous operation and maintenance of transmission lines by integrating three-dimensional terrain data of the transmission line inspection area, tower distribution coordinates, preset inspection task list, and UAV high-altitude detection feedback information, and constructing a three-dimensional virtual operation model based on this dataset. Using this model as a foundation, the initial paths of the UAV and robot dog are obtained through step-by-step gradient deduction, path iteration, multi-dimensional passage cost evaluation, and time-series matching. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a path air-ground cooperative control method for drone and robot dog operations provided by an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of a path air-ground cooperative control system for drone and robot dog operations provided by an embodiment of the present invention. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0019] like Figure 1 As shown, an embodiment of the present invention proposes a path air-ground cooperative control method for drone and robot dog operations, the method comprising the following steps: Step 1: Obtain the 3D terrain data, tower distribution coordinates, and preset inspection tasks of the transmission line inspection area. By fusing the obstacle coordinates and hidden danger location information transmitted back by UAV high-altitude detection, the fused multidimensional spatial data is obtained. The fused multidimensional spatial data is preprocessed to obtain the initial point cloud dataset. Step 2: Based on the spatial adjacency relationship and elevation continuity of the initial point cloud dataset, perform grid reconstruction and semantic feature mapping to construct a three-dimensional virtual operation model; Step 3: Using the three-dimensional virtual operation model as the spatial optimization basis, a step-by-step gradient deduction and path iteration processing mechanism is adopted. Through multi-dimensional passage cost evaluation and temporal beat matching calculation, the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog are obtained. Step 4: Parse the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog into cooperative control commands; send the cooperative control commands to the execution terminals of the UAV and the robot dog, collect the flight status data of the UAV and the ground pose data of the robot dog; perform spatiotemporal alignment and motion trend matching processing on the flight status data of the UAV and the ground pose data of the robot dog and the time window labels in the cooperative control commands to obtain the real-time air-ground cooperative state flow. Step 5: Based on the real-time air-ground cooperative state flow, the trajectory timing is compared with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog to obtain the actual air-ground operation progress deviation; when the progress deviation exceeds the preset safety threshold, the unfinished section is recalculated to obtain the cooperative control command sequence.

[0020] In this embodiment of the invention, the invention acquires three-dimensional terrain data, tower distribution coordinates, and a preset inspection task list for the transmission line inspection area. After preprocessing this data and integrating obstacle coordinates and potential hazard location information transmitted back from high-altitude drone detection, an initial point cloud dataset is obtained. Based on the spatial adjacency relationship and elevation continuity of this initial point cloud dataset, a three-dimensional virtual operation model is constructed. Using this model as the spatial optimization basis, the initial inspection trajectory of the drone and the initial ground movement sequence of the robot dog are obtained through step-by-step gradient deduction, path iteration, multi-dimensional passage cost evaluation, and time-series beat matching. This process is then used to analyze and coordinate control... This technology involves issuing commands, collecting equipment operating status data, and performing spatiotemporal alignment and motion trend matching to obtain a real-time air-ground collaborative status flow. When the progress deviation exceeds a preset safety threshold, the unfinished section is replanned. Therefore, it overcomes the technical problems of existing UAV and robot dog operation path control being independent, lacking an air-ground collaborative optimization mechanism, easily leading to operation timing misalignment, low collaborative efficiency, long inspection time, and difficulty in adapting to the inspection needs of power transmission lines. Thus, it achieves air-ground path collaborative control of UAV and robot dog, avoids operation timing misalignment, improves inspection collaborative efficiency, and shortens the inspection cycle.

[0021] In a preferred embodiment of the present invention, step 1 above may include: Step 1.1: Acquire 3D terrain data, tower distribution coordinates, and preset inspection tasks for the transmission line inspection area. Receive obstacle coordinates and potential hazard location information transmitted back from UAV high-altitude detection. Integrate the terrain data, coordinates, task list, and transmitted information from multiple sources to obtain fused multi-dimensional spatial data. Specifically, this includes: acquiring 3D terrain data of the transmission line inspection area through a combination of satellite remote sensing and ground-based lidar scanning. This data includes the elevation, slope, and aspect information of all terrain points within the inspection area; exporting tower distribution coordinates from the transmission line operation and maintenance management system. These coordinates represent the 3D coordinates of each transmission tower within the inspection area, including longitude, latitude, and elevation, ensuring the traceability of each tower's location information; and obtaining a preset inspection task list through pre-entry by maintenance personnel. This list clearly includes core information such as inspection points, inspection priorities for each point, required inspection duration, and inspection accuracy standards. Meanwhile, the drone is equipped with a high-definition camera and a lidar detector to conduct high-altitude detection and transmit the coordinates of obstacles in the inspection area in real time. The three-dimensional coordinates of obstacles such as trees, landslides, and buildings, as well as the location information of potential hazards, are transmitted back in real time. The location information of potential hazards such as icing on power transmission lines, damage to tower components, and foreign objects hanging on power lines are also transmitted back in real time to ensure the timeliness of the data.

[0022] The acquired 3D terrain data, tower distribution coordinates, pre-set inspection task list, and obstacle coordinates and hazard location information transmitted by the UAV are subjected to multi-source data fusion processing. During the fusion process, all data are formatted uniformly, converting all types of data into the same data format to ensure that the data can be mutually recognized. Then, a weighted fusion algorithm is used to perform fusion calculations on the various types of data. The fusion formula is as follows: ,in, For the fused multidimensional spatial data, Weights for 3D terrain data, For three-dimensional terrain data, The weights of the tower distribution coordinates, For tower distribution coordinate data, To preset the weights of the inspection task list, To pre-set the inspection task list data, The weights of the obstacle coordinate data, The obstacle coordinate data transmitted back by the drone. Weighting of information on potential hazard locations. to The weights are summed to 1, and the weight values ​​are set according to the accuracy and importance of each type of data. Among them, obstacle coordinates and three-dimensional terrain data have higher weights. After fusion, multi-dimensional spatial data containing five major categories of information, including terrain, towers, tasks, obstacles, and hidden dangers, is obtained.

[0023] Step 1.2 involves performing multi-source coordinate system registration on the fused multi-dimensional spatial data to unify the spatial reference and eliminate positional deviations between different detection sources, resulting in reference-aligned spatial data. Specifically, this includes: since the multi-source data originates from different detection sources and uses different coordinate systems, direct use would result in positional deviations, affecting the accuracy of subsequent data processing. Therefore, multi-source coordinate system registration is performed on the fused multi-dimensional spatial data to unify the spatial reference. The registration process uses a seven-parameter coordinate transformation method, selecting at least three common control points. These common control points must exist simultaneously in all coordinate systems, and their coordinates must be known and highly accurate. The coordinate transformation formula is: ; In the formula, , , To unify the coordinate values ​​in the coordinate system after registration, , , To obtain the coordinate values ​​of the original coordinate system before registration, to , to , to There are 7 registration parameters. When calculating the registration parameters, the original coordinates of the common control points are used. , , ) and target coordinates ( , , Substituting the above formula into the equations, we establish a system of equations and solve for the registration parameters using the least squares method. During the solution process, we calculate the coordinate deviation of each common control point. The deviation calculation formula is as follows: The registration parameters are iteratively adjusted until the coordinate deviations of all common control points are less than a preset deviation threshold to ensure registration accuracy. After registration, the fused multidimensional spatial data is converted into a unified UTM coordinate system to eliminate positional deviations between different detection sources, resulting in reference-aligned spatial data.

[0024] Step 1.3 involves noise filtering and outlier removal of the benchmark-aligned spatial data to extract effective geographic features and obtain clean spatial feature data. Specifically, this includes: performing noise filtering and outlier removal on the benchmark-aligned spatial data, extracting effective geographic features, and performing noise filtering using a Gaussian filtering algorithm. The filtering formula is as follows: ,in, This is the Gaussian filter kernel function. , These are the relative coordinates within the filtering window. To determine the standard deviation, during the filtering process, a 3×3 filtering window is selected, centered on each data point in the baseline aligned spatial data. The weighted average of the coordinates of all data points within the window is calculated. The weights are calculated using a Gaussian kernel function, with the central data point having the highest weight and decreasing weight towards the window edges. The formula for calculating the weighted average is: ,in These are the coordinates of the filtered data points. This represents the number of data points within the filter window. For the first The weight of each data point For the first The original coordinates of each data point are used to replace the original center data point coordinates with the calculated weighted average. All data points are then filtered sequentially to remove noise and outliers. The Grubbs criterion is used to calculate the mean and standard deviation of the feature data in the baseline alignment spatial data. The formula for calculating the mean is: The formula for calculating the standard deviation is: ,in, This is the average of the data. For the number of data points, For the first The value of each data point. To calculate the standard deviation, the deviation value for each data point is determined using the following formula: Select the Grubbs coefficient G. If the deviation value of a certain data point is... If the value is greater than G×s, the data point is determined to be an outlier. After noise filtering and outlier removal, the effective geographic features in the benchmark aligned spatial data are extracted, including terrain undulation features, obstacle distribution features, tower structure features, and inspection target features. These effective geographic features are then summarized to obtain clean spatial feature data.

[0025] Step 1.4 involves discretizing and voxelizing the clean spatial feature data according to a preset spatial resolution to obtain an initial point cloud dataset containing elevation attributes, obstacle distribution attributes, and task location attributes. Specifically, this includes: the clean spatial feature data is continuous spatial data; the clean spatial feature data is discretized and voxelized according to a preset spatial resolution, transforming the continuous spatial data into discrete point cloud data to obtain the initial point cloud dataset; the initial point cloud dataset undergoes discretization processing; the preset spatial resolution is set according to the accuracy requirements of transmission line inspection, and is 0.5 m × 0.5 m × 0.5 m, along the three-dimensional space... axis, axis, Along the axial direction, a discrete unit is divided every 0.5 meters. During the discretization calculation process, the spatial range of the inspection area is determined, and the calculation area is within the specified range. axis, axis, The minimum and maximum values ​​of the axis will axis, axis, The axes are divided according to a preset resolution, and the coordinates of each discrete unit are calculated as follows: , , ,in, , , The coordinates of the discrete element. , , These are the original coordinates of a data point in the clean spatial feature data. , , For inspection area axis, axis, Minimum value of the axis, For preset spatial resolution, As a floor function, the above calculation assigns all data points in the pure spatial feature data to the corresponding discrete units, thus completing the discretization representation.

[0026] The clean spatial feature data is segmented into voxels, converting each discrete unit into a voxel. Each voxel corresponds to a three-dimensional spatial unit, and all attribute information within that voxel is recorded. The voxel attribute values ​​are calculated using the average method. For the elevation attribute, the calculation formula is as follows: ,in The elevation attribute value of the voxel. This represents the number of data points contained within the voxel. For the first voxel The elevation values ​​of each data point; for obstacle distribution attributes, if an obstacle data point exists within the voxel, the voxel is determined to be an obstacle voxel, and the obstacle distribution attribute value is recorded as 1; otherwise, it is recorded as 0. If some data points within the voxel are obstacle data points, the obstacle distribution attribute value is the ratio of the number of obstacle data points within the voxel to the total number of data points. For task location attributes, if the voxel contains inspection task locations, the task location attribute value is recorded as the inspection priority of that location; otherwise, it is recorded as 0. If the voxel contains multiple task locations, the value with the highest priority is taken as the task location attribute value of the voxel. For empty voxels that do not contain any data points, linear interpolation is used to supplement their attribute values. The interpolation calculation formula is: ,in, The interpolated attribute value for an empty voxel. , The attribute values ​​of the two non-empty voxels adjacent to the empty voxel. Using the interpolation coefficients, after voxel segmentation, all voxels are arranged according to their three-dimensional coordinates to obtain an initial point cloud dataset containing elevation attributes, obstacle distribution attributes, and task location attributes.

[0027] In this embodiment of the invention, by employing multi-source data weighted fusion, multi-source coordinate system registration, Gaussian filtering and Grubbs criterion outlier removal, preset resolution discretization and voxel segmentation techniques, various types of data in the transmission line inspection area are systematically processed. This overcomes the technical problems in the prior art, such as inconsistent multi-source data formats, large coordinate deviations, noise and outliers, resulting in low data purity and difficulty in providing reliable support for the construction of 3D virtual operation models and path planning, thus affecting the efficiency and accuracy of UAV and robot dog collaborative inspections. The invention achieves the technical effect of obtaining a clean, complete, and uniformly formatted initial point cloud dataset, effectively preserving the core spatial features of the inspection area such as terrain, obstacles, towers, and tasks, providing high-quality data support for the construction of 3D virtual operation models and air-ground collaborative path planning, improving the accuracy and rationality of path planning, and laying the foundation for efficient collaborative inspections of UAVs and robot dogs.

[0028] In a preferred embodiment of the present invention, step 2 above may include: Step 2.1: Extract the spatial adjacency topology and elevation variation parameters of each discrete point in the initial point cloud dataset, and construct a point cloud feature connectivity network. Specifically, this includes: based on the initial point cloud dataset, defining a three-dimensional spatial neighborhood for each discrete point, with the neighborhood radius set to twice the point cloud voxel resolution, denoted as . Iterate through all discrete points and search for points falling within the neighborhood to determine the spatial adjacency topology between points. For any pair of adjacent points, let the three-dimensional coordinates of the two points be... The spatial distance between the two points is: The rate of change of elevation is the ratio of the difference in elevation between two points to the spatial distance: The elevation change rate of a single center point is averaged over all its adjacent points to obtain the continuous elevation change parameter for that point. In the formula The number of adjacent nodes. For the first The elevation change rate corresponding to each adjacent point is used to construct a point cloud feature connection network covering the entire inspection area, with discrete points as nodes, spatial adjacency relationships as connecting edges, and continuous elevation change parameters as edge weights.

[0029] Step 2.2 involves reconstructing the point cloud feature network into a grid, transforming the discrete point cloud into a continuous terrain surface grid to obtain a gridded terrain base. Specifically, this includes: using the point cloud feature network as a basis, performing grid reconstruction using an irregular triangular mesh partitioning method; selecting three adjacent nodes whose elevation variation parameters are less than a preset abrupt change threshold, combining them to form triangular mesh units; traversing the entire domain to complete mesh stitching; and transforming the discrete point cloud into a continuous terrain surface grid. For any triangular mesh, the coordinates of the three vertices are... , , Its center coordinates are: , , By removing deformed small grids and completing the broken grids at the edges, a continuous and regular gridded terrain base is obtained.

[0030] Step 2.3 involves semantic feature mapping of the gridded terrain base, classifying and labeling each grid cell as passable areas, obstacle-prohibited areas, tower structure areas, and inspection target areas, obtaining the fused classification and labeling results. Specifically, this includes calculating the terrain slope and obstacle ratio for each grid cell of the gridded terrain base. The slope calculation formula is: , The tangent function is usually approximated using Taylor series expansion, and the formula is: ,in That is, in the formula , must meet , : Number of iteration terms; the larger the value, the higher the calculation accuracy. The average slope representing the local terrain, and the percentage of obstacle voxels within the grid are: In the formula The number of obstacle voxels within the grid. The total number of point cloud voxels within the grid is used to classify and label the grid based on the calculation results: areas where the proportion of obstacle voxels exceeds the threshold are prohibited areas; areas where the coordinates coincide with those of poles and towers are pole and tower structure areas; areas containing inspection task points are inspection target areas; and areas where both slope and obstacles meet the passage conditions are passable areas. Multi-attribute grids are uniquely labeled according to the priority of inspection target areas, pole and tower structure areas, prohibited areas, and passable areas, resulting in a fused classification and labeling result.

[0031] Step 2.4 involves fusing the classification and annotation results with the grid spatial coordinates to construct a 3D virtual operation model that includes terrain undulations, obstacle spatial distribution, and task node locations. Specifically, this includes: Based on the gridded terrain base and the fused classification and annotation results, formally constructing the 3D virtual operation model. The specific construction process is as follows: Coordinate and annotation binding: binding the 3D spatial coordinates of each grid cell... By associating and binding with corresponding semantic classification labels, each spatial grid simultaneously possesses geometric location information and environmental semantic information, a set of spatial grid units with semantic attributes is obtained, resulting in the spatial aggregation of tower structures. For the grids labeled as tower structure areas, spatial clustering and aggregation are performed according to the tower distribution coordinates. All grids corresponding to the same tower are spatially associated and integrated. By superimposing grid elevations and fitting contours, the three-dimensional shape of the tower is reconstructed. The calculation formula is the coordinates of the tower aggregation center: In the formula This represents the number of grid cells corresponding to a single tower. The first one below this tower The center coordinates of each grid are used to construct a three-dimensional structure consistent with the real tower.

[0032] Inspection target location calibration: For the grid marked as the inspection target area, extract the grid center coordinates as the standard spatial location of the task node to complete the inspection task node calibration, and present the target points, terrain and obstacle features of the operation task in the model: using the elevation coordinates of all grids. The continuous combination fully represents the terrain undulation of the inspection area; the grid distribution of the obstacle-restricted area intuitively reflects the position, range and distribution of obstacles in three-dimensional space. The model is integrated as a whole: the spatial geometric skeleton of the gridded terrain base, the three-dimensional structure of the tower, the location of the inspection task node, the terrain undulation, the spatial distribution of obstacles and the semantic annotation information of the whole domain are uniformly integrated to obtain a three-dimensional spatial computing carrier that is highly consistent with the actual transmission line inspection scenario. Finally, a three-dimensional virtual operation model containing spatial geometric attributes, terrain undulation attributes, obstacle distribution attributes, tower structure attributes and task node attributes is constructed.

[0033] In this embodiment of the invention, by employing the technical means of extracting the spatial adjacency topology and elevation continuity parameters of point clouds to construct a feature connection network, obtaining a gridded terrain base through triangulation reconstruction, calculating the slope and obstacle ratio to complete grid semantic annotation, and integrating coordinates, annotations, tower aggregation, task calibration, and terrain obstacle features to construct a three-dimensional virtual operation model, the technical problems of discrete point clouds being unable to directly represent real operation scenarios, spatial semantic deficiencies leading to ambiguous environmental judgments, and difficulty in supporting unified planning of air-ground collaborative paths are overcome. Thus, the invention achieves high-precision restoration of the three-dimensional environment and operational constraints of the inspection area, providing a unified and standardized spatial optimization base for drones and robot dogs, and improving the reliability of path planning collaborative operations.

[0034] In a preferred embodiment of the present invention, step 3 above may include: Step 3.1: Using the spatial nodes in the 3D virtual operation model as the optimization calculation objects, a step-by-step gradient deduction method is used to calculate the multi-dimensional passage cost of each node under the constraints of high-altitude detection field of view coverage requirements and ground obstacle crossing capability, resulting in a node cost distribution matrix. Specifically, this includes: calculating the basic high-altitude passage cost of the UAV and the basic ground passage cost of the robot dog for all spatial nodes. Both types of costs are calculated based on terrain features: Robot dog basic ground passage cost: Taking into account the slope and elevation undulation of the node terrain, the calculation formula is as follows: ,in, The value of the ground foundation for the robot dog. For slope weight, The slope angle corresponding to the node. Weighting for elevation fluctuations, The average elevation of the node is a continuously varying parameter; the cost of UAV high-altitude access is calculated by comprehensively considering the node's flight altitude deviation and terrain obstruction. ,in, This provides a fundamental value for high-altitude drones. For high deviation weight, This represents the deviation between the node's actual flight altitude and the preset optimal inspection altitude. To obscure weight, The line-of-sight occlusion rate of a node is calculated by adding specific costs to the global base cost, specifically the costs for UAV high-altitude inspection field-of-sight coverage constraints and robot dog ground obstacle-crossing capability constraints. The UAV field-of-sight coverage cost is calculated as follows: For the UAV high-altitude inspection field-of-sight coverage requirements, the cost corresponding to insufficient node field-of-sight coverage is calculated using the following formula: ,in, The value of expanding the field of view for drones For field of view coverage weight, The obstacle-crossing capability cost of the robot dog is calculated based on the node's field of view coverage: Considering the obstacle-crossing capability constraints of the robot dog's ground movement, the cost corresponding to the obstacle-crossing difficulty of the node is calculated using the following formula: ,in, Adding value to robot dogs' obstacle-crossing capabilities Weighting of obstacle course difficulty This represents the obstacle-crossing difficulty value of the node.

[0035] Based on the costs of the first two levels, an additional cost-benefit value for air-to-ground collaboration is added to ensure that nodes simultaneously meet the collaborative operation requirements of both drones and robotic dogs. The calculation formula is as follows: ,in, To collaboratively adapt and replace value, For collaborative weights, To assess the collaborative adaptability of nodes, after completing the simulation at three levels, the total travel value of the drone and the total travel value of the robot dog for each node are calculated using the following formula: , ,in, The total cost of drone traffic to the node. To determine the total cost of passage for the robot dog at each node, the total cost of all spatial nodes and the cost of each level are arranged in the order of the spatial coordinates of the nodes to construct a node cost distribution matrix. The rows of the matrix correspond to a single spatial node, and the columns correspond to the cost of each dimension.

[0036] Step 3.2 involves iteratively processing the node cost distribution matrix to dynamically remove redundant path branches that exceed the passage cost threshold, resulting in a set of candidate paths that satisfy terrain and safety constraints. Specifically, this includes generating initial path branches using a node-by-node expansion approach. For each expanded node, the cumulative passage cost of that path branch is calculated. The cumulative cost is the sum of the total costs corresponding to all nodes on the path, calculated using the following formula: ,in, The cumulative passage cost for the path branch. This represents the number of nodes contained in the path branch. For the first in the path The total cost of passage for each node is determined by setting two types of passage cost thresholds: the maximum cost threshold for a single node and the maximum cumulative cost threshold for the path. During the path iteration process, each generated path branch is dynamically judged: if the total cost of any node in the branch exceeds the maximum cost threshold for a single node, or the cumulative cost of the branch exceeds the maximum cumulative cost threshold for the path, the redundant path branch is directly removed and no further node expansion is performed. After completing the iteration and removal of all path branches, the remaining effective path branches are optimized: redundant paths containing duplicate nodes or detours are removed, and similar paths with consistent directions and similar costs are merged. Finally, a set of air-ground candidate paths that simultaneously satisfies terrain constraints and safety constraints is obtained. The set contains multiple candidate UAV inspection tracks and multiple candidate robot dog ground movement sequences.

[0037] Step 3.3 involves performing timing and rhythm matching calculations between the set of candidate air-to-ground paths and the time requirements in the preset inspection task list. The arrival times of each candidate path node are synchronized and aligned to obtain the initial UAV inspection track and the robot dog's initial ground movement sequence that meet the timing requirements of air-to-ground collaborative operations. Specifically, this includes: for each candidate path, calculating the theoretical arrival time of each node based on the path node spacing and equipment operating speed. The calculation formula for the UAV candidate track is as follows: ; In the formula, For the drone flight path The theoretical arrival time of each node, To ensure a unified start time for the tasks, The first in the flight path The node to the first The flight distance of each node, The average flight speed of the drone, for the candidate movement sequence of the robot dog, is calculated using the following formula: ;in, The robot dog's movement sequence number The theoretical arrival time of each node, For the first in the sequence The node to the first The distance traveled by each node. To determine the robot dog's average travel speed, the arrival times of the drone and the robot dog for each inspection target node are compared with the time requirements in the preset inspection task list. Three types of time deviations are calculated using the following formula: ; ; ; In the formula, For the arrival time deviation of the drone, The arrival time deviation of the robot dog For the time difference of air-to-ground coordination, , These represent the times when the drone and the robot dog arrive at the target node, respectively. For the specified inspection time of the target node, for paths with time deviations exceeding the preset threshold, timing beat matching calculations are performed. By adjusting the flight speed of the UAV and the movement speed of the robot dog, or by adding temporary waiting nodes in the path, the arrival time of each node is iteratively corrected until the three types of time deviations of all inspection target nodes meet the preset requirements. The candidate paths that have completed beat alignment are comprehensively evaluated, and the path with the lowest cumulative travel cost and the smallest time deviation is selected as the final initial inspection track of the UAV and the initial ground movement sequence of the robot dog.

[0038] In this embodiment of the invention, by employing a three-dimensional virtual operation model as the spatial optimization basis, and through step-by-step gradient deduction to calculate the multi-dimensional passage cost value covering the high-altitude field of view, ground obstacle crossing, and collaborative adaptation, a node cost distribution matrix is ​​generated. Based on the matrix, path iteration processing is carried out to dynamically eliminate redundant branches and obtain a set of air-to-ground candidate paths. Combined with the task time requirements, timing rhythm matching and arrival time calibration are performed. Therefore, this overcomes the technical problems in the prior art where the path planning of UAVs and robot dogs are independent, lacking air-to-ground spatiotemporal coupling constraints and collaborative optimization mechanisms, resulting in misaligned operation timing, low collaborative efficiency, long overall inspection time, and difficulty in adapting to the needs of efficient and synchronous operation and maintenance of transmission lines in complex mountainous areas. Thus, it achieves the technical effect of realizing collaborative optimization planning of UAV and robot dog air-to-ground paths, meeting the constraints of high-altitude detection field of view coverage and ground obstacle crossing capability, synchronously calibrating the operation timing rhythm, avoiding misaligned air-to-ground operation timing, improving collaborative inspection efficiency, shortening the inspection cycle, and ensuring efficient and synchronous operation and maintenance of transmission lines in complex mountainous and hilly areas.

[0039] In a preferred embodiment of the present invention, step 4 above may include: Step 4.1 involves parsing and segmenting the initial UAV inspection trajectory and the initial ground movement sequence of the robot dog, extracting the spatiotemporal control parameters of each trajectory node and movement node to obtain collaborative control commands with time window labels. Specifically, this includes parsing the initial UAV inspection trajectory and the initial ground movement sequence of the robot dog as the processing objects, extracting basic control information such as spatial coordinates, target movement speed, and operation dwell requirements for each node according to the node order of the trajectory and movement sequence. After parsing, segmentation encoding is performed according to the distribution characteristics of the inspection task nodes, dividing the continuous and complete path into multiple sequentially executed control segments. Each segment corresponds to an independent execution unit. For each execution unit, spatiotemporal control parameters are extracted, including the node's three-dimensional spatial coordinates, the equipment target movement speed, and the node's operation requirements. The corresponding time window parameters are also calculated, with the formula for calculating the time window length as follows: In the formula, The length of the time window. The planned start time for the device to arrive at the current node. The planned end time is when the equipment completes its current node's work and departs. The planned execution time for a single node is also calculated using the following formula: In the formula, The execution duration of the node plan, This represents the path length between the current node and the next node. For the rated movement speed of the drone or robot dog, the above spatiotemporal control parameters are bound with the calculated time window information, a unique time window label is added to each control command, and the encoding and encapsulation are completed according to the common communication protocol of air and ground equipment, so as to obtain the cooperative control command suitable for synchronous execution.

[0040] Step 4.2: The collaborative control command is sent to the execution terminal of the UAV and robot dog via the communication link, triggering the status acquisition function of the UAV and robot dog platform to enter the synchronous operation mode. Specifically, this includes: sending the encoded collaborative control command to the onboard execution terminal of the UAV and robot dog via the wireless communication link; after the command is sent, a global trigger signal is sent synchronously to initiate the clock synchronization calibration process between the devices; to eliminate the timing deviation between the scheduling end reference clock and the device's local clock, the clock synchronization compensation value is calculated, and the calculation formula is: In the formula, This is the clock synchronization compensation value. The reference clock time for the scheduling end. The local clock time of the drone or robot dog is used to send the clock synchronization compensation value back to the execution terminal to complete the clock correction, so that the status acquisition module of the drone and robot dog can start under a unified timing reference and automatically enter the synchronous operation mode.

[0041] Step 4.3: During the execution of the collaborative control command, the flight attitude, speed, and altitude information fed back by the UAV execution terminal in the triggered state, as well as the pose, joint angle, and travel distance information fed back by the robot dog execution terminal, are read in real time. The flight status data of the UAV and the ground pose data of the robot dog are then summarized. Specifically, this includes: during the execution of the collaborative control command, the operational information fed back by the UAV execution terminal is read in real time according to a fixed acquisition cycle, including flight attitude data composed of roll angle, pitch angle, and yaw angle, real-time flight speed data, and ground altitude data; the three-dimensional pose coordinates, rotation angle data of each drive joint, and travel distance data fed back by the robot dog execution terminal are read in real time. The collected robot dog travel data is accumulated and calculated to obtain the real-time cumulative travel distance. The calculation formula is as follows: In the formula, This represents the current cumulative distance traveled. This is the cumulative mileage from the previous data collection cycle. For the single-segment travel mileage within the current cycle, the flight attitude, speed, and altitude information at the same collection moment are integrated into UAV flight status data, and the pose coordinates, joint angles, and cumulative mileage information at the same moment are integrated into robot dog ground pose data, thus completing the unified summary of the real-time operating status of the two types of equipment.

[0042] In this embodiment of the invention, by using technical means to parse the initial paths of the UAV and the robot dog, segment and encode them, and calculate time window parameters to generate collaborative control commands with time window labels, and by using clock synchronization compensation calibration to achieve synchronous command issuance and trigger the synchronous acquisition mode of the equipment, and by reading and calculating and summarizing the flight status of the UAV and the ground pose data of the robot dog in real time, the technical problems of asynchronous command execution between air and ground equipment, chaotic timing of status data acquisition, and misalignment of collaborative operation timing caused by lack of a unified spatiotemporal reference are overcome. Thus, a unified execution and acquisition timing reference between air and ground is established to ensure the synchronization of command execution and status feedback.

[0043] In a preferred embodiment of the present invention, step 4 above may include: Step 4.4 involves performing timestamp synchronization compensation and spatial coordinate system transformation on the time window labels in the flight status data, ground pose data, and cooperative control commands. This eliminates the time difference and coordinate reference difference between the UAV and robot dog data acquisition, resulting in a spatiotemporally aligned data stream. Specifically, this includes extracting the actual acquisition timestamp corresponding to each set of UAV flight status data and robot dog ground pose data, using UAV flight status data, robot dog ground pose data, and cooperative control commands with time window labels as the core processing objects. Simultaneously, extract the time window label reference timestamp corresponding to this set of data from the collaborative control instructions. The timing deviation is obtained by calculating the difference between the two. The calculation formula is: ,in, This is the actual timestamp of data collection. The base timestamp for the time window label. For timing deviation; if >0 indicates that the data acquisition of the corresponding device lags behind the reference time of the collaborative control command; if <0 indicates that the data acquisition of the corresponding device is ahead of the reference time of the collaborative control command, and synchronization compensation is required in both cases.

[0044] For the calculated timing deviation All collected flight status data and ground pose data are time-stamp calibrated and compensated. The compensation calculation formula is as follows: ,in To compensate for the timestamps, this calculation process unifies and calibrates the timestamps of all UAV flight status data and robot dog ground pose data to the reference time axis of the collaborative control commands, eliminating the timing difference in data acquisition between the two types of devices. After completing the timestamp synchronization compensation, the coordinate reference difference between UAV and robot dog data is eliminated. Since the UAV flight status data acquisition uses the unified UTM coordinate system for high-altitude detection, while the robot dog ground pose data acquisition uses the local ground coordinate system, the two coordinate systems have different references, and direct comparison will produce positional deviations. Therefore, the robot dog's ground pose data is uniformly converted to the UTM coordinate system, and the conversion parameters between the local ground coordinate system and the UTM coordinate system are calculated, mainly including the translation ΔS and the rotation angle α, where the translation ΔS... The calculation formula is: ,in The coordinates of the origin of the UTM coordinate system are... The coordinates are taken as the origin of the local ground coordinate system. The rotation angle is determined based on the azimuth deviation between the two coordinate systems to ensure that the transformed coordinates correspond to the actual spatial position. Based on the calculated translation ΔS and rotation angle, the ground pose data of the robot dog is transformed using the following formula: ,in These are the transformed UTM coordinates. These are the original local coordinates of the robot dog.

[0045] After completing the timestamp synchronization compensation and spatial coordinate system transformation processes, the calibrated UAV flight status data and robot dog ground pose data are arranged sequentially according to the reference timestamp order of the collaborative control commands, and integrated to form a spatiotemporally aligned data stream containing a unified time reference and a unified coordinate reference. This data stream completely preserves the real-time operating status information of the two types of devices.

[0046] Step 4.5: Using the spatiotemporally aligned data stream as the input for trend calculation, short-term motion trend extrapolation and dynamic trajectory matching degree calculation are performed based on the actual pose change rate and planned trajectory curvature to obtain the motion trend matching result. Specifically, this includes: using the spatiotemporally aligned data stream as the input for trend calculation, extracting the real-time pose data of the UAV and robot dog one by one from the data stream, including core information such as spatial coordinates and attitude angles, calculating the actual pose change rate of the two types of equipment respectively, and simultaneously extracting the planned trajectory curvature from the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog. Through short-term motion trend extrapolation and dynamic trajectory matching degree calculation, the operating trend of the equipment and the deviation between the actual trajectory and the planned trajectory are fully understood to obtain the motion trend matching result. The calculation process of the actual pose change rate is as follows: select pose data from 3 consecutive acquisition cycles, with a preset acquisition cycle length of [value missing], and the pose data for the current acquisition cycle is [value missing]. The pose data from the previous acquisition cycle was Calculate the pose difference between the two. Divide the difference by the duration of the data collection period. The actual pose change rate is obtained, and the calculation formula is: This calculation reflects the device's pose change every second, providing a basis for predicting motion trends. The planned trajectory curvature R is obtained in two ways: if the planned trajectory curvature is preset during the initial path, the preset parameter is directly extracted; if not preset, it is calculated using the node coordinates of the planned trajectory. The calculation formula is as follows: ,in Let be the perpendicular distance between a given node on the planned trajectory and the lines connecting its two adjacent nodes. The linear distance between this node and its two adjacent nodes is used to quantify the curvature of the planned trajectory, providing a reference for extrapolating the movement trend.

[0047] Short-term motion trend extrapolation calculation based on actual pose change rate and planned trajectory curvature Based on this, the device pose for the next data acquisition cycle is predicted. The calculation formula is: ,in To predict pose, For the current pose, This refers to the data acquisition cycle duration. This calculation allows for the prediction of the equipment's short-term operational trajectory, enabling timely detection of potential trajectory deviations. The dynamic trajectory matching calculation quantifies the degree of deviation between the actual and planned trajectories of the equipment. The specific calculation process involves comparing the actual pose data of the equipment in the spatiotemporally aligned data stream. Number of poses corresponding to the planned trajectory Calculate the spatial deviation between the two. The calculation formula is: ; In the formula , , These are the actual pose coordinates. , , The planned pose coordinates are then used; the calculated spatial deviation values ​​are then used as the basis for the calculation. Divide by the maximum permissible deviation of the planned trajectory Subtracting this ratio from 1 yields the dynamic trajectory matching degree. The calculation formula is: The dynamic trajectory matching degree K ranges from 0 to 1. The closer the value is to 1, the higher the matching degree between the actual trajectory and the planned trajectory, and the smaller the deviation. The closer the value is to 0, the greater the deviation. The predicted pose obtained by extrapolating the short-term motion trend is compared with the dynamic trajectory matching degree calculation result. The results are then integrated to obtain a complete motion trend matching result.

[0048] Step 4.6 involves feature fusion and sequence packaging of the spatiotemporal aligned data stream and motion trend matching results to obtain a real-time air-ground collaborative state stream showing the relative positional relationship between the UAV and the robot dog and the degree of deviation from the planned execution. Specifically, this includes: extracting core features from the spatiotemporal aligned data stream, such as the UAV's real-time flight speed, altitude, and attitude, and the robot dog's real-time pose, joint angles, and cumulative mileage; and extracting core features from the motion trend matching results, such as the device's predicted pose and dynamic trajectory matching degree. Feature fusion calculation is then performed using a weighted fusion method, assigning corresponding weights based on the importance of each feature. For dynamic trajectory matching weights, For real-time pose weights, To predict pose weights, The fusion calculation formula is as follows: (The formula is not provided in the original text.) ; In the formula To fuse feature values, For real-time pose data, this calculation process integrates various scattered feature data into unified fusion feature parameters, achieving deep fusion of the two types of data, eliminating data redundancy, and improving the usability of the data.

[0049] After feature fusion is completed, sequence packaging processing is performed. According to the base timestamp order of the collaborative control instructions, the fused feature values ​​corresponding to each timestamp are packaged. The original core feature data and the relative positional relationship between the drone and the robot dog are packaged and integrated. The relative positional relationship is calculated using the spatial coordinates of the two types of devices. Let the drone's coordinates be... The robot dog's coordinates are Relative position distance The calculation formula is: The relative azimuth angle β between the two types of equipment is calculated to comprehensively characterize the spatial positional relationship between the drone and the robot dog, providing a reference for collaborative operation control. All packaged sequence data are arranged in order according to the reference timestamp to obtain the real-time air-ground collaborative state flow.

[0050] In this embodiment of the invention, by employing time-stamp synchronization compensation and spatial coordinate system transformation of UAV flight status data and robot dog ground pose data, the difference in data acquisition timing and coordinate reference is eliminated to obtain a spatiotemporally aligned data stream. Based on the actual pose change rate and planned trajectory curvature, short-term motion trend extrapolation and dynamic trajectory matching degree calculation are performed. Then, the spatiotemporally aligned data stream and motion trend matching results are fused and sequence packaged. Therefore, this invention overcomes the technical problems in the prior art, such as chaotic data timing and inconsistent coordinate references between air and ground equipment, making it difficult to accurately compare the motion status of the two types of equipment and predict the equipment operation trend. Consequently, it is impossible to grasp the overall status of air-ground collaborative operations in real time, affecting the timeliness and accuracy of collaborative control. This invention achieves spatiotemporal unification of UAV and robot dog data, accurately predicts the short-term motion direction of equipment, quantifies the deviation between the actual trajectory and the planned trajectory, and grasps the relative positional relationship and plan execution status of the two types of equipment in real time. It provides high-quality, standardized real-time data support for trajectory timing comparison, deviation correction, and path replanning, ensuring the stability, accuracy, and efficiency of air-ground collaborative operations in power transmission line inspection.

[0051] In a preferred embodiment of the present invention, step 5 above may include: Step 5.1 involves comparing the real-time air-to-ground collaborative state stream with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog by overlaying their trajectory spatial projections onto the time axis. This process summarizes the deviation in actual air-to-ground operational progress. Specifically, this includes: projecting the actual flight and movement trajectories of the UAV and robot dog in the real-time air-to-ground collaborative state stream onto the three-dimensional spatial planes containing the initial inspection trajectory and the initial ground movement sequence, respectively, to eliminate comparison errors caused by minor deviations in the spatial coordinate system; comparing the actual trajectory with the planned trajectory on the same spatial projection plane; completely overlapping the timestamp of the real-time air-to-ground collaborative state stream with the planned time axis of the initial path; and comparing the actual flight position of the UAV with the planned position of the initial inspection trajectory, and the actual movement position of the robot dog with the planned position of the initial ground movement sequence, one by one, at the same time node. The spatial position deviation of the two types of equipment is calculated using the following formula: In the formula This refers to the spatial deviation between the actual and planned positions of the equipment. , , The actual three-dimensional coordinates of the equipment. , , Based on the planned three-dimensional coordinates of the equipment and the comparison of positional deviations, the planned operation progress and actual operation progress of the two types of equipment are calculated separately. The planned operation progress is the ratio of the current running time to the total operation time, and the calculation formula is as follows: In the formula, To plan the work schedule, For the current moment, The start time of the operation. The preset total operation time is used as the basis for determining the actual operation progress, which is the ratio of the number of completed inspection nodes to the total number of inspection nodes. The calculation formula is as follows: In the formula, This refers to the actual work progress. This represents the number of inspection nodes that have been completed. Given the total number of inspection nodes, calculate the deviation of the work progress for a single piece of equipment based on the planned and actual work progress. The calculation formula is as follows: In the formula, The schedule deviation of a single device is calculated by summing the schedule deviations of both the drone and the robot dog, resulting in the overall schedule deviation for air-ground collaborative operations. The calculation formula is as follows: In the formula This refers to the overall deviation in the progress of air-ground coordinated operations. This represents the deviation in the progress of drone operations. The deviation in the robot dog's operational progress is represented by an average value, which comprehensively reflects the overall progress deviation of the air-ground collaborative operation.

[0052] Step 5.2: The actual deviation of the air-to-ground operation progress is compared with a preset safety threshold. When the deviation exceeds the preset safety threshold, the remaining inspection nodes not covered by the current drone and robot dog, along with terrain access constraints, are extracted to obtain a replanning trigger signal for the incomplete section. Specifically, this involves comparing the actual air-to-ground operation progress deviation with a preset safety threshold to determine if the current progress deviation exceeds the safety range. If it does, a path replanning process is triggered. The preset safety threshold is a fixed value, and the judgment rule is implemented through absolute value comparison. The judgment formula is: In the formula, A safety threshold is preset for schedule deviation. If the formula is met, the schedule deviation is determined to be outside the safety range and replanning needs to be initiated. If the formula is not met, the original collaborative control instructions continue to be executed.

[0053] After determining that the threshold has been exceeded, the remaining inspection nodes are spatially filtered and their boundaries locked using a radial sector partitioning elliptical constraint geometric algorithm. A composite geometric model of polar coordinate sector partitioning and planar elliptical constraints is established, with the UAV's current spatial coordinates as the sector center pole and the robot dog's current position as the ellipse focus. First, the radial angle range of the sector partition is calculated using the following formula: In the formula, The polar angle of the remaining inspection nodes relative to the center of the sector. The current coordinates of the drone. The coordinates of the remaining inspection nodes are given. The ground elliptical constraint boundary equations are then constructed: In the formula, The current coordinates of the robot dog. It is the semi-major axis of the ellipse. Using the minor semi-axis of the ellipse, this geometric equation is used to determine whether the remaining inspection nodes fall within the effective geometric area where air-ground collaboration is feasible. Invalid nodes that exceed the geometric constraints are eliminated. After completing the geometric screening, the remaining inspection nodes that fall within the effective geometric area are extracted, along with the corresponding terrain slope, obstacle distribution, traffic restrictions, and other terrain-specific error constraints of the section. The screened node information and constraint parameters are integrated and encoded to obtain the replanning trigger signal for the incomplete section.

[0054] Step 5.3: Based on the replanning trigger signal, taking the current actual pose of the UAV and robot dog as the optimization starting point, and combining terrain accessibility and air-to-ground communication constraints, the online iterative solution is performed on the access order, dynamic waiting and dwell time, and terrain-following detour strategy of the remaining inspection nodes to obtain the cooperative control command sequence to replace the original command. Specifically, this includes: taking the current actual pose of the UAV and robot dog as the optimization starting point, and combining terrain accessibility and air-to-ground communication constraints, the online iterative solution is performed on the access order, dynamic waiting and dwell time, and terrain-following detour strategy of the remaining inspection nodes. For the access order of the remaining inspection nodes, the priority weight of each node is calculated. The higher the weight, the higher the access priority. The calculation formula is: In the formula, As the priority weight of the inspection nodes, This is the inherent priority coefficient of the node. The spatial distance between the node and the device's current pose. Given the total path length of the remaining inspection section, the optimal node access order is determined based on weighted sorting. The dynamic waiting and dwell time is calculated in conjunction with the overall progress deviation. The work progress deviation is compensated by adjusting the dwell time. The calculation formula is as follows: In the formula, For dynamic waiting and dwell time, This refers to the standard operation time for a single node.

[0055] For the terrain-following detour strategy, the detour path length and detour time are calculated separately. The detour path length is the sum of the original planned path and the additional path due to obstacles. The calculation formula is as follows: In the formula, This represents the total length of the detour route. This is the original planned path length. The additional path length for obstacle detour. The detour time is the ratio of the detour path length to the equipment's speed, calculated using the following formula: In the formula, The time required for detours. To adjust the operation sequence based on the equipment movement speed and detour time, the progress deviation is gradually compensated. After multiple rounds of iterative solutions, the access order, dwell time, and detour strategy that meet the optimization objectives are integrated and encoded to obtain a sequence of collaborative control instructions to replace the original instructions. This sequence is then sent to the equipment execution terminal to complete the remaining inspection work.

[0056] In this embodiment of the invention, by comparing the real-time air-ground collaborative state flow with the initial path through trajectory space projection and time axis overlap, the overall progress deviation of the air-ground collaborative operation is calculated. The deviation is then numerically judged against a preset safety threshold. When the threshold is exceeded, the remaining inspection nodes and terrain constraints are extracted to generate a replanning signal. Starting from the current pose, the node access order, dwell time, and detour strategy are iteratively optimized online in combination with the constraints to obtain a new collaborative control command sequence. Therefore, this invention overcomes the technical problems of existing air-ground collaborative inspections, such as the inability to perceive progress deviations in real time, the inability to replan paths under abnormal conditions, and the tendency to miss inspections, disrupt timing, and reduce operational efficiency. This invention achieves real-time closed-loop control of the air-ground collaborative operation status, corrects progress deviations and trajectory deviations, ensures full coverage of inspection tasks, and improves the stability and adaptability of UAV and robot dog collaborative inspections in complex terrains.

[0057] like Figure 2 As shown, embodiments of the present invention also provide a path-based air-ground cooperative control system for drone and robot dog operations, comprising: The acquisition module is used to acquire three-dimensional terrain data, tower distribution coordinates, and preset inspection tasks of the transmission line inspection area. By fusing obstacle coordinates and hidden danger location information transmitted back by UAV high-altitude detection, the fused multi-dimensional spatial data is obtained. The fused multi-dimensional spatial data is preprocessed to obtain the initial point cloud dataset. The building module is used to perform grid-based reconstruction and semantic feature mapping based on the spatial adjacency relationship and elevation continuity of the initial point cloud dataset to build a three-dimensional virtual operation model; The calculation module is used to obtain the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog by using a three-dimensional virtual operation model as the spatial optimization basis, adopting a step-by-step gradient deduction and path iteration processing mechanism, and through multi-dimensional passage cost evaluation and temporal beat matching calculation. The parsing module is used to parse the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog into cooperative control commands; send the cooperative control commands to the execution terminals of the UAV and the robot dog; collect the flight status data of the UAV and the ground pose data of the robot dog; and perform spatiotemporal alignment and motion trend matching processing on the flight status data of the UAV and the ground pose data of the robot dog and the time window labels in the cooperative control commands to obtain the real-time air-ground cooperative state flow. The comparison module is used to compare the trajectory timing with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog based on the real-time air-ground cooperative state flow, and obtain the actual air-ground operation progress deviation; when the progress deviation exceeds the preset safety threshold, the unfinished section is recalculated to obtain the cooperative control command sequence.

[0058] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0059] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0060] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0061] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A path-based air-ground cooperative control method for drone and robot dog operations, characterized in that, The method includes: The system acquires 3D terrain data, tower distribution coordinates, and preset inspection tasks for the transmission line inspection area. By fusing obstacle coordinates and hazard location information transmitted back from UAV high-altitude detection, it obtains fused multidimensional spatial data. The fused multidimensional spatial data is then preprocessed to obtain an initial point cloud dataset. A three-dimensional virtual operation model is constructed by performing grid-based reconstruction and semantic feature mapping based on the spatial adjacency relationship and elevation continuity of the initial point cloud dataset. Using a three-dimensional virtual operation model as the spatial optimization basis, a step-by-step gradient deduction and path iteration processing mechanism is adopted. Through multi-dimensional passage cost evaluation and temporal beat matching calculation, the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog are obtained. The initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog are parsed into cooperative control commands; the cooperative control commands are sent to the execution terminals of the UAV and the robot dog, and the flight status data of the UAV and the ground pose data of the robot dog are collected; the flight status data of the UAV and the ground pose data of the robot dog are spatiotemporally aligned and motion trend matched with the time window labels in the cooperative control commands to obtain the real-time air-ground cooperative state flow. The actual progress deviation of air-ground operations is obtained by comparing the real-time air-ground cooperative state flow with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog. When the progress deviation exceeds the preset safety threshold, the unfinished section is recalculated to obtain the cooperative control command sequence.

2. The path air-ground cooperative control method for UAV and robot dog operations according to claim 1, characterized in that, The system acquires three-dimensional terrain data, tower distribution coordinates, and preset inspection tasks for the transmission line inspection area. By integrating obstacle coordinates and hidden danger location information transmitted back from UAV high-altitude detection, it obtains fused multi-dimensional spatial data. The fused multidimensional spatial data is preprocessed to obtain an initial point cloud dataset, including: The system acquires three-dimensional terrain data, tower distribution coordinates, and preset inspection tasks for the transmission line inspection area. It also receives obstacle coordinates and hidden danger location information transmitted back from UAV high-altitude detection. The system integrates the terrain data, coordinates, task list, and transmitted information from multiple sources to obtain fused multi-dimensional spatial data. The fused multidimensional spatial data is then registered using a multi-source coordinate system to unify the spatial reference and eliminate positional deviations between different detection sources, resulting in reference-aligned spatial data. The benchmark-aligned spatial data is subjected to noise filtering and outlier removal to extract effective geographic features and obtain clean spatial feature data. The clean spatial feature data is discretized and segmented into voxels according to a preset spatial resolution to obtain an initial point cloud dataset containing elevation attributes, obstacle distribution attributes, and task location attributes.

3. The path air-ground cooperative control method for UAV and robot dog operations according to claim 2, characterized in that, Based on the spatial adjacency relationships and elevation continuity of the initial point cloud dataset, a grid-based reconstruction and semantic feature mapping process are performed to construct a 3D virtual operation model, including: Extract the spatial adjacency topology and elevation variation parameters of each discrete point in the initial point cloud dataset, and construct a point cloud feature connectivity network; The point cloud features are connected to a network and then reconstructed into a grid, transforming the discrete point cloud into a continuous terrain surface grid, thus obtaining a gridded terrain base. The gridded terrain base is processed by semantic feature mapping, and each grid unit is classified and labeled as passable area, obstacle-restricted area, tower structure area and inspection target area to obtain the fused classification and labeling result. By integrating the classification and labeling results with grid spatial coordinates, a three-dimensional virtual operation model is constructed, which includes the terrain undulation, obstacle spatial distribution, and task node locations.

4. The path air-ground cooperative control method for UAV and robot dog operations according to claim 3, characterized in that, Using a 3D virtual operation model as the spatial optimization basis, and employing a stepped gradient deduction and path iteration processing mechanism, the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog are obtained through multi-dimensional passage cost evaluation and temporal rhythm matching calculation, including: Using spatial nodes in the three-dimensional virtual operation model as the optimization calculation objects, a step-by-step gradient deduction method is used to calculate the multi-dimensional passage cost of each node under the constraints of high-altitude detection field of view coverage requirements and ground obstacle crossing ability, and the node cost distribution matrix is ​​obtained. The node cost distribution matrix is ​​processed by path iteration to dynamically remove redundant path branches that exceed the passage cost threshold, thereby obtaining a set of open-field candidate paths that meet the terrain and safety constraints. By matching the time requirements in the set of candidate air-ground paths with the preset inspection task list, the arrival time of each candidate path node is synchronously calibrated and aligned with the time requirements to obtain the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog that meet the time requirements of air-ground collaborative operation.

5. The path air-ground cooperative control method for UAV and robot dog operations according to claim 4, characterized in that, The initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog are parsed into cooperative control commands; The collaborative control commands are sent to the execution terminals of the drone and the robot dog, and the flight status data of the drone and the ground pose data of the robot dog are collected, including: The initial inspection track of the UAV and the initial ground movement sequence of the robot dog are processed by instruction parsing and segmented encoding. The spatiotemporal control parameters of each track node and movement node are extracted to obtain the cooperative control instructions with time window labels. The collaborative control command is sent to the execution terminal of the drone and robot dog through the communication link, triggering the status acquisition function of the drone and robot dog platform to enter the synchronous operation mode; During the execution of collaborative control commands, the flight attitude, speed and altitude information fed back by the UAV execution terminal under the trigger state are read in real time, as well as the pose, joint angle and travel distance information fed back by the robot dog execution terminal. The flight status data of the UAV and the ground pose data of the robot dog are then summarized.

6. The path air-ground cooperative control method for UAV and robot dog operations according to claim 5, characterized in that, The flight status data of the UAV, the ground pose data of the robot dog, and the time window labels in the cooperative control commands are spatiotemporally aligned and matched for motion trends to obtain the real-time air-ground cooperative state flow, including: The time window labels in flight status data, ground pose data and cooperative control commands are time-stamped and compensated for, and spatial coordinate system is transformed to eliminate the time difference and coordinate reference difference in data acquisition between UAV and robot dog, and obtain a spatiotemporally aligned data stream. Using the spatiotemporally aligned data stream as the input for trend calculation, short-term motion trend extrapolation and dynamic trajectory matching degree calculation are performed based on the actual pose change rate and the planned trajectory curvature to obtain motion trend matching results. By performing feature fusion and sequence packaging processing on the spatiotemporal aligned data stream and motion trend matching results, a real-time air-ground collaborative state stream is obtained, which shows the relative positional relationship between the UAV and the robot dog and the degree of deviation from the plan execution.

7. The path air-ground cooperative control method for UAV and robot dog operations according to claim 6, characterized in that, The actual air-ground operation progress deviation is obtained by comparing the real-time air-ground collaborative state flow with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog. When the schedule deviation exceeds the preset safety threshold, the unfinished section is recalculated to obtain a sequence of collaborative control instructions, including: The real-time air-ground collaborative status flow is compared with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog by projecting the trajectory space onto the time axis and summarizing the deviation of the actual air-ground operation progress. The actual progress deviation of the air-ground operation is compared with the preset safety threshold. When the progress deviation exceeds the preset safety threshold, the remaining inspection nodes and terrain access constraints not covered by the current drone and robot dog are extracted to obtain the replanning trigger signal for the unfinished section. Based on the replanning trigger signal, taking the current actual pose of the UAV and robot dog as the optimization starting point, and combining terrain accessibility and air-to-ground communication constraints, the access order, dynamic waiting dwell time and terrain following detour strategy of the remaining inspection nodes are solved online iteratively to obtain the cooperative control command sequence to replace the original command.

8. A path-based air-ground cooperative control system for drone and robot dog operations, wherein the system implements the method as described in any one of claims 1 to 7, characterized in that, include: The acquisition module is used to acquire three-dimensional terrain data, tower distribution coordinates and preset inspection tasks of the transmission line inspection area. By integrating obstacle coordinates and hidden danger point information transmitted back by UAV high-altitude detection, multi-dimensional spatial data is obtained. The fused multidimensional spatial data is preprocessed to obtain an initial point cloud dataset. The module is used to perform grid-based reconstruction and semantic feature mapping based on the spatial adjacency relationship and elevation continuity of the initial point cloud dataset to build a three-dimensional virtual operation model. The calculation module is used to obtain the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog by using a three-dimensional virtual operation model as the spatial optimization basis, adopting a step-by-step gradient deduction and path iteration processing mechanism, and through multi-dimensional passage cost evaluation and temporal beat matching calculation. The parsing module is used to parse the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog into cooperative control commands; send the cooperative control commands to the execution terminals of the UAV and the robot dog; collect the flight status data of the UAV and the ground pose data of the robot dog; and perform spatiotemporal alignment and motion trend matching processing on the flight status data of the UAV and the ground pose data of the robot dog and the time window labels in the cooperative control commands to obtain the real-time air-ground cooperative state flow. The comparison module is used to compare the trajectory timing with the initial inspection trajectory of the UAV and the initial ground movement sequence of the robot dog based on the real-time air-ground cooperative state flow, and obtain the actual air-ground operation progress deviation; when the progress deviation exceeds the preset safety threshold, the unfinished section is recalculated to obtain the cooperative control command sequence.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images