Power distribution facility inspection method and system using collaborative unmanned vehicle, robot dog, and robotic arm

Abstract

The present invention relates to the field of information technology, and discloses a distribution facility inspection method and system that collaborates with an unmanned vehicle, a robot dog, and a robotic arm. The unmanned vehicle is used to obtain three-dimensional distribution data of cables in a fault area, and through path planning and dynamic adjustment of the chassis height, the unmanned vehicle can be accurately docked in a confined space; the robot dog is used to detect blocked areas to extract the characteristics of faulty components and determine their precise coordinates; the reachable range of the robotic arm carried by the unmanned vehicle is quantified, and then the change in the joint angle of the robotic arm is calculated. After the unmanned vehicle fine-tunes the position to eliminate deviations, the operation instructions are determined; based on the predicted real-time position change trend of the robot dog relative to the fault point, the fault position data synchronized with the operation instructions is determined, so that the robotic arm can generate a grasping plan and accurately implement emergency repair actions; automated and precise emergency repairs are achieved in complex cable environments, improving work efficiency and safety.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a distribution facility inspection method and system using an unmanned vehicle, a robot dog, and a robotic arm in coordination. Background Art

[0002] Distribution facility inspections are a core component of power system operations and maintenance. Existing technologies typically conduct inspections of distribution facilities, especially those in complex environments like high-density cable areas, by human workers. However, traditional manual inspections are subject to low efficiency, high risk, and incomplete coverage.

[0003] Therefore, how to improve the inspection efficiency and safety of power distribution facilities in high-density cable areas has become a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0004] The present invention provides a distribution facility inspection method and system that collaborates with an unmanned vehicle, a robot dog, and a robotic arm to solve the problem of how to improve the inspection efficiency and safety of distribution facilities in high-density cable areas.

[0005] To solve the above technical problems, the first aspect of the present invention provides a distribution facility inspection method using an unmanned vehicle, a robot dog, and a robotic arm in collaboration, comprising:

[0006] Determining a preliminary parking area for the unmanned vehicle based on three-dimensional cable distribution data corresponding to a fault area of the power distribution facility, and quantifying the movement trajectory of the unmanned vehicle in the fault area based on the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point;

[0007] The occlusion information data of the fault area collected by the robot dog is processed by a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and the range of motion of the robot arm is quantified based on the relative relationship between the precise docking position and the precise three-dimensional coordinates;

[0008] Acquire real-time coordinate offset data when the robot dog adjusts its posture, so as to quantify dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generate operation instructions for the robot arm based on the dynamic adjustment parameters;

[0009] The real-time coordinate offset data is processed using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point, thereby obtaining fault location data synchronized with the operation instruction;

[0010] Capture data is generated based on the operation instruction and the fault location data to separate the faulty component from the fault area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

[0011] A second aspect of the present invention provides a power distribution facility inspection system coordinated by an unmanned vehicle, a robot dog, and a robotic arm, comprising:

[0012] a position determination module, configured to determine a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point;

[0013] a range quantification module, configured to process the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and quantify the motion range of the robot arm based on the relative relationship between the precise docking position and the precise three-dimensional coordinates;

[0014] an instruction generation module, configured to obtain real-time coordinate offset data when the robot dog adjusts its posture, quantify dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generate an operation instruction for the robot arm based on the dynamic adjustment parameters;

[0015] a position synchronization module, configured to process the real-time coordinate offset data using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point and obtain fault position data synchronized with the operation instruction;

[0016] A fault repair module is used to generate captured data based on the operation instruction and the fault location data, so as to separate the faulty component from the fault area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

[0017] Compared with the prior art, the embodiments of the present invention have the following advantages:

[0018] (1) By processing the three-dimensional distribution data of cables, the precise parking area of the unmanned vehicle in complex fault areas is determined; in an obstructed environment, the robot dog extracts the contour features of the faulty component through a convolutional neural network, accurately locates the three-dimensional coordinates of the fault point, and dynamically adjusts its posture to avoid obstacles, reducing inspection risks;

[0019] (2) Through the collaborative operation of unmanned vehicles, robot dogs and robotic arms, coordinate offset correction, fault location synchronization, etc. are carried out to achieve seamless collaboration, shorten the overall operation time, and adapt to different terrains and complex environments, realizing efficient, accurate, intelligent and automated inspection and repair of the fault area of distribution facilities. This not only improves the efficiency of repairs, reduces labor costs and risks, but also ensures the safe and stable operation of distribution facilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the implementation. Obviously, the drawings described below are only some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is a flow chart of a distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm, provided by one embodiment of the present invention;

[0022] Figure 2 This is a structural diagram of a power distribution facility inspection system that is coordinated by an unmanned vehicle, a robot dog, and a robotic arm, provided by a certain embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings and embodiments. Obviously, the embodiments described are only some embodiments of the present invention, rather than all embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0024] In the description of this application, the terms "first," "second," "third," etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first," "second," "third," etc. may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0025] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be a communication between the two components. The terms "vertical", "horizontal", "left", "right", "up", "down" and similar expressions used herein are for illustrative purposes only, and do not indicate or imply that the system or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0026] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this application have the same meanings as those commonly understood by those skilled in the art. The terms used in this specification are only for describing specific embodiments and are not intended to limit the present invention. Those skilled in the art will understand the specific meanings of the above terms in this application according to specific circumstances.

[0027] In one embodiment, if Figure 1 As shown, the first aspect of the present invention provides a distribution facility inspection method using an unmanned vehicle, a robot dog, and a robotic arm in collaboration, comprising:

[0028] S1. Determine a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area based on the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point;

[0029] In one embodiment, determining the preliminary parking area for the unmanned vehicle based on the three-dimensional cable distribution data corresponding to the fault area of the power distribution facility includes:

[0030] The multi-view laser radar carried by the unmanned vehicle collects three-dimensional distribution data of cables corresponding to a fault area of the power distribution facility; the fault area is a ground space-constrained environment area including a high-density cable interlaced area of the power distribution facility;

[0031] The three-dimensional distribution data of the cables is processed using a point cloud distance calculation algorithm to obtain the height data and position coordinates of each cable in the fault area, so as to analyze the density distribution law of each cable in the fault area and obtain cable density distribution characteristic data;

[0032] Dividing the fault area based on the cable density distribution characteristic data, and marking areas where the cable height is lower than a preset height threshold as candidate parking areas for the unmanned vehicle, to obtain a set of candidate areas;

[0033] Extracting target candidate areas that match the height requirement of the unmanned vehicle from the candidate area set, and using a geometric constraint algorithm to determine the spatial connectivity of each target candidate area to obtain a connected area range;

[0034] According to the range of the connected area and the scanning boundary conditions of the multi-view laser radar, the coordinates of the docking area are determined so as to optimize and group the target candidate areas within the range of the connected area using a machine learning clustering algorithm to generate a preliminary docking area for the unmanned vehicle.

[0035] Specifically, since the unmanned vehicle needs to move on the ground to carry tools and spare parts during the inspection process, the ground space in the high-density cable area is limited and the cables are hung at different heights, it is difficult for the unmanned vehicle to accurately park near the fault point.

[0036] Based on this, the present invention uses the multi-view laser radar carried by the unmanned vehicle to scan the fault area corresponding to the distribution facility (that is, the high-density cable interlaced area) to obtain the three-dimensional distribution data of the cables from the confined ground space environment of the high-density cable interlaced area; such as in an old industrial park, the cables are crisscrossed at different heights and the ground space is limited. The multi-view laser radar installed on the unmanned vehicle collects data from multiple directions such as the side and top at a scanning frequency of 200,000 points per second, captures the reflected signal of the cable, and forms a three-dimensional point cloud distribution data of the cable to determine the preliminary spatial distribution information of the cable in the fault area, so as to clearly display the three-dimensional layout of the cable. For example, there are 5 cables in a certain area, located at heights of 1.5 meters, 2.0 meters, 3.0 meters, etc., laying the foundation for subsequent analysis.

[0037] A point cloud distance calculation algorithm is used to extract cable features. Based on the three-dimensional cable distribution data collected by the unmanned vehicle, the algorithm can calculate the vertical distance between each cable and the ground through the point cloud data, and obtain the height data, diameter data, and location coordinates of each cable in the fault area. For example, in the aforementioned industrial park scenario, the algorithm analyzes the point cloud in real time and determines that the diameter of a cable at a height of 1.5 meters is approximately 10 centimeters, and the diameter of a cable at a height of 2.0 meters is 15 centimeters. At the same time, the precise coordinates of each cable are determined, such as (x1, y1, z1) is (10, 20, 1.5), to distinguish cable height differences and provide data support for space occupancy analysis. The fault area is divided into several grids (such as 1m×1m) to count the number of cables and the average height in each grid. Kernel density estimation is then used to generate a two-dimensional density heat map to identify the density distribution pattern of each cable in the fault area and obtain cable density distribution characteristic data. Furthermore, assuming the scanning area is 50 meters by 50 meters, analysis shows that cables with a height of less than 2.0 meters are concentrated in the northwest corner of the area, while cables with a height of more than 3.0 meters are mostly distributed in the southeast corner. It can be seen that low-height cables have a more obvious restriction on ground space. For example, under the low cables in the northwest corner, the space height is only 1.5 meters, which restricts the passage of large equipment. However, the space in the southeast corner is relatively open. These analysis results can provide a basis for subsequent area division and help optimize the docking planning of unmanned vehicles.

[0038] The fault area is divided according to the cable density distribution characteristic data. If the cable height is lower than the preset height threshold, the area below it is marked as a candidate area for the unmanned vehicle to dock. The areas with an area smaller than the minimum turning radius of the unmanned vehicle and the areas where the cable height does not meet the standard are eliminated. Then, a 0.3m buffer zone is extended outward to ensure that the unmanned vehicle maintains a safe distance from the cable when docking, so as to generate a set of candidate areas. Among them, the preset height threshold can be 2.5 meters. The division of the fault area utilizes the height distribution characteristics to screen out potential available docking space and improve space utilization.

[0039] Target candidate areas that match the height requirements of unmanned vehicles (that is, areas that allow unmanned vehicles to pass smoothly) are extracted from the candidate area set, and a geometric constraint algorithm is used to construct a topological graph model with candidate areas as nodes and reachable paths between areas as edges. The path width must be ≥ the width of the unmanned vehicle. The Dijkstra algorithm is used to search for the longest continuous path and mark the connected subgraph. If the connected area is blocked by temporary obstacles (such as maintenance equipment), the topological graph is updated and recalculated through real-time point cloud until the spatial connectivity of all target candidate areas is determined. The range of connected areas that can be passed by unmanned vehicles is generated to ensure the practicality of the docking area and avoid spatial fragmentation.

[0040] Based on the scanning boundary conditions of the multi-view LiDAR on the unmanned vehicle, the coordinates of the initial docking area and the docking coordinates of the unmanned vehicle can be determined from the connected area. These coordinates are then extracted to identify candidate areas with features such as area, shape index (aspect ratio), and Euclidean distance from the fault point. Constrained by the vehicle's scanning boundary conditions, the DBSCAN algorithm is used to merge adjacent areas with similar attributes within the connected area to determine the initial docking area for the unmanned vehicle. These attributes refer to the multi-dimensional features of the candidate areas, which quantify their geometric properties (area, shape index), spatial location (the Euclidean distance from the fault point must meet a safety radius constraint), and environmental constraints (minimum turning radius). For example, if the height of a cable in a sub-area suddenly drops to 2.0 meters, clustering can be used to identify and adjust the boundaries in advance. Furthermore, docking areas can be adjusted based on real-time environmental changes. For example, if a 2.2-meter-high cable is added to an industrial park, the algorithm can dynamically update the candidate area set after the LiDAR rescan, removing the affected areas to ensure the adaptability of the solution. The entire process, from scanning to optimization, achieves efficient utilization of spatial resources, providing reliable support for unmanned vehicle docking in complex environments while reducing manual intervention costs.

[0041] The present invention uses multi-view lidar and density analysis to accurately identify safe stopping points in high-density cable crossing areas to avoid collision risks; combines geometric constraints with machine learning clustering to efficiently divide feasible areas in restricted ground environments and improve path planning efficiency; screens stopping areas based on preset height thresholds to ensure a safe distance between the unmanned vehicle and the cables and reduce the risk of electromagnetic interference; and uses point cloud processing and clustering algorithms to reduce manual intervention and achieve automated and intelligent stopping area selection.

[0042] In one embodiment, quantifying the movement trajectory of the unmanned vehicle in the fault area based on the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point includes:

[0043] quantifying the movement trajectory of the unmanned vehicle under the spatially restricted conditions corresponding to the fault area based on the preliminary docking area, the cable density distribution characteristic data, and the height data of each cable, to obtain an initial trajectory coordinate sequence;

[0044] When it is determined that there is a deviation between the initial trajectory coordinate sequence and the fault area, an adjustment parameter for the chassis height of the unmanned vehicle is generated by a model predictive control method to obtain a trajectory adjustment coordinate sequence;

[0045] Based on the trajectory adjustment coordinate sequence and the height data of each cable, the cable height change characteristics of the fault point and its preset area are extracted to determine the area division range of the fault point through a clustering algorithm;

[0046] According to the area division range and the spatial restriction condition, a target trajectory adjustment coordinate sequence that meets the connectivity requirements is selected from the trajectory adjustment coordinate sequence and combined to generate a continuous movement trajectory path to be combined with the cable density distribution characteristic data to determine the precise parking position of the unmanned vehicle relative to the fault point.

[0047] Specifically, the present invention takes the coordinates of the preliminary docking area, the cable density distribution characteristic data of the area, and the height data of the cables contained in the area as input, and adopts a rapid exploration random tree algorithm to convert the cable height data into a three-dimensional obstacle model, limit the moving height of the unmanned vehicle (such as the chassis height must be ≥ cable height + 0.3m), output the moving trajectory of the unmanned vehicle under spatially constrained conditions, and obtain an initial trajectory coordinate sequence (including position, speed, attitude angle, etc.). Other path planning algorithms may also be used. It should be noted that the present invention only adopts these algorithms and does not improve them. Therefore, the specific steps of the algorithm are not described in detail. For example, in an old industrial park, the unmanned vehicle needs to move from the starting point (0,0,0) to the fault point (40,40,0). Combined with the cable height data obtained by the aforementioned lidar scanning, for example, the cable height in the northwest corner is 1.5 meters and the cable height in the southeast corner is 3.0 meters. The path planning algorithm can analyze the spatial limitations and avoid areas with a height lower than 1.2 meters from the unmanned vehicle chassis. The algorithm prioritizes the path under the cable at a height of 3.0 meters in the southeast corner to generate an initial trajectory coordinate sequence, such as (0,0,0), (10,10,0), (20,20,0), (30,30,0), and (40,40,0). By making full use of the height distribution pattern of the cables, the trajectory is initially feasible.

[0048] A deviation detection algorithm analyzes the degree of match between the movement trajectory and the actual environment. This involves using LiDAR SLAM and IMU fusion positioning to calculate the trajectory tracking error (lateral error, heading angle error) and compare it with a preset deviation threshold. If the deviation error exceeds the preset deviation threshold, a deviation is determined between the initial trajectory coordinate sequence and the fault area. A model predictive control approach is then used to establish the unmanned vehicle's kinematic model. With the goal of minimizing the tracking error and chassis height adjustment (to prevent frequent lifting and lowering), the vehicle outputs the chassis height adjustment amount and trajectory coordinate offset compensation as the trajectory adjustment coordinate sequence. If the trajectory tracking error is within the preset deviation threshold, the vehicle's original chassis height is used. This real-time adjustment improves the vehicle's environmental adaptability and avoids failures caused by static planning.

[0049] The three-dimensional point cloud distribution data of the cables collected by the unmanned vehicle is updated based on the trajectory adjustment coordinate sequence. The cable height change characteristics (reflecting the degree of fluctuation) and height gradient (for example, a height difference of >0.5m between adjacent points is considered a mutation point) within the fault point and its preset area (such as a radius of 2m) are extracted according to the height of each cable. The extracted cable height change characteristics are processed by the density-based OPTICS clustering algorithm to group the fault point and its preset area, and the cable height mutation area is marked as a potential fault sub-area (such as a break point or loose joint) to obtain the regional division range of the fault point.

[0050] Based on the spatial constraints of the regional division range and the fault area, the trajectory adjustment coordinate sequence is determined to meet connectivity requirements. Constrained by the cable density not exceeding a preset density threshold and the cable height not exceeding a preset height threshold, target trajectory adjustment coordinate sequences that meet connectivity requirements are selected from the trajectory adjustment coordinate sequence and combined. The combined results are mapped to graph nodes. A check is performed to determine whether there are reachable paths between the nodes. Trajectory breakpoints are filled using cubic spline interpolation to ensure path continuity. The interpolation step size is reduced (e.g., 0.1m) in densely cabled areas and increased (e.g., 0.5m) in sparsely cabled areas. A continuous moving trajectory path and its coordinate set are generated. Connectivity verification avoids spatial fragmentation and ensures path integrity.

[0051] The Euclidean distance between each point in the continuous trajectory and the fault point is calculated. Using the objective function of minimizing the weighted sum of the total path length and the maximum distance difference (weight ratio 6:4), a gradient descent method or genetic algorithm is used to optimize trajectory points whose distance difference exceeds a preset difference threshold, outputting the optimized trajectory path. Trajectory points whose distance difference does not exceed the preset difference threshold indicate that their distance from the fault point meets the requirements and can still be used. Based on the optimized trajectory path and cable density distribution characteristic data, the lowest density candidate area (density <2 cables / m²) is selected as the docking point, provided that the docking location is ≥0.5m from the nearest cable (to prevent electromagnetic interference and mechanical collisions). The center of the unmanned vehicle chassis is aligned with the coordinate system of the fault point. Fine-tuning is assisted by visual landmarks (ArUco codes) to obtain the precise docking position of the unmanned vehicle relative to the fault point and output it. For example, at (41,41,0), the cable height is 3.0 meters, and the spatial connectivity range is 5 meters × 4 meters, which meets the 3 meters × 2 meters requirement of the unmanned vehicle. The final docking coordinates are then determined to be (41,41,0) to (43,43,0). The precise positioning of the unmanned vehicle fully utilizes spatial resources and reduces the impact of deviations.

[0052] The present invention combines spatial constraints with dynamic adjustments to perform high-precision trajectory planning, ensuring the safe movement of unmanned vehicles in cable-dense areas; uses model predictive control to correct trajectories in real time to cope with environmental interference; avoids path breakage or unreachable problems through connectivity verification and trajectory optimization; and uses clustering and interpolation algorithms to quickly fill trajectory gaps, reducing computational time. At the same time, it enables efficient movement and docking of unmanned vehicles in complex environments, significantly improving the level of automation and safety.

[0053] S2. Processing the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and quantifying the motion range of the robot arm based on the relative relationship between the precise docking position and the precise three-dimensional coordinates;

[0054] In one embodiment, the processing of the occlusion information data of the fault area collected by the robot dog by a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point includes:

[0055] Based on the precise docking position, an infrared sensor and a depth camera deployed on the robot dog are used to obtain occlusion information data of the occlusion area below the cable, and a multimodal data set is obtained to be input into a convolutional neural network for processing, and a contour feature data set is output;

[0056] Determining the three-dimensional coordinate value of the faulty component according to the contour feature data set and parameter information of the depth camera;

[0057] Based on the three-dimensional coordinate values and the shielded area below the cable, a clustering algorithm is used to divide the regional boundary of the faulty component to obtain a boundary division coordinate sequence to determine the thermal characteristic distribution matrix of the faulty component according to the multimodal data set;

[0058] Based on the contour feature data set, a matching thermal feature distribution matrix is extracted from the thermal feature distribution matrix. When it is determined that there is a position deviation between the matching thermal feature distribution matrix and the three-dimensional coordinate value, the matching thermal feature distribution matrix is adjusted by a gradient descent algorithm to obtain a thermal feature distribution adjustment matrix to combine with the three-dimensional coordinate value to determine the rough three-dimensional coordinates of the fault point.

[0059] Specifically, since the robot dog needs to enter the narrow area under the cable to locate the fault during the inspection process, its visual sensor is easily interfered by the dense cables, making it difficult to accurately identify the three-dimensional coordinates of the faulty component.

[0060] Based on this, the present invention uses the infrared sensor and depth camera deployed on the robot dog to obtain the occlusion information data of the obstructed area under the cable according to the precise parking position of the unmanned vehicle, such as abnormal thermal characteristics such as overheating of the cable joint, 3D point cloud of the obstructed area, etc., and aligns the timestamps of the infrared and depth data through hardware trigger signals to form a multimodal data set containing temperature and three-dimensional position, so as to improve the comprehensiveness of the information and avoid missing key features by a single sensor; an improved Mask R-CNN is adopted, whose backbone network is ResNet-50-FPN, and an infrared channel input branch is added. The improved Mask R-CNN is trained with the historical multimodal data set and its fault labels as training data, and the multimodal data set collected by the robot dog based on the precise parking position of the unmanned vehicle is input into the trained improved Mask R-CNN, and the binary mask (initial contour feature) of the faulty component is output to obtain a contour feature data set. For multimodal data sets, separating the initial contour features of the faulty component from cable obstruction interference can also be achieved through image processing technology: cable-dense areas may obstruct the faulty component, and the high-temperature areas in the infrared data are combined with the mutation points in the depth data to preliminarily outline the contour of the faulty component.

[0061] The infrared image and the depth point cloud are aligned using an extrinsic parameter matrix to generate a 3D point cloud with a temperature label. 100 points are randomly sampled within the contour mask output by the improved Mask R-CNN, and the mean of their 3D coordinates is calculated as the initial value of the 3D coordinates. Then, with minimizing the projection error of the contour points (from the depth camera coordinate system to the image plane) as the objective function, the least squares method is used to iteratively adjust the translation vector and rotation matrix of the faulty component until convergence. The optimized 3D coordinate value of the faulty component is obtained as the 3D coordinate value of the faulty component.

[0062] The obstructed area below the cable, its point cloud data, and the optimized three-dimensional coordinates are used as input. Density-based DBSCAN clustering is used with parameter settings (eps=0.05m, min_samples=5) to process these data. The faulty component is divided into regions with regional boundaries, and a coordinate sequence of the boundary division of the faulty component (such as the high-temperature zone boundary coordinate set) is output. Based on the clustering results, the thermal feature distribution data of the faulty component collected by the infrared sensor is extracted from the multimodal data set to generate a thermal feature distribution matrix.

[0063] The contour mask output by Mask R-CNN is mapped to the thermal feature distribution matrix for contour-temperature alignment. If the distance between the contour center and the high-temperature area center is greater than the preset center distance, it is determined that the matching fails, and the thermal feature distribution matrix parameters are optimized by the optimization algorithm. Otherwise, it is determined that the matching is successful, and the thermal feature distribution matrix that passes the matching and carries the contour mask (the coordinate data can be calculated based on this) is used as the matching thermal feature distribution matrix to calculate the position deviation between it and the optimized three-dimensional coordinate value. When the position deviation exceeds the preset position deviation threshold, it is determined that there is a position deviation between the matching thermal feature distribution matrix and the three-dimensional coordinate value. The contour position is translated along the temperature gradient direction by the gradient descent algorithm with an iteration step of 0.01 meters and a maximum of 50 iterations to adjust the matching thermal feature distribution matrix to obtain the thermal feature distribution adjustment matrix. When the position deviation does not exceed the preset position deviation threshold, the original thermal feature distribution matrix is output as the thermal feature distribution adjustment matrix.

[0064] A weighted combination of the thermal signature distribution adjustment matrix and the optimized 3D coordinate values (0.4:0.46) is performed to determine the rough 3D coordinates of the fault point. This method utilizes multimodal data fusion from infrared and depth cameras to accurately locate the faulty component, penetrating cable obstructions. High-precision 3D reconstruction is achieved by combining CNN contour extraction with least-squares optimization to improve coordinate accuracy. The combined optimization of thermal signatures and geometric contours enhances robustness under complex environmental interference. Clustering-based boundary demarcation reduces reliance on manual annotation and adapts to diverse fault morphologies.

[0065] In one embodiment, the processing of the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point further includes:

[0066] Based on the rough three-dimensional coordinates, extracting the occlusion depth data collected by the depth camera from the multimodal data set, performing denoising processing using an image denoising algorithm to obtain denoised depth data, and processing the denoised depth data using a convolutional neural network to extract edge features of the faulty component under cable interference to obtain an edge feature dataset;

[0067] Determining whether the integrity of the edge feature dataset is lower than a preset integrity threshold, and when the integrity of the edge feature dataset is lower than the preset integrity threshold, optimizing the denoised depth data through an iterative optimization algorithm to obtain optimized depth data;

[0068] Determining initial three-dimensional coordinates of the faulty component according to the optimized depth data and the rough three-dimensional coordinates, and processing the edge feature dataset and the optimized depth data using an interpolation algorithm based on the initial three-dimensional coordinates to obtain a continuous three-dimensional coordinate path;

[0069] Based on the continuous three-dimensional coordinate path, the spatial distribution characteristics of the faulty component are extracted from the obscured area below the cable, and a distribution feature data set is obtained to divide the regional boundaries of the faulty component using a clustering algorithm to obtain the precise three-dimensional coordinates of the fault point and its corresponding boundary sequence data.

[0070] Specifically, when obtaining data on occluded areas captured by the robot dog's depth camera using coarse coordinates, there is often complex interference beneath the cables, such as dense cables, light variations, or dust. This method extracts the occluded depth data collected by the depth camera from the multimodal data set corresponding to the coarse three-dimensional coordinates. This data is filtered using a non-local means denoising algorithm (search window 21×21, similarity window 5×5), and a conditional generative adversarial network (U-Net generator, PatchGAN discriminator) is used to fill in missing depth pixels. This removes noise from the occluded depth data, such as depth value jumps caused by cable reflections and electromagnetic interference. The resulting denoised depth data is used as input and processed using a multi-scale edge detection network to extract edge features of faulty components under cable interference. The resulting edge probability map is then output to generate an edge feature dataset. Among them, the depth of a certain point is shown as 2.5 meters, but the surrounding points are all 2.2 meters. This sudden change may be caused by the reflection of the obstruction. When using the image denoising algorithm to process this data, you can use mean filtering to remove isolated noise points. For example, smooth the 2.5 meters to around 2.2 meters to form a denoised depth data set to improve data smoothness.

[0071] Completeness is defined as total closed edge length / theoretical maximum closed edge length (component size estimated based on rough three-dimensional coordinates) to assess the integrity of the edge feature dataset. If it is not less than the preset completeness threshold, the denoised depth data is output as the optimized depth data. Otherwise, the denoised depth data is optimized using a particle swarm optimization algorithm, with the fitness function maximizing the area and continuity of the edge closed region to generate optimized depth data. Furthermore, for the edge feature dataset, if the completeness is judged to be less than a preset threshold, for example, setting the threshold to 80%, if the edge points only cover 50% of the outline of the faulty component, the denoised depth dataset is adjusted through an iterative optimization algorithm. Assuming that the initial depth data omits some areas, iterative optimization can supplement the missing data by comparing edge points and depth values multiple times. For example, the depth near (42.1, 42, 2.2) is adjusted from an invalid value to 2.18 meters, resulting in an optimized depth data set.

[0072] The optimized depth map and rough 3D coordinates (from infrared-depth fusion) are registered using the ICP algorithm to align the coordinate systems. Harris corner points are extracted from the edge map and back-projected into 3D space. The center of mass is calculated as the initial 3D coordinates of the faulty component. Based on the initial 3D coordinates, B-spline interpolation is used to fit the edge feature dataset and the discrete edge points in the optimized depth data into a smooth curve. The density of control points is increased in areas with large curvature (such as corners) to obtain a continuous 3D coordinate path formed by a continuous set of path points.

[0073] Based on the position of each point set in the continuous three-dimensional coordinate path, the spatial distribution characteristics of the faulty component, such as geometric characteristics and depth characteristics, are extracted from the obscured area below the cable and combined to generate a distribution feature set. The distribution feature set is processed using a spectral clustering algorithm, and a similarity matrix is constructed based on the Euclidean distance of the feature vectors to divide the regional boundaries of the faulty component. The precise three-dimensional coordinates of the fault point and its corresponding boundary sequence data, such as a polygon vertex coordinate set, are obtained to describe the shape of the faulty component. In addition, when extracting the spatial distribution characteristics of the faulty component from the obscured area based on the continuous three-dimensional coordinate path data, the height distribution of the path points can be analyzed. For example, the points on the west side are concentrated at 2.2 meters and the points on the east side are concentrated at 2.5 meters to generate a distribution feature data set. Furthermore, if an abnormal concentration of depth is found in a certain area, it may indicate the shape characteristics of the faulty component, such as a circle or a strip.

[0074] Through multi-stage denoising and optimization, the present invention effectively eliminates the influence of cable obstruction and electromagnetic noise on depth data, improves the integrity of edge features, and enhances anti-interference capability. It combines interpolation and clustering algorithms for high-precision 3D reconstruction, achieves precise coordinate positioning of faulty components, and breaks through the accuracy limitations of traditional single sensors. Iterative optimization based on integrity criteria ensures that reliable 3D models can still be generated in extreme obstruction scenarios. The combination of spatial distribution characteristics and clustering allows for adaptive boundary division to adapt to the diversity of fault morphologies in complex cable-interlaced environments.

[0075] For the range of motion of the robotic arm, that is, the boundary data of the reachable range of the robotic arm, the present invention uses the Euclidean distance and direction vector between the precise parking position of the unmanned vehicle and the precise three-dimensional coordinates of the fault point to determine it; wherein, the Euclidean distance is used to quantify the straight-line distance between the unmanned vehicle and the fault point to guide the planning of the robotic arm's extension length, and the direction vector is used to determine the spatial direction that the robotic arm needs to cover to guide the adjustment of the joint angle. The maximum extension range of the robotic arm is calculated according to the length of each link of the robotic arm and the joint angle limit as a length constraint, and then combined with the lidar point cloud data to identify obstacles in the fault area to plan the obstacle avoidance path of the robotic arm; the DH parameter method is used to establish the robotic arm kinematic model to define the transformation relationship between each joint coordinate system, and the forward kinematics calculation method is used to calculate the position of the robotic arm end effector according to the joint angle to verify whether the fault point is covered; finally, with the Euclidean distance as the cost function, the A* algorithm is used to plan the shortest path of the robotic arm to cover the fault point, and the gradient projection method is used to adjust the Cartesian path to ensure that the robotic arm can still complete the task when a joint fault or obstacle exists, thereby obtaining the range of motion of the robotic arm.

[0076] S3. Acquire real-time coordinate offset data when the robot dog adjusts its posture, quantify dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generate operation instructions for the robot arm based on the dynamic adjustment parameters;

[0077] In one embodiment, obtaining real-time coordinate offset data when the robot dog adjusts its posture, and quantifying dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, includes:

[0078] Acquiring real-time coordinate offset data of the robot dog when it frequently adjusts its posture and processing it through a filtering algorithm to obtain denoised offset data, which is then combined with the operating platform boundary data within the motion range to calculate initial values of dynamic adjustment parameters of the robot arm at the current posture adjustment frequency, thereby obtaining an initial dynamic adjustment parameter set;

[0079] When a deviation between the initial dynamic adjustment parameter set and a preset adjustment threshold exceeds a preset deviation limit range, iteratively optimizing the initial dynamic adjustment parameter set using a gradient descent method to generate an optimized dynamic adjustment parameter set when the robotic arm is aligned with the fault point;

[0080] Based on the optimized dynamic adjustment parameter set and the real-time coordinate offset data, a clustering algorithm is used to partition the motion range, obtain motion range boundary sequence data for combination with the precise three-dimensional coordinates, calculate a mapping relationship between the optimized dynamic adjustment parameter set and the unmanned vehicle, and generate a mapping relationship data set;

[0081] Based on the mapping relationship data set and the denoised offset data, a linear regression method is used to predict the changing trend of the dynamic adjustment parameters of the robotic arm in the next time period, and a predicted adjustment parameter set is generated as the dynamic adjustment parameter output when the robotic arm is aligned with the fault point.

[0082] Specifically, if the fault is at the junction of cables, the robot dog needs to frequently adjust its posture to avoid obstacles, so there will be a real-time deviation between the fault location data it transmits back and the fault coordinates received by the robotic arm.

[0083] Based on this, the present invention utilizes infrared sensors and depth cameras deployed on a robot dog to obtain real-time coordinate offset data when the robot dog frequently adjusts its posture, and uses an adaptive Kalman filter algorithm to process these offset data to obtain denoised offset data. This data is combined with the operating platform boundary data within the dynamic range of the robotic arm, and a fuzzy control algorithm or other dynamic adjustment algorithm is used to calculate the initial value of the joint angular velocity adjustment of the robotic arm at the current posture adjustment frequency based on the offset, thereby obtaining an initial dynamic adjustment parameter set. The robot dog transmits back offset data (0.1, 0.1, 0) and (0.5, 0.5, 0.2) within 1 second, but (0.5, 0.5, 0.2) may be an outlier caused by vibration. Using a filtering algorithm such as a Kalman filter to process these data, the outlier can be smoothed to (0.2, 0.2, 0.1), forming a denoised offset data set, thereby improving the reliability of the data and providing accurate input for the calculation of dynamic adjustment parameters. When calculating the initial values of the robot arm's dynamic adjustment parameters using the denoised offset data set and the operating platform boundary data, the current posture adjustment frequency must be considered. For example, if the offset is (0.2, 0.2, 0.1) and the operating platform boundary limits the robot arm height to within 2.5 meters, if the frequency is 5 times per second, the initial values can be set to an angle adjustment speed of 10 degrees per second and an extension distance of 0.3 meters per second, forming the initial dynamic adjustment parameter set.

[0084] If the difference between the data in the initial dynamic adjustment parameter set and the preset adjustment threshold exceeds the preset deviation limit, the gradient descent method is used to iteratively optimize the data in the initial dynamic adjustment parameter set to generate an optimized dynamic adjustment parameter set when the robotic arm is aligned with the fault point; otherwise, the initial dynamic adjustment parameter set is output as the optimized dynamic adjustment parameter set.

[0085] The optimized dynamic adjustment parameter set and real-time coordinate offset data collected by the robot dog are used as input to extract features such as the trajectory point density of the manipulator's end point, the rate of change of joint angles, and the variance of the pose offset. The manipulator's range of motion is partitioned using a Gaussian mixture model (GMM), outputting a range boundary sequence to describe the geometric extent of each partition. The manipulator's end coordinates in each partition and the unmanned vehicle's precise three-dimensional coordinates are converted to the same coordinate system. Based on the converted data, a nonlinear regression model is constructed using the random forest regression algorithm (this model takes the optimized dynamic adjustment parameter set and the unmanned vehicle's precise three-dimensional coordinates as input and outputs the manipulator's end coordinates in each partition). This model quantifies the mapping relationship between the optimized dynamic adjustment parameter set and the unmanned vehicle's precise three-dimensional coordinates, generating a mapping relationship dataset. Alternatively, K-means clustering can be used to partition the manipulator's reachable range adjustment region within the operating platform boundary, dividing the offset and boundary data into proximal and distal groups, generating adjustment region boundary sequence data such as (51,51,1) and (53,53,2).

[0086] Taking the denoised offset data and mapping relationship data set as input, the linear regression method is used to predict the changing trend of the dynamic adjustment parameters of the robotic arm in the next time period. For example, the adjustment speed may increase from 10 degrees per second to 12 degrees per second. The predicted adjustment parameter set is obtained as the dynamic adjustment parameter output when the robotic arm is aligned with the fault point. The trajectory of the robotic arm is corrected in advance through trend prediction, which reduces the deviation between the fault location data sent back by the robot dog and the fault coordinates received by the robotic arm. It also improves the response foresight of the robotic arm, reduces the delay of real-time adjustment, and improves overall efficiency.

[0087] The present invention uses real-time filtering and iterative parameter optimization to ensure that the robotic arm can still respond quickly when the robot dog's posture changes frequently, thereby improving alignment accuracy. It combines clustering and regression prediction to effectively suppress the influence of environmental noise (such as vibration and electromagnetic interference) on the adjustment parameters. It establishes a mapping relationship between the robot dog, the robotic arm, and the unmanned vehicle to achieve precise coordination under multi-device motion coupling. It uses trend prediction to correct the robotic arm's trajectory in advance, reduce action delays, and improve emergency repair efficiency.

[0088] In one embodiment, generating an operation instruction for the robotic arm based on the dynamic adjustment parameter includes:

[0089] Based on the dynamic adjustment parameters, an inverse kinematics algorithm is used to calculate a joint angle change sequence of the robotic arm, and the joint angle change sequence is adjusted according to a docking deviation between the precise docking position of the unmanned vehicle and the actual docking position to obtain a joint angle adjustment data set;

[0090] When the joint angle change in the joint angle adjustment data set exceeds the physical limit of the robotic arm, fine-tuning the precise docking position to update the joint angle adjustment data set to obtain a deviation elimination angle change set;

[0091] The deviation elimination angle change set is integrated with the motion range to obtain a joint control sequence after dynamic parameter compensation, which is combined with the precise docking position of the unmanned vehicle after fine-tuning. The operation instruction set is divided into boundary areas by clustering algorithm processing to generate a preliminary instruction set;

[0092] Based on the preliminary instruction set and the joint control sequence, the linear regression method is used to predict the joint angle change trend of the robotic arm in the next time period, and a predicted angle adjustment set is obtained to be combined with the real-time docking deviation of the unmanned vehicle. After processing by a filtering algorithm, a denoised instruction set is obtained as the operation instruction output for the robotic arm.

[0093] Specifically, since the robotic arm relies on the unmanned vehicle platform to provide stable support for inspection and repair, if the position of the unmanned vehicle deviates, the operating range of the robotic arm is limited, making it difficult to perform repair actions on the fault point.

[0094] Based on this, the present invention uses the Jacobian matrix pseudo-inverse method to select the optimal solution by minimizing the weighted norm of the joint angle according to the obtained dynamic adjustment parameters, obtains the joint angle change sequence of the manipulator, and calculates the docking deviation based on the precise docking position of the unmanned vehicle and the actual docking position. The docking deviation is converted into a displacement case in the manipulator base coordinate system to update the joint angle change sequence and output the joint angle adjustment data set. Among them, the core of calculating the manipulator joint angle change sequence is to convert the position requirement of the end effector into the angle adjustment of each joint. Assuming that the end of the manipulator needs to be aligned with the fault point coordinate (52,52,2), and the current docking position of the unmanned vehicle is (50,50,0), inverse kinematics derives the joint angle sequence from the initial posture to the target posture through the known manipulator length and joint degrees of freedom, such as the shoulder joint is adjusted from 30 degrees to 35 degrees, and the elbow joint is changed from 45 degrees to 50 degrees. When obtaining compensation data from the unmanned vehicle's parking deviation, it is assumed that the actual parking position of the unmanned vehicle is offset to (50.2, 50.1, 0.1) due to uneven ground, and the deviation is (0.2, 0.1, 0.1). The compensation data adjusts the initial value of the joint angle based on the direction and magnitude of the deviation, so that the end of the robot arm remains aligned with the fault point, thereby improving adaptability to external disturbances.

[0095] When the joint angle changes in the joint angle adjustment dataset exceed the physical limitations of the robot arm, the robot arm's joint angles are constrained to minimize the amount of docking position adjustment. A sequential quadratic programming method is used to fine-tune the precise docking position and iteratively update the joint angle adjustment dataset. The iteration ends when all joint angles in the updated joint angle adjustment dataset meet the constraints. The resulting updated joint angle adjustment dataset is then output as the deviation-eliminating angle change set. Fine-tuning the unmanned vehicle's position avoids the risk of the robot arm shutting down due to exceeding limits and improves operational continuity.

[0096] According to the range of motion of the robotic arm, speed limits are imposed on the deviation elimination angle change set to obtain the joint control sequence after dynamic parameter compensation. The joint angle, speed, acceleration statistics (mean, variance), etc. are extracted from it, combined with the precise docking position after fine-tuning of the unmanned vehicle as features. The K-means algorithm is used to process these features to divide the boundary area of the operation instruction set. The clustering algorithm divides the instruction subsets according to feature similarity. For example, the joint angle data is divided into the proximal area (angle less than 40 degrees) and the distal area (angle greater than 40 degrees), generating boundary points such as (51,51,1) and (53,53,2), and then outputting a preliminary instruction set.

[0097] The preliminary instruction set and joint control sequence are taken as input, and the linear regression method is used to predict the trend of joint angle changes of the robot arm in the next time period, and the predicted angle adjustment set is output to improve the forward-looking adjustment capability of the robot arm; the predicted angle adjustment set is combined with the real-time docking deviation of the unmanned vehicle and denoised by Kalman filtering, and the smoothed denoised instruction set is output as the operation instruction output for the robot arm.

[0098] The present invention realizes high-precision dynamic compensation and high-precision positioning of the end of the robotic arm through the joint optimization of inverse kinematics and docking deviation; ensures that the joint angle does not exceed the limit through iterative adjustment to avoid mechanical damage or motion failure; improves the operational reliability in complex environments by integrating the position of the unmanned vehicle and the motion range of the robotic arm; combines trend prediction and filtering noise reduction to generate predictive instructions, reduce action delay and jitter, and enhance stability; adopts a multi-faceted collaborative approach, from deviation compensation to instruction mapping, step by step, to jointly support the high-precision and high-stability operation goals, which not only improves the response speed and alignment accuracy of the robotic arm, but also reduces the failure probability caused by environmental changes, and is suitable for complex maintenance scenarios.

[0099] S4. Processing the real-time coordinate offset data using a time series analysis method and a Kalman filter algorithm to predict a real-time position change trend of the robot dog relative to the fault point, and obtaining fault location data synchronized with the operation instruction;

[0100] In one embodiment, step S4 includes:

[0101] The real-time coordinate offset data is processed by a time series analysis method to obtain a real-time offset change trend, which is then processed by a Kalman filter algorithm to output the predicted coordinates of the robot dog relative to the fault point and fuse them with the operation instructions to obtain a synchronized fault location data set;

[0102] Processing the synchronized fault location dataset using a clustering algorithm to obtain a spatial distribution dataset of coordinate offsets, combining the dataset with the real-time coordinate offset data and processing the dataset using a linear regression algorithm to calculate the trend offset of the fault point coordinates, thereby obtaining a trend-adjusted fault coordinate prediction dataset;

[0103] Extracting real-time noise data from the real-time coordinate offset data, and processing the real-time noise data using a Kalman filter algorithm, so as to adjust the fault coordinate prediction data set according to the processing result to obtain a denoised coordinate prediction data set;

[0104] The mapping relationship between the denoised coordinate prediction data set and the operation instruction is quantified to adjust the operation instruction to obtain the adjusted operation instruction, so as to generate a continuous sequence of coordinate predictions of the robot dog relative to the fault point through an interpolation algorithm and serve as fault location data synchronized with the operation instruction.

[0105] Specifically, the present invention adopts the ARIMA (autoregressive integrated moving average) model, takes the real-time coordinate offset sequence of the robot dog as input, predicts and outputs the real-time offset change trend within a preset time period in the future, and processes it through the Kalman filter algorithm. The output is the predicted coordinates of the fault point and is aligned with the timestamp of the operation instruction for fusion to obtain a synchronized fault location data set.

[0106] The predicted coordinates in the synchronized fault location dataset are used as input and the K-means clustering algorithm is used to divide the dataset into proximal and distal regions. Assuming that the center of the proximal region is (52, 52, 2) and the center of the distal region is (53, 53, 2.2) after division, the spatial distribution dataset of the coordinate offset is determined, and the distribution characteristics of the fault points are clarified. The spatial distribution dataset of the coordinate offset is used as the dependent variable, and linear regression modeling is performed with time as the independent variable. The weights are solved by the least squares method, the trend offset of the fault point coordinates is calculated, and the fault coordinate prediction dataset after trend adjustment is output.

[0107] Real-time noise data is extracted from the real-time coordinate offset data and processed using the Kalman filter algorithm. The processed data is then used to correct the fault coordinate prediction dataset, resulting in a denoised coordinate prediction dataset. For example, if the coordinate prediction dataset is (52.18, 52.18, 2.18) and the noise is 0.03 meters, filtering adjusts it to 0.01 meters, generating a denoised fault location dataset such as (52.17, 52.17, 2.17).

[0108] A correlation function is defined between the operation instruction and the denoised coordinate prediction dataset for modeling through a multi-layer perceptron. When the predicted coordinate change of the denoised coordinate prediction dataset is greater than a preset change threshold, the instruction parameters are scaled proportionally to obtain the adjusted operation instruction. A continuous sequence of coordinate predictions of the robot dog relative to the fault point is generated through an interpolation algorithm as the fault location data (mainly the location data of the faulty component in the fault point) synchronized with the operation instruction, which is output as the fault location data.

[0109] The present invention achieves millisecond-level synchronization between the robot dog's position and operating instructions through the dynamic fusion of time series analysis and Kalman filtering, ensuring movement accuracy; combines clustering with noise modeling to effectively suppress the impact of environmental interference (such as vibration and electromagnetic noise) on coordinate prediction; calculates trend offsets based on linear regression to adaptively correct prediction deviations and improve long-term tracking stability; generates smooth prediction sequences through interpolation algorithms to avoid mechanical jitter or mutation risks during instruction execution; and the entire process from coordinate offset acquisition to instruction optimization is progressive, which not only improves the accuracy of fault point positioning, but also enhances the system's adaptability to complex environments and reduces the risk of misoperation due to noise or deviations.

[0110] S5. Generate captured data based on the operation instruction and the fault location data to isolate the faulty component from the fault area and perform emergency repairs, thereby implementing a patrol inspection of the power distribution facilities;

[0111] In one embodiment, step S5 includes:

[0112] Based on the fault location data, a closed-loop control system of the robotic arm is controlled to generate initial gripping angle and force data of the end effector, and environmental data of the faulty component is obtained according to the fault location data to perform regional division using a clustering algorithm to obtain a faulty component region;

[0113] An interpolation algorithm is used to process the initial grasping angle and force data to generate a continuous end-effector motion trajectory to construct an implementation action sequence data set;

[0114] Acquire real-time environmental data of the fault component area and perform denoising on the data using a Kalman filter algorithm, so as to adjust the implementation action sequence data set according to the denoising result to obtain a denoised repair action data set;

[0115] Extracting dynamic environmental change features from the real-time environmental data to process the fault location data through an interpolation algorithm, generating a continuous fault component separation path to adjust the denoised repair action data set, and obtaining a final execution action data set as captured data;

[0116] The robotic arm is controlled according to the captured data to separate the faulty component from the faulty area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

[0117] Specifically, when generating grasping data, the core lies in converting position information into executable mechanical actions. The present invention uses fault location data and the geometric parameters (size, material) of the faulty component as input data. The closed-loop control system that controls the robotic arm uses inverse kinematics to solve the initial gripper angle based on the faulty component's position data and the end-of-arm posture. The initial gripping angle and force data for the robotic arm's end effector are obtained by presetting the gripping force based on the material hardness (e.g., rubber / metal). The system also obtains environmental data (such as cable distribution and temperature field around the faulty component) within the location area based on the faulty component's position data. OPTICS clustering is then used to divide the area into safe operating zones (low temperature, low density) and risk zones (high temperature, high density), obtaining the faulty component area and its boundary data.

[0118] The discrete grasping angle sequence in the initial grasping angle and force data is converted into a time-angle curve, and cubic spline interpolation is used to ensure the continuity of angle, angular velocity, and angular acceleration to avoid mechanical shock. The clamping force data is linearly interpolated to ensure that the force change rate is within the motor response range, and a continuous end-effector motion trajectory is generated to construct an implementation action sequence dataset.

[0119] Real-time environmental data from the faulty component area is acquired and denoised using a Kalman filter algorithm. This filtered data is then used to adjust the action sequence dataset, avoiding risk areas and outputting a denoised repair action dataset. When acquiring real-time interference data from a complex environment, it is assumed that wind or vibration can cause a position offset of 0.02 meters. Using the Kalman filter algorithm to address environmental noise, the offset is smoothed from 0.02 meters to 0.01 meters by fusing predicted and observed values, resulting in a denoised repair action dataset.

[0120] Dynamic environmental change characteristics are extracted from real-time environmental data, such as spatial characteristics: cable movement speed, fault component displacement vector, temporal characteristics: temperature change rate, vibration spectrum main frequency, etc., and multidimensional feature vectors are generated through PCA dimensionality reduction to characterize the dynamic changes of the environment. Based on the multidimensional feature vectors, a potential field model is constructed around the fault component, with attraction pointing in the separation direction and repulsion coming from high-risk areas. B-spline curves are used to fit path points to ensure curvature continuity and smooth movement of the robotic arm joints, generating a continuous separation path for the fault component. The dynamic time warping method is used to align the separation path with the action sequence timeline, compensate for the robotic arm delay, and adjust the denoised repair action dataset to obtain the final execution action dataset as the grasping data, which contains time-synchronized trajectory and force instructions. Finally, according to the grasping data, the robotic arm is controlled to separate the faulty component from the fault area and perform emergency repairs, thereby realizing inspection of the distribution facilities.

[0121] The present invention achieves high-precision grasping and separation of faulty components through closed-loop control and dynamic path planning, avoiding secondary damage to surrounding cables; real-time environmental perception and data denoising ensure stable operation in complex electromagnetic interference and vibration environments; based on interpolation and feature extraction, a smooth separation path is generated to reduce robot arm jitter and improve movement continuity; the full process automation from positioning to separation can significantly shorten the fault recovery time of distribution facilities.

[0122] In the embodiment of the present application, based on the problem of how to improve the inspection efficiency and safety of distribution facilities in high-density cable areas, a distribution facility inspection method that cooperates with an unmanned vehicle, a robot dog, and a robotic arm is designed. The method determines the precise parking position of the unmanned vehicle based on the three-dimensional cable distribution data collected by the unmanned vehicle in the high-density cable fault area; the occlusion information data collected by the robot dog when it enters the narrow area under the cable in the fault area is processed by a convolutional neural network to eliminate the interference of cable occlusion to accurately identify the three-dimensional coordinates of the faulty component and determine the motion range of the robotic arm that relies on the stable support provided by the unmanned vehicle platform; the coordinate offset data (i.e., the coordinate offset data transmitted by the robot dog when it needs to frequently adjust its posture to avoid obstacles) is used. The system combines the fault location data returned by the robot dog with the fault coordinates received by the robotic arm (the motion range of the robotic arm is used to quantify the dynamic adjustment parameters of the robotic arm when aligning with the fault point, and then generates operating instructions for the robotic arm; the coordinate offset data returned by the robot dog is processed using time series analysis and filtering algorithms to predict the accurate position of the robot dog relative to the fault point and synchronized with the operating instructions when the unmanned vehicle needs to temporarily adjust its position to adapt to the needs of the robotic arm. The position is combined with the operating instructions to control the robotic arm to generate a grasping plan to separate and repair the faulty components in the fault area, thereby realizing effective inspection of the power distribution facilities. This solution can realize automated and precise repairs in complex cable environments, improving operational efficiency and safety.

[0123] It should be noted that although the steps in the above flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.

[0124] In another embodiment, Figure 2 As shown, the second aspect of the present invention provides a power distribution facility inspection system coordinated by an unmanned vehicle, a robot dog, and a robotic arm, comprising:

[0125] A position determination module 10 is configured to determine a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of the power distribution facility, and to quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point;

[0126] a range quantification module 20 for processing the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and quantifying the motion range of the robot arm based on the relative relationship between the precise docking position and the precise three-dimensional coordinates;

[0127] an instruction generation module 30 for acquiring real-time coordinate offset data when the robot dog adjusts its posture, quantifying dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generating an operation instruction for the robot arm based on the dynamic adjustment parameters;

[0128] a position synchronization module 40 for processing the real-time coordinate offset data using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point and obtain fault position data synchronized with the operation instruction;

[0129] The fault repair module 50 is used to generate captured data based on the operation instruction and the fault location data, so as to separate the faulty component from the fault area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

[0130] It should be noted that each module in the above-mentioned distribution facility inspection system in which an unmanned vehicle, a robot dog and a robotic arm cooperate can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. For the specific definition of a distribution facility inspection system in which an unmanned vehicle, a robot dog and a robotic arm cooperate, please refer to the definition of a distribution facility inspection method in which an unmanned vehicle, a robot dog and a robotic arm cooperate. The two have the same functions and effects and will not be repeated here.

[0131] In summary, the present invention relates to the field of information technology, and discloses a distribution facility inspection method and system that cooperates with an unmanned vehicle, a robot dog, and a robotic arm. The unmanned vehicle is used to obtain three-dimensional distribution data of cables in a fault area, and through path planning and dynamic adjustment of the chassis height, the unmanned vehicle can be accurately docked in a confined space; the robot dog is used to detect blocked areas to extract the characteristics of faulty components and determine their precise coordinates; the reachable range of the robotic arm carried on the unmanned vehicle is quantified, and then the joint angle changes of the robotic arm are calculated, and the operation instructions are determined after the unmanned vehicle fine-tunes the position to eliminate deviations; based on the predicted real-time position change trend of the robot dog relative to the fault point, the fault position data synchronized with the operation instructions is determined, so that the robotic arm can generate a grasping plan and accurately implement emergency repair actions; automated and precise emergency repairs are achieved in complex cable environments, improving work efficiency and safety.

[0132] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the various technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0133] The above-described embodiments merely represent several preferred implementations of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art could make several improvements and substitutions without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be based on the scope of protection of the claims.

Claims

1. A power distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm, characterized in that: include: Determining a preliminary parking area for the unmanned vehicle based on three-dimensional cable distribution data corresponding to a fault area of the power distribution facility, and quantifying the movement trajectory of the unmanned vehicle in the fault area based on the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point; The occlusion information data of the fault area collected by the robot dog is processed by a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, so as to quantify the motion range of the robot arm together with the precise docking position; Acquire real-time coordinate offset data when the robot dog adjusts its posture, so as to quantify dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generate operation instructions for the robot arm based on the dynamic adjustment parameters; The real-time coordinate offset data is processed using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point, thereby obtaining fault location data synchronized with the operation instruction; Generating captured data based on the operation instruction and the fault location data to isolate the faulty component from the faulty area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities; The process of processing the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point includes: Based on the precise docking position, an infrared sensor and a depth camera deployed on the robot dog are used to obtain occlusion information data of the occlusion area below the cable, and a multimodal data set is obtained to be input into a convolutional neural network for processing, and a contour feature data set is output; Determining the three-dimensional coordinate value of the faulty component according to the contour feature data set and parameter information of the depth camera; Based on the three-dimensional coordinate values and the shielded area below the cable, a clustering algorithm is used to divide the regional boundary of the faulty component to obtain a boundary division coordinate sequence to determine the thermal characteristic distribution matrix of the faulty component according to the multimodal data set; Based on the contour feature data set, a matching thermal feature distribution matrix is extracted from the thermal feature distribution matrix. When it is determined that there is a position deviation between the matching thermal feature distribution matrix and the three-dimensional coordinate value, the matching thermal feature distribution matrix is adjusted by a gradient descent algorithm to obtain a thermal feature distribution adjustment matrix to combine with the three-dimensional coordinate value to determine the rough three-dimensional coordinates of the fault point.

2. The power distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm according to claim 1, characterized in that: The determining of a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of the power distribution facility includes: The multi-view laser radar carried by the unmanned vehicle collects three-dimensional distribution data of cables corresponding to a fault area of the power distribution facility; the fault area is a ground space-constrained environment area including a high-density cable interlaced area of the power distribution facility; The three-dimensional distribution data of the cables is processed using a point cloud distance calculation algorithm to obtain the height data and position coordinates of each cable in the fault area, so as to analyze the density distribution law of each cable in the fault area and obtain cable density distribution characteristic data; Dividing the fault area based on the cable density distribution characteristic data, and marking areas where the cable height is lower than a preset height threshold as candidate parking areas for the unmanned vehicle, to obtain a set of candidate areas; Extracting target candidate areas that match the height requirement of the unmanned vehicle from the candidate area set, and using a geometric constraint algorithm to determine the spatial connectivity of each target candidate area to obtain a connected area range; According to the range of the connected area and the scanning boundary conditions of the multi-view laser radar, the coordinates of the docking area are determined so as to optimize and group the target candidate areas within the range of the connected area using a machine learning clustering algorithm to generate a preliminary docking area for the unmanned vehicle.

3. The power distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm according to claim 2, characterized in that: quantifying the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point includes: quantifying the movement trajectory of the unmanned vehicle under the spatially restricted conditions corresponding to the fault area based on the preliminary docking area, the cable density distribution characteristic data, and the height data of each cable, to obtain an initial trajectory coordinate sequence; When it is determined that there is a deviation between the initial trajectory coordinate sequence and the fault area, an adjustment parameter for the chassis height of the unmanned vehicle is generated by a model predictive control method to obtain a trajectory adjustment coordinate sequence; Based on the trajectory adjustment coordinate sequence and the height data of each cable, the cable height change characteristics of the fault point and its preset area are extracted to determine the area division range of the fault point through a clustering algorithm; According to the area division range and the spatial restriction condition, a target trajectory adjustment coordinate sequence that meets the connectivity requirements is selected from the trajectory adjustment coordinate sequence and combined to generate a continuous movement trajectory path to be combined with the cable density distribution characteristic data to determine the precise parking position of the unmanned vehicle relative to the fault point.

4. The power distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm according to claim 1, characterized in that: The method further includes processing the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point: Based on the rough three-dimensional coordinates, extracting the occlusion depth data collected by the depth camera from the multimodal data set, performing denoising processing using an image denoising algorithm to obtain denoised depth data, and processing the denoised depth data using a convolutional neural network to extract edge features of the faulty component under cable interference to obtain an edge feature dataset; Determining whether the integrity of the edge feature dataset is lower than a preset integrity threshold, and when the integrity of the edge feature dataset is lower than the preset integrity threshold, optimizing the denoised depth data through an iterative optimization algorithm to obtain optimized depth data; Determining initial three-dimensional coordinates of the faulty component according to the optimized depth data and the rough three-dimensional coordinates, and processing the edge feature dataset and the optimized depth data using an interpolation algorithm based on the initial three-dimensional coordinates to obtain a continuous three-dimensional coordinate path; Based on the continuous three-dimensional coordinate path, the spatial distribution characteristics of the faulty component are extracted from the obscured area below the cable, and a distribution feature data set is obtained to divide the regional boundaries of the faulty component using a clustering algorithm to obtain the precise three-dimensional coordinates of the fault point and its corresponding boundary sequence data.

5. The power distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm according to claim 1, characterized in that: The acquiring of real-time coordinate offset data when the robot dog adjusts its posture, and quantifying the dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, includes: Acquiring real-time coordinate offset data of the robot dog when it frequently adjusts its posture and processing it through a filtering algorithm to obtain denoised offset data, which is then combined with the operating platform boundary data within the motion range to calculate initial values of dynamic adjustment parameters of the robot arm at the current posture adjustment frequency, thereby obtaining an initial dynamic adjustment parameter set; When a deviation between the initial dynamic adjustment parameter set and a preset adjustment threshold exceeds a preset deviation limit range, iteratively optimizing the initial dynamic adjustment parameter set using a gradient descent method to generate an optimized dynamic adjustment parameter set when the robotic arm is aligned with the fault point; Based on the optimized dynamic adjustment parameter set and the real-time coordinate offset data, a clustering algorithm is used to partition the motion range, obtain motion range boundary sequence data for combination with the precise three-dimensional coordinates, calculate a mapping relationship between the optimized dynamic adjustment parameter set and the unmanned vehicle, and generate a mapping relationship data set; Based on the mapping relationship data set and the denoised offset data, a linear regression method is used to predict the changing trend of the dynamic adjustment parameters of the robotic arm in the next time period, and a predicted adjustment parameter set is generated as the dynamic adjustment parameter output when the robotic arm is aligned with the fault point.

6. The power distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm according to claim 1, characterized in that: The generating of an operation instruction for the robotic arm based on the dynamic adjustment parameter includes: Based on the dynamic adjustment parameters, an inverse kinematics algorithm is used to calculate a joint angle change sequence of the robotic arm, and the joint angle change sequence is adjusted according to a docking deviation between the precise docking position of the unmanned vehicle and the actual docking position to obtain a joint angle adjustment data set; When the joint angle change in the joint angle adjustment data set exceeds the physical limit of the robotic arm, fine-tuning the precise docking position to update the joint angle adjustment data set to obtain a deviation elimination angle change set; The deviation elimination angle change set is integrated with the motion range to obtain a joint control sequence after dynamic parameter compensation, which is combined with the precise docking position of the unmanned vehicle after fine-tuning. The operation instruction set is divided into boundary areas by clustering algorithm processing to generate a preliminary instruction set; Based on the preliminary instruction set and the joint control sequence, the linear regression method is used to predict the joint angle change trend of the robotic arm in the next time period, and a predicted angle adjustment set is obtained to be combined with the real-time docking deviation of the unmanned vehicle. After processing by a filtering algorithm, a denoised instruction set is obtained as the operation instruction output for the robotic arm.

7. The power distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm according to claim 1, characterized in that: The method of processing the real-time coordinate offset data using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point and obtain fault location data synchronized with the operation instruction includes: The real-time coordinate offset data is processed by a time series analysis method to obtain a real-time offset change trend, which is then processed by a Kalman filter algorithm to output the predicted coordinates of the robot dog relative to the fault point and fuse them with the operation instructions to obtain a synchronized fault location data set; Processing the synchronized fault location dataset using a clustering algorithm to obtain a spatial distribution dataset of coordinate offsets, combining the dataset with the real-time coordinate offset data and processing the dataset using a linear regression algorithm to calculate the trend offset of the fault point coordinates, thereby obtaining a trend-adjusted fault coordinate prediction dataset; Extracting real-time noise data from the real-time coordinate offset data, and processing the real-time noise data using a Kalman filter algorithm, so as to adjust the fault coordinate prediction data set according to the processing result to obtain a denoised coordinate prediction data set; The mapping relationship between the denoised coordinate prediction data set and the operation instruction is quantified to adjust the operation instruction to obtain the adjusted operation instruction, so as to generate a continuous sequence of coordinate predictions of the robot dog relative to the fault point through an interpolation algorithm and serve as fault location data synchronized with the operation instruction.

8. The power distribution facility inspection method using a collaborative unmanned vehicle, a robot dog, and a robotic arm according to claim 1, characterized in that: The generating of captured data based on the operation instruction and the fault location data to separate the faulty component from the faulty area and perform emergency repairs to realize inspection of the power distribution facilities includes: Based on the fault location data, a closed-loop control system of the robotic arm is controlled to generate initial gripping angle and force data of the end effector, and environmental data of the faulty component is obtained according to the fault location data to perform regional division using a clustering algorithm to obtain a faulty component region; An interpolation algorithm is used to process the initial grasping angle and force data to generate a continuous end-effector motion trajectory to construct an implementation action sequence data set; Acquire real-time environmental data of the fault component area and perform denoising on the data using a Kalman filter algorithm, so as to adjust the implementation action sequence data set according to the denoising result to obtain a denoised repair action data set; Extracting dynamic environmental change features from the real-time environmental data to process the fault location data through an interpolation algorithm, generating a continuous fault component separation path to adjust the denoised repair action data set, and obtaining a final execution action data set as captured data; The robotic arm is controlled according to the captured data to separate the faulty component from the faulty area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

9. A power distribution facility inspection system that uses an unmanned vehicle, a robot dog, and a robotic arm to coordinate, characterized in that: include: a position determination module, configured to determine a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point; a range quantification module, configured to process the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and quantify the motion range of the robot arm based on the relative relationship between the precise docking position and the precise three-dimensional coordinates; an instruction generation module, configured to obtain real-time coordinate offset data when the robot dog adjusts its posture, quantify dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generate an operation instruction for the robot arm based on the dynamic adjustment parameters; a position synchronization module, configured to process the real-time coordinate offset data using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point and obtain fault position data synchronized with the operation instruction; A fault repair module, configured to generate captured data based on the operation instruction and the fault location data, so as to isolate the faulty component from the fault area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities; The process of processing the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point includes: Based on the precise docking position, an infrared sensor and a depth camera deployed on the robot dog are used to obtain occlusion information data of the occlusion area below the cable, and a multimodal data set is obtained to be input into a convolutional neural network for processing, and a contour feature data set is output; Determining the three-dimensional coordinate value of the faulty component according to the contour feature data set and parameter information of the depth camera; Based on the three-dimensional coordinate values and the shielded area below the cable, a clustering algorithm is used to divide the regional boundary of the faulty component to obtain a boundary division coordinate sequence to determine the thermal characteristic distribution matrix of the faulty component according to the multimodal data set; Based on the contour feature data set, a matching thermal feature distribution matrix is extracted from the thermal feature distribution matrix. When it is determined that there is a position deviation between the matching thermal feature distribution matrix and the three-dimensional coordinate value, the matching thermal feature distribution matrix is adjusted by a gradient descent algorithm to obtain a thermal feature distribution adjustment matrix to combine with the three-dimensional coordinate value to determine the rough three-dimensional coordinates of the fault point.

Images