Power distribution facility inspection method and system based on cooperation of unmanned vehicle, robot dog and mechanical arm

Abstract

The invention relates to the technical field of information, and discloses an unmanned vehicle, robot dog and mechanical arm coordinated power distribution facility inspection method and system, the unmanned vehicle is utilized to obtain three-dimensional distribution data of a cable in a fault area, and accurate parking of the unmanned vehicle in a limited space is realized through path planning and dynamic adjustment of the height of a chassis; utilizing a robot dog to detect the shielding area so as to extract fault component features and determine accurate coordinates of the fault component features; the reachable range of the mechanical arm carried on the unmanned vehicle is quantified, then the joint angle change of the mechanical arm is calculated, and an operation instruction is determined after the position is finely adjusted through the unmanned vehicle to eliminate deviation; based on the predicted real-time position change trend of the robot dog relative to the fault point, fault position data synchronous with the operation instruction is determined, and a grabbing scheme is generated through the mechanical arm to accurately implement the emergency repair action; automatic and accurate first-aid repair in a complex cable environment is realized, and the operation efficiency and safety are improved.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a distribution facility inspection method and system using an unmanned vehicle, a robot dog and a mechanical arm to coordinate. Background Art

[0002] Distribution facility inspection is the core link of power system operation and maintenance. In the existing technology, the inspection of distribution facilities, especially the inspection of distribution facilities in complex environments such as high-density cable areas, is usually carried out by staff. However, traditional manual inspection has problems such as low efficiency, high risk and incomplete coverage.

[0003] It can be seen that how to improve the inspection efficiency and safety of power distribution facilities in high-density cable areas has become a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0004] The present invention provides a distribution facility inspection method and system in which an unmanned vehicle, a robot dog and a mechanical arm cooperate, so as to solve the problem of how to improve the inspection efficiency and safety of distribution facilities in high-density cable areas.

[0005] In order to solve the above technical problems, the first aspect of the present invention provides a distribution facility inspection method coordinated by an unmanned vehicle, a robot dog and a mechanical arm, comprising: Determine a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of ​​the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point; The occlusion information data of the fault area collected by the robot dog is processed by a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and the motion range of the robot arm is quantified based on the relative relationship between the precise docking position and the precise three-dimensional coordinates; Acquire real-time coordinate offset data when the robot dog adjusts its posture, so as to quantify dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generate operation instructions for the robot arm based on the dynamic adjustment parameters; The real-time coordinate offset data is processed by using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point, and obtain fault position data synchronized with the operation instruction; Capture data is generated based on the operation instruction and the fault location data to separate the faulty component from the fault area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

[0006] A second aspect of the present invention provides a power distribution facility inspection system coordinated by an unmanned vehicle, a robot dog and a mechanical arm, comprising: A position determination module, used to determine a preliminary parking area of ​​the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of ​​the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point; A range quantification module is used to process the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and quantify the motion range of the robot arm based on the relative relationship between the precise docking position and the precise three-dimensional coordinates; An instruction generation module, used for acquiring real-time coordinate offset data when the robot dog adjusts its posture, so as to quantify the dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and to generate an operation instruction for the robot arm based on the dynamic adjustment parameters; A position synchronization module, used to process the real-time coordinate offset data using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point and obtain fault position data synchronized with the operation instruction; A fault repair module is used to generate captured data based on the operation instruction and the fault location data, so as to separate the faulty component from the fault area and perform emergency repairs, thereby realizing inspection of the power distribution facilities.

[0007] Compared with the prior art, the embodiments of the present invention have the following advantages: (1) By processing the three-dimensional distribution data of the cables, the precise parking area of ​​the unmanned vehicle in the complex fault area is determined; in an obstructed environment, the robot dog extracts the contour features of the faulty component through a convolutional neural network, accurately locates the three-dimensional coordinates of the fault point, and dynamically adjusts its posture to avoid obstacles, reducing inspection risks; (2) Through the collaborative operation of unmanned vehicles, robot dogs and robotic arms, seamless collaboration is achieved to perform coordinate offset correction, fault location synchronization, etc., shortening the overall operation time and adapting to different terrains and complex environments. Efficient, accurate, intelligent and automated inspection and repair of fault areas of distribution facilities is achieved, which not only improves the efficiency of repairs, reduces labor costs and risks, but also ensures the safe and stable operation of distribution facilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the implementation mode will be briefly introduced below. Obviously, the drawings described below are only some implementation modes of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 It is a flow chart of a distribution facility inspection method using an unmanned vehicle, a robot dog and a robotic arm in collaboration, provided by a certain embodiment of the present invention; Figure 2 It is a structural diagram of a power distribution facility inspection system that is coordinated by an unmanned vehicle, a robot dog, and a robotic arm, provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0010] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings and embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0011] In the description of this application, the terms "first", "second", "third", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the feature. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0012] In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two components. The terms "vertical", "horizontal", "left", "right", "upper", "lower" and similar expressions used herein are for illustrative purposes only, and do not indicate or imply that the system or component referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0013] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art. The terms used in the specification of the present invention are only for describing specific embodiments and are not intended to limit the present invention. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0014] In one embodiment, if Figure 1 As shown, the first aspect of the present invention provides a distribution facility inspection method coordinated by an unmanned vehicle, a robot dog and a mechanical arm, comprising: S1. Determine a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of ​​the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point; In one embodiment, determining the preliminary parking area of ​​the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of ​​the power distribution facilities includes: The three-dimensional distribution data of cables in the fault area of ​​the power distribution facility corresponding to the power distribution facility is collected by a multi-view laser radar carried by the unmanned vehicle; the fault area is a ground space-constrained environment area including a high-density cable interlaced area of ​​the power distribution facility; The three-dimensional distribution data of the cables are processed by using a point cloud distance calculation algorithm to obtain the height data and position coordinates of each cable in the fault area, so as to analyze the density distribution law of each cable in the fault area and obtain the cable density distribution characteristic data; Dividing the fault area based on the cable density distribution characteristic data, and marking the area where the cable height is lower than a preset height threshold as the candidate parking area for the unmanned vehicle, to obtain a candidate area set; Extracting target candidate regions that match the height requirement of the unmanned vehicle from the candidate region set, and using a geometric constraint algorithm to determine the spatial connectivity of each of the target candidate regions to obtain a connected region range; According to the range of the connected area and the scanning boundary conditions of the multi-view laser radar, the coordinates of the docking area are determined so as to optimize and group the target candidate areas within the range of the connected area using a machine learning clustering algorithm to generate a preliminary docking area for the unmanned vehicle.

[0015] Specifically, since the unmanned vehicle needs to move on the ground to carry tools and spare parts during the inspection process, but the ground space in the high-density cable area is limited and the cables are hung at different heights, it is difficult for the unmanned vehicle to accurately park near the fault point.

[0016] Based on this, the present invention uses the multi-view laser radar carried by the unmanned vehicle to scan the fault area corresponding to the distribution facility (that is, the high-density cable interlacing area) to obtain the three-dimensional distribution data of the cables from the limited ground space environment of the high-density cable interlacing area; such as in an old industrial park, the cables are crisscrossed at different heights and the ground space is limited. The multi-view laser radar installed on the unmanned vehicle collects data from multiple directions such as the side and top at a scanning frequency of 200,000 points per second, captures the reflected signal of the cable, and forms a three-dimensional point cloud distribution data of the cable to determine the preliminary spatial distribution information of the cable in the fault area, so as to clearly display the three-dimensional layout of the cables. For example, there are 5 cables in a certain area, which are located at heights of 1.5 meters, 2.0 meters, 3.0 meters, etc., respectively, laying the foundation for subsequent analysis.

[0017] A point cloud distance calculation algorithm is used to extract cable features. Based on the three-dimensional cable distribution data collected by the unmanned vehicle, the vertical distance between each cable and the ground can be calculated through the point cloud data to obtain the height data, diameter data and position coordinates of each cable in the fault area. For example, in the aforementioned industrial park scenario, the algorithm analyzes the point cloud in real time and determines that the diameter of a cable with a height of 1.5 meters is about 10 centimeters, and the diameter of a cable with a height of 2.0 meters is 15 centimeters. At the same time, the precise coordinates of each cable are determined, such as (x1, y1, z1) is (10, 20, 1.5), to distinguish the height differences of cables, and to provide data support for space occupancy analysis; the fault area is divided into several grids (such as 1m×1m) to count the number of cables and the average height in each grid, and then kernel density estimation is used to generate a two-dimensional density heat map to identify the density distribution law of each cable in the fault area and obtain the cable density distribution characteristic data. Furthermore, assuming the scanning area is 50 meters × 50 meters, the analysis found that cables with a height of less than 2.0 meters are concentrated in the northwest corner of the area, while cables with a height of more than 3.0 meters are mostly distributed in the southeast corner. It can be seen that low-height cables have a more obvious restriction on ground space; for example, under the low cables in the northwest corner, the space height is only 1.5 meters, which restricts the passage of large equipment, while the space in the southeast corner is relatively open. These analysis results can provide a basis for subsequent area division and help optimize the parking planning of unmanned vehicles.

[0018] The fault area is divided according to the cable density distribution characteristic data. If the cable height is lower than the preset height threshold, the area below it is marked as a candidate area for the unmanned vehicle to park, and the areas with an area smaller than the minimum turning radius of the unmanned vehicle and the areas where the cable height does not meet the standard are eliminated. A 0.3m buffer zone is then expanded outward to ensure that the unmanned vehicle maintains a safe distance from the cable when parking, so as to generate a set of candidate areas; among which, the preset height threshold can be 2.5 meters. The division of the fault area utilizes the height distribution characteristics to screen out potential available parking space, thereby improving space utilization.

[0019] Target candidate areas that match the height requirements of the unmanned vehicle (that is, areas that allow the unmanned vehicle to pass smoothly) are extracted from the candidate area set, and a geometric constraint algorithm is used to construct a topological graph model with candidate areas as nodes and reachable paths between areas as edges. The path width must be ≥ the width of the unmanned vehicle. The Dijkstra algorithm is used to search for the longest continuous path and mark the connected subgraph. If the connected area is blocked by temporary obstacles (such as maintenance equipment), the topological graph is updated and recalculated through real-time point cloud until the spatial connectivity of all target candidate areas is determined, and the range of connected areas that the unmanned vehicle can pass is generated, thereby ensuring the practicality of the parking area and avoiding spatial fragmentation.

[0020] Based on the scanning boundary conditions of the multi-view laser radar carried by the unmanned vehicle, the initial docking area coordinates and docking coordinates of the unmanned vehicle can be determined from the connected area range to extract the area, shape index (aspect ratio), Euclidean distance from the fault point and other features of the candidate area. The DBSCAN algorithm is used to merge adjacent areas with similar attributes within the connected area with the unmanned vehicle scanning boundary conditions as constraints to obtain the initial docking area of ​​the unmanned vehicle. The attributes here refer to the multi-dimensional features of the candidate area, which are used to quantify the geometric characteristics (area, shape index), spatial position (Euclidean distance from the fault point must meet the safety radius constraint) and environmental constraints (minimum turning radius limit) of the area. If the cable height in a sub-area suddenly drops to 2.0 meters, the clustering operation can identify and adjust the boundary in advance. In addition, the docking area can be adjusted in combination with real-time environmental changes. For example, if a cable with a height of 2.2 meters is added to the industrial park, the algorithm can dynamically update the candidate area set after the laser radar is rescanned to eliminate the affected part to ensure the adaptability of the solution. The entire process from scanning to optimization realizes the efficient use of spatial resources, provides reliable support for the docking of unmanned vehicles in complex environments, and reduces the cost of manual intervention.

[0021] The present invention uses multi-view laser radar and density analysis to accurately identify safe stopping points in high-density cable crossing areas to avoid collision risks; combines geometric constraints with machine learning clustering to efficiently divide feasible areas in restricted ground environments and improve path planning efficiency; screens stopping areas based on preset height thresholds to ensure a safe distance between unmanned vehicles and cables and reduce electromagnetic interference risks; and reduces manual intervention through point cloud processing and clustering algorithms to achieve automated and intelligent stopping area selection.

[0022] In one embodiment, quantifying the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point includes: Based on the preliminary parking area, the cable density distribution characteristic data and the height data of each cable, quantify the movement trajectory of the unmanned vehicle under the spatially restricted conditions corresponding to the fault area to obtain an initial trajectory coordinate sequence; When it is determined that there is a deviation between the initial trajectory coordinate sequence and the fault area, an adjustment parameter for the chassis height of the unmanned vehicle is generated by a model predictive control method to obtain a trajectory adjustment coordinate sequence; Based on the trajectory adjustment coordinate sequence and the height data of each of the cables, the height variation characteristics of the cables within the fault point and its preset area are extracted to determine the area division range of the fault point through a clustering algorithm; According to the area division range and the space restriction condition, a target trajectory adjustment coordinate sequence that meets the connectivity requirements is selected from the trajectory adjustment coordinate sequence and combined to generate a continuous moving trajectory path to be combined with the cable density distribution characteristic data to determine the precise parking position of the unmanned vehicle relative to the fault point.

[0023] Specifically, the present invention takes the coordinates of the preliminary docking area and the cable density distribution characteristic data of the area and the height data of the cables contained in the area as input, and adopts a fast exploration random tree algorithm to convert the cable height data into a three-dimensional obstacle model, limits the moving height of the unmanned vehicle (such as the chassis height must be ≥ cable height + 0.3m), outputs the moving trajectory of the unmanned vehicle under space-constrained conditions, and obtains an initial trajectory coordinate sequence (including position, speed, attitude angle, etc.). Other path planning algorithms may also be used. It should be noted that the present invention only adopts these algorithms and does not improve them, so the specific steps of the algorithm are not described in detail. For example, in an old industrial park, the unmanned vehicle needs to move from the starting point (0,0,0) to the fault point (40,40,0). Combined with the cable height data obtained by the aforementioned lidar scanning, for example, the cable height in the northwest corner is 1.5 meters and the cable height in the southeast corner is 3.0 meters. The path planning algorithm can analyze the spatial limitations and avoid areas with a height lower than 1.2 meters from the chassis of the unmanned vehicle. The algorithm prioritizes the path under the cable at a height of 3.0 meters in the southeast corner to generate an initial trajectory coordinate sequence, such as (0,0,0), (10,10,0), (20,20,0), (30,30,0), and (40,40,0). By making full use of the height distribution pattern of the cable, the trajectory is initially feasible.

[0024] The deviation detection algorithm is used to analyze the matching degree between the mobile trajectory and the actual environment, that is, through the fusion positioning of LiDAR SLAM and IMU, the trajectory tracking error (lateral error, heading angle error) is calculated to compare with the preset deviation threshold, and when the deviation error is greater than the preset deviation threshold, it is determined that there is a deviation between the initial trajectory coordinate sequence and the fault area. The kinematic model of the unmanned vehicle is established by the model predictive control method, with the goal of minimizing the tracking error and the chassis height adjustment amplitude (to prevent frequent lifting), and the chassis height adjustment amount and the trajectory coordinate offset compensation amount are output as the trajectory adjustment coordinate sequence; if the trajectory tracking error is not greater than the preset deviation threshold, the original chassis height of the unmanned vehicle is also used. The environmental adaptability of the unmanned vehicle is improved through real-time adjustment, avoiding failures caused by static planning.

[0025] The three-dimensional point cloud distribution data of the cable collected by the unmanned vehicle is updated based on the trajectory adjustment coordinate sequence, and the cable height change characteristics (reflecting the degree of fluctuation) and height gradient (such as the height difference between adjacent points>0.5m is a mutation point) within the fault point and its preset area (such as a radius of 2m) are extracted according to the height of each cable. The extracted cable height change characteristics are processed based on the density-based OPTICS clustering algorithm to group the fault point and its preset area, and the cable height mutation area is marked as a potential fault sub-area (such as a break point, a loose joint), and the regional division range of the fault point is obtained.

[0026] Based on the spatial constraints of the area division range and the fault area, it is determined whether the trajectory adjustment coordinate sequence meets the connectivity requirements. The cable density is not greater than the preset density threshold and the cable height is not greater than the preset height threshold. The target trajectory adjustment coordinate sequence that meets the connectivity requirements is selected from the trajectory adjustment coordinate sequence and combined to map the combined result to a graph node. It is checked whether there is a reachable path between the nodes, and cubic spline interpolation is used to fill the trajectory breakpoints to ensure the continuity of the path. The interpolation step size is reduced (such as 0.1m) in the cable dense area and increased (such as 0.5m) in the sparse area to generate a continuous moving trajectory path and its coordinate set. The connectivity verification avoids the spatial fragmentation problem and ensures the integrity of the path.

[0027] Calculate the Euclidean distance between each point in the continuous moving trajectory and the fault point, and take the weighted sum of the minimization of the total path length and the maximum distance difference (weight ratio 6:4) as the objective function, use the gradient descent method or genetic algorithm to optimize the moving trajectory points whose distance difference exceeds the preset difference threshold, and output the optimized moving trajectory path; and the moving trajectory points whose distance difference does not exceed the preset difference threshold indicate that their distance from the fault point meets the requirements and can still be used. Based on the optimized moving trajectory path and cable density distribution feature data, on the basis that the parking position needs to be ≥0.5m away from the nearest cable (to prevent electromagnetic interference and mechanical collision), select the candidate area with the lowest density (density <2 cables / m²) as the parking point, and align the center of the unmanned vehicle chassis with the coordinate system of the fault point. Fine-tune with the aid of visual markers (ArUco code) to obtain the precise parking position of the unmanned vehicle relative to the fault point and output it. For example, at (41,41,0), the cable height is 3.0 meters, and the spatial connectivity range is 5 meters × 4 meters, which meets the 3 meters × 2 meters requirement of the unmanned vehicle. The final docking coordinates are then determined to be (41,41,0) to (43,43,0). The precise positioning of the unmanned vehicle fully utilizes spatial resources and reduces the impact of deviations.

[0028] The present invention combines spatial constraints with dynamic adjustments to perform high-precision trajectory planning to ensure the safe movement of unmanned vehicles in cable-dense areas; uses model predictive control to correct trajectories in real time to cope with environmental interference; avoids path breakage or unreachable problems through connectivity verification and trajectory optimization; uses clustering and interpolation algorithms to quickly fill trajectory gaps and reduce calculation time, while achieving efficient movement and docking of unmanned vehicles in complex environments, significantly improving the level of automation and safety.

[0029] S2. Processing the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and quantifying the motion range of the robot arm based on the relative relationship between the precise docking position and the precise three-dimensional coordinates; In one embodiment, the occlusion information data of the fault area collected by the robot dog is processed by a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, including: Based on the precise docking position, an infrared sensor and a depth camera deployed on the robot dog are used to obtain occlusion information data of the occlusion area below the cable, and a multimodal data set is obtained to be input into a convolutional neural network for processing, and a contour feature data set is output; Determining the three-dimensional coordinate value of the faulty component according to the contour feature data set and parameter information of the depth camera; Based on the three-dimensional coordinate values ​​and the shielded area below the cable, a clustering algorithm is used to divide the regional boundary of the faulty component to obtain a boundary division coordinate sequence to determine the thermal characteristic distribution matrix of the faulty component according to the multimodal data set; Based on the contour feature data set, a matching thermal feature distribution matrix is ​​extracted from the thermal feature distribution matrix, and when it is determined that there is a position deviation between the matching thermal feature distribution matrix and the three-dimensional coordinate value, the matching thermal feature distribution matrix is ​​adjusted through a gradient descent algorithm to obtain a thermal feature distribution adjustment matrix to combine with the three-dimensional coordinate value to determine the rough three-dimensional coordinates of the fault point.

[0030] Specifically, since the robot dog needs to enter the narrow area under the cable to locate faults during the inspection process, its visual sensor is easily interfered by the dense cables, making it difficult to accurately identify the three-dimensional coordinates of the faulty components.

[0031] Based on this, the present invention uses the infrared sensor and depth camera deployed on the robot dog to obtain the occlusion information data of the occlusion area under the cable according to the precise parking position of the unmanned vehicle, such as abnormal thermal features such as overheating of the cable joint, 3D point cloud of the occlusion area, etc., and aligns the timestamps of the infrared and depth data through hardware trigger signals to form a multimodal data set containing temperature and three-dimensional position to improve the comprehensiveness of the information and avoid missing key features from a single sensor; an improved Mask R-CNN is adopted, whose backbone network is ResNet-50-FPN, and an infrared channel input branch is added. The improved Mask R-CNN is trained with the historical multimodal data set and its fault labels as training data, and the multimodal data set collected by the robot dog based on the precise parking position of the unmanned vehicle is input into the trained improved Mask R-CNN, and the binary mask (initial contour feature) of the faulty component is output to obtain a contour feature data set. For multimodal data sets, separating the initial contour features of the faulty component from the cable obstruction interference can also be achieved through image processing technology: cable-dense areas may obstruct the faulty component, and the high-temperature areas in the infrared data are combined with the mutation points in the depth data to preliminarily outline the contour of the faulty component.

[0032] The infrared image and the depth point cloud are aligned through the external parameter matrix to generate a 3D point cloud with a temperature label. 100 points are randomly sampled in the contour mask output by the improved Mask R-CNN, and the mean of their 3D coordinates is calculated as the initial value of the 3D coordinates. Then, taking minimizing the projection error of the contour points (from the depth camera coordinate system to the image plane) as the objective function, the least squares method is used to iteratively adjust the translation vector and rotation matrix of the faulty component until convergence, and the optimized 3D coordinate value of the faulty component is obtained as the 3D coordinate value of the faulty component.

[0033] The blocked area under the cable, the point cloud data of the area, and the optimized three-dimensional coordinates are taken as input, and the density-based DBSCAN clustering is used with parameter settings (eps=0.05m, min_samples=5) to process these data. The faulty component is divided according to its regional boundary, and the boundary division coordinate sequence of the faulty component (such as the high-temperature zone boundary coordinate set) is output. Based on the clustering results, the thermal feature distribution data of the faulty component collected by the infrared sensor is extracted from the multimodal data set to generate a thermal feature distribution matrix.

[0034] The contour mask output by Mask R-CNN is mapped to the thermal feature distribution matrix for contour-temperature alignment. If the distance between the contour center and the high temperature area center is greater than the preset center distance, it is determined that the match fails, and the thermal feature distribution matrix parameters are optimized through the optimization algorithm. Otherwise, it is determined that the match is successful, and the thermal feature distribution matrix that passes the match and carries the contour mask (the coordinate data can be calculated based on this) is used as the matching thermal feature distribution matrix to calculate the position deviation between it and the three-dimensional coordinate optimization value. When the position deviation exceeds the preset position deviation threshold, it is determined that there is a position deviation between the matching thermal feature distribution matrix and the three-dimensional coordinate value. The contour position is translated along the temperature gradient direction through the gradient descent algorithm, with an iteration step of 0.01 meters and a maximum of 50 iterations to adjust the matching thermal feature distribution matrix to obtain the thermal feature distribution adjustment matrix. When the position deviation does not exceed the preset position deviation threshold, the original thermal feature distribution matrix is ​​output as the thermal feature distribution adjustment matrix.

[0035] The thermal feature distribution adjustment matrix and the three-dimensional coordinate optimization value (0.4:0.46) are weighted combined to determine the rough three-dimensional coordinates of the fault point. The present invention uses multimodal data fusion of infrared and depth cameras to penetrate cable shielding to accurately locate the faulty component; combines CNN contour extraction with least squares optimization for high-precision three-dimensional reconstruction to improve coordinate accuracy; jointly optimizes thermal features and geometric contours to improve robustness under complex environmental interference. Cluster-based boundary division can reduce reliance on manual annotation and adapt to variable fault forms.

[0036] In one embodiment, the processing of the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the accurate three-dimensional coordinates of the fault point also includes: Based on the rough three-dimensional coordinates, extract the occlusion depth data collected by the depth camera from the multimodal data set, perform denoising processing using an image denoising algorithm to obtain denoised depth data, and process the denoised depth data using a convolutional neural network to extract edge features of the faulty component under cable interference to obtain an edge feature data set; Determining whether the integrity of the edge feature data set is lower than a preset integrity threshold, and when the integrity of the edge feature data set is lower than the preset integrity threshold, optimizing the denoised depth data through an iterative optimization algorithm to obtain optimized depth data; Determine the initial three-dimensional coordinates of the faulty component according to the optimized depth data and the rough three-dimensional coordinates, and process the edge feature data set and the optimized depth data using an interpolation algorithm based on the initial three-dimensional coordinates to obtain a continuous three-dimensional coordinate path; Based on the continuous three-dimensional coordinate path, the spatial distribution characteristics of the faulty component are extracted from the shielded area below the cable, and a distribution characteristic data set is obtained to divide the regional boundaries of the faulty component using a clustering algorithm to obtain the precise three-dimensional coordinates of the fault point and its corresponding boundary sequence data.

[0037] Specifically, when obtaining the occluded area data collected by the robot dog depth camera through rough coordinates, there are usually complex interferences under the cable, such as dense cables, light changes or dust effects, etc. The present invention extracts the occluded depth data collected by the depth camera from the multimodal data set corresponding to the rough three-dimensional coordinates, filters it through the non-local mean denoising algorithm (search window 21×21, similarity window 5×5), and uses the conditional generative adversarial network to fill in the depth missing pixels (generator is U-Net, discriminator is PatchGAN), removes the noise such as depth value jump caused by cable reflection and electromagnetic interference in the occluded depth data, obtains the denoised depth data and uses it as input, uses a multi-scale edge detection network for processing, extracts the edge features of the faulty component under cable interference, and outputs the edge probability map to generate an edge feature data set. Among them, the depth of a certain point is shown as 2.5 meters, but the surrounding points are all 2.2 meters. This mutation may be caused by the reflection of the obstruction. When using the image denoising algorithm to process these data, you can use mean filtering to remove isolated noise points. For example, smooth 2.5 meters to around 2.2 meters to form a denoised depth data set to improve data smoothness.

[0038] Completeness is defined as total length of closed edge / theoretical maximum closed edge length (component size estimated based on rough three-dimensional coordinates) to evaluate the integrity of the edge feature data set. If it is not less than the preset complete threshold, the denoised depth data is output as the optimized depth data; otherwise, the denoised depth data is optimized using the particle swarm optimization algorithm to maximize the area and continuity of the edge closed area as the fitness function to generate optimized depth data. Furthermore, for the edge feature data set, when the integrity is judged to be lower than the preset threshold, for example, the threshold is set to 80%, if the edge point only covers 50% of the contour of the faulty component, the denoised depth data set is adjusted through the iterative optimization algorithm. Assuming that the initial depth data misses some areas, the iterative optimization can supplement the missing data by comparing the edge points and depth values ​​multiple times, for example, the depth near (42.1, 42, 2.2) is adjusted from an invalid value to 2.18 meters to obtain the optimized depth data set.

[0039] The optimized depth map and the rough three-dimensional coordinates (from infrared-depth fusion) are registered with the ICP algorithm to align the coordinate system, and Harris corner points are extracted from the edge map, back-projected into three-dimensional space, and the center of mass is calculated as the initial three-dimensional coordinates of the faulty component; based on the initial three-dimensional coordinates, B-spline interpolation is used to fit the edge feature data set and the discrete edge points in the optimized depth data into a smooth curve, and the density of control points is increased in areas with large curvature (such as corners) to obtain a continuous three-dimensional coordinate path formed by a continuous path point set.

[0040] Based on the position of each point set in the continuous three-dimensional coordinate path, the spatial distribution characteristics of the faulty component, such as geometric characteristics, depth characteristics, etc., are extracted from the obstructed area below the cable and combined to generate a distribution feature set; the distribution feature set is processed using a spectral clustering algorithm, and a similarity matrix is ​​constructed based on the Euclidean distance of the feature vector to divide the regional boundaries of the faulty component, and obtain the precise three-dimensional coordinates of the fault point and its corresponding boundary sequence data, such as a polygon vertex coordinate set, to describe the shape of the faulty component. In addition, when extracting the spatial distribution characteristics of the faulty component from the obstructed area based on the continuous three-dimensional coordinate path data, the height distribution of the path points can be analyzed. For example, the points on the west side are concentrated at 2.2 meters and the points on the east side are 2.5 meters to generate a distribution feature data set. Furthermore, if it is found that the depth of a certain area is abnormally concentrated, it may indicate the shape characteristics of the faulty component, such as a circle or a strip.

[0041] Through multi-stage denoising and optimization, the present invention effectively eliminates the influence of cable obstruction and electromagnetic noise on depth data, improves the integrity of edge features, and enhances anti-interference ability; combines interpolation and clustering algorithms for high-precision 3D reconstruction to achieve accurate coordinate positioning of faulty components, breaking through the accuracy limitations of traditional single sensors; iterative optimization based on integrity criteria ensures that reliable 3D models can still be generated in extreme obstruction scenarios; combines spatial distribution features with clustering for adaptive boundary division to adapt to the diversity of fault morphologies in complex cable interlaced environments.

[0042] For the range of motion of the robot arm, that is, the boundary data of the reachable range of the robot arm, the present invention adopts the Euclidean distance and direction vector between the precise parking position of the unmanned vehicle and the precise three-dimensional coordinates of the fault point to determine it; wherein, the Euclidean distance is used to quantify the straight-line distance between the unmanned vehicle and the fault point to guide the planning of the extension length of the robot arm, the direction vector is used to determine the spatial direction that the robot arm needs to cover to guide the adjustment of the joint angle, and the maximum extension range of the robot arm is calculated according to the length of each link of the robot arm and the joint angle limit as a length constraint, and then combined with the laser radar point cloud data to identify obstacles in the fault area to plan the obstacle avoidance path of the robot arm; the DH parameter method is used to establish the kinematic model of the robot arm to define the conversion relationship between each joint coordinate system, and the forward kinematics calculation method is used to calculate the position of the end effector of the robot arm according to the joint angle to verify whether the fault point is covered; finally, the Euclidean distance is used as the cost function, the A* algorithm is used to plan the shortest path of the robot arm to cover the fault point, and the gradient projection method is used to adjust the Cartesian path to ensure that the robot arm can still complete the task when there is a joint failure or an obstacle, thereby obtaining the range of motion of the robot arm.

[0043] S3, obtaining real-time coordinate offset data when the robot dog adjusts its posture, so as to quantify the dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generating an operation instruction for the robot arm based on the dynamic adjustment parameters; In one embodiment, the real-time coordinate offset data of the robot dog when adjusting its posture is obtained to quantify the dynamic adjustment parameters of the robot arm when aligning with the fault point in combination with the motion range includes: The real-time coordinate offset data of the robot dog when the robot dog frequently adjusts its posture is obtained and processed by a filtering algorithm to obtain denoised offset data to be combined with the boundary data of the operating platform in the motion range, and the initial value of the dynamic adjustment parameter of the robot arm at the current posture adjustment frequency is calculated to obtain an initial dynamic adjustment parameter set; When the deviation between the initial dynamic adjustment parameter set and the preset adjustment threshold exceeds the preset deviation limit range, the initial dynamic adjustment parameter set is iteratively optimized using a gradient descent method to generate an optimized dynamic adjustment parameter set when the mechanical arm is aligned with the fault point; Based on the optimized dynamic adjustment parameter set and the real-time coordinate offset data, a clustering algorithm is used to partition the motion range, and motion range boundary sequence data is obtained to be combined with the precise three-dimensional coordinates, and a mapping relationship between the optimized dynamic adjustment parameter set and the unmanned vehicle is calculated to generate a mapping relationship data set; According to the mapping relationship data set and the denoised offset data, a linear regression method is used to predict the change trend of the dynamic adjustment parameters of the robotic arm in the next time period, and a predicted adjustment parameter set is generated as the dynamic adjustment parameter output when the robotic arm is aligned with the fault point.

[0044] Specifically, if the fault is at the junction of the cables, the robot dog needs to frequently adjust its posture to avoid obstacles, so there will be a real-time deviation between the fault location data it transmits back and the fault coordinates received by the robotic arm.

[0045] Based on this, the present invention uses infrared sensors and depth cameras deployed on the robot dog to obtain real-time coordinate offset data when the robot dog frequently adjusts its posture, and uses an adaptive Kalman filter algorithm to process these offset data to obtain denoised offset data, which is combined with the operating platform boundary data in the dynamic range of the robot arm, and the initial value of the joint angular velocity adjustment of the robot arm at the current posture adjustment frequency is calculated according to the offset through a fuzzy control algorithm or other dynamic adjustment algorithms to obtain an initial dynamic adjustment parameter set. Among them, the robot dog returns offset data (0.1, 0.1, 0) and (0.5, 0.5, 0.2) within 1 second, but (0.5, 0.5, 0.2) may be an abnormal value caused by vibration. Using a filtering algorithm such as a Kalman filter to process these data, the abnormal value can be smoothly adjusted to (0.2, 0.2, 0.1), forming a denoised offset data set, so as to improve the reliability of the data and provide accurate input for the calculation of dynamic adjustment parameters. When calculating the initial values ​​of the dynamic adjustment parameters of the robot arm through the denoised offset data set and the operating platform boundary data, the current posture adjustment frequency must be considered. For example, the offset is (0.2, 0.2, 0.1), and the operating platform boundary limits the height of the robot arm to 2.5 meters. If the frequency is 5 times per second, the initial value can be set to an angle adjustment speed of 10 degrees per second and an extension distance of 0.3 meters per second to form an initial dynamic adjustment parameter set.

[0046] If the difference between the data in the initial dynamic adjustment parameter set and the preset adjustment threshold exceeds the preset deviation limit range, the gradient descent method is used to iteratively optimize the data in the initial dynamic adjustment parameter set to generate an optimized dynamic adjustment parameter set when the robot arm is aligned with the fault point; otherwise, the initial dynamic adjustment parameter set is output as the optimized dynamic adjustment parameter set.

[0047] The optimized dynamic adjustment parameter set and the real-time coordinate offset data collected by the robot dog are used as input to extract the features such as the trajectory point density, joint angle change rate, and posture offset variance of the end of the robot arm. The motion range of the robot arm is partitioned by GMM (Gaussian mixture model), and the motion range boundary sequence data is output to describe the geometric range of each partition. The end coordinates of the robot arm in each partition and the precise three-dimensional coordinates of the unmanned vehicle are converted to the same coordinate system, and a nonlinear regression model is constructed based on the converted data using the random forest regression algorithm (the model takes the optimized dynamic adjustment parameter set and the precise three-dimensional coordinates of the unmanned vehicle as input and the end coordinates of the robot arm in each partition as output) to quantify the mapping relationship between the optimized dynamic adjustment parameter set and the precise three-dimensional coordinates of the unmanned vehicle, and obtain a mapping relationship data set. In addition, K-means clustering can also be used to divide the reachable range adjustment area of ​​the robot arm within the boundary of the operating platform to divide the offset and boundary data into two groups, the proximal and distal, and generate adjustment area boundary sequence data such as (51,51,1) and (53,53,2).

[0048] With denoised offset data and mapping relationship data sets as input, the linear regression method is used to predict the changing trend of the dynamic adjustment parameters of the robotic arm in the next time period. For example, the adjustment speed may increase from 10 degrees per second to 12 degrees per second, and the predicted adjustment parameter set is obtained as the dynamic adjustment parameter output when the robotic arm is aligned with the fault point. The robotic arm trajectory is corrected in advance through trend prediction, which reduces the deviation between the fault location data sent back by the robot dog and the fault coordinates received by the robotic arm, improves the response foresight of the robotic arm, reduces the delay in real-time adjustment, and improves overall efficiency.

[0049] The present invention ensures that the robotic arm can respond quickly when the robot dog's posture changes frequently through real-time filtering and iterative parameter optimization, thereby improving the alignment accuracy. It combines clustering and regression prediction to effectively suppress the influence of environmental noise (such as vibration and electromagnetic interference) on the adjustment parameters. It establishes a mapping relationship between the robot dog, the robotic arm, and the unmanned vehicle to achieve precise coordination under multi-device motion coupling. It corrects the trajectory of the robotic arm in advance through trend prediction to reduce action delays and improve emergency repair efficiency.

[0050] In one embodiment, the generating an operation instruction for the robotic arm based on the dynamic adjustment parameter includes: Based on the dynamic adjustment parameters, an inverse kinematics algorithm is used to calculate a joint angle change sequence of the robotic arm, and the joint angle change sequence is adjusted according to a parking deviation between the precise parking position of the unmanned vehicle and the actual parking position to obtain a joint angle adjustment data set; When the joint angle change in the joint angle adjustment data set exceeds the physical limit of the robot arm, fine-tuning the precise docking position to update the joint angle adjustment data set to obtain a deviation elimination angle change set; The deviation elimination angle change set is integrated with the motion range to obtain a joint control sequence after dynamic parameter compensation, which is combined with the precise parking position after fine-tuning of the unmanned vehicle, and processed by a clustering algorithm to divide the boundary area of ​​the operation instruction set to generate a preliminary instruction set; Based on the preliminary instruction set and the joint control sequence, the linear regression method is used to predict the joint angle change trend of the robotic arm in the next time period, and a predicted angle adjustment set is obtained to be combined with the real-time docking deviation of the unmanned vehicle, and processed by a filtering algorithm to obtain a denoised instruction set as the operation instruction output for the robotic arm.

[0051] Specifically, since the robotic arm relies on the unmanned vehicle platform to provide stable support for inspection and repair, if the position of the unmanned vehicle deviates, the operating range of the robotic arm is limited, and it is difficult to perform repair actions on the fault point.

[0052] Based on this, the present invention uses the Jacobian matrix pseudo-inverse method to select the optimal solution by minimizing the weighted norm of the joint angle according to the obtained dynamic adjustment parameters, obtains the joint angle change sequence of the robot arm, and calculates the docking deviation according to the precise docking position of the unmanned vehicle and the actual docking position, converts the docking deviation into a displacement case in the robot arm base coordinate system to update the joint angle change sequence, and outputs the joint angle adjustment data set. Among them, the core of calculating the robot arm joint angle change sequence is to convert the position requirement of the end effector into the angle adjustment of each joint. Assuming that the end of the robot arm needs to be aligned with the coordinates of the fault point (52,52,2), and the current docking position of the unmanned vehicle is (50,50,0), inverse kinematics derives the joint angle sequence from the initial posture to the target posture through the known length of the robot arm and the degree of freedom of the joints, such as the shoulder joint is adjusted from 30 degrees to 35 degrees, and the elbow joint is changed from 45 degrees to 50 degrees. When obtaining compensation data from the unmanned vehicle's parking deviation, it is assumed that the actual parking position of the unmanned vehicle is offset to (50.2, 50.1, 0.1) due to uneven ground, and the deviation is (0.2, 0.1, 0.1). The compensation data adjusts the initial value of the joint angle through the deviation direction and magnitude, so that the end of the robotic arm remains aligned with the fault point to improve adaptability to external disturbances.

[0053] When the joint angle change in the joint angle adjustment data set exceeds the physical limit of the robot arm, the joint angle of the robot arm can be constrained with the goal of minimizing the adjustment amount of the docking position. The sequential quadratic programming method is used to fine-tune the precise docking position to iteratively update the joint angle adjustment data set until all joint angles in the updated joint angle adjustment data set meet the constraints, and the iteration is terminated. The final updated joint angle adjustment data set is output as the deviation-eliminating angle change set. Among them, the fine-tuning of the position of the unmanned vehicle avoids the risk of downtime of the robot arm due to over-limit and improves the continuity of operation.

[0054] According to the motion range of the robot arm, speed limiting is applied to the deviation elimination angle change set to obtain the joint control sequence after dynamic parameter compensation. The joint angle, speed, acceleration statistics (mean, variance), etc. are extracted from it and combined with the precise parking position after fine-tuning of the unmanned vehicle as features. The K-means algorithm is used to process these features to divide the boundary area of ​​the operation instruction set. The clustering algorithm divides the instruction subsets according to the feature similarity. For example, the joint angle data is divided into the proximal area (angle less than 40 degrees) and the distal area (angle greater than 40 degrees), generating boundary points such as (51,51,1) and (53,53,2), and then outputting the preliminary instruction set.

[0055] The preliminary instruction set and joint control sequence are taken as input, and the linear regression method is used to predict the change trend of the joint angle of the robot arm in the next period of time, and the predicted angle adjustment set is output to improve the forward-looking adjustment ability of the robot arm; the predicted angle adjustment set is combined with the real-time docking deviation of the unmanned vehicle and denoised by Kalman filtering, and the smoothed denoised instruction set is output as the operation instruction output for the robot arm.

[0056] The present invention realizes high-precision dynamic compensation and high-precision positioning of the end of the robot arm through the joint optimization of inverse kinematics and docking deviation; ensures that the joint angle does not exceed the limit through iterative adjustment to avoid mechanical damage or motion failure; improves the operational reliability in complex environments by integrating the position of the unmanned vehicle and the motion range of the robot arm; generates predictive instructions by combining trend prediction and filtering noise reduction to reduce motion delay and jitter and enhance stability; adopts a multi-faceted collaborative approach, from deviation compensation to instruction mapping, step by step, to jointly support high-precision and high-stability operation goals, which not only improves the response speed and alignment accuracy of the robot arm, but also reduces the failure probability caused by environmental changes, and is suitable for complex maintenance scenarios.

[0057] S4, using a time series analysis method and a Kalman filter algorithm to process the real-time coordinate offset data to predict the real-time position change trend of the robot dog relative to the fault point, and obtain fault position data synchronized with the operation instruction; In one embodiment, step S4 includes: The real-time coordinate offset data is processed by a time series analysis method to obtain a real-time offset change trend and processed by a Kalman filter algorithm, and the predicted coordinates of the fault point of the robot dog relative to the fault point are output and merged with the operation instruction to obtain a synchronous fault position data set; The synchronous fault location data set is processed by a clustering algorithm to obtain a spatial distribution data set of coordinate offsets, which is combined with the real-time coordinate offset data and processed by a linear regression algorithm to calculate the trend offset of the fault point coordinates, thereby obtaining a fault coordinate prediction data set after trend adjustment; Extracting real-time noise data from the real-time coordinate offset data, and processing the real-time noise data using a Kalman filter algorithm, so as to adjust the fault coordinate prediction data set according to the processing result to obtain a denoised coordinate prediction data set; The mapping relationship between the denoised coordinate prediction data set and the operation instruction is quantified to adjust the operation instruction to obtain the adjusted operation instruction, so as to generate a continuous sequence of coordinate predictions of the robot dog relative to the fault point through an interpolation algorithm and serve as fault location data synchronized with the operation instruction.

[0058] Specifically, the present invention adopts the ARIMA (autoregressive integrated moving average) model, takes the real-time coordinate offset sequence of the robot dog as input, predicts and outputs the real-time offset change trend within a preset time period in the future and processes it through the Kalman filter algorithm, outputs the predicted coordinates of the fault point and aligns them with the timestamp of the operation instruction for fusion, and obtains a synchronized fault location data set.

[0059] The predicted coordinates in the synchronized fault location dataset are used as input and the K-means clustering algorithm is used to divide the fault location into proximal and distal regions. Assuming that the center of the proximal region is (52, 52, 2) and the center of the distal region is (53, 53, 2.2) after division, the spatial distribution dataset of the coordinate offset is determined, the distribution characteristics of the fault point are clarified, and the spatial distribution dataset of the coordinate offset is used as the dependent variable. Linear regression modeling is performed with time as the independent variable. The weights are solved by the least squares method, the trend offset of the fault point coordinates is calculated, and the fault coordinate prediction dataset after trend adjustment is output.

[0060] Real-time noise data is extracted from the real-time coordinate offset data, and the real-time noise data is processed using the Kalman filter algorithm to correct the fault coordinate prediction data set through the processing results to obtain a denoised coordinate prediction data set. Assuming that the coordinate prediction data set is (52.18, 52.18, 2.18), the noise data is 0.03 meters, and after filtering, it is adjusted to 0.01 meters to generate a denoised fault location data set, such as (52.17, 52.17, 2.17).

[0061] The correlation function between the operation instruction and the denoised coordinate prediction data set is defined to be modeled through a multi-layer perceptron, and when the predicted coordinate change of the denoised coordinate prediction data set is greater than a preset change threshold, the instruction parameters are scaled proportionally to obtain the adjusted operation instruction and generate a continuous sequence of coordinate predictions of the robot dog relative to the fault point through an interpolation algorithm as the fault location data (mainly the location data of the faulty component in the fault point) output synchronized with the operation instruction.

[0062] The present invention realizes millisecond-level synchronization between the robot dog's position and operating instructions through the dynamic fusion of time series analysis and Kalman filtering, ensuring the accuracy of the action; combines clustering and noise modeling to effectively suppress the influence of environmental interference (such as vibration, electromagnetic noise) on coordinate prediction; calculates trend offset based on linear regression, adaptively corrects prediction deviation, and improves long-term tracking stability; generates a smooth prediction sequence through an interpolation algorithm to avoid the risk of mechanical jitter or mutation in instruction execution; the whole process from coordinate offset acquisition to instruction optimization is progressive, which not only improves the accuracy of fault point positioning, but also enhances the system's adaptability to complex environments and reduces the risk of misoperation caused by noise or deviation.

[0063] S5. Generate captured data based on the operation instruction and the fault location data to separate the faulty component from the faulty area and perform emergency repairs to implement inspection of the power distribution facilities; In one embodiment, step S5 includes: Based on the fault location data, a closed-loop control system of the robot arm is controlled to generate initial gripping angle and force data of the end effector, and environmental data of the faulty component is acquired according to the fault location data to perform regional division using a clustering algorithm to obtain a faulty component region; An interpolation algorithm is used to process the initial grasping angle and force data to generate a continuous end-effector motion trajectory to construct an implementation action sequence data set; Acquire real-time environmental data of the fault component area and perform denoising through a Kalman filter algorithm, so as to adjust the implementation action sequence data set according to the denoising result to obtain a denoised repair action data set; Extracting dynamic environmental change features from the real-time environmental data, processing the fault location data through an interpolation algorithm, generating a continuous fault component separation path to adjust the denoised repair action data set, and obtaining a final execution action data set as captured data; The robot arm is controlled according to the captured data to separate the faulty component from the faulty area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

[0064] Specifically, when generating the grasping data, the core lies in converting the position information into executable mechanical actions. The present invention uses the fault position data and the geometric parameters (size, material) of the faulty component as input data, and controls the closed-loop control system of the robotic arm. Based on the position data of the faulty component and the end posture of the robotic arm, the initial gripping angle is solved by inverse kinematics, and the clamping force is preset according to the material hardness (such as rubber / metal), so as to obtain the initial gripping angle and force data of the end effector of the robotic arm; according to the position data of the faulty component, the environmental data of the faulty component in the position area (cable distribution and temperature field around the faulty component, etc.) are obtained, and OPTICS clustering is used to divide the safe operation area (low temperature, low density) and the risk area (high temperature, high density) to obtain the faulty component area and its boundary data.

[0065] The discrete grasping angle sequence in the initial grasping angle and force data is converted into a time-angle curve, and cubic spline interpolation is used to ensure the continuity of angle, angular velocity, and angular acceleration to avoid mechanical shock. The clamping force data is linearly interpolated to ensure that the force change rate is within the motor response range, and a continuous end-effector motion trajectory is generated to construct an implementation action sequence data set.

[0066] The real-time environmental data of the faulty component area is obtained and denoised by the Kalman filter algorithm. The filtered environmental data is used to adjust the action sequence data set, avoid the risk area, and output the denoised repair action data set. When obtaining real-time interference data from a complex environment, it is assumed that wind or vibration causes a position offset of 0.02 meters. When the Kalman filter algorithm is used to process environmental noise, the offset is smoothed from 0.02 meters to 0.01 meters by fusing the predicted value and the observed value, and the denoised repair action data set is obtained.

[0067] Dynamic environmental change characteristics are extracted from real-time environmental data, such as spatial characteristics: cable movement speed, fault component displacement vector, time characteristics: temperature change rate, vibration spectrum main frequency, etc., and multi-dimensional feature vectors are generated through PCA dimensionality reduction to characterize the dynamic changes of the environment; based on the multi-dimensional feature vectors, a potential field model is constructed around the faulty component, with the attraction pointing to the separation direction and the repulsion coming from the high-risk area. The B-spline curve is used to fit the path points to ensure the continuity of the curvature and the smooth movement of the robot arm joints, and a continuous separation path for the faulty component is generated. The dynamic time warping method is used to align the separation path with the action sequence time axis, compensate for the robot arm delay, and adjust the denoised repair action data set to obtain the final execution action data set as the grasping data, which contains the time-synchronized trajectory and force instructions; finally, according to the grasping data, the robot arm is controlled to separate the faulty component from the faulty area and perform repairs, thereby realizing the inspection of the distribution facilities.

[0068] The present invention realizes high-precision grasping and separation of faulty components through closed-loop control and dynamic path planning, avoiding secondary damage to surrounding cables; real-time environmental perception and data denoising ensure stable operation under complex electromagnetic interference and vibration environments; generates a smooth separation path based on interpolation and feature extraction, reduces robot arm jitter and improves action continuity; and automates the entire process from positioning to separation, which can significantly shorten the fault recovery time of distribution facilities.

[0069] In the embodiment of the present application, based on the problem of how to improve the inspection efficiency and safety of distribution facilities in high-density cable areas, a distribution facility inspection method that cooperates with an unmanned vehicle, a robot dog, and a mechanical arm is designed. The method determines the precise parking position based on the three-dimensional cable distribution data collected by the unmanned vehicle in the high-density cable fault area; the occlusion information data collected by the robot dog when it enters the narrow area below the cable in the fault area is processed by a convolutional neural network to eliminate the interference of cable occlusion to accurately identify the three-dimensional coordinates of the faulty component, and determine the motion range of the mechanical arm that relies on the stable support provided by the unmanned vehicle platform; the coordinate offset data (i.e., the coordinate offset data transmitted by the robot dog when it needs to frequently adjust its posture to avoid obstacles) is used. The fault location data returned by the robot dog and the fault coordinates received by the robotic arm are combined with the motion range of the robotic arm to quantify the dynamic adjustment parameters when the robotic arm is aligned with the fault point, and then generate operation instructions for the robotic arm; the coordinate offset data returned by the robot dog is processed by time series analysis and filtering algorithm to predict the accurate position of the robot dog relative to the fault point and synchronized with the operation instruction time when the unmanned vehicle needs to temporarily adjust its position to adapt to the needs of the robotic arm, and combine it with the operation instruction to control the robotic arm to generate a grasping plan to separate and repair the faulty parts in the fault area, thereby realizing effective inspection of the distribution facilities. This solution can realize automated and accurate repairs in complex cable environments, improving work efficiency and safety.

[0070] It should be noted that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.

[0071] In another embodiment, if Figure 2 As shown, the second aspect of the present invention provides a distribution facility inspection system coordinated by an unmanned vehicle, a robot dog and a mechanical arm, comprising: A position determination module 10 is used to determine a preliminary parking area of ​​the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of ​​the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point; The range quantification module 20 is used to process the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and quantify the motion range of the robot arm based on the relative relationship between the precise parking position and the precise three-dimensional coordinates; The instruction generation module 30 is used to obtain the real-time coordinate offset data when the robot dog adjusts its posture, so as to quantify the dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and to generate an operation instruction for the robot arm based on the dynamic adjustment parameters; The position synchronization module 40 is used to process the real-time coordinate offset data by using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point and obtain fault position data synchronized with the operation instruction; The fault repair module 50 is used to generate captured data based on the operation instruction and the fault location data, so as to separate the faulty component from the fault area and perform emergency repairs, thereby realizing inspection of the power distribution facilities.

[0072] It should be noted that each module in the above-mentioned distribution facility inspection system in which an unmanned vehicle, a robot dog and a mechanical arm cooperate can be fully or partially implemented through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. For the specific definition of a distribution facility inspection system in which an unmanned vehicle, a robot dog and a mechanical arm cooperate, please refer to the above definition of a distribution facility inspection method in which an unmanned vehicle, a robot dog and a mechanical arm cooperate. The two have the same functions and effects, which will not be repeated here.

[0073] In summary, the present invention relates to the field of information technology, and discloses a distribution facility inspection method and system in which an unmanned vehicle, a robot dog and a mechanical arm cooperate. The unmanned vehicle is used to obtain three-dimensional distribution data of cables in a fault area, and the unmanned vehicle is accurately docked in a confined space through path planning and dynamic adjustment of the chassis height; the robot dog is used to detect the blocked area to extract the characteristics of the faulty component and determine its precise coordinates; the reachable range of the mechanical arm carried by the unmanned vehicle is quantified, and then the change in the joint angle of the mechanical arm is calculated, and the operation instruction is determined after the unmanned vehicle fine-tunes the position to eliminate the deviation; based on the predicted real-time position change trend of the robot dog relative to the fault point, the fault position data synchronized with the operation instruction is determined, so that the mechanical arm generates a grasping plan and accurately implements the emergency repair action; automated and precise emergency repairs are achieved in complex cable environments, improving work efficiency and safety.

[0074] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A distribution facility inspection method using unmanned vehicles, robot dogs and robotic arms, characterized in that: include: Determine a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of ​​the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point; The occlusion information data of the fault area collected by the robot dog is processed by a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, so as to quantify the motion range of the robot arm together with the precise docking position; Acquire real-time coordinate offset data when the robot dog adjusts its posture, so as to quantify dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and generate operation instructions for the robot arm based on the dynamic adjustment parameters; The real-time coordinate offset data is processed by using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point, and obtain fault position data synchronized with the operation instruction; Capture data is generated based on the operation instruction and the fault location data to separate the faulty component from the fault area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

2. The power distribution facility inspection method using unmanned vehicles, robot dogs and robotic arms in collaboration according to claim 1 is characterized in that: The method of determining a preliminary parking area for the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of ​​the power distribution facilities includes: The three-dimensional distribution data of cables in the fault area of ​​the power distribution facility corresponding to the power distribution facility is collected by a multi-view laser radar carried by the unmanned vehicle; the fault area is a ground space-constrained environment area including a high-density cable interlaced area of ​​the power distribution facility; The three-dimensional distribution data of the cables are processed by using a point cloud distance calculation algorithm to obtain the height data and position coordinates of each cable in the fault area, so as to analyze the density distribution law of each cable in the fault area and obtain the cable density distribution characteristic data; Dividing the fault area based on the cable density distribution characteristic data, and marking the area where the cable height is lower than a preset height threshold as the candidate parking area for the unmanned vehicle, to obtain a candidate area set; Extracting target candidate regions that match the height requirement of the unmanned vehicle from the candidate region set, and using a geometric constraint algorithm to determine the spatial connectivity of each of the target candidate regions to obtain a connected region range; According to the range of the connected area and the scanning boundary conditions of the multi-view laser radar, the coordinates of the docking area are determined so as to optimize and group the target candidate areas within the range of the connected area using a machine learning clustering algorithm to generate a preliminary docking area for the unmanned vehicle.

3. The power distribution facility inspection method using unmanned vehicles, robot dogs and mechanical arms in collaboration according to claim 2 is characterized in that: The step of quantifying the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point includes: Based on the preliminary parking area, the cable density distribution characteristic data and the height data of each cable, quantify the movement trajectory of the unmanned vehicle under the spatially restricted conditions corresponding to the fault area to obtain an initial trajectory coordinate sequence; When it is determined that there is a deviation between the initial trajectory coordinate sequence and the fault area, an adjustment parameter for the chassis height of the unmanned vehicle is generated by a model predictive control method to obtain a trajectory adjustment coordinate sequence; Based on the trajectory adjustment coordinate sequence and the height data of each of the cables, the height variation characteristics of the cables within the fault point and its preset area are extracted to determine the area division range of the fault point through a clustering algorithm; According to the area division range and the space restriction condition, a target trajectory adjustment coordinate sequence that meets the connectivity requirements is selected from the trajectory adjustment coordinate sequence and combined to generate a continuous moving trajectory path to be combined with the cable density distribution characteristic data to determine the precise parking position of the unmanned vehicle relative to the fault point.

4. The power distribution facility inspection method using unmanned vehicles, robot dogs and robotic arms in collaboration according to claim 1 is characterized in that: The method of processing the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the accurate three-dimensional coordinates of the fault point includes: Based on the precise docking position, an infrared sensor and a depth camera deployed on the robot dog are used to obtain occlusion information data of the occlusion area below the cable, and a multimodal data set is obtained to be input into a convolutional neural network for processing, and a contour feature data set is output; Determining the three-dimensional coordinate value of the faulty component according to the contour feature data set and parameter information of the depth camera; Based on the three-dimensional coordinate values ​​and the shielded area below the cable, a clustering algorithm is used to divide the regional boundary of the faulty component to obtain a boundary division coordinate sequence to determine the thermal characteristic distribution matrix of the faulty component according to the multimodal data set; Based on the contour feature data set, a matching thermal feature distribution matrix is ​​extracted from the thermal feature distribution matrix, and when it is determined that there is a position deviation between the matching thermal feature distribution matrix and the three-dimensional coordinate value, the matching thermal feature distribution matrix is ​​adjusted through a gradient descent algorithm to obtain a thermal feature distribution adjustment matrix to combine with the three-dimensional coordinate value to determine the rough three-dimensional coordinates of the fault point.

5. The power distribution facility inspection method using unmanned vehicles, robot dogs and mechanical arms as described in claim 4 is characterized in that: The method further includes: processing the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the accurate three-dimensional coordinates of the fault point; Based on the rough three-dimensional coordinates, extract the occlusion depth data collected by the depth camera from the multimodal data set, perform denoising processing using an image denoising algorithm to obtain denoised depth data, and process the denoised depth data using a convolutional neural network to extract edge features of the faulty component under cable interference to obtain an edge feature data set; Determining whether the integrity of the edge feature data set is lower than a preset integrity threshold, and when the integrity of the edge feature data set is lower than the preset integrity threshold, optimizing the denoised depth data through an iterative optimization algorithm to obtain optimized depth data; Determine the initial three-dimensional coordinates of the faulty component according to the optimized depth data and the rough three-dimensional coordinates, and process the edge feature data set and the optimized depth data using an interpolation algorithm based on the initial three-dimensional coordinates to obtain a continuous three-dimensional coordinate path; Based on the continuous three-dimensional coordinate path, the spatial distribution characteristics of the faulty component are extracted from the shielded area below the cable, and a distribution characteristic data set is obtained to divide the regional boundaries of the faulty component using a clustering algorithm to obtain the precise three-dimensional coordinates of the fault point and its corresponding boundary sequence data.

6. The power distribution facility inspection method using unmanned vehicles, robot dogs and robotic arms in collaboration according to claim 1 is characterized in that: The acquiring of real-time coordinate offset data when the robot dog adjusts its posture, so as to combine the motion range to quantify the dynamic adjustment parameters when the robot arm is aligned with the fault point, includes: The real-time coordinate offset data of the robot dog when the robot dog frequently adjusts its posture is obtained and processed by a filtering algorithm to obtain denoised offset data to be combined with the boundary data of the operating platform in the motion range, and the initial value of the dynamic adjustment parameter of the robot arm at the current posture adjustment frequency is calculated to obtain an initial dynamic adjustment parameter set; When the deviation between the initial dynamic adjustment parameter set and the preset adjustment threshold exceeds the preset deviation limit range, the initial dynamic adjustment parameter set is iteratively optimized using a gradient descent method to generate an optimized dynamic adjustment parameter set when the mechanical arm is aligned with the fault point; Based on the optimized dynamic adjustment parameter set and the real-time coordinate offset data, a clustering algorithm is used to partition the motion range, and motion range boundary sequence data is obtained to be combined with the precise three-dimensional coordinates, and a mapping relationship between the optimized dynamic adjustment parameter set and the unmanned vehicle is calculated to generate a mapping relationship data set; According to the mapping relationship data set and the denoised offset data, a linear regression method is used to predict the change trend of the dynamic adjustment parameters of the robotic arm in the next time period, and a predicted adjustment parameter set is generated as the dynamic adjustment parameter output when the robotic arm is aligned with the fault point.

7. The power distribution facility inspection method using unmanned vehicles, robot dogs and robotic arms in collaboration according to claim 1 is characterized in that: The generating an operation instruction for the robotic arm based on the dynamic adjustment parameter comprises: Based on the dynamic adjustment parameters, an inverse kinematics algorithm is used to calculate a joint angle change sequence of the robotic arm, and the joint angle change sequence is adjusted according to a parking deviation between the precise parking position of the unmanned vehicle and the actual parking position to obtain a joint angle adjustment data set; When the joint angle change in the joint angle adjustment data set exceeds the physical limit of the robot arm, fine-tuning the precise docking position to update the joint angle adjustment data set to obtain a deviation elimination angle change set; The deviation elimination angle change set is integrated with the motion range to obtain a joint control sequence after dynamic parameter compensation, which is combined with the precise parking position after fine-tuning of the unmanned vehicle, and processed by a clustering algorithm to divide the boundary area of ​​the operation instruction set to generate a preliminary instruction set; Based on the preliminary instruction set and the joint control sequence, the linear regression method is used to predict the joint angle change trend of the robotic arm in the next time period, and a predicted angle adjustment set is obtained to be combined with the real-time docking deviation of the unmanned vehicle, and processed by a filtering algorithm to obtain a denoised instruction set as the operation instruction output for the robotic arm.

8. The power distribution facility inspection method using unmanned vehicles, robot dogs and mechanical arms in coordination according to claim 1 is characterized in that: The method of using a time series analysis method and a Kalman filter algorithm to process the real-time coordinate offset data to predict the real-time position change trend of the robot dog relative to the fault point and obtain the fault position data synchronized with the operation instruction includes: The real-time coordinate offset data is processed by a time series analysis method to obtain a real-time offset change trend and processed by a Kalman filter algorithm, and the predicted coordinates of the fault point of the robot dog relative to the fault point are output and merged with the operation instruction to obtain a synchronous fault position data set; The synchronous fault location data set is processed by a clustering algorithm to obtain a spatial distribution data set of coordinate offsets, which is combined with the real-time coordinate offset data and processed by a linear regression algorithm to calculate the trend offset of the fault point coordinates, thereby obtaining a fault coordinate prediction data set after trend adjustment; Extracting real-time noise data from the real-time coordinate offset data, and processing the real-time noise data using a Kalman filter algorithm, so as to adjust the fault coordinate prediction data set according to the processing result to obtain a denoised coordinate prediction data set; The mapping relationship between the denoised coordinate prediction data set and the operation instruction is quantified to adjust the operation instruction to obtain the adjusted operation instruction, so as to generate a continuous sequence of coordinate predictions of the robot dog relative to the fault point through an interpolation algorithm and serve as fault location data synchronized with the operation instruction.

9. The power distribution facility inspection method using unmanned vehicles, robot dogs and mechanical arms in coordination according to claim 1 is characterized in that: The generating of the captured data based on the operation instruction and the fault location data to separate the faulty component from the faulty area and perform emergency repairs to realize the inspection of the power distribution facilities includes: Based on the fault location data, a closed-loop control system of the robot arm is controlled to generate initial gripping angle and force data of the end effector, and environmental data of the faulty component is acquired according to the fault location data to perform regional division using a clustering algorithm to obtain a faulty component region; An interpolation algorithm is used to process the initial grasping angle and force data to generate a continuous end-effector motion trajectory to construct an implementation action sequence data set; Acquire real-time environmental data of the fault component area and perform denoising through a Kalman filter algorithm, so as to adjust the implementation action sequence data set according to the denoising result to obtain a denoised repair action data set; Extracting dynamic environmental change features from the real-time environmental data, processing the fault location data through an interpolation algorithm, generating a continuous fault component separation path to adjust the denoised repair action data set, and obtaining a final execution action data set as captured data; The robot arm is controlled according to the captured data to separate the faulty component from the faulty area and perform emergency repairs, thereby realizing patrol inspection of the power distribution facilities.

10. A power distribution facility inspection system coordinated by an unmanned vehicle, a robot dog and a mechanical arm, characterized in that: include: A position determination module, used to determine a preliminary parking area of ​​the unmanned vehicle based on the three-dimensional distribution data of cables corresponding to the fault area of ​​the power distribution facility, and quantify the movement trajectory of the unmanned vehicle in the fault area according to the preliminary parking area to determine the precise parking position of the unmanned vehicle relative to the fault point; A range quantification module is used to process the occlusion information data of the fault area collected by the robot dog through a convolutional neural network to obtain the precise three-dimensional coordinates of the fault point, and quantify the motion range of the robot arm based on the relative relationship between the precise docking position and the precise three-dimensional coordinates; An instruction generation module, used for acquiring real-time coordinate offset data when the robot dog adjusts its posture, so as to quantify the dynamic adjustment parameters when the robot arm is aligned with the fault point in combination with the motion range, and to generate an operation instruction for the robot arm based on the dynamic adjustment parameters; A position synchronization module, used to process the real-time coordinate offset data using a time series analysis method and a Kalman filter algorithm to predict the real-time position change trend of the robot dog relative to the fault point and obtain fault position data synchronized with the operation instruction; A fault repair module is used to generate captured data based on the operation instruction and the fault location data, so as to separate the faulty component from the fault area and perform emergency repairs, thereby realizing inspection of the power distribution facilities.

Images