An overhead line obstacle detection and information feedback system based on a patrol robot

Abstract

The application relates to the technical field of electric power inspection robots, and discloses an overhead line obstacle detection and information feedback system based on an inspection robot, which comprises a perception execution module, a fusion processing module, a classification and identification module, a transmission feedback module and a background module. The perception execution module is configured to collect multi-source perception data corresponding to obstacles around overhead lines through a plurality of sensing units carried on the inspection robot and output the multi-source perception data. The classification and identification module is configured to respectively perform living body attribute identification and flexible / hard attribute identification according to the fusion feature data, cross-match the living body attribute identification result and the flexible / hard attribute identification result, determine an obstacle classification label and a risk level, and the transmission feedback module is configured to transmit the obstacle classification label and the risk level to a user terminal. The background module is configured to store the obstacle classification label and the risk level. The application can realize multi-dimensional accurate subdivision of three categories of obstacles, namely animal living bodies, plant living bodies and non-living bodies, and flexible / hard attributes.

Description

Technical Field

[0001] This invention relates to the field of power line inspection robot technology, and more specifically, to an overhead line obstacle detection and information feedback system based on an inspection robot. Background Technology

[0002] Overhead transmission lines are the core carriers of power transmission and are widely distributed in complex areas such as mountainous areas, suburbs, and forests. Hazards such as floating foreign objects, tree obstacles, bird nests, conductor damage, and hardware defects attached to the lines can easily induce short circuits and line breaks, threatening the safe operation of the power grid. Regular inspections and checks are necessary.

[0003] Traditional overhead power line inspection is primarily manual, with workers climbing towers or using aerial work platforms for inspection. This method suffers from drawbacks such as high risks associated with working at heights, large blind spots in mountainous areas, low efficiency, and high labor intensity, with limitations being particularly pronounced in complex terrain. While robotics technology is leading to the gradual replacement of manual labor with robots for overhead power line inspection, existing obstacle detection solutions for these robots still have significant shortcomings: First, obstacle classification is limited by a single dimension, failing to provide precise differentiation. Existing equipment mostly employs single-vision or simple multi-sensor fusion solutions, which can only determine the presence of obstacles but cannot distinguish between living and non-living objects, or hard and flexible obstacles. Some solutions identify tree obstacles using infrared temperature difference, but the results are greatly affected by sunlight and ambient temperature, making it difficult to reliably distinguish between living and dead trees, resulting in unreliable classification results. This lack of classification information leads to a lack of targeted maintenance and handling: different types of obstacles correspond to different handling solutions, but existing robots only output a single result indicating the presence of an obstacle. Maintenance personnel cannot pre-select a solution and must conduct on-site inspections, delaying the timeliness of fault handling.

[0004] Second, single-sensor identification is prone to confusing obstacle features, and multi-source fusion solutions lack targeted design. Visible light cameras struggle to distinguish between floating objects and tree branches in rain, snow, fog, or backlight conditions; LiDAR can only acquire size and outline, unable to determine material or biological properties; infrared thermal imaging can only identify constant-temperature heat sources, unable to distinguish room-temperature foreign objects. Existing general data fusion solutions do not extract features specifically for obstacle classification, nor do they enhance material and biological characteristics, exacerbating the problems of missed and false detections in classification and identification.

[0005] Third, information feedback is not bound to category tags, resulting in a broken operation and maintenance loop. The existing transmission module only uploads images and coordinate data, without simultaneously pushing obstacle type tags. The backend cannot automatically push disposal work orders according to their classification, and inspection data is disconnected from operation and maintenance management, failing to form an efficient disposal loop.

[0006] The core root cause of the above problems is the inability to accurately and reliably classify obstacles.

[0007] Therefore, it is necessary to design an overhead line obstacle detection and information feedback system based on inspection robots to solve the problems existing in the current technology. Summary of the Invention

[0008] In view of this, the present invention proposes an overhead line obstacle detection and information feedback system based on an inspection robot, which aims to solve the problems of low obstacle recognition accuracy, coarse classification dimensions, and disconnect between information feedback and operation and maintenance in the existing technology.

[0009] This invention proposes an overhead power line obstacle detection and information feedback system based on an inspection robot, comprising: The perception and execution module is configured to collect multi-source perception data corresponding to obstacles around overhead lines through multiple sensing units mounted on the inspection robot, and output the multi-source perception data. The fusion processing module is configured to perform region filtering, feature extraction, and feature fusion processing on the multi-source sensing data to generate fused feature data for characterizing the liveness and physical properties of obstacles. The classification and recognition module is configured to perform liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, and to perform cross-matching of the liveness attribute recognition results and flexible / rigid attribute recognition results to determine the obstacle classification label and risk level; The transmission feedback module is configured to associate and transmit the obstacle classification labels, risk levels and corresponding detection data, and adjust the corresponding detection operation of the inspection robot according to the received feedback instructions. The backend module is configured to receive obstacle detection information sent by the transmission feedback module, and to display obstacles, push early warnings, and manage operation and maintenance tasks according to the obstacle classification labels and risk levels.

[0010] Furthermore, the perception execution module includes a high-definition visible light camera, an infrared thermal imager, a lidar, and an ultrasonic ranging sensor.

[0011] Furthermore, when performing region filtering, feature extraction, and feature fusion processing on the multi-source sensing data to generate fused feature data for characterizing the liveness and physical properties of obstacles, the process includes: Based on the spatial coordinates of the overhead line, the multi-source sensing data is filtered for regions of interest to obtain target areas containing obstacles, and background data corresponding to normal conductors, fittings and towers are removed. The image data acquired by the high-definition visible light camera, the temperature field data acquired by the infrared thermal imager, the three-dimensional point cloud data acquired by the lidar, and the sound echo data acquired by the ultrasonic ranging sensor are preprocessed to obtain the corresponding visual feature data, temperature feature data, morphological structure feature data, and material feature data. Based on the visual feature data, temperature feature data, and morphological structure feature data, feature fusion is performed to generate liveness attribute fusion features for liveness attribute recognition. Based on the morphological structure feature data, material feature data, and visual feature data, feature fusion is performed to generate physical attribute fusion features for flexible / rigid attribute recognition.

[0012] Further, when performing feature fusion based on the visual feature data, temperature feature data, and morphological structure feature data to generate liveness attribute fusion features for liveness attribute recognition, the process includes: Extract the texture gradient, edge continuity, morphological structure features of obstacles, as well as the texture of plant leaves and branching features from the visual feature data; Extract the temperature range, area ratio, and temperature stability parameters of the high-temperature region corresponding to the obstacle from the temperature feature data; Extract the point cloud discrete features, spatial contour features, and plant branch divergence structure features corresponding to the obstacles from the morphological and structural feature data; The living attribute fusion feature is generated by fusing the texture gradient, edge continuity, morphological structure features, plant leaf texture and branching features, temperature range of high-temperature region, area ratio, temperature stability parameters and point cloud discrete features.

[0013] Further, when performing feature fusion based on the visual feature data, temperature feature data, and morphological structure feature data to generate liveness attribute fusion features for liveness attribute recognition, the process includes: Based on the high-temperature region features in the temperature feature data, the plant texture features in the visual feature data, and the branch structure features in the morphological structure feature data, the corresponding temperature feature weights, visual feature weights, and morphological structure feature weights are determined. Based on the temperature feature weight, visual feature weight, and morphological structure feature weight, the temperature feature data, visual feature data, and morphological structure feature data are weighted respectively. The weighted temperature feature data, visual feature data, and morphological structure feature data are fused to generate the liveness attribute fusion feature.

[0014] Furthermore, when performing liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, the process includes: The live attribute fusion features are input into the live attribute recognition branch, and the live attribute is determined based on the temperature features, visual morphology features and plant structure features in the live attribute fusion features; When the temperature feature in the liveness attribute fusion feature satisfies the animal liveness heat source feature, and the visual morphology feature matches the animal outline feature, the liveness attribute corresponding to the obstacle is determined to be an animal liveness. When the temperature feature in the liveness attribute fusion feature does not have a stable heat source feature, and the visual morphology feature matches the plant texture feature and the branch structure feature in the morphological structure feature, the liveness attribute corresponding to the obstacle is determined to be a plant-type liveness. When the temperature feature in the liveness attribute fusion feature does not detect a stable heat source feature corresponding to a live animal and does not meet the matching condition for a live plant feature, the liveness attribute corresponding to the obstacle is determined to be non-live.

[0015] Furthermore, when performing liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, the method further includes: The physical property fusion features are input into the flexible / rigid property recognition branch, and the flexible / rigid properties are determined based on the morphological change features and material response features in the physical property fusion features. When the point cloud discrete feature and the acoustic echo attenuation feature in the physical attribute fusion feature meet the flexible obstacle determination condition, the physical attribute corresponding to the obstacle is determined to be flexible. When the spatial contour feature and acoustic echo stability feature in the physical attribute fusion feature meet the hard obstacle determination conditions, the physical attribute corresponding to the obstacle is determined to be hard.

[0016] Furthermore, when cross-matching the liveness attribute recognition results and the flexible / rigid attribute recognition results to determine the obstacle classification label and risk level, the process includes: The results of the liveness attribute recognition are correlated and matched with the results of the flexible / rigid attribute recognition to establish the obstacle attribute combination relationship; When the liveness attribute identification result is a live animal, the obstacle classification label is determined to be a live animal obstacle; When the liveness attribute identification result is a plant-type live organism, and the flexibility / rigidity attribute identification result is flexible or rigid respectively, the obstacle classification label is determined as a flexible plant-type live obstacle or a rigid plant-type live obstacle. When the liveness attribute identification result is non-liveness, and the flexibility / rigidity attribute identification result is flexible or rigid respectively, the obstacle classification label is determined as non-liveness flexible obstacle or non-liveness rigid obstacle respectively. The corresponding risk level is determined based on the obstacle classification label, the spatial relationship between the obstacle and the conductor, and the number of conductor phases covered by the obstacle.

[0017] Furthermore, when determining the corresponding risk level based on the obstacle classification label, the spatial relationship between the obstacle and the conductor, and the number of conductor phases covered by the obstacle, the following steps are included: The corresponding obstacle risk type is determined based on the obstacle classification label, and the risk level is determined based on the distance relationship between the obstacle and the conductor, the contact state, and the number of phases of the conductor covered. When the obstacle classification label corresponds to a non-living flexible obstacle, and the obstacle covers two or more phase conductors, the risk level is determined to be high risk. When the obstacle classification label corresponds to a live animal obstacle, a broken wire obstacle, or when the obstacle is in continuous contact with the wire, the risk level is determined to be high risk. When the obstacle classification label corresponds to a non-living flexible obstacle and is wrapped around a single wire, an animal-type living obstacle does not cause cross-phase impact, or a plant-type rigid living obstacle is close to the wire, the risk level is determined to be medium risk. When the obstacle classification label corresponds to a flexible living obstacle such as a plant or a small hard obstacle far from the wire, and there is no contact or entanglement with the wire, the risk level is determined to be low risk.

[0018] Furthermore, when transmitting the obstacle classification labels, risk levels, and corresponding detection data in association, the process includes: The corresponding classification code is determined based on the obstacle classification label, and the classification code, risk level, handling guidelines, and corresponding visual image data, temperature field data, three-dimensional point cloud data and sound wave echo data are associated and encapsulated to generate an obstacle detection data package with classification labels; The transmission priority of the obstacle detection data packet is determined according to the risk level. Data packets containing high-risk obstacle information are transmitted to the backend module first, and the corresponding multi-source detection data are transmitted synchronously. When a weak network or communication interruption is detected in the communication link, the obstacle detection data packet is stored in the local storage unit of the inspection robot and retransmitted according to the risk level priority after the communication link is restored. The system receives inspection adjustment instructions generated by the backend module based on the obstacle classification labels and risk levels, and controls the inspection robot to perform corresponding detection operations according to the inspection adjustment instructions.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) Comprehensive classification dimensions and reliable recognition path It can accurately classify obstacles into three categories: live animals, live plants, and non-living objects, as well as flexible / hard attributes. Among them, live animals are identified by infrared heat source, and live plants are identified by visual texture and point cloud morphology. It effectively solves the defect of traditional infrared solutions that cannot reliably distinguish between live trees and dead trees. The technical path is adapted to the needs of engineering field applications, the classification results are highly stable, and it can fully cover high-risk scenarios of overhead lines.

[0020] (2) Targeted feature fusion for accurate and efficient identification The system employs a dual-branch weighted feature fusion scheme, matching core sensing weights for liveness attributes and soft / hard attributes respectively, thereby strengthening the core classification features and effectively shielding against interference from outdoor environments such as rain, snow, and strong light. Combined with a lightweight deep learning model, it can complete classification calculations locally on the robot in real time, while maintaining high accuracy and low latency.

[0021] (3) Category tags are bound to the entire chain, and information feedback is stable. By embedding classification codes in the header field of the transmitted data packets, the backend can directly parse the obstacle type and match the handling plan without the need for secondary identification. Relying on the dual links of 4G / 5G public network and power wireless private network, and with the hierarchical caching and retransmission mechanism, the delivery of high-risk obstacle information can be prioritized. At the same time, it supports downlink classified inspection and control commands, realizing the integrated linkage of identification and inspection and control.

[0022] (4) Categorized closed-loop operation and maintenance effectively reduces costs and increases efficiency. The backend can implement tiered early warning, automatic work order dispatch, and categorized data statistics based on the classification results. Maintenance personnel can match handling tools and solutions in advance to reduce unnecessary on-site inspections. It can also accurately locate high-risk areas, assist in optimizing inspection plans, and build a closed loop of "identification-early warning-handling-statistics", effectively shortening the response cycle for hazard handling and reducing the manpower and vehicle costs of on-site inspections.

[0023] (5) Modular architecture with strong adaptability and scalability The decoupled design of each module allows it to adapt to overhead lines with multiple voltage levels and diverse terrains; the classification model supports the addition of new obstacle subclasses without requiring a complete architecture reconstruction, making engineering implementation and iterative upgrades more convenient. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1This is a structural block diagram of an overhead line obstacle detection and information feedback system based on an inspection robot, provided in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] See Figure 1 As shown in some embodiments of this application, this embodiment provides an overhead line obstacle detection and information feedback system based on an inspection robot, including: The perception and execution module is configured to collect multi-source perception data corresponding to obstacles around overhead lines through multiple sensing units mounted on the inspection robot, and output the multi-source perception data. The fusion processing module is configured to perform region filtering, feature extraction, and feature fusion processing on the multi-source sensing data to generate fused feature data for characterizing the liveness and physical properties of obstacles. The classification and recognition module is configured to perform liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, and to perform cross-matching of the liveness attribute recognition results and flexible / rigid attribute recognition results to determine the obstacle classification label and risk level; The transmission feedback module is configured to associate and transmit the obstacle classification labels, risk levels and corresponding detection data, and adjust the corresponding detection operation of the inspection robot according to the received feedback instructions. The backend module is configured to receive obstacle detection information sent by the transmission feedback module, and to display obstacles, push early warnings, and manage operation and maintenance tasks according to the obstacle classification labels and risk levels.

[0029] It is understood that this embodiment is applied to the inspection of 110kV overhead transmission lines in mountainous areas. The overhead line obstacle detection and information feedback system based on the inspection robot provided in this embodiment includes a perception and execution module, a fusion processing module, a classification and recognition module, a transmission and feedback module, and a backend module.

[0030] The perception and execution module is located on the inspection robot body and includes a high-definition visible light camera, an infrared thermal imager, a lidar, and an ultrasonic ranging sensor: the high-definition visible light camera is equipped with a 3x optical zoom lens to collect visual image data of obstacles around overhead lines; the infrared thermal imager is used to collect temperature field data of obstacles; the lidar is used to collect three-dimensional point cloud data of obstacles; and the ultrasonic ranging sensor is used to collect sound wave echo data of obstacles.

[0031] The fusion processing module, classification and recognition module, and transmission feedback module are all deployed in the embedded processor of the inspection robot. The processor adopts the ARM Cortex-A76 architecture and has 4 TOPS computing power. It can complete the region screening, feature extraction, feature fusion, obstacle attribute recognition, and detection data transmission processing of multi-source perception data.

[0032] The transmission feedback module also connects the communication module and the local storage unit: the communication module includes a 4G / 5G communication module and a power wireless private network module, which is responsible for transmitting obstacle classification labels, risk levels and corresponding detection data to the background module; the local storage unit has a capacity of 32GB, which can store obstacle detection data packets when the communication link is in a weak network state or communication is interrupted, and complete data retransmission after communication is restored.

[0033] The backend module is deployed in the operation and maintenance monitoring center and includes a server, a 3D line monitoring platform, and an early warning push terminal. It is used to receive obstacle detection information sent by the transmission feedback module and to perform obstacle display, early warning push and operation and maintenance task management according to obstacle classification labels and risk levels.

[0034] Specifically, the perception execution module includes a high-definition visible light camera, an infrared thermal imager, a lidar, and an ultrasonic ranging sensor.

[0035] Understandably, the inspection robot is equipped with multiple sensing units. Among them, a high-definition visible light camera is used to collect visible light image data of obstacles around overhead lines to obtain information on the texture, edges, and shape of the obstacles; an infrared thermal imager is used to collect temperature field data corresponding to the obstacles to identify the heat source characteristics that may be generated by living animals; a lidar is used to collect three-dimensional point cloud data corresponding to the obstacles to obtain spatial contour and structural information of the obstacles; and an ultrasonic ranging sensor is used to collect sound wave echo data corresponding to the obstacles to assist in analyzing the material response characteristics of the obstacles.

[0036] Specifically, when performing region filtering, feature extraction, and feature fusion processing on the multi-source sensing data to generate fused feature data for characterizing the liveness and physical properties of obstacles, the process includes: Based on the spatial coordinates of the overhead line, the multi-source sensing data is filtered for regions of interest to obtain target areas containing obstacles, and background data corresponding to normal conductors, fittings and towers are removed. The image data acquired by the high-definition visible light camera, the temperature field data acquired by the infrared thermal imager, the three-dimensional point cloud data acquired by the lidar, and the sound echo data acquired by the ultrasonic ranging sensor are preprocessed to obtain the corresponding visual feature data, temperature feature data, morphological structure feature data, and material feature data. Based on the visual feature data, temperature feature data, and morphological structure feature data, feature fusion is performed to generate liveness attribute fusion features for liveness attribute recognition. Based on the morphological structure feature data, material feature data, and visual feature data, feature fusion is performed to generate physical attribute fusion features for flexible / rigid attribute recognition.

[0037] Understandably, the fusion processing module adopts a modular design, with each feature extraction subunit being independent and having standardized interfaces. This facilitates flexible configuration or replacement of data processing flows for specific sensors based on the actual line environment. For example, in areas with high vegetation cover, the weighting of visual and point cloud plant structure feature extraction can be enhanced, while in areas with frequent animal activity, the sensitivity of infrared thermal source analysis can be dynamically improved. Furthermore, this module supports the introduction of new obstacle subclass recognition capabilities through a model plugin mechanism. When adding typical hazard categories such as "bird's nest," "kite," or "plastic film wrapping," only the corresponding feature templates and discrimination rules need to be loaded, without altering the system's main architecture. This significantly improves the system's adaptability and continuous evolution capabilities in different regions and seasonal scenarios.

[0038] Specifically, when performing feature fusion based on the visual feature data, temperature feature data, and morphological structure feature data to generate liveness attribute fusion features for liveness attribute recognition, the process includes: Extract the texture gradient, edge continuity, morphological structure features of obstacles, as well as the texture of plant leaves and branching features of branches from the visual feature data. Extract the temperature range, area ratio, and temperature stability parameters of the high-temperature region corresponding to the obstacle from the temperature feature data; Extract the point cloud discrete features, spatial contour features, and plant branch divergence structure features corresponding to the obstacles from the morphological and structural feature data; The living attribute fusion feature is generated by fusing the texture gradient, edge continuity, morphological structure features, plant leaf texture and branching features, temperature range of high-temperature region, area ratio, temperature stability parameters and point cloud discrete features.

[0039] It is understandable that the temperature stability parameter refers to the statistical index of the temperature fluctuation range of the obstacle surface within a preset time window, which is used to characterize whether the heat source has continuous metabolic activity characteristics.

[0040] Understandably, this modular architecture is not only reflected in the feature processing and model invocation at the software level, but also permeates the deployment and integration of the hardware system. All sensors in the perception and execution module adopt a plug-and-play interface design, supporting flexible selection and combination based on line voltage levels or terrain complexity. The fusion processing and classification recognition module is deployed as a containerized microservice, with each functional unit interacting through standard APIs. This facilitates the dynamic starting and stopping of non-critical sub-modules under resource-constrained edge computing conditions, ensuring the real-time performance of the core recognition process. When expanding to include new obstacle types (such as adding scenarios like "hanging advertising banners" or "broken wires"), only the corresponding feature fusion strategy and discrimination threshold rules need to be injected into the classification recognition module, and the backend processing knowledge base updated; no structural adjustments to the perception layer, transmission layer, or backend management logic are required.

[0041] Specifically, when performing feature fusion based on the visual feature data, temperature feature data, and morphological structure feature data to generate liveness attribute fusion features for liveness attribute recognition, the process includes: Based on the high-temperature region features in the temperature feature data, the plant texture features in the visual feature data, and the branch structure features in the morphological structure feature data, the corresponding temperature feature weights, visual feature weights, and morphological structure feature weights are determined. Based on the temperature feature weight, visual feature weight, and morphological structure feature weight, the temperature feature data, visual feature data, and morphological structure feature data are weighted respectively. The weighted temperature feature data, visual feature data, and morphological structure feature data are fused to generate the liveness attribute fusion feature.

[0042] Understandably, the specific steps for determining the corresponding temperature feature weights, visual feature weights, and morphological structure feature weights based on the high-temperature region features in the temperature feature data, the plant texture features in the visual feature data, and the branch structure features in the morphological structure feature data are as follows: First, the high-temperature region features are quantitatively analyzed, including the duration of the heat source, the rate of change of the temperature gradient, and the significance of the temperature difference with the environment. If the threshold of animal living metabolic features is met, the temperature feature weight is increased. Second, the plant texture features are jointly analyzed in the frequency and time domains to extract periodic leaf vein structures and non-rigid swaying patterns. If they conform to typical vegetation responses, the visual feature weight is enhanced. Simultaneously, based on the spatial fractal dimension and point cloud density distribution of the branch structure features, it is determined whether they exhibit a natural growth form. If they are typical plant topological structures, the morphological structure feature weight is increased accordingly. After normalization, the three types of weights are applied to the corresponding feature vectors, and after weighted fusion, discriminative living attribute fusion features are generated and input into the classification and recognition module to complete the final judgment. This mechanism ensures that the most discriminative perception dimension is automatically focused under different obstacle scenarios, avoiding redundant calculations and ensuring classification robustness, fully demonstrating the adaptability and engineering scalability of modular architecture in dynamic environments.

[0043] It is understood that the threshold for animal living metabolic characteristics refers to the condition for determining a heat source where the surface temperature of an obstacle is consistently higher than the ambient background temperature A℃ within a preset observation period, and the standard deviation of temperature fluctuation is less than B℃. In this embodiment, the preferred value for A is 2.5, and the preferred value for B is 0.8.

[0044] In this embodiment, the weight of temperature feature is set to 0.5, the weight of visual feature is set to 0.3, and the weight of morphological structure feature is set to 0.2. By weighting and fusing the above features, a liveness attribute fusion feature vector is generated and input into the liveness attribute recognition branch of the classification and recognition module.

[0045] Specifically, when performing liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, the process includes: The live attribute fusion features are input into the live attribute recognition branch, and the live attribute is determined based on the temperature features, visual morphology features and plant structure features in the live attribute fusion features; When the temperature feature in the liveness attribute fusion feature satisfies the animal liveness heat source feature, and the visual morphology feature matches the animal outline feature, the liveness attribute corresponding to the obstacle is determined to be an animal liveness. When the temperature feature in the liveness attribute fusion feature does not have a stable heat source feature, and the visual morphology feature matches the plant texture feature and the branch structure feature in the morphological structure feature, the liveness attribute corresponding to the obstacle is determined to be a plant-type liveness. When the temperature feature in the liveness attribute fusion feature does not detect a stable heat source feature corresponding to a live animal and does not meet the matching condition for a live plant feature, the liveness attribute corresponding to the obstacle is determined to be non-live.

[0046] Understandably, when performing liveness attribute recognition on fused feature data, the system inputs the fused liveness attribute features into the liveness attribute recognition branch. Through comprehensive analysis of the temperature features, visual morphological features, and plant structural features contained within, it determines the liveness attribute of obstacles surrounding overhead power lines. For animal-type live obstacles, since animals possess continuous heat production characteristics, the system extracts liveness heat source features based on temperature field data collected by an infrared thermal imager and combines this with visual morphological features collected by a high-definition visible light camera to match whether the obstacle possesses animal outline features. When a stable temperature distribution characteristic meeting preset conditions is detected in the area corresponding to the obstacle, and the visual morphological features match the animal's external shape, the obstacle can be determined to be a live animal. For plant-type live obstacles, since plants typically do not possess the stable heat source characteristics corresponding to live animals, but have leaf textures, branch structures, and continuous growth patterns that distinguish them from non-live obstacles, the system matches plant texture features from the visual feature data with branch structure features from the morphological structure feature data. When no stable heat source features corresponding to a living animal are detected in an obstacle, and its visual morphological features match the texture features of a plant, while the spatial structure corresponding to the point cloud data conforms to the branching structure feature, the obstacle can be determined to be a living plant. For non-living obstacles, when no stable heat source features corresponding to a living animal are detected in the obstacle, and its visual morphological features and spatial structure features do not meet the matching conditions for a living plant, it indicates that the obstacle does not possess the living characteristics corresponding to an animal or plant, and it is determined to be a non-living obstacle.

[0047] Understandably, by using the above methods, this system leverages the correlation between visible light image features, infrared temperature features, and three-dimensional structural features to distinguish the liveness attributes of obstacles around overhead lines, avoiding misidentification caused by single sensor data and effectively improving the accuracy of obstacle category judgment.

[0048] Specifically, when performing liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, the method further includes: The physical property fusion features are input into the flexible / rigid property recognition branch, and the flexible / rigid properties are determined based on the morphological change features and material response features in the physical property fusion features. When the point cloud discrete feature and the acoustic echo attenuation feature in the physical attribute fusion feature meet the flexible obstacle determination condition, the physical attribute corresponding to the obstacle is determined to be flexible. When the spatial contour feature and acoustic echo stability feature in the physical attribute fusion feature meet the hard obstacle determination conditions, the physical attribute corresponding to the obstacle is determined to be hard.

[0049] Understandably, when performing flexible / rigid attribute recognition based on fused feature data, the system inputs the fused physical attribute features into the flexible / rigid attribute recognition branch. By analyzing the morphological change features and material response features corresponding to the obstacle, the system determines the obstacle's physical attributes. For flexible obstacles, because they are prone to morphological changes under external forces and their spatial structure usually has a certain degree of discreteness, the system extracts the discrete features of the point cloud corresponding to the obstacle based on the 3D point cloud data collected by LiDAR to characterize the continuity and spatial distribution of the obstacle's surface. Simultaneously, based on the sound wave echo data collected by the ultrasonic ranging sensor, the system extracts the sound wave echo attenuation features to reflect the obstacle's response to the sound wave propagation process. When the discrete features of the point cloud are dispersed and the sound wave echo signal shows significant attenuation changes, the obstacle can be determined to have a strong morphological change capability and is thus identified as a flexible obstacle. For rigid obstacles, due to their stable structure, minimal changes in spatial contour, and stable reflection response to sound wave propagation, spatial contour features are extracted from 3D point cloud data collected by lidar to characterize the overall structural stability of the obstacle. Simultaneously, sound echo stability features are extracted from sound echo data collected by ultrasonic ranging sensors to reflect the obstacle's consistent reflection characteristics of sound signals. When the spatial contour features remain continuous and stable, and the sound echo signal stability is high, the obstacle can be determined to possess strong structural stability and thus be classified as a rigid obstacle.

[0050] Understandably, by fusing and analyzing the spatial structural features and acoustic response features of obstacles in the above manner, it is possible to further distinguish the flexible or rigid properties of obstacles on the basis of completing the identification of living attributes, thus providing support for the subsequent generation of obstacle classification labels and the determination of risk levels.

[0051] Specifically, when cross-matching the liveness attribute recognition results and the flexible / rigid attribute recognition results to determine the obstacle classification label and risk level, the process includes: The results of the liveness attribute recognition are correlated and matched with the results of the flexible / rigid attribute recognition to establish the obstacle attribute combination relationship; When the liveness attribute identification result is a live animal, the obstacle classification label is determined to be a live animal obstacle; When the liveness attribute identification result is a plant-type live organism, and the flexibility / rigidity attribute identification result is flexible or rigid respectively, the obstacle classification label is determined as a flexible plant-type live obstacle or a rigid plant-type live obstacle. When the liveness attribute identification result is non-liveness, and the flexibility / rigidity attribute identification result is flexible or rigid respectively, the obstacle classification label is determined as non-liveness flexible obstacle or non-liveness rigid obstacle respectively. The corresponding risk level is determined based on the obstacle classification label, the spatial relationship between the obstacle and the conductor, and the number of conductor phases covered by the obstacle.

[0052] Understandably, when conducting cross-matching of liveness attribute recognition results and flexible / rigid attribute recognition results, it is necessary to correlate the two types of recognition results to generate obstacle attribute combination information. By jointly analyzing the life attributes and physical attributes of obstacles, it is possible to distinguish between obstacles with only a single attribute feature and composite obstacles with multiple attribute features. Specifically, considering the characteristics of animal liveness, which include heat source and activity features, when the liveness attribute recognition result determines that the animal is live, an animal liveness obstacle label is directly generated; when it is determined to be plant liveness, a plant flexible liveness obstacle label or a plant hard liveness obstacle label is generated based on the flexible / rigid attribute recognition results and the corresponding physical features of the plant's leaf and stem structure; when it is determined to be non-liveness, a non-liveness flexible obstacle label or a non-liveness hard obstacle label is generated based on the flexible or hard features corresponding to the non-liveness obstacle. After obtaining the obstacle classification label, the spatial relationship between the obstacle and the conductor, as well as the number of conductor phases covered by the obstacle, are used to analyze the potential impact range of the obstacle on the overhead line. Finally, the corresponding risk level is determined by combining the obstacle type, the degree of proximity to the conductor, and the number of conductors affected.

[0053] Specifically, when determining the corresponding risk level based on the obstacle classification label, the spatial relationship between the obstacle and the conductor, and the number of conductor phases covered by the obstacle, the following are included: The corresponding obstacle risk type is determined based on the obstacle classification label, and the risk level is determined based on the distance relationship between the obstacle and the conductor, the contact state, and the number of phases of the conductor covered. When the obstacle classification label corresponds to a non-living flexible obstacle, and the obstacle covers two or more phase conductors, the risk level is determined to be high risk. When the obstacle classification label corresponds to a live animal obstacle, a broken wire obstacle, or when the obstacle is in continuous contact with the wire, the risk level is determined to be high risk. When the obstacle classification label corresponds to a non-living flexible obstacle and is wrapped around a single wire, an animal-type living obstacle does not cause cross-phase impact, or a plant-type rigid living obstacle is close to the wire, the risk level is determined to be medium risk. When the obstacle classification label corresponds to a flexible living obstacle such as a plant or a small hard obstacle far from the wire, and there is no contact or entanglement with the wire, the risk level is determined to be low risk.

[0054] Understandably, the risk level determination logic is deeply coupled with the configurable nature of the modular architecture. The system uses a parameterized rule engine to flexibly define and dynamically load risk assessment strategies. Different operation and maintenance areas can customize risk thresholds corresponding to various obstacle combinations based on local climate characteristics, line design specifications, and historical fault data. For example, in coastal high-wind-speed areas, the default medium risk of "non-living flexible obstacles entangled in a single conductor" can be upgraded to high risk; while in areas with frequent lightning activity, "vegetation-type rigid living obstacles approaching conductors" is given higher weight. Such strategy adjustments only require updating the configuration file in the rule base, without modifying the underlying recognition or fusion algorithm, fully demonstrating the decoupling advantages of modular design at the business logic level.

[0055] Specifically, when transmitting the obstacle classification labels, risk levels, and corresponding detection data in association, it includes: The corresponding classification code is determined based on the obstacle classification label, and the classification code, risk level, handling guidelines, and corresponding visual image data, temperature field data, three-dimensional point cloud data and sound wave echo data are associated and encapsulated to generate an obstacle detection data package with classification labels; The transmission priority of the obstacle detection data packet is determined according to the risk level. Data packets containing high-risk obstacle information are transmitted to the backend module first, and the corresponding multi-source detection data are transmitted synchronously. When a weak network or communication interruption is detected in the communication link, the obstacle detection data packet is stored in the local storage unit of the inspection robot and retransmitted according to the risk level priority after the communication link is restored. The system receives inspection adjustment instructions generated by the backend module based on the obstacle classification labels and risk levels, and controls the inspection robot to perform corresponding detection operations according to the inspection adjustment instructions.

[0056] Understandably, this data association and transmission mechanism relies on a modular communication interface design to achieve unified encapsulation and hierarchical scheduling of multi-source heterogeneous data. The system maps obstacle classification labels to standardized classification codes (such as "AL-01" representing live animal obstacles, "PL-F" representing flexible live plant obstacles, etc.), and structurally binds them with risk levels, handling suggestions (such as "immediate re-inspection," "included in monthly tracking," "arrange manual removal") and raw detection data (including visible light images, infrared temperature fields, laser point clouds, ultrasonic echo timing, etc.) to form obstacle detection data packets with complete semantic information. When the inspection robot enters a signal blind zone or encounters a temporary link interruption, the local storage unit automatically caches unsent data packets and establishes a retransmission queue based on the risk level. High-risk data packets are retransmitted immediately after communication is restored, and the remaining data are uploaded sequentially according to priority to avoid storage overflow caused by data accumulation.

[0057] Understandably, the backend module can generate inspection adjustment instructions in real time based on the received obstacle information. For example, it can initiate a high-frequency re-inspection mode for high-risk areas, adjust the gimbal's perspective to focus on specific obstacle details, or schedule nearby robots for collaborative verification. These instructions are transmitted to the inspection end via the same communication link, where the task scheduling module parses them and drives the sensing unit to reconfigure parameters (such as increasing the infrared frame rate and increasing the density of point cloud sampling), forming a closed-loop control for recognition feedback optimization. The entire process requires no manual intervention, and the various functional modules interact through standard APIs, facilitating future integration of new sensors or upgrades to the recognition algorithm, further demonstrating the system's openness and scalability at the architectural level.

[0058] Understandably, when the inspection robot detects a plastic film obstacle near the overhead line, the perception and execution module collects the corresponding high-definition visible light image data, infrared temperature field data, lidar 3D point cloud data, and ultrasonic echo data, and sends them to the fusion processing module.

[0059] The fusion processing module filters the regions of interest from the multi-source sensing data based on the spatial coordinates of the overhead line, obtains the target area containing obstacles, and extracts the corresponding features. Specifically, the infrared thermal imaging data did not detect any stable high-temperature areas; the visual feature data exhibited high light transmittance and irregular edge features; the point cloud discrete feature parameter corresponding to the 3D point cloud data acquired by the lidar was 0.18 m²; and the echo attenuation feature parameter corresponding to the acoustic echo data acquired by the ultrasonic ranging sensor was 0.8 dB / cm.

[0060] The classification and recognition module performs liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, determining that the obstacle's liveness attribute is non-liveness and its physical attribute is flexible, and further determining the obstacle's classification label as a non-liveness flexible obstacle. Based on the obstacle classification label, the spatial relationship between the obstacle and the conductor, and the number of conductor phases covered, the corresponding risk level is determined to be high risk.

[0061] The transmission feedback module associates and encapsulates the obstacle classification labels, risk levels, and corresponding detection data to generate an obstacle detection data packet, which is then preferentially transmitted to the backend module. Upon receiving the obstacle detection information, the backend module displays obstacles, sends warnings, and manages maintenance tasks based on the obstacle classification labels and risk levels.

[0062] For example, when the inspection robot detects a bird's nest containing live birds, the fusion processing module extracts the corresponding temperature features, visual morphological features, and morphological structure features. Among them, the infrared thermal imaging data detects a stable high temperature area of ​​about 36°C, the visual contour features match the features of live animals, and the point cloud discrete feature parameters collected by the lidar are 0.08m².

[0063] The classification and recognition module identifies the obstacle based on the fusion features of the liveness attribute and the fusion features of the physical attribute. It determines the liveness attribute of the obstacle to be animal-like and the physical attribute to be rigid, and assigns the obstacle classification label as an animal-like liveness obstacle. Based on the obstacle classification label and the spatial relationship between the obstacle and the guide wire, the risk level is determined to be medium risk. This information is then sent to the backend module via the transmission feedback module, which generates the corresponding maintenance task.

[0064] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this invention, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0065] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A system for detecting and feeding back information about overhead power line obstacles based on an inspection robot, characterized in that, include: The perception and execution module is configured to collect multi-source perception data corresponding to obstacles around overhead lines through multiple sensing units mounted on the inspection robot, and output the multi-source perception data. The fusion processing module is configured to perform region filtering, feature extraction, and feature fusion processing on the multi-source sensing data to generate fused feature data for characterizing the liveness and physical properties of obstacles. The classification and recognition module is configured to perform liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, and to perform cross-matching of the liveness attribute recognition results and flexible / rigid attribute recognition results to determine the obstacle classification label and risk level; The transmission feedback module is configured to associate and transmit the obstacle classification labels, risk levels and corresponding detection data, and adjust the corresponding detection operation of the inspection robot according to the received feedback instructions. The backend module is configured to receive obstacle detection information sent by the transmission feedback module, and to display obstacles, push early warnings, and manage operation and maintenance tasks according to the obstacle classification labels and risk levels.

2. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 1, characterized in that, The perception and execution module includes a high-definition visible light camera, an infrared thermal imager, a lidar, and an ultrasonic ranging sensor.

3. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 2, characterized in that, When performing region filtering, feature extraction, and feature fusion processing on the multi-source sensing data to generate fused feature data for characterizing the liveness and physical properties of obstacles, the process includes: Based on the spatial coordinates of the overhead line, the multi-source sensing data is filtered for regions of interest to obtain target areas containing obstacles, and background data corresponding to normal conductors, fittings and towers are removed. The image data acquired by the high-definition visible light camera, the temperature field data acquired by the infrared thermal imager, the three-dimensional point cloud data acquired by the lidar, and the sound echo data acquired by the ultrasonic ranging sensor are preprocessed to obtain the corresponding visual feature data, temperature feature data, morphological structure feature data, and material feature data. Based on the visual feature data, temperature feature data, and morphological structure feature data, feature fusion is performed to generate liveness attribute fusion features for liveness attribute recognition. Based on the morphological structure feature data, material feature data, and visual feature data, feature fusion is performed to generate physical attribute fusion features for flexible / rigid attribute recognition.

4. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 3, characterized in that, When performing feature fusion based on the visual feature data, temperature feature data, and morphological structure feature data to generate liveness attribute fusion features for liveness attribute recognition, the following are included: Extract the texture gradient, edge continuity, morphological structure features of obstacles, as well as the texture of plant leaves and branching features from the visual feature data; Extract the temperature range, area ratio, and temperature stability parameters of the high-temperature region corresponding to the obstacle from the temperature feature data; Extract the point cloud discrete features, spatial contour features, and plant branch divergence structure features corresponding to the obstacles from the morphological and structural feature data; The living attribute fusion feature is generated by fusing the texture gradient, edge continuity, morphological structure features, plant leaf texture and branching features, temperature range of high-temperature region, area ratio, temperature stability parameters and point cloud discrete features.

5. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 4, characterized in that, When performing feature fusion based on the visual feature data, temperature feature data, and morphological structure feature data to generate liveness attribute fusion features for liveness attribute recognition, the following are included: Based on the high-temperature region features in the temperature feature data, the plant texture features in the visual feature data, and the branch structure features in the morphological structure feature data, the corresponding temperature feature weights, visual feature weights, and morphological structure feature weights are determined. Based on the temperature feature weight, visual feature weight, and morphological structure feature weight, the temperature feature data, visual feature data, and morphological structure feature data are weighted respectively. The weighted temperature feature data, visual feature data, and morphological structure feature data are fused to generate the liveness attribute fusion feature.

6. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 5, characterized in that, When performing liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, the following are included: The live attribute fusion features are input into the live attribute recognition branch, and the live attribute is determined based on the temperature features, visual morphology features and plant structure features in the live attribute fusion features; When the temperature feature in the liveness attribute fusion feature satisfies the animal liveness heat source feature, and the visual morphology feature matches the animal outline feature, the liveness attribute corresponding to the obstacle is determined to be an animal liveness. When the temperature feature in the liveness attribute fusion feature does not have a stable heat source feature, and the visual morphology feature matches the plant texture feature and the branch structure feature in the morphological structure feature, the liveness attribute corresponding to the obstacle is determined to be a plant-type liveness. When the temperature feature in the liveness attribute fusion feature does not detect a stable heat source feature corresponding to a live animal and does not meet the matching condition for a live plant feature, the liveness attribute corresponding to the obstacle is determined to be non-live.

7. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 6, characterized in that, When performing liveness attribute recognition and flexible / rigid attribute recognition based on the fused feature data, the method further includes: The physical property fusion features are input into the flexible / rigid property recognition branch, and the flexible / rigid properties are determined based on the morphological change features and material response features in the physical property fusion features. When the point cloud discrete feature and the acoustic echo attenuation feature in the physical attribute fusion feature meet the flexible obstacle determination condition, the physical attribute corresponding to the obstacle is determined to be flexible. When the spatial contour feature and acoustic echo stability feature in the physical attribute fusion feature meet the hard obstacle determination conditions, the physical attribute corresponding to the obstacle is determined to be hard.

8. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 7, characterized in that, When cross-matching the liveness attribute recognition results and the flexible / rigid attribute recognition results to determine the obstacle classification label and risk level, the following steps are included: The results of the liveness attribute recognition are correlated and matched with the results of the flexible / rigid attribute recognition to establish the obstacle attribute combination relationship; When the liveness attribute identification result is a live animal, the obstacle classification label is determined to be a live animal obstacle; When the liveness attribute identification result is a plant-type live organism, and the flexibility / rigidity attribute identification result is flexible or rigid respectively, the obstacle classification label is determined as a flexible plant-type live obstacle or a rigid plant-type live obstacle. When the liveness attribute identification result is non-liveness, and the flexibility / rigidity attribute identification result is flexible or rigid respectively, the obstacle classification label is determined as non-liveness flexible obstacle or non-liveness rigid obstacle respectively. The corresponding risk level is determined based on the obstacle classification label, the spatial relationship between the obstacle and the conductor, and the number of conductor phases covered by the obstacle.

9. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 8, characterized in that, When determining the corresponding risk level based on the obstacle classification label, the spatial relationship between the obstacle and the conductor, and the number of conductor phases covered by the obstacle, the following are included: The corresponding obstacle risk type is determined based on the obstacle classification label, and the risk level is determined based on the distance relationship between the obstacle and the conductor, the contact state, and the number of phases of the conductor covered. When the obstacle classification label corresponds to a non-living flexible obstacle, and the obstacle covers two or more phase conductors, the risk level is determined to be high risk. When the obstacle classification label corresponds to a live animal obstacle, a broken wire obstacle, or when the obstacle is in continuous contact with the wire, the risk level is determined to be high risk. When the obstacle classification label corresponds to a non-living flexible obstacle and is wrapped around a single wire, an animal-type living obstacle does not cause cross-phase impact, or a plant-type rigid living obstacle is close to the wire, the risk level is determined to be medium risk. When the obstacle classification label corresponds to a flexible living obstacle such as a plant or a small hard obstacle far from the wire, and there is no contact or entanglement with the wire, the risk level is determined to be low risk.

10. The overhead line obstacle detection and information feedback system based on an inspection robot according to claim 9, characterized in that, When transmitting the obstacle classification labels, risk levels, and corresponding detection data in association, the following is included: The corresponding classification code is determined based on the obstacle classification label, and the classification code, risk level, handling guidelines, and corresponding visual image data, temperature field data, three-dimensional point cloud data and sound wave echo data are associated and encapsulated to generate an obstacle detection data package with classification labels; The transmission priority of the obstacle detection data packet is determined according to the risk level. Data packets containing high-risk obstacle information are transmitted to the backend module first, and the corresponding multi-source detection data are transmitted synchronously. When a weak network or communication interruption is detected in the communication link, the obstacle detection data packet is stored in the local storage unit of the inspection robot and retransmitted according to the risk level priority after the communication link is restored. The system receives inspection adjustment instructions generated by the backend module based on the obstacle classification labels and risk levels, and controls the inspection robot to perform corresponding detection operations according to the inspection adjustment instructions.

Images