Method and device for detecting obstacles on a power grid inspection route, electronic equipment and computer readable storage medium

Abstract

A method and device for detecting obstacles on a power grid inspection route, an electronic device and a computer readable storage medium are provided. The method comprises: in response to a trigger detection condition, obtaining a to-be-detected visible light image and a to-be-detected infrared image of the power grid inspection route; fusing the to-be-detected visible light image and the to-be-detected infrared image based on a collection angle, respectively; performing feature recognition on the fused to-be-detected visible light image and the fused to-be-detected infrared image, respectively; and performing obstacle detection based on first to-be-detected contour features and second to-be-detected contour features obtained through the feature recognition, to obtain an obstacle detection result. The method and device provided in the embodiments improve the accuracy and efficiency of power grid inspection.

Description

Technical Field

[0001] This invention relates to the field of power grid inspection technology, and more specifically, to a method, apparatus, electronic device, and computer-readable storage medium for detecting obstacles along a power grid inspection route. Background Technology

[0002] In power grid inspection work, traditional obstacle detection methods often rely on manual visual identification. Inspectors observe the transmission line corridor and its surrounding environment with the naked eye, recording any obstacles they find, such as houses, membrane structures, and wind turbine blades. During routine inspections, inspectors walk along the transmission lines or use vehicles to move around, checking the conditions within the line corridor section by section.

[0003] The main drawbacks of this method are its low detection efficiency and susceptibility to subjective factors. Due to the limited speed of manual inspections, completing a comprehensive inspection of long-distance transmission lines requires a significant amount of time and manpower. Furthermore, differences in the eyesight, experience, and concentration of different inspectors can lead to some obstacles being missed or misidentified. For example, in low light conditions or when obstacles are concealed, it is difficult for humans to accurately detect and identify obstacles, posing a potential threat to the safe and stable operation of the power grid. Summary of the Invention

[0004] In view of this, the present invention proposes a method, apparatus, electronic device and computer-readable storage medium for detecting obstacles along a power grid inspection route, aiming to solve one or more of the technical problems mentioned in the background section above.

[0005] In a first aspect, embodiments of the present invention provide a method for detecting obstacles along a power grid inspection route. The method includes: in response to a triggered detection condition, acquiring a visible light image and an infrared image of the power grid inspection route to be detected; fusing the visible light image and the infrared image based on the acquisition angle to obtain a fused visible light image and a fused infrared image; performing feature recognition on the fused visible light image and the fused infrared image to be detected to obtain a first contour feature and a second contour feature to be detected; and performing obstacle detection based on the first contour feature and the second contour feature to obtain an obstacle detection result.

[0006] Furthermore, the power grid inspection route is obtained in the following way: based on the layout information and geographical information of the power grid transmission lines in the target inspection area and the frequency of historical obstacles, several power grid inspection routes are planned to obtain them.

[0007] Furthermore, the layout information includes the route and tower location, and the geographical information includes topographic information. Route planning is performed based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacle occurrences, to obtain several power grid inspection routes. This includes: dividing the target inspection area into several sub-regions based on the route and topographic information of the power grid transmission lines; using the tower location of the power grid transmission lines in each sub-region as a route node, and determining core inspection nodes among these nodes based on the importance of the power grid transmission lines; performing preliminary route planning based on the core inspection nodes in each sub-region to obtain the first inspection route for each sub-region; performing path filtering on the first inspection route in each sub-region based on the spatiotemporal distribution model of obstacles in each sub-region to obtain the second inspection route in each sub-region; and optimizing the second inspection route in each sub-region with the objectives of minimizing route risk, minimizing route length, and minimizing inspection time to obtain the power grid inspection route for each sub-region.

[0008] Furthermore, the trigger detection conditions include: acquiring visible light images and infrared images of the power grid inspection route; performing texture analysis on the visible light images to obtain the texture complexity of the visible light images, and performing temperature gradient analysis on the infrared images to obtain the temperature gradient of the infrared images; determining visual change information based on the texture complexity of the visible light images, and determining temperature change information based on the temperature gradient of the infrared images; performing correlation analysis on the visual change information and the temperature change information according to a time series, and if they do not conform to a preset normal correlation pattern, then triggering the detection conditions.

[0009] Furthermore, both the fused visible light image and the fused infrared image to be detected are obtained in the following manner: based on the distance from each camera device to the starting point of the power grid inspection route, the acquisition angle of each camera device is mapped to obtain the transformed spatial coordinates; based on the transformed spatial coordinates and a preset neighborhood division step size, each camera device is classified into the corresponding angular spatial neighborhood; for each angular spatial neighborhood, the object edge direction related to the acquisition angle in each image to be detected is extracted, and the angular similarity is determined according to the acquisition angle between any two images to be detected; based on the object edge direction and the angular similarity, the two images to be detected are fused to obtain the fused image of each angular spatial neighborhood; the fused images of each angular spatial neighborhood are integrated to obtain the fused image to be detected.

[0010] Furthermore, the two images to be detected are fused based on the object edge direction and angular similarity to obtain a fused image for each angular spatial neighborhood. This includes: for each angular spatial neighborhood, determining a target image pair based on the first object edge direction of the first image to be detected, the second object edge direction of the second image to be detected, and the angular similarity between the first and second images to be detected; performing image transformation on the target image pair based on the angular difference between the acquisition angles of the images in the target image pair to generate an auxiliary image; performing fusion region analysis based on the acquisition angle and image content features of the target image pair to obtain a target fusion region; and fusing the target image pair based on the auxiliary image and the target fusion region to obtain a fused image for each angular spatial neighborhood.

[0011] Further, obstacle detection is performed based on the first and second detectable contour features to obtain obstacle detection results, including: determining a direction consistency index, a length ratio index, and a topological structure difference degree based on the contour line segment direction, contour length, and contour topology of the first and second detectable contour features, respectively; and performing obstacle detection based on the direction consistency index, the length ratio index, and the topological structure difference degree, combined with the contour feature patterns of historical obstacles, to obtain obstacle detection results.

[0012] Further, obstacle detection is performed based on the directional consistency index, the length ratio index, and the topological structure difference, combined with the contour feature patterns of historical obstacles, to obtain obstacle detection results. This includes: matching the first and second contour features to be detected with the contour feature patterns of historical obstacles to obtain corresponding pattern matching degrees; performing association space and feature association analysis on the first and second contour features to be detected to determine the collaborative relationship index between the two contour features; and performing obstacle detection based on the directional consistency index, the length ratio index, the topological structure difference, the pattern matching degree, and the collaborative relationship index to obtain obstacle detection results.

[0013] Secondly, embodiments of the present invention also provide an apparatus for detecting obstacles along a power grid inspection route. The apparatus includes: an acquisition unit, configured to acquire a visible light image and an infrared image of the power grid inspection route to be detected in response to a triggered detection condition; a fusion unit, configured to fuse the visible light image and the infrared image to be detected based on the acquisition angle, respectively, to obtain a fused visible light image and a fused infrared image to be detected; an identification unit, configured to perform feature identification on the fused visible light image and the fused infrared image to be detected, respectively, to obtain a first detection contour feature and a second detection contour feature; and a detection unit, configured to detect obstacles based on the first detection contour feature and the second detection contour feature, to obtain an obstacle detection result.

[0014] Furthermore, the power grid inspection route is obtained in the following way: based on the layout information and geographical information of the power grid transmission lines in the target inspection area and the frequency of historical obstacles, several power grid inspection routes are planned to obtain them.

[0015] Furthermore, the layout information includes the route and tower location, and the geographical information includes topographic information. Route planning is performed based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacle occurrences, to obtain several power grid inspection routes. This includes: dividing the target inspection area into several sub-regions based on the route and topographic information of the power grid transmission lines; using the tower location of the power grid transmission lines in each sub-region as a route node, and determining core inspection nodes among these nodes based on the importance of the power grid transmission lines; performing preliminary route planning based on the core inspection nodes in each sub-region to obtain the first inspection route for each sub-region; performing path filtering on the first inspection route in each sub-region based on the spatiotemporal distribution model of obstacles in each sub-region to obtain the second inspection route in each sub-region; and optimizing the second inspection route in each sub-region with the objectives of minimizing route risk, minimizing route length, and minimizing inspection time to obtain the power grid inspection route for each sub-region.

[0016] Furthermore, the trigger detection conditions include: acquiring visible light images and infrared images of the power grid inspection route; performing texture analysis on the visible light images to obtain the texture complexity of the visible light images, and performing temperature gradient analysis on the infrared images to obtain the temperature gradient of the infrared images; determining visual change information based on the texture complexity of the visible light images, and determining temperature change information based on the temperature gradient of the infrared images; performing correlation analysis on the visual change information and the temperature change information according to a time series, and if they do not conform to a preset normal correlation pattern, then triggering the detection conditions.

[0017] Furthermore, both the fused visible light image and the fused infrared image to be detected are obtained in the following manner: based on the distance from each camera device to the starting point of the power grid inspection route, the acquisition angle of each camera device is mapped to obtain the transformed spatial coordinates; based on the transformed spatial coordinates and a preset neighborhood division step size, each camera device is classified into the corresponding angular spatial neighborhood; for each angular spatial neighborhood, the object edge direction related to the acquisition angle in each image to be detected is extracted, and the angular similarity is determined according to the acquisition angle between any two images to be detected; based on the object edge direction and the angular similarity, the two images to be detected are fused to obtain the fused image of each angular spatial neighborhood; the fused images of each angular spatial neighborhood are integrated to obtain the fused image to be detected.

[0018] Furthermore, the two images to be detected are fused based on the object edge direction and angular similarity to obtain a fused image for each angular spatial neighborhood. This includes: for each angular spatial neighborhood, determining a target image pair based on the first object edge direction of the first image to be detected, the second object edge direction of the second image to be detected, and the angular similarity between the first and second images to be detected; performing image transformation on the target image pair based on the angular difference between the acquisition angles of the images in the target image pair to generate an auxiliary image; performing fusion region analysis based on the acquisition angle and image content features of the target image pair to obtain a target fusion region; and fusing the target image pair based on the auxiliary image and the target fusion region to obtain a fused image for each angular spatial neighborhood.

[0019] Further, obstacle detection is performed based on the first and second detectable contour features to obtain obstacle detection results, including: determining a direction consistency index, a length ratio index, and a topological structure difference degree based on the contour line segment direction, contour length, and contour topology of the first and second detectable contour features, respectively; and performing obstacle detection based on the direction consistency index, the length ratio index, and the topological structure difference degree, combined with the contour feature patterns of historical obstacles, to obtain obstacle detection results.

[0020] Further, obstacle detection is performed based on the directional consistency index, the length ratio index, and the topological structure difference, combined with the contour feature patterns of historical obstacles, to obtain obstacle detection results. This includes: matching the first and second contour features to be detected with the contour feature patterns of historical obstacles to obtain corresponding pattern matching degrees; performing association space and feature association analysis on the first and second contour features to be detected to determine the collaborative relationship index between the two contour features; and performing obstacle detection based on the directional consistency index, the length ratio index, the topological structure difference, the pattern matching degree, and the collaborative relationship index to obtain obstacle detection results.

[0021] Thirdly, embodiments of the present invention also provide an electronic device, including: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the methods provided in the above embodiments.

[0022] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods provided in the above embodiments.

[0023] The method and apparatus for detecting obstacles along power grid inspection routes provided in this invention acquire, in response to a triggered detection condition, a visible light image and an infrared image of the power grid inspection route to be detected. The visible light and infrared images are then fused based on their acquisition angles. Feature recognition is performed on both the fused images, and obstacle detection is conducted based on the first and second outline features obtained after feature recognition. This results in obstacle detection results, reducing missed detections and misjudgments caused by subjective factors of inspection personnel. Furthermore, the camera equipment can accurately capture obstacle features, enabling timely detection even in poor lighting or when obstacles are concealed, thus improving the accuracy of power grid inspections. Simultaneously, because image capture can quickly cover large areas of transmission lines and can operate 24 hours a day with a wide shooting range, it significantly shortens power grid inspection time and improves inspection efficiency. Therefore, by improving both the efficiency and accuracy of power grid inspections, the safe and stable operation of the power grid is comprehensively guaranteed. Attached Figure Description

[0024] Figure 1 A flowchart illustrating a method for detecting obstacles along a power grid inspection route according to an embodiment of the present invention; Figure 2 for Figure 1 A flowchart of a preferred embodiment of the method shown; Figure 3 for Figure 1 A flowchart of a preferred embodiment of the method shown; Figure 4 for Figure 1 A flowchart of a preferred embodiment of the method shown; Figure 5 for Figure 1 A flowchart of a preferred embodiment of the method shown; Figure 6 This is a schematic diagram of the structure of a device for detecting obstacles along a power grid inspection route according to an embodiment of the present invention; Figure 7 A block diagram of an electronic device provided as an exemplary embodiment of the present invention. Detailed Implementation

[0025] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0026] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0027] Figure 1 This is a flowchart of a method for detecting obstacles along a power grid inspection route, provided as an embodiment of the present invention.

[0028] like Figure 1 As shown, the method includes: Step S101: In response to the triggered detection condition, acquire the visible light image and the infrared image to be detected along the power grid inspection route.

[0029] Optionally, the camera device is a high-definition camera device, and the acquired visible light image is a high-definition image.

[0030] Furthermore, whether the detection condition is triggered is as described in steps S301 to S304. For example, the detection condition is that the infrared camera detects an abnormal temperature change in the image or / and the camera detects a suspected object moving in the image.

[0031] Furthermore, if the triggering conditions are determined, the power grid inspection system controls the video camera and infrared camera to acquire images of the current scene, respectively obtaining the visible light image and the infrared image to be detected in the current scene.

[0032] In one embodiment, on a power grid inspection route, the triggering conditions are: an abnormal temperature change appears in the image, and a suspected object is seen moving in the image. When the inspection equipment travels along the route to a certain location, the high-definition camera detects a rapidly moving object in the image. This movement is not characteristic of normal moving objects such as birds, triggering image acquisition and capturing a clear high-definition image of the object to be detected. The image shows an unidentified object approaching the power transmission line. At the same time, the infrared camera detects a sudden temperature increase in a section of the power transmission line, exceeding the normal operating temperature threshold. This also triggers image acquisition, obtaining an infrared image of the area with the abnormally high temperature, which appears as a different color from its surroundings.

[0033] Furthermore, the power grid inspection route is obtained in the following way: Based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacles, route planning is carried out to obtain several power grid inspection routes.

[0034] Optionally, the layout information in this embodiment includes the route and tower location, and the geographical information includes topographic information. Therefore, the power grid inspection system acquires the route, tower location, topographic information, and historical obstacle frequency of the power grid transmission lines within the target inspection area, and plans multiple power grid inspection routes based on these information, as described in steps S201 to S205. The principle of route planning is to comprehensively consider the actual route of the lines to ensure that the routes cover all key transmission line segments, while also setting reasonable stopping points or observation points based on tower locations. Geographic information is used to avoid complex terrain such as mountains, large lakes, and swamps, which are unfavorable for inspection. Areas with a high frequency of historical obstacles require focused route planning for better monitoring.

[0035] In one embodiment, the target inspection area is a region encompassing both mountainous and plain areas. The transmission line starts in the plain area, passes through part of the mountainous region, and reaches another plain area. The line runs in a northwest-southeast direction, with towers distributed at regular intervals along the line. Geographic information shows that the mountainous areas have steep terrain and rivers flowing through them. Historical data indicates that obstacles such as trees are frequently encountered near certain valleys in the mountains. When planning the route, the power grid inspection system plans a relatively straight route along the line for the plain area to facilitate rapid inspection of most of the line. For the mountainous area, considering the steep terrain and river obstructions, it avoids directly crossing steep peaks and rivers, instead detouring along the edges of the valleys. Simultaneously, for valley areas with a high frequency of historical obstacles, a dedicated branch line is planned to ensure close monitoring of the area surrounding the transmission line, ultimately generating multiple power grid inspection routes including the main line and branch lines.

[0036] Figure 2 for Figure 1 A flowchart of a preferred embodiment of the method shown.

[0037] like Figure 2 As shown, in a preferred embodiment, route planning is performed based on the layout information and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacles, to obtain several power grid inspection routes, including: Step S201: Based on the route and topographic information of the power grid transmission lines, the target inspection area is divided into several sub-areas.

[0038] Optionally, the power grid inspection system acquires detailed topographic data, such as the distribution of mountains, rivers, and plains, and combines this with the route of the power transmission lines to divide the entire target inspection area into multiple sub-regions, each with relatively independent characteristics. For example, areas with complex terrain and significant changes in line direction are divided more finely, while areas with flat terrain and relatively regular line directions are divided more broadly. In one embodiment, the target inspection area is a zone containing mountains, plains, and rivers. The transmission line starts in a plain area, crosses mountains, passes a river, and reaches another plain area. The power grid inspection system acquires digital elevation model (DEM) data of this area to understand the topographic information and the detailed direction coordinates of the transmission lines. The power grid inspection system finds that the mountainous section of the line is winding and has significant topographic relief, so it divides the mountainous area into multiple sub-regions based on the mountain's direction and the line's turning points. For example, between a valley and an adjacent peak, an independent sub-region is defined based on the line's direction from the bottom of the valley to the side of the peak. In plains areas, due to the flat terrain and relatively straight routes, several sub-regions are divided according to a certain distance range, such as every 10 kilometers, combined with the route direction. Ultimately, the entire target inspection area is divided into multiple sub-regions with different terrain and route distribution characteristics.

[0039] Step S202: Using the tower positions of the power grid transmission lines in each sub-region as line nodes, determine the core inspection nodes among the line nodes based on the importance of the power grid transmission lines.

[0040] Furthermore, for each sub-region, the power grid inspection system uses the tower positions of the power grid transmission lines as line nodes, and performs critical node analysis on the line nodes according to the importance of the power grid transmission lines to determine the core inspection nodes. The importance of the line can be determined by various factors, such as the size of the power load carried by the line, whether it is a critical transmission channel, and the degree of impact of power outages on the surrounding areas.

[0041] In one embodiment, in a mountainous sub-region, there are five transmission towers serving as line nodes. Tower A connects to a main power supply channel to a major city, carrying a large electrical load. While tower B connects to a line with a relatively smaller load, a power outage there would affect the operation of several important industrial facilities in the surrounding area. Towers C, D, and E are general transmission line connection nodes. The power grid inspection system analyzes the electrical load data of each line and the impact range of a power outage to identify towers A and B as core inspection nodes. Tower A carries 60% of the total load in this sub-region, and its outage would cause a large-scale power outage in the city, with a significant impact. Although tower B's load accounts for only 15%, it plays a crucial role in the normal operation of surrounding industrial facilities. Towers C, D, and E, due to their smaller loads and limited impact range from power outages, are not considered core inspection nodes.

[0042] Step S203: Based on the core inspection nodes in each sub-region, perform preliminary route planning to obtain the first inspection route for each sub-region.

[0043] Furthermore, the power grid inspection system, based on the principle of connected graphs, performs preliminary route planning by combining the identified core inspection nodes within each sub-region. A connected graph is a graph structure representing the connections between nodes. In this embodiment, nodes represent the locations of transmission line towers, and edges represent the transmission line connections between towers. The power grid inspection system focuses on core inspection nodes. Starting from a core inspection node, it uses a connected graph search algorithm (such as depth-first search) to find the shortest or optimal path connecting other core inspection nodes and some important non-core inspection nodes. This yields the first inspection route for each sub-region, ensuring coverage of core inspection nodes while efficiently traversing some critical lines within the sub-region.

[0044] Continuing with the above embodiment, in the mountainous sub-region where tower A and tower B are identified as core inspection nodes, the power grid inspection system constructs a connectivity graph. Nodes in the graph represent the locations of various towers, and the edge weights can be set to the actual length of the lines between towers. Starting from tower A, the power grid inspection system uses a depth-first search algorithm to search along the edges connecting the transmission lines. During the search, edges connecting core inspection nodes or important non-core inspection nodes are prioritized. After the search, a path is found that starts from tower A, passes through tower D (a relatively important non-core inspection node because it connects to a backup line leading to other areas), and finally reaches tower B. This path is the first inspection route for this sub-region, ensuring that core inspection nodes A and B are covered, while also taking into account the important non-core node D.

[0045] Step S204: Based on the spatiotemporal distribution model of obstacles in each sub-region, perform path filtering on the first inspection route of each sub-region to obtain the second inspection route in each sub-region.

[0046] Optionally, in this embodiment of the invention, a spatiotemporal distribution model of obstacles is pre-modeled based on the frequency of historical obstacle occurrences, using time and space as dimensions. Therefore, the power grid inspection system compares each location on the first inspection route with the spatiotemporal distribution model of obstacles to determine the probability of encountering obstacles at different time and spatial locations. For path segments with high-frequency obstacle occurrence times and locations, the power grid inspection system adjusts or replaces them, selecting paths with a lower probability of encountering obstacles, thereby obtaining a second inspection route for each sub-region.

[0047] In one embodiment, within a certain sub-region, the first inspection route traverses a forested area. An obstacle spatiotemporal distribution model indicates that during spring and autumn, this forested area experiences a higher frequency of obstacles due to rapid tree growth and branches close to power transmission lines. Based on this model information, the power grid inspection system adjusts the first inspection route. In spring and autumn, the route bypasses the forested area, selecting a backup route along the forest edge as the new path. Although this backup route is slightly longer, the model predicts a significantly reduced likelihood of encountering obstacles during these two seasons. In other seasons, since the frequency of obstacles in the forested area is lower, the original first inspection route can still traverse the forested area. Through this selection process, second inspection routes for this sub-region in different seasons are obtained.

[0048] Step S205: Optimize the second inspection route in each sub-region with the objectives of minimizing route risk, minimizing route length, and minimizing inspection time to obtain the power grid inspection route in each sub-region.

[0049] Furthermore, to minimize route risk, the power grid inspection system further adjusts the route by combining the spatiotemporal distribution model of obstacles and terrain risk information (such as landslide risk areas in mountainous regions), avoiding high-risk areas. For minimizing route length, the system uses optimization algorithms (such as variants of Dijkstra's algorithm) to find the shortest path connecting all inspection nodes, considering risk and other constraints. For minimizing inspection time, the system considers speed limits on different road sections (such as slower speeds on mountain roads and faster speeds on plains) and the detection time of equipment in different environments, comprehensively optimizing the route to ultimately obtain the power grid inspection route for each sub-region.

[0050] In one embodiment, within a sub-region that includes both mountainous and plain areas, the second inspection route originally passed through a section of road near a landslide-prone area in the mountainous section. To minimize route risk, the power grid inspection system analyzed terrain risk data and historical landslide records to adjust the route, bypassing the landslide-prone area and selecting a slightly longer but lower-risk mountain road. Regarding route length minimization, the system used a variant of Dijkstra's algorithm, considering the actual length and connectivity of the adjusted mountainous route and the plain section, to recalculate the shortest path from the starting point to the end point, passing through each inspection node. It found that adjusting the route connections in the plain section could shorten the overall route length. Regarding minimizing inspection time, the system comprehensively optimized the route, taking into account factors such as the 30 km / h speed limit on mountain roads, the 80 km / h speed limit on plain roads, and the fact that the equipment requires 5 minutes longer to inspect each tower in the complex mountainous environment compared to the plain. Ultimately, a power grid inspection route that achieves a good balance in terms of risk, length, and time was determined. For example, in mountainous areas, a low-risk and relatively short mountain road was selected, while in plains areas, a route that allows for rapid passage and connects inspection nodes reasonably was selected. This achieved the comprehensive goal of minimizing route risk, shortening route length, and minimizing inspection time.

[0051] Guided by the goals of minimizing route length and inspection time, the optimized route of this invention not only has a shorter distance but also takes into account the travel speed and equipment testing time of different road sections, resulting in a significant reduction in overall inspection time and improved inspection efficiency. It can complete a comprehensive inspection of the entire target inspection area in a shorter time, promptly identify and address potential problems, and ensure the stable operation of the power grid.

[0052] Figure 3 for Figure 1 A flowchart of a preferred embodiment of the method shown.

[0053] like Figure 3 As shown, the triggering detection conditions include: Step S301: Acquire visible light and infrared images of the power grid inspection route.

[0054] Furthermore, each power grid inspection route is equipped with corresponding video and infrared cameras, which can be deployed at preset locations or on drones. Therefore, for each power grid inspection route, the power grid inspection system controls the video and infrared cameras to identify the current scene to determine whether the image identified by the cameras triggers detection conditions.

[0055] Step S302: Perform texture analysis on the visible light image to obtain the texture complexity of the visible light image, and perform temperature gradient analysis on the infrared image to obtain the temperature gradient of the infrared image.

[0056] Furthermore, texture analysis is performed based on the pixel value of each pixel in the image identified by the high-definition camera device and the pixel values ​​of the pixels in the preset neighborhood of each pixel to obtain the texture complexity of the high-definition image.

[0057] Optionally, after acquiring images using high-definition cameras, the power grid inspection system traverses every pixel in the image. For a selected pixel, a neighborhood of a preset size is defined around it, such as a 3×3 or 5×5 pixel neighborhood. Further, the power grid inspection system uses a specific algorithm to analyze the pixel values ​​of this pixel and its neighboring pixels to perform texture analysis, obtaining the texture complexity of the high-definition image. High texture complexity indicates frequent and irregular changes in pixel values, reflecting a complex texture structure in the image; low texture complexity indicates relatively smooth changes in pixel values, and a relatively simple image texture. The specific formula for the algorithm is as follows: .in, Indicates the texture complexity of a high-resolution image. This represents the pixel value of each pixel in the image. This represents the pixel value of the pixels within the preset neighborhood of each pixel. Indicates the width of the image. Indicates the height of the image.

[0058] Furthermore, based on the temperature value of each pixel in the image identified by the infrared camera device and the temperature values ​​of pixels in the preset neighborhood of each pixel, temperature gradient analysis is performed on different image regions to obtain the infrared image temperature gradient.

[0059] Furthermore, the power grid inspection system, for images acquired by infrared cameras, traverses each pixel of the infrared image and sets a neighborhood range for each pixel. Further, the system calculates the temperature gradient of the area by measuring the temperature difference between a pixel and its neighboring pixels, and then combining this with the spatial relationship of the neighboring pixels. The temperature gradient reflects the rate of temperature change in space; a large temperature gradient indicates rapid temperature changes in the area, potentially indicating the presence of heat sources or areas of temperature anomalies; a small temperature gradient indicates a more uniform temperature distribution. In one embodiment, in an image captured by an infrared camera, for example, a pixel Q represents a location on a transmission line, and its neighborhood is also set to a 3×3 range, with pixels Q1-Q8 within the neighborhood. The temperature value of Q is known to be 30℃, the temperature value of Q1 is 32℃, the temperature value of Q2 is 31℃, etc. The system calculates the temperature difference between Q and Q1-Q8, such as the temperature difference between Q and Q1 being 32℃-30℃=2℃. Considering the spatial location of pixels in the neighborhood, if Q1 is to the right of Q, the temperature gradient in the horizontal direction of the area where point Q is located is calculated based on the spatial relationship and temperature difference. The entire infrared image is traversed to obtain temperature gradient information for different regions. If there is a heated area in the image caused by poor line contact, the temperature difference between a pixel and its neighboring pixels in this area is large, and the calculated temperature gradient will be significantly higher than that of a normal transmission line area. Normal transmission line areas have a relatively uniform temperature distribution and a smaller temperature gradient.

[0060] Step S303: Determine visual change information based on the texture complexity of the visible light image, and determine temperature change information based on the temperature gradient of the infrared image.

[0061] Furthermore, the power grid inspection system determines the visual change information of the image recognized by the camera equipment based on the texture complexity of the visible light image.

[0062] When the texture complexity of a visible light image changes significantly within a certain time period, it indicates a change in the visual scene within the image, such as an object moving into the frame or a change in the state of a previously stationary object. In one embodiment, the camera continuously captures images over a period of time. Initially, the power line and its surrounding environment are relatively stable, and the texture complexity of the visible light image is at a relatively stable, low level. When a bird approaches the power line and enters the frame, the complex texture of the bird's feathers causes the visible light image texture complexity of its area to increase rapidly. The system determines that a visual change has occurred by comparing the changes in texture complexity between the before and after images.

[0063] Furthermore, the power grid inspection system determines the temperature change information in the image identified by the infrared camera based on the temperature gradient of the infrared image. If the temperature gradient suddenly increases or exhibits an abnormal temperature gradient distribution, it indicates an abnormal temperature change in the image, which may be related to equipment failure or potential hazards. In one embodiment, the temperature distribution of various parts of the transmission line was originally uniform, with a small and stable temperature gradient. When a line joint begins to heat up due to increased contact resistance, the temperature gradient in that area increases rapidly. Based on this abnormal change in the temperature gradient, the system determines that a temperature change has occurred, indicating an abnormal temperature in that area.

[0064] Step S304: Perform correlation analysis on visual change information and temperature change information according to time series. If they do not meet the preset normal correlation mode, trigger the detection condition.

[0065] Furthermore, the power grid inspection system performs correlation analysis on visual change information and temperature change information according to time series to determine whether the detection conditions are triggered, as described in steps S3041 to S3043.

[0066] This invention, through analysis of image texture complexity and temperature gradients, can keenly capture visual scene changes and abnormal temperature changes in images. Further time-series correlation analysis effectively eliminates the possibility of misjudgment by a single device, improving inspection accuracy. When a situation truly affecting the safety of transmission lines occurs, such as foreign objects approaching and causing abnormal line temperatures, the detection conditions can be triggered promptly and accurately, enhancing the early warning capability for potential risks and ensuring the safe and stable operation of the power grid.

[0067] In one embodiment, step S304 includes: Step S3041: Perform correlation analysis on the visual change information and temperature change information according to the time series to determine the correlation degree of the image time series between the infrared camera devices.

[0068] Optionally, the power grid inspection system timestamps the visual and temperature change information acquired at different times to ensure the accuracy of the data's temporal order.

[0069] Furthermore, the power grid inspection system compares the characteristics and magnitudes of visual changes and temperature changes at the same or similar time points to determine the temporal correlation between infrared camera devices. For example, it observes the relationship between the change in the speed of an object moving in the visual scene and the rate of change of the temperature gradient in the infrared image over a certain period of time. If visual changes and temperature changes show similar trends or are closely related in time at multiple time points, the temporal correlation is high; otherwise, the correlation is low. In one embodiment, over a period of time, a high-definition camera captures an image every second and records visual change information, while an infrared camera similarly records temperature change information every second. At t=5 seconds, an object is detected in the high-definition image starting to approach the transmission line at a gradually accelerating speed, and the visual change is manifested as the image texture complexity gradually increasing over time. At the same time, after t=5 seconds, the temperature gradient in the corresponding area of ​​the infrared image where the object is approaching the transmission line also begins to gradually increase, and the rate of increase in the temperature gradient is similar to the trend of the object's approach speed in the visual scene. Similar synchronous trends between visual and temperature changes were observed at subsequent time points such as t=6 seconds and t=7 seconds. By analyzing the characteristics of visual and temperature changes at these time points, the power grid inspection system determined that the time series correlation between images from infrared and high-definition cameras was high during this period.

[0070] Step S3042: Determine whether the visual change information and temperature change information conform to the preset normal correlation mode based on the image time series correlation degree.

[0071] Furthermore, the power grid inspection system determines whether visual and temperature change information conforms to a preset normal correlation pattern. This preset normal correlation pattern is derived from historical data and monitoring analysis of transmission lines under normal operating conditions, and includes a reasonable correspondence between visual scenes and temperature changes under different environmental conditions and time periods. Therefore, the power grid inspection system compares the currently calculated image time series correlation degree with the correlation degree range in the preset normal correlation pattern, and analyzes whether the specific characteristics of visual and temperature changes match the characteristics in the preset pattern. For example, under normal circumstances, when a small bird briefly flies near a transmission line, there will be a brief change in the complexity of the image texture. The temperature in the infrared image may also show a very small and brief change due to the bird's body temperature. The magnitude of this change, its duration, and the correlation between the two are all recorded in the preset normal correlation pattern.

[0072] In one embodiment, during a certain monitoring period, the image time-series correlation calculated by the power grid inspection system showed that visual changes and temperature changes were closely correlated in time, and the magnitudes of the changes corresponded to each other. Further analysis revealed that visually, a small bird rapidly flew across the transmission line area, causing a momentary increase in image texture complexity, which then quickly returned to normal. Simultaneously, the temperature of the corresponding area in the infrared image experienced a slight, brief increase when the bird flew by, before returning to normal levels. Comparing these characteristics with a preset normal correlation pattern revealed that the temporal synchronicity, magnitude, and duration of the visual and temperature changes all matched the description of the small bird flying by in the preset normal correlation pattern. Therefore, the system determined that the visual and temperature change information conformed to the preset normal correlation pattern.

[0073] Step S3043: If the preset normal association mode is met, it is determined that the detection condition has not been triggered. If the preset normal association mode is not met, it is determined that the detection condition has been triggered.

[0074] Furthermore, if the visual and temperature change information conforms to the preset normal correlation pattern, the power grid inspection system determines that no detection conditions have been triggered. This means that the currently monitored visual and temperature changes are common phenomena under normal operating conditions of the transmission line, and no further detection procedures need to be initiated. If they do not conform to the preset normal correlation pattern, the power grid inspection system determines that detection conditions have been triggered, indicating that an abnormal situation may have occurred that affects the safe operation of the transmission line, and subsequent image acquisition needs to be initiated promptly.

[0075] In one embodiment, during another monitoring session, a high-definition camera detected a large, unidentified object slowly approaching a power transmission line. Visual changes showed a continuous increase in image texture complexity with significant fluctuations. Simultaneously, an infrared camera detected a sharp increase in the temperature gradient in the area approaching the object, indicating a rapid rise in temperature. Comparing these visual and temperature changes with a preset normal correlation pattern revealed that no known normal scenario would exhibit such a large and sustained synchronous abnormal change in both visual and temperature. Therefore, the power grid inspection system determined that the situation did not conform to the preset normal correlation pattern, triggered the detection conditions, and subsequently initiated detailed image acquisition of the current scene.

[0076] This invention, by determining the correlation degree of image time series, can quantitatively analyze the relationship between visual changes and temperature changes, avoiding the one-sidedness of judging based on a single factor. By comparing the correlation degree with a preset normal correlation pattern, and using historical data and experience-based normal patterns as the judgment criteria, it effectively distinguishes between normal environmental changes and abnormal situations. Therefore, when a situation that does not conform to the normal correlation pattern occurs, the detection conditions are accurately triggered, enabling timely detection of potential factors threatening the safety of transmission lines. This allows for timely warnings before potential problems develop into serious faults, ensuring the stable operation of the power grid.

[0077] Step S102: The visible light image and the infrared image to be detected are fused based on the acquisition angle to obtain the fused visible light image and the fused infrared image to be detected.

[0078] Furthermore, due to the different deployment positions and angles of the video cameras and infrared cameras, the acquired images have different viewing angles. Therefore, the power grid inspection system acquires the acquisition angle of each video camera and infrared camera, and fuses each visible light image to be detected based on the acquisition angle of each camera to obtain a fused visible light image to be detected. Similarly, it fuses each infrared image to be detected based on the acquisition angle of each infrared camera to obtain a fused infrared image to be detected. In other words, the image fusion targets images of the same type.

[0079] Figure 4 for Figure 1 A flowchart of a preferred embodiment of the method shown.

[0080] like Figure 4 As shown, in a preferred embodiment, both the fused visible light image to be detected and the fused infrared image to be detected are obtained in the following manner: Step S401: Based on the distance from each camera device to the starting point of the power grid inspection route, map the acquisition angle of each camera device to obtain the converted spatial coordinates.

[0081] Optionally, for each power grid inspection route, the power grid inspection system determines the starting point of each route and then measures the distance of each camera device relative to that starting point. For each camera device, its acquisition angle is a direction vector in three-dimensional space, which describes the direction in which the device is shooting. Therefore, the power grid inspection system uses mathematical methods such as trigonometric functions, combined with the distance from the device to the starting point, to transform the acquisition angle from the original angular coordinate system to a spatial rectangular coordinate system with the starting point of the inspection route as the origin, obtaining the coordinate representation of the device in this spatial coordinate system, i.e., the transformed spatial coordinates. In one embodiment, there is a power grid inspection route with the starting point coordinates set to (0,0,0). Three high-definition camera devices A, B, and C are used to film the transmission lines along this route. Device A is 100 meters away from the starting point, and its acquisition angle is 30° to the right and upward in its own coordinate system. Based on trigonometric relationships, in a rectangular coordinate system with the starting point of the inspection route as the origin, the coordinates of device A in the x-axis direction are approximately 100 × cos30° ≈ 86.6 meters, in the y-axis direction (assuming the horizontal direction is the y-axis) are approximately 100 × sin30° = 50 meters, and in the z-axis direction are 0 (assuming the device is on a horizontal plane). Similarly, device B is 150 meters from the starting point, with a sampling angle 15° downwards to the left. After calculation, its coordinates in the rectangular coordinate system are approximately (-150 × cos15°, -150 × sin15°, 0) ≈ (-144.9, -38.8, 0). Device C is 200 meters from the starting point, with a sampling angle vertically upwards, and its spatial coordinates are (0, 0, 200). Through this calculation, the sampling angle of each high-definition camera device is successfully mapped to the transformed spatial coordinates.

[0082] Step S402: Based on the transformed spatial coordinates and the preset neighborhood division step size, classify each camera device into the corresponding angular spatial neighborhood.

[0083] Furthermore, the power grid inspection system pre-sets a neighborhood partitioning step size, which determines the size and range of the spatial neighborhood for each angle. For each camera's converted spatial coordinates, the system uses these coordinates as a reference and partitions a neighborhood within the space according to the preset step size. For example, if the preset step size is 50 meters in the x, y, and z directions, then for a device with coordinates (x, y, z), its angular spatial neighborhood is a cubic space (assuming the space is three-dimensional) with (x-25, y-25, z-25) as its lower left vertex and (x+25, y+25, z+25) as its upper right vertex. The power grid inspection system categorizes all camera devices into their corresponding angular spatial neighborhoods based on their converted spatial coordinates. Devices within the same neighborhood have relatively similar acquisition angles and spatial positions.

[0084] Continuing with the example of the three camera devices A, B, and C, the preset neighborhood partitioning step size is 50 meters in all three directions. The transformed spatial coordinates of device A are (86.6, 50, 0). Therefore, its angular spatial neighborhood is a cube space with (86.6-25, 50-25, 0-25) = (61.6, 25, -25) as the lower left vertex and (86.6+25, 50+25, 0+25) = (111.6, 75, 25) as the upper right vertex. The coordinates of device B are (-144.9, -38.8, 0). Its angular space neighborhood is a cube space with (-144.9-25, -38.8-25, 0-25) = (-169.9, -63.8, -25) as the lower left vertex and (-144.9+25, -38.8+25, 0+25) = (-119.9, -13.8, 25) as the upper right vertex. Device C has coordinates (0,0,200). Its angular space neighborhood is a cube with (0-25,0-25,200-25) = (-25,-25,175) as the lower left vertex and (0+25,0+25,200+25) = (25,25,225) as the upper right vertex. Through this division, devices A, B, and C are respectively classified into their corresponding angular space neighborhoods.

[0085] Step S403: For each angular spatial neighborhood, extract the object edge direction related to the acquisition angle in each image to be detected, and determine the angular similarity degree based on the acquisition angle between any two images to be detected.

[0086] It is important to understand that the images to be detected in this embodiment are of the same type. That is, for the fused visible light image to be detected, the image to be detected here is the visible light image to be detected; and for the fused infrared image to be detected, the image to be detected here is the infrared image to be detected.

[0087] Furthermore, for each angular spatial neighborhood, the power grid inspection system performs edge detection on the images to be inspected captured by each camera within the neighborhood, identifying the edges of objects in the images using a specific algorithm (such as the Canny edge detection algorithm). For each edge, its volume edge direction in the image is determined, and this direction is related to the acquisition angle of the camera. Further, the power grid inspection system compares the acquisition angles of any two images to be inspected within the neighborhood (denoted as the first and second images to be inspected) and calculates the angle difference between them.

[0088] Furthermore, the power grid inspection system calculates the angle similarity score based on the angle similarity function, combined with the angle difference between the acquired angles and the consistency of the object edge directions in the images. For example, if the object edge directions in two images are mostly similar and the acquired angle difference is within a small range, then the angle similarity score is high; otherwise, it is low. The specific formula for the angle similarity function is as follows: .

[0089] in, Indicates the degree of angular similarity. This represents the difference in the acquisition angles of the two images. and These represent the first and second images to be detected, respectively. The direction of the object's edge, This represents the total number of object edges in the image. and This indicates the preset weighting coefficient.

[0090] In one embodiment, the similarity threshold is 0.75. , Within a certain spatial neighborhood, there are images of the object to be detected captured by cameras D and E. After Canny edge detection, the object edges in the image captured by device D are mainly concentrated in the horizontal and vertical directions, while the object edges in the image captured by device E also have many horizontal and vertical edges, but with slight differences. The acquisition angle of device D is 20° upward and to the right of the horizontal, while the acquisition angle of device E is 25° upward and to the right of the horizontal. The difference in acquisition angle is significant. , , By calculating the differences in the direction of object edges in the image and substituting them into the aforementioned angular similarity function, the angular similarity score is obtained. This indicates that the images captured by devices D and E have a high degree of similarity in terms of acquisition angle and object edge direction.

[0091] Step S404: Based on the similarity of object edge direction and angle, fuse the two images to be detected to obtain a fused image of the spatial neighborhood of each angle.

[0092] Furthermore, the power grid inspection system fuses the first image to be detected and the second image to be detected based on the similarity of the object's edge direction and angle, to obtain a fused image of the spatial neighborhood of each angle, as described in steps S4041 to S4044.

[0093] Further, step S404 includes: Step S4041: For each angular spatial neighborhood, a target image pair is determined based on the first object edge direction of the first image to be detected, the second object edge direction of the second image to be detected, and the angular similarity between the first image to be detected and the second image to be detected.

[0094] Optionally, for each angular spatial neighborhood, within that neighborhood, the power grid inspection system acquires the first object edge direction of the first image to be detected and the second object edge direction of the second image to be detected. Further, the power grid inspection system determines the similarity of object edge directions based on the first and second object edge directions, and selects the most suitable image pair for fusion based on the similarity of object edge directions and the degree of angular similarity between different image pairs, thus identifying this as the target image pair. Generally, image pairs with similar object edge directions and high angular similarity can more effectively retain and integrate information after fusion, presenting clearer and more accurate scene details.

[0095] In one embodiment, images A, B, and C exist in a certain angular spatial neighborhood. The object edges in image A are primarily horizontal and tilted at approximately 45°. The object edges in image B are also primarily horizontal and tilted at approximately 45°, while the object edges in image C are mostly vertical. The angular similarity between images A and B is calculated to be 0.8, between images A and C to be 0.4, and between images B and C to be 0.3. After comparing these data, the power grid inspection system found that images A and B not only have high similarity in the object edge directions but also the highest angular similarity. Therefore, images A and B are determined to be the target image pair in this angular spatial neighborhood.

[0096] Step S4042: Based on the angular difference between the acquisition angles of the target image centering image, perform image transformation on the target image centering image to generate an auxiliary image.

[0097] Furthermore, based on the angular difference between the acquisition angles of the two images in the target image pair, the power grid inspection system performs image transformation on the images in the target image pair to generate an auxiliary image. Specifically, the power grid inspection system first accurately calculates the difference in acquisition angles between the two images in the target image pair. Based on this angular difference, a specific image transformation algorithm, such as an affine transformation algorithm, is applied to transform one or both images. The affine transformation can maintain the parallelism and straightness of objects in the image while adjusting the image by rotating, scaling, etc., according to the angular difference, making the two images more angularly consistent. The resulting transformed image is the auxiliary image.

[0098] In one embodiment, the target image pair consists of image M and image N. Image M is acquired at an angle 15° upward and to the right of the horizontal, while image N is acquired at an angle 25° upward and to the right of the horizontal, with a difference of 10° between the two angles. The power grid inspection system uses an affine transformation algorithm to transform image M. Through the calculation of the affine transformation matrix, image M is rotated 10° counterclockwise around its center, making the acquisition angle of image M closer to that of image N. The rotated image M is the generated auxiliary image. During the rotation, the parallelism and straight-line characteristics of objects in image M, such as transmission lines and towers, are maintained; only the overall angle is adjusted to better integrate with image N.

[0099] Step S4043: Based on the acquisition angle of the target image and the content features of the image itself, perform fusion region analysis to obtain the target fusion region.

[0100] Furthermore, considering the acquisition angle, different acquisition angles may lead to information duplication or loss in some areas of the image, necessitating the determination of which areas truly require fusion and can be effectively fused. On the other hand, the image's inherent features are analyzed, such as the shape, position, and texture of objects. Specific algorithms, such as edge detection and region growing algorithms, are used to first determine the contours of objects in the image using edge detection, and then perform region growing based on these contours. According to the characteristics and distribution of objects, representative and important areas in the image are identified. Therefore, the power grid inspection system performs fusion region analysis based on the acquisition angle of the target image and the image's inherent features to obtain the target fusion region. This target fusion region contains key information from the image and can retain and integrate useful information to the greatest extent possible during the fusion process.

[0101] In one embodiment, for the target image pair image X and image Y, image X is acquired at a slightly lower angle than image Y. In image X, edge detection reveals that the edges of the transmission line tower are clear, and there is a forest area surrounding it. Image Y also shows the transmission line tower and some forest, but due to the different acquisition angles, the display range and details of the forest area differ. The system utilizes an algorithm based on edge detection and region growing. First, it performs region growing in image X, starting from the tower edge, to determine the tower and its surrounding area as key regions. Simultaneously, considering the difference in acquisition angles, a similar analysis is performed on the corresponding regions in image Y. After comprehensive judgment, the regions in images X and Y containing the transmission line tower and surrounding important trees are determined as the target fusion region. This region, during fusion, can integrate the clear information of the tower from different angles and comprehensively display the spatial relationship between the surrounding trees and the tower.

[0102] Step S4044: Based on the auxiliary image of the target image centering image and the target fusion region, fuse the target image centering image to obtain the fused image of the spatial neighborhood of each angle.

[0103] Furthermore, the power grid inspection system aligns the auxiliary image with another original image (the untransformed image) to ensure accurate spatial correspondence. Then, for the target fusion region, a feature-matching-based fusion algorithm is employed. This algorithm meticulously matches corresponding regions of the two images within the target fusion region based on features such as texture and edges. For example, for each small image patch within the target fusion region, its texture feature vector is calculated, and the most similar texture feature vector is found in the corresponding region of the other image to determine the matching relationship. Based on the matching results, pixel values ​​within the target fusion region are appropriately fused, such as using a weighted average method based on pixel features to calculate the fused pixel values. For non-target fusion regions, the corresponding pixel values ​​from one of the images are directly used. Finally, the fused target and non-target fusion regions are combined to obtain a fused image of the spatial neighborhood at each angle.

[0104] In one embodiment, the target image pair is image P and its auxiliary image P', and the target fusion region is the portion of the image containing power transmission lines and key obstacles. The auxiliary image P' is aligned with image P so that the power transmission lines and other objects in the two images coincide spatially. Within the target fusion region, for a small patch of image P, the texture feature vector of that small patch is... Find the texture feature vector in the corresponding region of image P'. and The most similar small patch is selected. Weights are determined based on the degree of feature matching; for example, regions with high matching scores have weights of 0.6 (image P') and 0.4 (image P). The pixel values ​​of this small patch are then fused. For non-target fusion regions, the corresponding pixel values ​​from image P are used. After processing the target fusion region and combining it with non-target fusion regions, a fused image of the spatial neighborhood at that angle is obtained. This fused image clearly displays the comprehensive information of the transmission line and key obstacles at different angles.

[0105] This invention generates auxiliary images by correcting for differences in acquisition angles, reducing deviations caused by angle issues during image fusion and making the fusion process more accurate. Determining the target fusion region accurately locates key and suitable parts of the image for fusion, avoiding fusion interference from unnecessary areas and improving fusion efficiency and information utilization. The fusion algorithm based on the auxiliary image and the target fusion region fully considers image features and can meticulously integrate image information. This allows the fused image to retain key information from each image while achieving a natural transition, reducing stitching artifacts. The resulting fused image of each angular spatial neighborhood can more comprehensively and clearly present the corresponding area along the power grid inspection route, providing richer and more accurate data support for subsequent feature recognition, obstacle detection, and other tasks. This helps to more efficiently discover potential problems around transmission lines and ensure the safe and stable operation of the power grid.

[0106] Step S405: Integrate the fused images of the spatial neighborhood of each angle to obtain the fused image to be detected.

[0107] Furthermore, the power grid inspection system stitches together images from various spatial neighborhoods according to a specific spatial order (e.g., from left to right, from front to back, determined based on the inspection route and equipment layout). During the stitching process, overlapping areas between neighborhoods are considered, and image matching and fusion techniques are used to eliminate stitching gaps, ensuring that the final fused high-definition image appears visually continuous and complete. For example, for overlapping areas of adjacent neighborhood fused images, a feature-matching-based fusion method is used by comparing the pixel values ​​and object features of the overlapping parts to ensure a natural transition between the overlapping images without obvious stitching marks, thus obtaining a fused image of the area to be inspected that encompasses information captured from different angles along the entire power grid inspection route.

[0108] In one embodiment, there are three angular spatial neighborhoods, whose fused images are I1, I2, and I3, arranged spatially from left to right. During the integration process, I1 and I2 have a certain overlapping area. The system uses a feature matching algorithm to find the corresponding object feature points in the overlapping area, such as the top of the transmission line tower or the turning point of the wire. Based on these feature points, the pixel values ​​of the overlapping area are adjusted and fused to make the transition between I1 and I2 natural at the splicing point. Similarly, the overlapping area of ​​I2 and I3 is processed. Finally, the processed I1, I2, and I3 are spliced ​​together in left-to-right order to obtain a complete fused image to be detected. This image clearly shows the overall view of the transmission line and its surrounding environment taken from different angles along the power grid inspection route.

[0109] This invention, through mapping the acquisition angle to transformed spatial coordinates and dividing the image into neighborhoods, rationally groups camera devices at different positions and angles. By determining the angle similarity and performing image fusion based on this similarity, the angular relationships and object edge features between images are fully considered. This ensures that the fused image retains key information from each device's image while minimizing information redundancy and conflicts. The resulting fused image comprehensively and clearly displays the situation along the power grid inspection route, providing a higher quality data foundation for subsequent feature recognition and obstacle detection processes. This improves the accuracy of power grid inspections and ensures the stable operation of the power grid.

[0110] Step S103: Perform feature recognition on the fused visible light image and the fused infrared image to be detected respectively to obtain the first and second contour features to be detected.

[0111] Furthermore, the power grid inspection system performs feature recognition on the fused visible light image and the fused infrared image to obtain a first and a second detectable contour feature, respectively. In the fused visible light image, image recognition algorithms, such as edge detection algorithms (Canny algorithm, etc.), are used to extract the edge contour information of objects in the image. This contour information reflects the shape characteristics of the objects and forms the first detectable contour feature. For the fused infrared image, based on the characteristics of infrared images, a feature extraction algorithm based on temperature distribution is used to identify the contour boundaries of temperature anomaly regions, obtaining the second detectable contour feature. This feature reflects the shape and range of the temperature anomaly region.

[0112] In one embodiment, for a fused visible light image, the power grid inspection system employs the Canny edge detection algorithm. This algorithm smooths the image using Gaussian filtering to reduce noise interference, then calculates the image gradient to determine the intensity and direction of the edges. After removing non-edge pixels through non-maximum suppression, and then through steps such as double threshold detection and edge connection, the edge contours of transmission lines, surrounding trees, and suspected obstacles in the image are successfully extracted, forming the first detection contour feature. For example, an irregularly shaped suspected obstacle contour is identified. For the corresponding fused infrared image, the system uses a temperature threshold-based segmentation method. A suitable temperature threshold is set, and areas above this threshold are identified as temperature anomaly areas. Then, morphological processing (erosion, dilation, etc.) is used to determine the precise contour boundary of the temperature anomaly area, obtaining the second detection contour feature. For example, if a circular temperature anomaly area contour is found, this area corresponds to the location of local overheating of the transmission line.

[0113] Step S104: Perform obstacle detection based on the first and second contour features to be detected, and obtain the obstacle detection result.

[0114] This invention utilizes the collaborative work of multiple devices to quickly cover large areas of power transmission lines. Furthermore, the high-definition camera and infrared camera can work continuously for 24 hours, covering a wide shooting range, which greatly shortens the power grid inspection time and improves the inspection efficiency of the power grid.

[0115] On the other hand, the use of multiple devices for shooting reduces missed inspections and misjudgments caused by subjective factors of inspection personnel. Moreover, video and infrared cameras can accurately capture the characteristics of obstacles, and can detect them in time even in poor lighting or when obstacles are hidden, thus improving the accuracy of power grid inspections. In this way, by improving the efficiency and accuracy of power grid inspections, the safe and stable operation of the power grid is comprehensively guaranteed.

[0116] Figure 5 for Figure 1 A flowchart of a preferred embodiment of the method shown.

[0117] like Figure 5 As shown, in a preferred embodiment, step S104 includes: Step S1041: Based on the contour line segment direction, contour length, and contour topology of the first and second contour features to be detected, respectively, determine the direction consistency index, length ratio index, and topology difference degree.

[0118] Furthermore, based on the contour line segment directions of the first and second detectable contour features, a direction consistency index is determined, including: Optionally, for each power grid inspection route, the power grid inspection system analyzes the direction of the contour segments of the first and second contour features to be detected corresponding to that route. For each contour segment in the first contour feature to be detected, its direction angle in the image coordinate system is determined, for example, by calculating the angle between the line segment and the positive direction of the horizontal axis. Similarly, the direction angle of the contour segments in the second contour feature to be detected is also calculated. Then, the power grid inspection system compares the direction angles of the two sets of contour segments and uses a specific algorithm to measure their degree of consistency. A high direction consistency index indicates strong similarity in the directions of the two sets of contour segments; conversely, a low index indicates weak similarity. For example, the direction consistency can be initially measured by calculating the mean of the difference in direction angles between the two sets of contour segments, and then the final direction consistency index can be obtained after normalization and other processing.

[0119] In one embodiment, in the fused image analysis corresponding to a power grid inspection route, the first detectable contour feature comes from visible light image analysis, which includes the contour of a transmission line and a suspected obstacle. The contour segment L1 of the suspected obstacle makes an angle of 30° with the positive direction of the horizontal axis. The other segment is 120 The second contour feature to be detected comes from infrared image analysis; the contour line segment L2 of the corresponding region makes an angle of 35° with the positive direction of the horizontal axis. and 125 The system first calculates the angle difference between each pair of corresponding line segments. The angle difference between the first segment of L1 and the first segment of L2 is |30-35|=5. The angle difference for the second segment is |120-125|=5. The mean of the angle difference is calculated to be (5+5) / 2=5. After normalization, it is assumed that the maximum possible angle difference in this scenario is 90°. The directional consistency index is 1-5 / 90. The value of 0.94 indicates that the first and second detected contour features have a high degree of consistency in the direction of the contour line segment.

[0120] Furthermore, based on the contour lengths of the first and second detectable contour features, a length ratio index is determined, including: Furthermore, the power grid inspection system measures the sum of the lengths of all contour segments in the first contour feature to be detected, and obtains the length of the first contour. Similarly, the length of the second contour is obtained by measuring the sum of the lengths of all contour segments in the second contour feature to be detected. Then, the length ratio index is determined by calculating the ratio of the two lengths. The length ratio index can be expressed as... (like , If a value is not specified, special handling is required, such as setting it to a maximum or minimum value to indicate an anomaly. This indicator reflects the relative relationship between two contour features in terms of overall length, which helps determine whether two contours may belong to the same object or related objects.

[0121] Continuing with the above embodiments, in the same power grid inspection route, the sum of the measured lengths of all contour segments in the first detectable contour feature is... Pixels (assuming pixels are the unit of length in the image), the sum of the lengths of all contour segments in the second contour feature to be detected. Pixels. The length ratio is then... This indicates that the length of the first detected contour feature is slightly longer than that of the second detected contour feature, but their lengths are relatively close.

[0122] Furthermore, based on the contour topology of the first and second contour features to be detected, the topology difference is determined, including: Furthermore, the topology describes the connections between different parts of the contour, the presence of holes, and other features. Therefore, the power grid inspection system extracts the topological information of two contour features. For example, for a closed contour, if it contains internal holes, the number and size of the holes, as well as the connection methods between contours, all belong to the topological information. The power grid inspection system further compares the topological structures of two contour features and determines the degree of topological dissimilarity by calculating the difference between them. The dissimilarity calculation can be based on graph theory and other methods, transforming the contour topology into a graph structure. Nodes represent key parts of the contour (such as line segment endpoints, contour intersections, etc.), and edges represent the connections between them. Then, the degree of topological dissimilarity is measured by calculating the edit distance between the two graph structures. A low degree of topological dissimilarity indicates that the two contours have similar topological structures; a high degree of dissimilarity indicates a large difference in topological structures.

[0123] In one embodiment, the first contour feature to be detected is an approximately rectangular contour with four vertices and four sides, without internal holes. Its topological structure can be represented as a simple quadrilateral graph structure. The second contour feature to be detected is also a similar rectangular contour, but with a small indentation at one corner. This indentation results in two additional vertices and two additional sides, and the indented portion forms a very small internal hole. After converting the two contour features into graph structures, the topological structure difference is determined by calculating the graph edit distance. Assuming that after a series of calculations, the graph edit distance is 3 (the specific calculation process involves complex graph matching and operation cost calculations), in this scenario, setting the maximum possible graph edit distance to 10, the topological structure difference is 3 / 10 = 0.3, indicating that the topological structures of the two contours have some differences, but not very large ones.

[0124] Step S1042: Based on the direction consistency index, length ratio index, and topological structure difference, obstacle detection is performed in combination with the contour feature patterns of historical obstacles to obtain obstacle detection results.

[0125] This invention determines directional consistency index, length ratio index, and topological structure difference degree, and performs quantitative analysis on the first and second contour features to be detected from multiple dimensions, comprehensively describing the relationship between the two contour features. It combines the contour feature patterns of historical obstacles for judgment, and utilizes previously accumulated experience and data to accurately identify known types of obstacles, improve the accuracy of power grid inspection, and ensure the safe and stable operation of the power grid.

[0126] Further, step S1042 includes: Step S10421: Match the first and second detectable contour features with the contour feature patterns of historical obstacles to obtain the corresponding pattern matching degree.

[0127] Optionally, the pattern matching degree characterizes whether the current contour features represent obstacles of a known type.

[0128] Furthermore, the power grid inspection system pre-stores a large number of contour feature patterns of different types of historical obstacles (such as tree branches, kites, bird nests, etc.). These patterns contain typical features of various obstacles in terms of shape, size, and topology. For the first contour feature to be detected, the power grid inspection system traverses all historical obstacle contour feature patterns and calculates the similarity between the first contour feature to be detected and each historical pattern using a specific matching algorithm, such as a shape context-based matching algorithm. The shape context algorithm represents the contour shape with a set of discrete points and measures the similarity by calculating the distribution difference between these points and the corresponding points in the historical patterns. The same operation is performed on the second contour feature to be detected, and finally, the similarity values ​​between the first and second contour features to be detected and each historical obstacle contour feature pattern are obtained. The highest similarity value is taken as the corresponding pattern matching degree. The higher the pattern matching degree, the more likely the current contour feature represents a known type of obstacle.

[0129] In one embodiment, along a power grid inspection route, a first detectable contour feature exhibits an irregular, elongated shape with some branches. A second detectable contour feature also shows a similar elongated shape and abnormal temperature distribution in the corresponding area of ​​the infrared image. The power grid inspection system matches the first detectable contour feature with stored historical obstacle contour feature patterns. When matching with a tree branch-type obstacle contour feature pattern, the similarity calculated based on the shape context algorithm is 0.8; when matching with a kite-type obstacle contour feature pattern, the similarity is 0.3. Therefore, the pattern matching degree of the first detectable contour feature with the tree branch-type obstacle contour feature pattern is 0.8. Similarly, when matching the second detectable contour feature, the similarity with the tree branch-type obstacle contour feature pattern is 0.75, while the similarity with other types of obstacle contour feature patterns is low. Therefore, the pattern matching degree of the second detectable contour feature with the tree branch-type obstacle contour feature pattern is 0.75, indicating that both the first and second detectable contour features have a high matching degree with the tree branch-type obstacle contour feature pattern, suggesting a high probability of the presence of tree branch-type obstacles.

[0130] Step S10422: Perform association space and feature association analysis on the first and second contour features to be detected respectively to determine the collaborative relationship index between the two contour features.

[0131] Furthermore, in terms of spatial correlation analysis, the power grid inspection system considers the positional relationship between two contour features in the image space, such as whether they partially overlap and their distance. If two contours have significant overlapping areas in space, it indicates that they are likely to correspond to the same object or related objects. In terms of feature correlation analysis, the power grid inspection system compares the similarity and complementarity of two contour features in terms of features such as shape, direction, and length. For example, whether the direction of a line segment in the first contour feature to be detected is consistent with the direction of the corresponding line segment in the second contour feature to be detected, or whether a missing part in one contour feature can be supplemented in another contour feature. By comprehensively analyzing information from both spatial correlation and feature correlation, a collaborative analysis algorithm based on a graph model is used to determine the collaborative relationship index. This algorithm transforms contour features into a graph structure, where nodes represent key feature points of the contour (such as line segment endpoints, contour intersections, etc.), and edges represent spatial or feature correlation relationships between feature points. The collaborative relationship index is obtained by calculating the similarity and correlation strength between the graph structures. The higher the collaborative relationship index, the stronger the collaboration between the two contour features, and the more likely they are to jointly represent a real object or obstacle. In the image analysis corresponding to the aforementioned power grid inspection route, a long, strip-shaped suspected obstacle contour is found near the transmission line in the first detectable contour feature, and a region with abnormally high temperature corresponds to it in the second detectable contour feature at the same location. From the spatial correlation perspective, the two contours are largely overlapping in the image, with the long strip-shaped portions completely overlapping. From the feature correlation perspective, the edge direction of the long strip-shaped object in the first detectable contour feature and the direction of temperature gradient change in the corresponding region in the second detectable contour feature are consistent in most areas. The power grid inspection system transforms this information into a graph structure and performs calculations using a graph model-based collaborative analysis algorithm. Assuming that in this algorithm, by calculating factors such as spatial distance between nodes and feature similarity, the collaborative relationship index between the two graph structures is 0.9, it indicates a strong collaborative relationship between the first and second detectable contour features, further supporting the judgment that they may represent the same obstacle.

[0132] Step S10423: Obstacle detection is performed based on directional consistency index, length ratio index, topology difference degree, pattern matching degree, and cooperative relationship index to obtain obstacle detection results.

[0133] Furthermore, the power grid inspection system sets a series of thresholds, determined based on extensive historical data and practical detection experience, to judge whether each indicator is within a reasonable range to determine the presence of obstacles. For the directional consistency index, if its value is above a certain lower threshold, it indicates that the two contour features have good directional consistency, which is conducive to identifying them as the same object. For the length ratio index, within a certain reasonable ratio range, it indicates that the length relationship between the two contour features conforms to the characteristics of common obstacles. A topological structure difference index below a certain upper threshold indicates that the topological structure of the two contours is highly similar. A pattern matching degree above a certain threshold indicates that the current contour feature has a high similarity to known historical obstacle contour feature patterns. A cooperation relationship index above a certain threshold indicates strong cooperation between the two contour features. Therefore, when these indicators simultaneously meet their respective threshold conditions, the power grid inspection system determines that an obstacle has been detected, determines the possible type of obstacle based on the historical obstacle contour feature pattern with the highest pattern matching degree, and determines the location of the obstacle by combining the position of the contour in the image and the geographical information of the inspection route, generating a detailed obstacle detection result report. If any indicator fails to meet the threshold condition, it is determined that no obvious obstacle was detected or an unknown type of anomaly was detected, and the relevant information is also recorded in the detection result report. Continuing with the above power grid inspection route as an example, the previously calculated directional consistency index is 0.94, length ratio index is 1.03, topology difference is 0.3, and the pattern matching degrees of the first and second detectable contour features for the tree branch obstacle contour feature pattern are 0.8 and 0.75, respectively, with a cooperative relationship index of 0.9. Assuming the system sets a lower threshold of 0.8 for the directional consistency index, a reasonable range of 0.8-1.2 for the length ratio index, an upper threshold of 0.4 for the topology difference, a threshold of 0.7 for the pattern matching, and a threshold of 0.8 for the cooperative relationship index, and all indicators currently meet their respective threshold conditions, it is determined that a tree branch obstacle has been detected on this power grid inspection route. Based on the position of the contour in the fused image and the geographical information of the inspection route, the obstacle is located 3 meters northeast of a transmission line tower. The system generates an obstacle detection result report, recording detailed information such as the obstacle's location and possible type (tree branch).

[0134] The pattern matching degree determined in this invention enables the system to quickly assess the similarity between the current contour feature and known obstacle types using historical experience, thus narrowing the judgment range. Determining the cooperative relationship index further analyzes the intrinsic connection between two contour features from a spatial and feature association perspective, enhancing the basis for judging real obstacles. Therefore, combining multiple indicators for obstacle detection can accurately identify situations that conform to common obstacle characteristics, promptly detect potential safety hazards, improve the accuracy of power grid inspections, and ensure the stable operation of the power grid.

[0135] The above embodiments, in response to triggering detection conditions, acquire visible light images and infrared images of the power grid inspection route to be detected. These images are then fused based on their acquisition angles. Feature recognition is performed on both the fused images, and obstacle detection is conducted based on the first and second outline features obtained after feature recognition. This reduces missed detections and misjudgments caused by subjective factors of inspection personnel. Furthermore, the camera equipment can accurately capture obstacle features, enabling timely detection even in poor lighting conditions or when obstacles are concealed, thus improving the accuracy of power grid inspections. Simultaneously, because image capture can quickly cover large areas of transmission lines and can operate 24 hours a day with a wide shooting range, it significantly shortens power grid inspection time and improves inspection efficiency. Therefore, by improving both the efficiency and accuracy of power grid inspections, the safe and stable operation of the power grid is comprehensively guaranteed.

[0136] Figure 6 A schematic diagram of a device for detecting obstacles along a power grid inspection route according to an embodiment of the present invention is shown.

[0137] like Figure 6 As shown, the device includes: The acquisition unit 601 is used to acquire the visible light image and the infrared image to be detected along the power grid inspection route in response to the trigger detection condition. The fusion unit 602 is used to fuse the visible light image to be detected and the infrared image to be detected based on the acquisition angle, respectively, to obtain the fused visible light image to be detected and the fused infrared image to be detected. The recognition unit 603 is used to perform feature recognition on the fused visible light image to be detected and the fused infrared image to be detected respectively, to obtain the first detection contour feature and the second detection contour feature. The detection unit 604 is used to perform obstacle detection based on the first and second contour features to be detected, and obtain the obstacle detection result.

[0138] Furthermore, the power grid inspection route is obtained in the following way: Based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacles, route planning is carried out to obtain several power grid inspection routes.

[0139] Furthermore, the layout information includes the route and tower locations, and the geographical information includes topographic information. Based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacles, route planning is performed to obtain several power grid inspection routes, including: Based on the route and topographic information of the power grid transmission lines, the target inspection area is divided into several sub-areas. Using the tower locations of power grid transmission lines in each sub-region as line nodes, and based on the importance of the power grid transmission lines, core inspection nodes are determined among the line nodes; Based on the core inspection nodes in each sub-region, preliminary route planning is performed to obtain the first inspection route for each sub-region. Based on the spatiotemporal distribution model of obstacles in each sub-region, the first inspection route in each sub-region is filtered to obtain the second inspection route in each sub-region. The second inspection route in each sub-region is optimized with the goals of minimizing route risk, shortening route length, and minimizing inspection time, thus obtaining the power grid inspection route in each sub-region.

[0140] Furthermore, the triggering conditions include: Acquire visible light and infrared images of the power grid inspection route; Texture analysis is performed on visible light images to obtain their texture complexity, and temperature gradient analysis is performed on infrared images to obtain their temperature gradient. Visual change information is determined based on the texture complexity of visible light images, and temperature change information is determined based on the temperature gradient of infrared images; Visual change information and temperature change information are correlated and analyzed according to time series. If they do not meet the preset normal correlation pattern, the detection condition is triggered.

[0141] Furthermore, both the fused visible light image and the fused infrared image to be detected are obtained using the following method: Based on the distance from each camera device to the starting point of the power grid inspection route, the acquisition angle of each camera device is mapped to obtain the transformed spatial coordinates; Based on the transformed spatial coordinates and the preset neighborhood division step size, each camera device is classified into the corresponding angular spatial neighborhood. For each angular spatial neighborhood, extract the object edge direction related to the acquisition angle in each image to be detected, and determine the angular similarity degree based on the acquisition angle between any two images to be detected; The two images to be detected are fused based on the similarity of the object edge direction and angle to obtain a fused image of the spatial neighborhood of each angle; By integrating the fused images of the spatial neighborhood of each angle, a fused image to be detected is obtained.

[0142] Furthermore, the two images to be detected are fused based on the similarity of object edge direction and angle to obtain a fused image of the spatial neighborhood of each angle, including: For each angular spatial neighborhood, a target image pair is determined based on the first object edge direction of the first image to be detected, the second object edge direction of the second image to be detected, and the angular similarity between the first image to be detected and the second image to be detected. Based on the angular difference between the acquisition angles of the target image and the centering image, image transformation is performed on the target image and centering image to generate an auxiliary image; Based on the acquisition angle of the target image and the content features of the image itself, the fusion region is analyzed to obtain the target fusion region; The target image centering image is fused with the auxiliary image and the target fusion region based on the target image centering image to obtain a fused image of the spatial neighborhood of each angle.

[0143] Furthermore, obstacle detection is performed based on the first and second target contour features to obtain obstacle detection results, including: Based on the contour line segment direction, contour length, and contour topology of the first and second detectable contour features, respectively, the direction consistency index, length ratio index, and topology difference degree are determined. Obstacle detection is performed based on directional consistency index, length ratio index, and topological structure difference, combined with the contour feature patterns of historical obstacles, to obtain obstacle detection results.

[0144] Furthermore, obstacle detection is performed based on directional consistency indicators, length ratio indicators, and topological structure differences, combined with the contour feature patterns of historical obstacles, to obtain obstacle detection results, including: The first and second contour features to be detected are matched with the contour feature patterns respectively to obtain the corresponding pattern matching degree. The first and second contour features to be detected are subjected to association space and feature association analysis respectively to determine the synergistic relationship index between the two contour features. Obstacle detection is performed based on directional consistency index, length ratio index, topological structure difference degree, pattern matching degree, and cooperative relationship index to obtain obstacle detection results.

[0145] The above embodiments, in response to triggering detection conditions, acquire visible light images and infrared images of the power grid inspection route to be detected. These images are then fused based on their acquisition angles. Feature recognition is performed on both the fused images, and obstacle detection is conducted based on the first and second outline features obtained after feature recognition. This reduces missed detections and misjudgments caused by subjective factors of inspection personnel. Furthermore, the camera equipment can accurately capture obstacle features, enabling timely detection even in poor lighting conditions or when obstacles are concealed, thus improving the accuracy of power grid inspections. Simultaneously, because image capture can quickly cover large areas of transmission lines and can operate 24 hours a day with a wide shooting range, it significantly shortens power grid inspection time and improves inspection efficiency. Therefore, by improving both the efficiency and accuracy of power grid inspections, the safe and stable operation of the power grid is comprehensively guaranteed.

[0146] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0147] Figure 7 This is a block diagram of an electronic device provided as an exemplary embodiment of the present invention. (See diagram below.) Figure 7 As shown, the electronic device includes one or more processors 710 and memory 720.

[0148] The processor 710 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0149] The memory 720 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 710 may execute the program instructions to implement the methods for data access based on dimensional information and / or other desired functions of the software programs of the various embodiments of the present invention described above. In one example, the electronic device may also include an input device 730 and an output device 740, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0150] In addition, the input device 730 may also include, for example, a keyboard, a mouse, etc.

[0151] The output device 740 can output various information to the outside. The output device 740 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0152] Of course, for the sake of simplicity, Figure 7 Only some of the components of the electronic device relevant to the present invention are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0153] In addition to the methods and apparatus described above, embodiments of the present invention may also be computer program products and computer-readable storage media, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods for detecting obstacles along a power grid inspection route according to various embodiments of the present invention described in the "Exemplary Methods" section of this specification.

[0154] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0155] Furthermore, embodiments of the present invention may also be computer-readable storage media storing a computer program thereon, which, when run by a processor, causes the processor to perform the steps of the method for detecting obstacles along a power grid inspection route according to various embodiments of the present invention as described in the "Exemplary Methods" section above.

[0156] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0157] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0158] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0159] The block diagrams of devices, apparatuses, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0160] The methods and apparatus of the present invention may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of the present invention are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present invention may also be implemented as a program recorded on a recording medium, the program comprising machine-readable instructions for implementing the methods according to the present invention. Thus, the present invention also covers recording media storing programs for performing the methods according to the present invention.

[0161] It should also be noted that in the apparatus, device, and method of the present invention, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered equivalents of the present invention. The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0162] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for detecting obstacles along a power grid inspection route, characterized in that, The method includes: In response to the triggered detection conditions, the system acquires the visible light image and the infrared image of the power grid inspection route to be detected. The visible light image and the infrared image to be detected are fused based on the acquisition angle to obtain the fused visible light image and the fused infrared image to be detected. Feature recognition is performed on the fused visible light image and the fused infrared image to be detected, respectively, to obtain the first and second contour features to be detected. Obstacle detection is performed based on the first and second contour features to be detected, and the obstacle detection result is obtained. The fused visible light image and the fused infrared image to be detected are both obtained in the following manner: Based on the distance from each camera device to the starting point of the power grid inspection route, the acquisition angle of each camera device is mapped to obtain the transformed spatial coordinates; Based on the transformed spatial coordinates and the preset neighborhood division step size, each camera device is classified into the corresponding angular spatial neighborhood. For each angular spatial neighborhood, extract the object edge direction related to the acquisition angle in each image to be detected, and determine the angular similarity degree based on the acquisition angle between any two images to be detected; The two images to be detected are fused based on the similarity of the object edge direction and angle to obtain a fused image of the spatial neighborhood of each angle; By integrating the fused images from the spatial neighborhood of each angle, a fused image to be detected is obtained; The process of fusing two images to be detected based on the similarity of object edge direction and angle to obtain a fused image of the spatial neighborhood of each angle includes: For each angular spatial neighborhood, based on the first object edge direction of the first image to be detected, the second object edge direction of the second image to be detected, and the angular similarity between the first image to be detected and the second image to be detected, a target image pair is determined, that is, image pairs with similar object edge directions and high angular similarity are selected. Based on the angular difference between the acquisition angles of the target image and the centering image, image transformation is performed on the target image and centering image to generate an auxiliary image; Based on the acquisition angle of the target image and the content features of the image itself, the fusion region is analyzed to obtain the target fusion region; The target image centering image is fused with the auxiliary image and the target fusion region based on the target image centering image to obtain a fused image of the spatial neighborhood of each angle.

2. The method according to claim 1, characterized in that, The power grid inspection route is obtained in the following way: Based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacles, route planning is carried out to obtain several power grid inspection routes.

3. The method according to claim 2, characterized in that, The layout information includes the route and tower locations, and the geographical information includes topographic information. Based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacles, route planning is performed to obtain several power grid inspection routes, including: Based on the route and topographic information of the power grid transmission lines, the target inspection area is divided into several sub-areas. Using the tower locations of power grid transmission lines in each sub-region as line nodes, and based on the importance of the power grid transmission lines, core inspection nodes are determined among these line nodes; Based on the core inspection nodes in each sub-region, preliminary route planning is performed to obtain the first inspection route for each sub-region. Based on the spatiotemporal distribution model of obstacles in each sub-region, the first inspection route in each sub-region is filtered to obtain the second inspection route in each sub-region. The second inspection route in each sub-region is optimized with the goals of minimizing route risk, shortening route length, and minimizing inspection time, thus obtaining the power grid inspection route in each sub-region.

4. The method according to claim 1, characterized in that, The trigger detection conditions include: Acquire visible light and infrared images of the power grid inspection route; The visible light image is subjected to texture analysis to obtain the texture complexity of the visible light image, and the infrared image is subjected to temperature gradient analysis to obtain the temperature gradient of the infrared image; Visual change information is determined based on the texture complexity of the visible light image, and temperature change information is determined based on the temperature gradient of the infrared image; The visual change information and the temperature change information are correlated and analyzed according to the time series. If they do not meet the preset normal correlation mode, the detection condition is triggered.

5. The method according to claim 1, characterized in that, The obstacle detection based on the first and second contour features to be detected, to obtain the obstacle detection result, includes: Based on the contour line segment direction, contour length, and contour topology of the first and second contour features to be detected, respectively, the direction consistency index, length ratio index, and topology difference degree are determined. Based on the directional consistency index, the length ratio index, and the topological structure difference, obstacle detection is performed in conjunction with the contour feature patterns of historical obstacles to obtain obstacle detection results.

6. The method according to claim 5, characterized in that, The obstacle detection is performed based on the directional consistency index, the length ratio index, and the topological structure difference, combined with the contour feature patterns of historical obstacles, to obtain the obstacle detection results, including: The first and second contour features to be detected are matched with the contour feature patterns of historical obstacles to obtain the corresponding pattern matching degree. The first and second contour features to be detected are respectively subjected to association space and feature association analysis to determine the collaborative relationship index between the two contour features; Obstacle detection is performed based on the directional consistency index, the length ratio index, the topology difference, the pattern matching degree, and the cooperative relationship index to obtain the obstacle detection results.

7. A device for detecting obstacles along a power grid inspection route, characterized in that, The device includes: The acquisition unit is used to acquire the visible light image and the infrared image to be detected along the power grid inspection route in response to the trigger detection condition. The fusion unit is used to fuse the visible light image to be detected and the infrared image to be detected based on the acquisition angle, respectively, to obtain the fused visible light image to be detected and the fused infrared image to be detected. The recognition unit is used to perform feature recognition on the fused visible light image to be detected and the fused infrared image to be detected respectively, to obtain the first detection contour feature and the second detection contour feature. The detection unit is used to perform obstacle detection based on the first detection contour feature and the second detection contour feature to obtain the obstacle detection result; The fused visible light image and the fused infrared image to be detected are both obtained in the following manner: Based on the distance from each camera device to the starting point of the power grid inspection route, the acquisition angle of each camera device is mapped to obtain the transformed spatial coordinates; Based on the transformed spatial coordinates and the preset neighborhood division step size, each camera device is classified into the corresponding angular spatial neighborhood. For each angular spatial neighborhood, extract the object edge direction related to the acquisition angle in each image to be detected, and determine the angular similarity degree based on the acquisition angle between any two images to be detected; The two images to be detected are fused based on the similarity of the object edge direction and angle to obtain a fused image of the spatial neighborhood of each angle; By integrating the fused images from the spatial neighborhood of each angle, a fused image to be detected is obtained; The process of fusing two images to be detected based on the similarity of object edge direction and angle to obtain a fused image of the spatial neighborhood of each angle includes: For each angular spatial neighborhood, based on the first object edge direction of the first image to be detected, the second object edge direction of the second image to be detected, and the angular similarity between the first image to be detected and the second image to be detected, a target image pair is determined, that is, image pairs with similar object edge directions and high angular similarity are selected. Based on the angular difference between the acquisition angles of the target image and the centering image, image transformation is performed on the target image and centering image to generate an auxiliary image; Based on the acquisition angle of the target image and the content features of the image itself, the fusion region is analyzed to obtain the target fusion region; The target image centering image is fused with the auxiliary image and the target fusion region based on the target image centering image to obtain a fused image of the spatial neighborhood of each angle.

8. The apparatus according to claim 7, characterized in that, The power grid inspection route is obtained in the following way: Based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacles, route planning is carried out to obtain several power grid inspection routes.

9. The apparatus according to claim 8, characterized in that, The layout information includes the route and tower locations, and the geographical information includes topographic information. Based on the layout and geographical information of the power grid transmission lines in the target inspection area, as well as the frequency of historical obstacles, route planning is performed to obtain several power grid inspection routes, including: Based on the route and topographic information of the power grid transmission lines, the target inspection area is divided into several sub-areas. Using the tower locations of power grid transmission lines in each sub-region as line nodes, and based on the importance of the power grid transmission lines, core inspection nodes are determined among these line nodes; Based on the core inspection nodes in each sub-region, preliminary route planning is performed to obtain the first inspection route for each sub-region. Based on the spatiotemporal distribution model of obstacles in each sub-region, the first inspection route in each sub-region is filtered to obtain the second inspection route in each sub-region. The second inspection route in each sub-region is optimized with the goals of minimizing route risk, shortening route length, and minimizing inspection time, thus obtaining the power grid inspection route in each sub-region.

10. The apparatus according to claim 7, characterized in that, The trigger detection conditions include: Acquire visible light and infrared images of the power grid inspection route; The visible light image is subjected to texture analysis to obtain the texture complexity of the visible light image, and the infrared image is subjected to temperature gradient analysis to obtain the temperature gradient of the infrared image; Visual change information is determined based on the texture complexity of the visible light image, and temperature change information is determined based on the temperature gradient of the infrared image; The visual change information and the temperature change information are correlated and analyzed according to the time series. If they do not meet the preset normal correlation mode, the detection condition is triggered.

11. The apparatus according to claim 7, characterized in that, The obstacle detection based on the first and second contour features to be detected, to obtain the obstacle detection result, includes: Based on the contour line segment direction, contour length, and contour topology of the first and second contour features to be detected, respectively, the direction consistency index, length ratio index, and topology difference degree are determined. Based on the directional consistency index, the length ratio index, and the topological structure difference, obstacle detection is performed in conjunction with the contour feature patterns of historical obstacles to obtain obstacle detection results.

12. The apparatus according to claim 11, characterized in that, The obstacle detection is performed based on the directional consistency index, the length ratio index, and the topological structure difference, combined with the contour feature patterns of historical obstacles, to obtain the obstacle detection results, including: The first and second contour features to be detected are matched with the contour feature patterns of historical obstacles to obtain the corresponding pattern matching degree. The first and second contour features to be detected are respectively subjected to association space and feature association analysis to determine the collaborative relationship index between the two contour features; Obstacle detection is performed based on the directional consistency index, the length ratio index, the topology difference, the pattern matching degree, and the cooperative relationship index to obtain the obstacle detection results.

13. An electronic device, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.

Images