Parallel line repeated alarm filtering method and system based on multi-sensor parameters
By using a multi-sensor parameter approach, combining visual and spatial features, the overlapping areas of parallel lines are identified and subjected to dual filtering, which solves the problem of repeated alarms in complex power grids and improves the accuracy of fault identification and the efficiency of power grid monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER CO
- Filing Date
- 2026-06-24
- Publication Date
- 2026-07-21
AI Technical Summary
In ultra-high voltage AC/DC hybrid power grids, the complex coupling relationship of multiple circuits on the same tower can cause a single fault to spread rapidly into a major power outage. Existing technologies are difficult to effectively distinguish between real faults and repeated alarms, and the computational load is large and the false alarm rate is high, which affects the efficiency of fault handling and power grid safety.
By employing multi-sensor parameters and acquiring multi-source data of parallel transmission lines, including visual images, ground point clouds, tower point clouds, and conductor point clouds, the overlapping areas of multi-view images are determined. Combined with visual and spatial features, dual filtering is performed to filter out non-repeating hidden dangers and identify duplicate alarms.
It improves the processing efficiency of duplicate alarm filtering, reduces the false alarm and missed alarm rates, realizes fully automatic and intelligent filtering of parallel lines, and improves the accuracy of fault identification and the real-time judgment efficiency of power grid dispatch monitoring.
Smart Images
Figure CN122435549A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent monitoring technology for power transmission lines, and specifically relates to a method and system for filtering repetitive alarms on parallel lines based on multi-sensor parameters. Background Technology
[0002] With the expansion of ultra-high voltage AC / DC hybrid power grids, the number of multi-circuit lines on the same tower is increasing, leading to complex coupling relationships. This can cause a single fault to rapidly escalate into a major blackout. Simultaneously, visible light equipment deployed along transmission corridors monitors only a single channel, continuously reporting data at high frequency. This results in a massive amount of duplicate, false, and fluctuating alarms, severely interfering with real-time judgment by dispatch and monitoring personnel. Traditional alarm mechanisms, such as manual identification or simple threshold stacking, cannot distinguish between genuine faults and multiple triggers of events from the same source, nor can they account for the electromagnetic coupling and spatial correlation between parallel lines. This results in a coexistence of "alarm surges" and "missed reporting blind spots," directly impacting fault handling efficiency and grid safety. Therefore, there is an urgent need for an intelligent algorithm that can integrate multi-sensor parameters, identify the authenticity of alarms online, and perform dual filtering of duplicate alarms from parallel lines. This would improve the accuracy of fault reporting, compress invalid information, and enable rapid perception and accurate decision-making regarding complex power grid faults in the context of integrated control.
[0003] In existing technologies, some solutions acquire a set of images to be detected in the target scene, extract and match two-dimensional feature points, and then perform three-dimensional reconstruction based on the matching relationships and images to obtain a three-dimensional scene point cloud. Finally, common-view region detection is performed on the defect-annotated areas to determine duplicate defect regions. The drawback is that it only uses two images from different shooting positions for hazard reconstruction, failing to consider the problem of large errors in feature matching and reconstructed point clouds when the shooting positions are far apart. Other solutions fuse laser point clouds with two-dimensional images, perform hazard identification bounding boxes on the real-time images, obtain three-dimensional coordinate information from the fused data, calculate the spatial coordinates of the hazard, and determine duplicate hazards based on all hazard spatial coordinates. The drawback is that it only uses spatial features as a factor for determining duplicate hazards, resulting in a single determination factor. Furthermore, it requires calculating the spatial coordinates of each hazard, leading to a large computational load. Additionally, large regression errors in the hazard target detection boxes can significantly affect the acquisition of hazard spatial information. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention provides a method and system for filtering repetitive alarms on parallel lines based on multiple sensor parameters.
[0005] The present invention adopts the following technical solution.
[0006] The first aspect of the present invention provides a method for filtering recurring alarms on parallel lines based on multiple sensor parameters, comprising: Acquire multi-source data of two parallel transmission lines at adjacent towers, including visualization images, ground point clouds, tower point clouds, conductor point clouds, and sensor parameters of the visualization devices, collected by visualization devices installed on the two parallel transmission lines respectively. Based on sensor parameters, ground point cloud, tower point cloud and conductor point cloud, the multi-view image overlap area between the visualization images of two parallel transmission lines is determined. The visualized image is subjected to hazard detection to generate hazard boxes, and the first layer of filtering is performed: for each hazard box, it is determined whether its preset representative point is located in the overlapping area of the corresponding multi-view image. If not, the hazard is determined to be a non-repeating hazard and filtered out. Visual and spatial features of the hazards retained after the first filtering are extracted. The spatial features include the three-dimensional physical dimensions of the hazards recovered based on the mapping relationship between monocular images and tower spans. A second layer of filtering is performed to identify suspected repeating hazards based on visual characteristics. When the three-dimensional physical size deviation between the suspected repeating hazard and the target hazard is within a preset threshold range, it is determined to be a repeating alarm.
[0007] Optionally, determining the multi-view image overlap region between the visualization images of two parallel transmission lines includes: The camera intrinsic parameter matrix is generated based on the focal length data, photosensitive element size data, and image resolution data from the sensor parameters of the visualization device, and the camera rotation matrix is generated based on the attitude data from the sensor parameters; the tower point cloud data and the conductor point cloud data are projected onto the two-dimensional image plane through the current camera parameters to obtain the projection points; The semantic segmentation algorithm is used to identify pixels in the tower area and the conductor area from the visualized image. With the goal of minimizing the total projection error between the projection point and the pixels in the tower area and the pixels in the conductor area, the camera translation matrix is iteratively optimized within the preset search space to obtain the optimized camera translation matrix. Each spatial point in the ground point cloud is projected and transformed using the camera intrinsic parameter matrix, camera rotation matrix, and optimized camera translation matrix of the corresponding transmission line to obtain the pixel coordinates of each spatial point in the visualization images of the two transmission lines. The set of pixel coordinates within the effective boundary range of each of the two power transmission lines' visualization images is determined as the overlapping area of the multi-view images.
[0008] Optionally, the preset representative point is the bottom center point of the hazard frame, and the execution of the first layer of filtering includes: Obtain the coordinates of the bottom center point of the hazard detection bounding box in the pixel coordinate system; Determine whether the coordinates of the bottom center point are within the set of pixel coordinates defined by the overlapping area of the multi-view images; If the coordinates of the bottom center point are not within the overlapping area of the multi-view images, it is determined that the hidden danger can only be observed by the visualization equipment of a single transmission line, and the hidden danger is determined to be a non-repeating hidden danger.
[0009] Optionally, obtaining the 3D physical dimensions of the potential hazard based on the mapping relationship between monocular images and tower spans includes: Input the visualized image containing potential hazards into the monocular depth estimation model to obtain a relative depth map that represents the relative distance between pixels; The known tower span parameters between the preset towers and the opposite towers in two parallel transmission lines are obtained, and the pixel coordinates of the tower center point in the visualization image are determined by the tower segmentation algorithm. The corresponding depth reference value of the pixel coordinates in the relative depth map is obtained. Based on the mapping ratio between the tower span parameter and the depth reference value, the relative depth value of each pixel in the relative depth map is converted into an absolute depth value representing the real spatial distance, thus generating an absolute depth map. The instance segmentation model is used to extract the pixel contour of the hazard from the hazard box and determine the hazard geometric feature points of the pixel contour of the hazard. The hazard geometric feature points include the highest point, lowest point, leftmost point and rightmost point. Based on the pixel coordinates of each hazard geometric feature point, the absolute depth value of the corresponding position in the absolute depth map, the camera intrinsic parameter matrix, and the extrinsic parameter matrix composed of the camera rotation matrix and the camera translation matrix, the physical coordinates of each hazard geometric feature point in the three-dimensional spatial coordinate system are calculated by collinearity equations. The height of the hazard is calculated based on the three-dimensional spatial coordinates of the highest and lowest points, and the width of the hazard is calculated based on the three-dimensional spatial coordinates of the leftmost and rightmost points.
[0010] Optionally, before obtaining the three-dimensional physical dimensions of the potential hazard, a self-verification process for the tower span is also included: Determine whether the x-coordinate of the bottom center point of the hazard box in the pixel coordinate system is greater than the x-coordinate of the tower center point; If the x-coordinate of the bottom center point is greater than the x-coordinate of the tower center point, the location distance of the hazard is determined to be within the effective range limited by the current tower span, and the three-dimensional physical dimension acquisition of the hazard continues; otherwise, the location distance of the hazard is determined to be beyond the effective range, and the hazard is filtered out.
[0011] Optionally, the two parallel transmission lines are designated as the first transmission line and the second transmission line, respectively. The extraction of visual features includes: The images of the hidden danger areas of the first transmission line and the second transmission line, which were retained after the first filtering, are respectively input into the feature matching model to extract the image feature points in each image. The image feature points in the images of the hidden danger areas of the first transmission line and the images of the hidden danger areas of the second transmission line are matched to obtain a list of visual feature matching point pairs between the two images of the hidden danger areas. In the list of visual feature matching points, the number of visual feature matching point pairs in which the pixel coordinates of the image feature points of the first power transmission line hazard area image fall within the boundary range of the first power transmission line hazard box, and the pixel coordinates of the image feature points of the second power transmission line hazard area image fall within the boundary range of the second power transmission line hazard box, is used as the visual feature quantification value between the hazard of the first power transmission line and the hazard of the second power transmission line.
[0012] Optionally, a second layer of filtering is performed to identify potential duplicate risks based on visual characteristics, including: For each target hazard retained after the first layer of filtering in the visualization image of the first transmission line, traverse all candidate hazard retained after the first layer of filtering in the visualization image of the second transmission line. The number of visual feature matching point pairs between the target hazard and each candidate hazard is obtained respectively; The candidate hidden danger of the second transmission line with the largest number of visual feature matching point pairs is identified as the suspected duplicate hidden danger corresponding to the target hidden danger.
[0013] Optionally, the condition for determining duplicate alarms in the second layer of filtering is: Obtain the height and width values of the target hazard, as well as the height and width values of suspected repeating hazards; Calculate the first ratio of the height of the suspected repeating hazard to the height of the target hazard, and the second ratio of the width of the suspected repeating hazard to the width of the target hazard; If both the first ratio and the second ratio are within the preset ratio range, the target hazard and the suspected duplicate hazard are determined to be a duplicate alarm of the same physical hazard; if either ratio is not within the preset ratio range, the target hazard and the suspected duplicate hazard are determined to be a non-duplicate hazard.
[0014] Optionally, the method further includes a process for determining the attribution of hazards: Extract conductor point cloud subsets corresponding to the first and second transmission lines from the multi-source data respectively; Obtain the coordinates of the reference feature point of the hidden danger in three-dimensional space, and calculate the first clearance distance from the reference feature point to the first transmission line conductor point cloud subset and the second clearance distance to the second transmission line conductor point cloud subset respectively. Calculate the ratio of the first clearance distance to the second clearance distance; If the ratio exceeds the preset attribution threshold, the hazard is determined to be a single-line threat that only threatens a single transmission line; if the ratio does not exceed the preset attribution threshold, the hazard is determined to be a cross-line threat that threatens two transmission lines simultaneously.
[0015] A second aspect of the present invention provides a parallel line repeat alarm filtering system based on multi-sensor parameters, used to implement the parallel line repeat alarm filtering method based on multi-sensor parameters described in the first aspect of the present invention, comprising: The module comprises a data acquisition module, an overlapping region determination module, a first filtering module, a feature extraction module, and a second filtering module, wherein: The data acquisition module is used to acquire multi-source data of two parallel transmission lines at adjacent towers. The multi-source data includes visualization images, ground point clouds, tower point clouds, conductor point clouds, and sensor parameters of the visualization equipment, which are collected by visualization devices installed on the two parallel transmission lines respectively. The overlapping area determination module is used to determine the multi-view image overlap area between the visualized images of two parallel transmission lines based on sensor parameters, ground point cloud, tower point cloud and conductor point cloud. The first filtering module is used to detect potential hazards in the visualized image to generate hazard boxes. For each hazard box, it determines whether its preset representative point is located in the overlapping area of the corresponding multi-view image. If not, the hazard is determined to be a non-repeating hazard and filtered out. The feature extraction module is used to extract the visual and spatial features of the hazards retained after the first layer of filtering. The spatial features include the three-dimensional physical dimensions of the hazards recovered based on the mapping relationship between monocular images and tower spans. The second filtering module is used to identify suspected repeating hazards based on visual features, and to determine a repeating alarm when the three-dimensional physical size deviation between the suspected repeating hazard and the target hazard is within a preset threshold range.
[0016] Compared with the prior art, the beneficial effects of the present invention include at least the following: 1. This invention uses a first-level filter to determine whether the representative point of the hazard box is located within the common viewing area based on the overlapping area of multi-view images. This can quickly filter out non-repeating hazards that can only be observed by the visualization equipment of a single transmission line, significantly reducing the computational load of subsequent feature extraction and matching, and improving the overall processing efficiency of repeated alarm filtering.
[0017] 2. This invention uses a second layer of filtering combined with visual feature matching points of potential hazards to make dual judgments on the quantity and three-dimensional physical size deviation. It adds image feature comparison on the basis of spatial features, which overcomes the defect that a single judgment factor is easily affected by the regression error of the target detection box, making the identification of repeated hazards more accurate and sufficient, and effectively reducing the false alarm and missed alarm rates.
[0018] 3. This invention employs a method for recovering the three-dimensional physical dimensions of potential hazards based on the mapping relationship between monocular images and tower spans. It uses the known tower spans as a scale benchmark to convert the relative depth map into an absolute depth map. The true height and width of the potential hazard can be calculated from a single image, avoiding the problem of large reconstruction errors in traditional binocular stereo matching when the shooting position distance is large, and improving the robustness of spatial feature extraction.
[0019] 4. This invention integrates multi-source data such as visualized images, laser point clouds, and sensor parameters, and combines camera intrinsic parameter matrix, rotation matrix, and optimized translation matrix to accurately determine overlapping areas. This achieves fully automatic and intelligent dual filtering of repeated alarms on parallel transmission lines, providing reliable technical support for intelligent inspection of transmission lines and improving the real-time judgment efficiency of dispatch and monitoring personnel. Attached Figure Description
[0020] Figure 1 This is a flowchart of a method provided according to an embodiment of the present invention. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.
[0022] In Embodiment 1, this invention provides a method for filtering recurring alarms on parallel lines based on multiple sensor parameters, such as... Figure 1 As shown, it includes the following steps: Step 1: Obtain multi-source data for the two parallel transmission lines at adjacent towers, including visualization images, ground point clouds, tower point clouds, conductor point clouds, and sensor parameters of the visualization devices, all collected by visualization equipment located on the two parallel transmission lines.
[0023] Preferably, the sensor parameters of the visualization device include the focal length, CCD size, and attitude data of the visualization sensor.
[0024] Preferably, the parallel lines refer to double-circuit or multi-circuit transmission lines on the same tower, as well as lines that are erected parallel to existing lines over a long distance. The adjacent towers refer to adjacent or corresponding towers in densely populated corridors as defined by the State Grid or China Southern Power Grid Company, belonging to the same direction but belonging to different transmission lines, and spatially close to each other, with horizontal distances conforming to regulations.
[0025] Preferably, the multi-source data further includes pole / tower point cloud data and conductor point cloud data. The pole / tower point cloud, conductor point cloud, and ground point cloud data are acquired by a UAV equipped with a lidar system, with the coordinate system established with the line pole / tower as the origin. The pole / tower point cloud mainly refers to the pole / tower facing the visualization device.
[0026] Specifically, the visualized image data is captured by visualization monitoring equipment installed on transmission line towers to monitor changes in the transmission corridor scene. The focal length and CCD size of the visualization sensor are determined by the camera hardware structure, describing the camera's own geometric characteristics and imaging relationship, and are generally fixed after leaving the factory. Attitude data, including yaw angle, pitch angle, and roll angle, can be acquired in real time through attitude sensors.
[0027] Step 2: Based on sensor parameters, ground point cloud, tower point cloud, and conductor point cloud, determine the multi-view image overlap area between the visualized images of the two parallel transmission lines.
[0028] Preferably, determining the multi-view image overlap region between the visualized images of two parallel transmission lines includes: The camera intrinsic parameter matrix is generated based on the focal length data, photosensitive element size data, and image resolution data from the sensor parameters of the visualization device, and the camera rotation matrix is generated based on the attitude data from the sensor parameters; the tower point cloud data and the conductor point cloud data are projected onto the two-dimensional image plane through the current camera parameters to obtain the projection points; The semantic segmentation algorithm is used to identify pixels in the tower area and the conductor area from the visualized image. With the goal of minimizing the total projection error between the projection point and the pixels in the tower area and the pixels in the conductor area, the camera translation matrix is iteratively optimized within the preset search space to obtain the optimized camera translation matrix. Each spatial point in the ground point cloud is projected and transformed using the camera intrinsic parameter matrix, camera rotation matrix, and optimized camera translation matrix of the corresponding transmission line to obtain the pixel coordinates of each spatial point in the visualization images of the two transmission lines. The set of pixel coordinates within the effective boundary range of each of the two power transmission lines' visualization images is determined as the overlapping area of the multi-view images.
[0029] More preferably, the attitude data includes yaw angle, pitch angle, and roll angle.
[0030] Specifically, the camera intrinsic parameter matrix is a 3×3 matrix composed of focal length, CCD size, and image resolution, describing the camera's own geometric characteristics. The camera rotation matrix is a 3×3 matrix obtained by transforming Euler angles acquired from the pose sensor, describing the camera's attitude.
[0031] More preferably, the pose optimization algorithm adopts the ant colony algorithm, and the semantic segmentation algorithm adopts the YOLOv8_seg model.
[0032] Specifically, the pose optimization algorithm employs an ant colony algorithm. The goal is to minimize the total projection error between the projection points of the tower and conductor point cloud data projected onto the 2D image using the current camera parameters and the corresponding cross-dimensional matching point pairs of the tower and conductor segmented from the visualized image using a semantic segmentation algorithm. This is achieved by optimizing the camera translation matrix within a defined search space. The cross-dimensional matching point pairs are derived from regional feature points extracted from the vertices, edges, and centroids of objects such as conductors and towers after obtaining the semantic segmentation results. These feature points are used for matching and error calculation with the point cloud projection points. The semantic segmentation algorithm can employ the YOLOv8_seg model, with its training dataset consisting of manually labeled transmission line images categorized as towers and conductors.
[0033] More preferably, the objective function for the projection error is defined as the sum of squared pixel coordinate errors of all cross-dimensional matching point pairs, i.e.:
[0034] Where n is the number of corresponding point pairs matched across dimensions. These are the actual pixel coordinates of the i-th tower region pixel or conductor region pixel identified from the visualized image using a semantic segmentation algorithm. For tower point cloud data or conductor point cloud data and The pixel coordinates of the corresponding 3D point are obtained by projecting the camera intrinsic parameters, rotation matrix, and translation matrix.
[0035] Specifically, the search space is set as follows: The variable to be optimized is Where r (roll angle) ranges from The range of p (pitch angle) is: The range of y (yaw angle) is: , The range is determined based on the extreme coordinate values of the point cloud of the tower where the visualization device is located.
[0036] The key parameters of the ant colony algorithm are set as follows: The ant colony has 80 members, a pheromone factor of 1.2, a heuristic function importance factor of 2.5, a pheromone volatility factor of 0.11, an initial pheromone concentration of 1, 300 iterations, a local search probability of 0.1, and 30 discretized nodes.
[0037] The termination condition is: The objective function value of the current optimal solution is less than 1 (empirical value), or the number of iterations exceeds 300, or the error change of the optimal solution is less than 0.02 (empirical value) over 15 consecutive iterations.
[0038] Specifically, the training parameters for the semantic segmentation model YOLOv8_seg are set as follows: training batch size 100, input image size 1200×900, batch size 2, optimizer SGD, initial learning rate 0.01, and momentum 0.93. Specifically, the projection transformation is based on the camera imaging principle, that is, for any point in the ground point cloud... Its pixel coordinates on the image satisfy:
[0039] in, For the camera intrinsic parameter matrix, Let be the extrinsic parameter matrix. For each ground point, let its mapped coordinates on the two route images be... and All fall within the effective size range of the corresponding image (i.e.) , If these pixels form part of the multi-view overlapping area of the two images, then these pixels constitute part of the multi-view overlapping area of the two images.
[0040] Step 3: Perform hazard detection on the visualized image to generate hazard boxes, and perform the first filtering: For each hazard box, determine whether its preset representative point is located in the overlapping area of the corresponding multi-view image. If not, the hazard is determined to be a non-repeating hazard and filtered out.
[0041] Preferably, the preset representative point is the bottom center point of the hazard frame, and the execution of the first layer of filtering includes: Obtain the coordinates of the bottom center point of the hazard detection bounding box in the pixel coordinate system; Determine whether the coordinates of the bottom center point are within the set of pixel coordinates defined by the overlapping area of the multi-view images; If the coordinates of the bottom center point are not within the overlapping area of the multi-view images, it is determined that the hidden danger can only be observed by the visualization equipment of a single transmission line, and the hidden danger is determined to be a non-repeating hidden danger.
[0042] More preferably, the hazard detection is implemented using a pre-trained hazard detection model. Specifically, the hazard detection model employs the RT-DETR algorithm, and its training dataset contains typical hazard labels for power transmission lines, such as cranes, excavators, trucks, and wildfires. Hazard bounding box The rectangle is an axis-aligned bounding box whose geometric features are uniquely determined by four basic pixel parameters. All parameters are based on the pixel coordinate system of the inspected image / video frame (origin is the upper left corner of the screen, X-axis is to the right, and Y-axis is down). The left boundary of the hazard box is located on the X-axis of the pixel coordinate system. The upper boundary of the potential hazard box is located on the Y-axis in the pixel coordinate system. The right boundary of the potential hazard box is located on the X-axis of the pixel coordinate system. The Y-axis coordinate of the lower boundary of the hazard box in the pixel coordinate system.
[0043] Specifically, the dataset format is the COCO dataset format, and the model training parameters are: training batch size of 200, batch size of 4, and learning rate of 0.0001. The formula for calculating the coordinates of the preset representative point (bottom center point) of the potential hazard is as follows:
[0044] Step 4: Extract the visual and spatial features of the hazards retained after the first filtering. The spatial features include the three-dimensional physical dimensions of the hazards recovered based on the mapping relationship between monocular images and tower spans.
[0045] Preferably, the process of obtaining the three-dimensional physical dimensions of the hidden danger based on the mapping relationship between monocular images and tower spans includes: Input the visualized image containing potential hazards into the monocular depth estimation model to obtain a relative depth map that represents the relative distance between pixels; Obtain the known tower span parameters between the preset tower and the opposite tower in the two parallel transmission lines, and determine the pixel coordinates of the tower center point in the visualization image and the corresponding depth reference value of the pixel coordinates in the relative depth map through the tower segmentation algorithm; Based on the mapping ratio between the tower span parameter and the depth reference value, the relative depth value of each pixel in the relative depth map is converted into an absolute depth value representing the real spatial distance, thereby generating an absolute depth map. The instance segmentation model is used to extract the pixel contour of the hazard from the hazard box, and the hazard geometric feature points of the pixel contour of the hazard are determined. The hazard geometric feature points include the highest point, the lowest point, the leftmost point and the rightmost point. Based on the pixel coordinates of each hazard geometric feature point, the absolute depth value of the corresponding position in the absolute depth map, the camera intrinsic parameter matrix, and the extrinsic parameter matrix composed of the camera rotation matrix and the camera translation matrix, the physical coordinates of each hazard geometric feature point in the three-dimensional spatial coordinate system are calculated by collinearity equations. The height of the hazard is calculated based on the three-dimensional spatial coordinates of the highest and lowest points, and the width of the hazard is calculated based on the three-dimensional spatial coordinates of the leftmost and rightmost points.
[0046] More preferably, the relative depth map is obtained through the SAM-3D model, and the absolute depth map is mapped by the known tower span and the depth value of the tower center point in the relative depth map; the hazard area segmentation is achieved through the SAM-2 model.
[0047] More preferably, before obtaining the three-dimensional physical dimensions of the potential hazard, a tower span self-verification process is also included: Determine whether the horizontal coordinate value of the bottom center point of the hazard box in the pixel coordinate system is greater than the horizontal coordinate value of the tower center point; If the x-coordinate value of the bottom center point is greater than the x-coordinate value of the tower center point, it is determined that the location distance of the hidden danger does not exceed the effective range limited by the current tower span, and the three-dimensional physical dimension acquisition of the hidden danger continues. If not, the location distance of the hazard is determined to be outside the effective range, and the hazard is filtered out.
[0048] Specifically, the tower span self-verification involves filtering potential hazards based on the tower's position in the image, eliminating hazards that are too far away (exceeding the current span, i.e., the x-coordinate of the hazard's bottom center point is greater than the x-coordinate of the tower's center point) and are prone to location failure. The acquisition of the three-dimensional dimensions needs to be performed separately for both parallel transmission lines.
[0049] Specifically, the detailed steps for obtaining the three-dimensional dimensions are as follows: The image containing the potential hazard is input into a monocular depth estimation algorithm to obtain a relative depth map of the same size as the input image. (m) The matrix n, where m and n are the length and width of the input image, and the relative depth map ranges from 0 to 1, where 0 represents the point closest to the camera and 1 represents the farthest point in the scene captured in the current image.
[0050] Obtain the span distance dd of the transmission line towers (here, the distance between the tower where the visualization device is located and the next tower opposite it). Segment the image using a tower segmentation algorithm to obtain the tower pixel set, and calculate the coordinates of the tower center point.
[0051] in, The set of x-axis coordinates representing the pixel set of the tower. The set of y-axis coordinates representing the set of pixels on the pole.
[0052] Obtain the depth value tower_d of the center point in the relative depth map. The depth value tower_d is directly read from the corresponding pixel coordinates in the relative depth map.
[0053] Mapping the relative depth map to the absolute depth map Maximum absolute depth The formula, where multiplying by 1 represents the theoretical maximum value of the relative depth map, 1, reflects the proportional mapping relationship of "actual maximum depth / relative maximum depth = actual depth / relative depth".
[0054] Then each pixel in the absolute depth map The value is .
[0055] Using an instance segmentation model, the image and the hazard bounding box are input to obtain the precise segmentation region of the hazard, danger_seg.
[0056] Determine the coordinates of the characteristic points of the potential hazard, including the highest point, lowest point, leftmost point, and rightmost point: Highest point:
[0057] Lowest point (center of bottom):
[0058] Leftmost point:
[0059] Rightmost point:
[0060] in, The precise segmentation of the potential hazard area is defined by the coordinates along the x-axis. The precise segmentation area for identifying potential hazards is defined by the coordinates along the y-axis.
[0061] Specifically, the three-dimensional spatial coordinates of the geometric feature points of the hidden danger are solved by the following collinearity equation:
[0062] in, In , , The three-dimensional spatial coordinates of each calculated geometric feature point of the potential hazard are: The pixel coordinates of the geometric feature points of the potential hazard. This is the absolute depth value corresponding to that pixel. For the camera intrinsic parameter matrix, This is the camera extrinsic parameter matrix.
[0063] Calculate the distance between the leftmost and rightmost points as the width of the hazard, and calculate the distance between the highest and lowest points as the height.
[0064] Preferably, the two parallel transmission lines are respectively designated as the first transmission line and the second transmission line, and the visual feature extraction process includes: The images of the hidden danger areas of the first transmission line and the second transmission line, which were retained after the first filtering, are respectively input into the feature matching model to extract the image feature points in each image. The image feature points in the images of the hidden danger areas of the first transmission line and the images of the hidden danger areas of the second transmission line are matched to obtain a list of visual feature matching point pairs between the two images of the hidden danger areas. In the list of visual feature matching points, the number of visual feature matching point pairs in which the pixel coordinates of the image feature points of the first power transmission line hazard area image fall within the boundary range of the first power transmission line hazard box, and the pixel coordinates of the image feature points of the second power transmission line hazard area image fall within the boundary range of the second power transmission line hazard box, is used as the visual feature quantification value between the hazard of the first power transmission line and the hazard of the second power transmission line.
[0065] Specifically, the visual features include the number of feature matching points. The number of feature matching points between potential hazards is obtained through a feature matching algorithm, and the feature matching model adopts the Roma algorithm. The input to the feature matching model is an image of the hazard area of the first transmission line and an image of the hazard area of the second transmission line, wherein the image of the hazard area of the first transmission line is used as the reference image, and the image of the hazard area of the second transmission line is used as the image to be matched. The output of the feature matching model is the image feature point set of the baseline image. Image feature point set of the image to be matched This involves the correspondence between image feature points in the baseline image and the image to be matched. Each pair of matching image feature points in the correspondence constitutes a visual feature matching point pair.
[0066] Specifically, the number of visual feature matching point pairs is counted as follows: After obtaining the visual feature matching point pairs, each pair of visual feature matching points is checked one by one to determine whether the pixel coordinates of the image feature points of the reference image are located within the pixel area defined by the first power transmission line hazard box, and whether the pixel coordinates of the image feature points of the image to be matched are located within the pixel area defined by the second power transmission line hazard box. The visual feature matching point pair is counted as valid only when the pixel coordinates of the two image feature points in the same visual feature matching point pair fall within the boundary range of their respective hazard boxes. The total number of valid visual feature matching point pairs obtained in the final statistics is the visual feature quantification value between the hidden dangers of the first transmission line and the hidden dangers of the second transmission line.
[0067] Step 5: Perform the second layer of filtering. Based on the visual features, identify suspected repeating hazards. When the three-dimensional physical size deviation between the suspected repeating hazard and the target hazard is within a preset threshold range, it is determined to be a repeating alarm.
[0068] Preferably, the process of determining suspected recurring risks based on the visual features includes: For each target hazard retained after the first layer of filtering in the visualization image of the first transmission line, traverse all candidate hazard retained after the first layer of filtering in the visualization image of the second transmission line. The number of visual feature matching point pairs between the target hazard and each candidate hazard is obtained respectively; The candidate hidden danger of the second transmission line with the largest number of visual feature matching point pairs is identified as the suspected duplicate hidden danger corresponding to the target hidden danger.
[0069] Preferably, the determination of repeated alarms includes: Obtain the height and width values of the target hazard, as well as the height and width values of suspected repeating hazards; Calculate the first ratio of the height of the suspected repeating hazard to the height of the target hazard, and the second ratio of the width of the suspected repeating hazard to the width of the target hazard; If both the first ratio and the second ratio are within the preset ratio range, the target hazard and the suspected duplicate hazard are determined to be a duplicate alarm of the same physical hazard; if either ratio is not within the preset ratio range, the target hazard and the suspected duplicate hazard are determined to be a non-duplicate hazard.
[0070] Preferably, the preset ratio range can be set according to the actual application scenario, for example, 90% to 110%.
[0071] Preferably, the method further includes a process for determining the attribution of potential hazards: Extract conductor point cloud subsets corresponding to the first and second transmission lines from the multi-source data respectively; Obtain the coordinates of the reference feature point of the hidden danger in three-dimensional space, and calculate the first clearance distance from the reference feature point to the first transmission line conductor point cloud subset and the second clearance distance to the second transmission line conductor point cloud subset respectively. Calculate the ratio of the first clearance distance to the second clearance distance; If the ratio exceeds the preset attribution threshold, the hazard is determined to be a single-line threat that only threatens a single transmission line; if the ratio does not exceed the preset attribution threshold, the hazard is determined to be a cross-line threat that threatens two transmission lines simultaneously.
[0072] Specifically, the reference feature points of the hazard in three-dimensional space can be selected from the geometric feature points of the hazard obtained in step 4. For example, the three-dimensional spatial coordinates of the geometric feature points of the hazard can be selected. The point with the largest value of the axis component is used as the reference feature point.
[0073] When calculating the first clearance distance and the second clearance distance, the spatial distance between the reference feature point and each point in the first transmission line conductor point cloud sub-set is calculated respectively, and the smallest spatial distance is taken as the first clearance distance; similarly, the spatial distance between the reference feature point and each point in the second transmission line conductor point cloud sub-set is calculated, and the smallest spatial distance is taken as the second clearance distance.
[0074] After obtaining the first and second clearance distances, the larger and smaller values are taken, and the ratio of the larger to the smaller value is calculated. The preset attribution determination threshold can be set according to the actual application scenario; for example, it can be set to 2. If the ratio is greater than 2, the hazard is determined to pose a threat only to the single transmission line with a smaller clearance distance. If the ratio is less than or equal to 2, the hazard is determined to pose a threat to both transmission lines simultaneously, i.e., a cross-line threat. Cross-line threats will be given priority in receiving alerts and will be addressed promptly.
[0075] In Embodiment 2, this invention provides a parallel line repetitive alarm filtering system based on multiple sensor parameters, used to implement the method described in Embodiment 1. The system includes a data acquisition module, an overlapping region determination module, a first filtering module, a feature extraction module, and a second filtering module, wherein: The data acquisition module is used to acquire multi-source data of two parallel transmission lines at adjacent towers. The multi-source data includes visualization images, ground point clouds, tower point clouds, conductor point clouds, and sensor parameters of the visualization devices respectively collected by visualization devices installed on the two parallel transmission lines. The overlapping area determination module is used to determine the multi-view image overlap area between the visualized images of two parallel transmission lines based on the sensor parameters, the ground point cloud, the tower point cloud, and the conductor point cloud. The first filtering module is used to perform hazard detection on the visualized image to generate hazard boxes, and for each hazard box, determine whether its preset representative point is located in the corresponding multi-view image overlap area. If not, the hazard is determined to be a non-repeating hazard and filtered out. The feature extraction module is used to extract the visual and spatial features of the hazards retained after the first filtering. The spatial features include the three-dimensional physical dimensions of the hazards recovered based on the mapping relationship between monocular images and tower spans. The second filtering module is used to determine suspected repeating hazards based on the visual features, and to determine a repeating alarm when the three-dimensional physical size deviation between the suspected repeating hazard and the target hazard is within a preset threshold range.
[0076] Preferably, the feature extraction module is also used to perform the tower span self-verification process and the hazard attribution determination process described in Embodiment 1.
[0077] To further illustrate the technical solution and beneficial effects of this invention, a detailed description is provided below with reference to a specific application example. The data in this example comes from an actual power transmission line inspection scenario in a certain region.
[0078] This example selects a certain span of two parallel transmission lines, 1 and 2, in a certain region as the test object.
[0079] Step 1: Acquire multi-source data Acquire point cloud data of conductors, towers, and ground for transmission lines 1 and 2. The point cloud data is acquired by a UAV equipped with a LiDAR. Visual image data of both lines is acquired, along with camera intrinsic parameter matrix, CCD size, camera focal length, and attitude data from the visualization monitoring equipment.
[0080] The intrinsic parameter matrix of the camera for the visual monitoring image capture device of transmission line 1 is as follows:
[0081] The intrinsic parameter matrix of the camera for the visual monitoring image capture equipment of transmission line 2 is as follows:
[0082] Step 2: Determine camera internal and external parameters The pose matrix of the visualization monitoring image equipment for transmission line 1 is obtained through the camera pose calculation module. Pose matrix of the visualization monitoring image equipment for transmission line 2 :
[0083] Step 3: Determine the overlapping region of multi-view images Ground point cloud data for:
[0084] Will By mapping the camera intrinsic parameter matrices of Line 1 and Line 2 to the device pose and then onto the images, the mapped depth information and the multi-view overlapping area on the two line devices are obtained. Specifically, the ground depth information of the converted Line 1 visual monitoring image and the converted Line 2 visual monitoring image are as follows:
[0085]
[0086] The multi-view overlapping areas on Line 1 and Line 2 are as follows:
[0087]
[0088] Filter out the pixel coordinates that fall within the effective size range of the image (pixel coordinates whose x and y coordinates are both smaller than the image width and height) to obtain the actual overlapping area:
[0089] Step 4: Hazard Detection The hidden danger 1 in the visual monitoring image of transmission line 1 was detected by the transmission line hidden danger detection module, and the coordinates of the hidden danger frame are as follows: Bottom center point The center point coordinates of the tower are [1086, 1084]. Hazard 1 is detected in the visual monitoring image of transmission line 2, and its bounding box coordinates are... Bottom center point The coordinates of the center point of the tower are [1070, 451].
[0090] Step 5: First Filtering judge lie in Inside, lie in Therefore, both potential hazards were retained.
[0091] Step 6: Extract hazard characteristics The hazard bounding box is obtained using the Roma feature matching algorithm. and The number of feature matching points between them is 29.
[0092] The dimensions of the hazard were obtained using vision-based 3D reconstruction and measurement technology. The span of the transmission line tower is dd = 300 meters, and the depth of the tower center point in the relative depth map is 0.6. The calculated height of hazard 1 is 3.75 meters and the width is 7.5 meters; the height of hazard 2 is 3.9 meters and the width is 7.3 meters.
[0093] Step 7: Second Filtering Since there is only one hidden danger in each of the two pictures, and the horizontal coordinate of the bottom center point of hidden danger 1, line1_cen1[1]=1426, is greater than the horizontal coordinate of the center point of the tower, 1084, and the horizontal coordinate of the bottom center point of hidden danger 2, line1_cen2[1]=968, is greater than the horizontal coordinate of the center point of the tower, 451, the self-check of the tower span of the hidden danger is fine, and hidden danger 2 is the suspected duplicate hidden danger of hidden danger 1. Comparing the two dimensions: the height of hidden danger 2, 3.9 meters, is within 90% to 110% of the height of hidden danger 1, 3.75 meters (3.375 meters to 4.125 meters), and the width, 7.3 meters, is within 90% to 110% of the width of hidden danger 1, 7.5 meters (6.75 meters to 8.25 meters). Therefore, it is determined that the two are the same physical hidden danger, that is, duplicate alarm.
[0094] Step 8: Determining the Attribution of Hazards Select the geometric feature points of hazard 1 in three-dimensional space coordinates. Using the highest point as a reference feature point, the distances between it and the point cloud of conductor 1 and the point cloud of conductor 2 are calculated to be 70.35m and 33.21m respectively. The ratio 70.35 / 33.21=2.11, indicating that the hidden danger only threatens a single line.
[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.
Claims
1. A method for filtering repetitive alarms on parallel lines based on multi-sensor parameters, characterized in that, include: Acquire multi-source data of two parallel transmission lines at adjacent towers, including visualization images, ground point clouds, tower point clouds, conductor point clouds, and sensor parameters of the visualization devices, collected by visualization devices installed on the two parallel transmission lines respectively; and determine the multi-view image overlap area between the visualization images of the two parallel transmission lines based on the sensor parameters, ground point clouds, tower point clouds, and conductor point clouds. The visualized image is used to detect potential hazards and generate hazard bounding boxes. The first layer of filtering is then performed: for each hazard bounding box, it is determined whether its preset representative point is located within the overlapping area of the corresponding multi-view image. If not, the hazard is identified as a non-repeating hazard and filtered out. The visual and spatial features of the hazards retained after the first layer of filtering are extracted. The spatial features include the three-dimensional physical dimensions of the hazard recovered based on the mapping relationship between the monocular image and the tower span. The second layer of filtering is then performed: suspected repetitive hazards are identified based on the visual features. When the deviation between the three-dimensional physical dimensions of the suspected repetitive hazard and the target hazard is within a preset threshold range, it is determined to be a repetitive alarm.
2. The method for filtering repetitive alarms on parallel lines based on multi-sensor parameters according to claim 1, characterized in that: Determining the overlapping region of multi-view images between visualizations of two parallel transmission lines includes: The camera intrinsic parameter matrix is generated based on the focal length data, photosensitive element size data, and image resolution data from the sensor parameters of the visualization device, and the camera rotation matrix is generated based on the attitude data from the sensor parameters; the tower point cloud data and the conductor point cloud data are projected onto the two-dimensional image plane through the current camera parameters to obtain the projection points; The semantic segmentation algorithm is used to identify pixels in the tower area and the conductor area from the visualized image. With the goal of minimizing the total projection error between the projection point and the pixels in the tower area and the pixels in the conductor area, the camera translation matrix is iteratively optimized within the preset search space to obtain the optimized camera translation matrix. Each spatial point in the ground point cloud is projected and transformed using the camera intrinsic parameter matrix, camera rotation matrix, and optimized camera translation matrix of the corresponding transmission line to obtain the pixel coordinates of each spatial point in the visualization images of the two transmission lines. The set of pixel coordinates within the effective boundary range of each of the two power transmission lines' visualization images is determined as the overlapping area of the multi-view images.
3. The method for filtering repetitive alarms on parallel lines based on multi-sensor parameters according to claim 2, characterized in that: The preset representative point is the bottom center point of the hazard frame, and the execution of the first layer of filtering includes: Obtain the coordinates of the bottom center point of the hazard detection bounding box in the pixel coordinate system; Determine whether the coordinates of the bottom center point are within the set of pixel coordinates defined by the overlapping area of the multi-view images; If the coordinates of the bottom center point are not within the overlapping area of the multi-view images, it is determined that the hidden danger can only be observed by the visualization equipment of a single transmission line, and the hidden danger is determined to be a non-repeating hidden danger.
4. The method for filtering repetitive alarms on parallel lines based on multi-sensor parameters according to claim 1, characterized in that: The acquisition of the 3D physical dimensions of potential hazards based on the mapping relationship between monocular images and tower spans includes: Input the visualized image containing potential hazards into the monocular depth estimation model to obtain a relative depth map that represents the relative distance between pixels; The known tower span parameters between the preset towers and the opposite towers in two parallel transmission lines are obtained, and the pixel coordinates of the tower center point in the visualization image are determined by the tower segmentation algorithm. The corresponding depth reference value of the pixel coordinates in the relative depth map is obtained. Based on the mapping ratio between the tower span parameter and the depth reference value, the relative depth value of each pixel in the relative depth map is converted into an absolute depth value representing the real spatial distance, thus generating an absolute depth map. The instance segmentation model is used to extract the pixel contour of the hazard from the hazard box and determine the hazard geometric feature points of the pixel contour of the hazard. The hazard geometric feature points include the highest point, lowest point, leftmost point and rightmost point. Based on the pixel coordinates of each hazard geometric feature point, the absolute depth value of the corresponding position in the absolute depth map, the camera intrinsic parameter matrix, and the extrinsic parameter matrix composed of the camera rotation matrix and the camera translation matrix, the physical coordinates of each hazard geometric feature point in the three-dimensional spatial coordinate system are calculated by collinearity equations. The height of the hazard is calculated based on the three-dimensional spatial coordinates of the highest and lowest points, and the width of the hazard is calculated based on the three-dimensional spatial coordinates of the leftmost and rightmost points.
5. The method for filtering repetitive alarms on parallel lines based on multi-sensor parameters according to claim 4, characterized in that: Before obtaining the three-dimensional physical dimensions of the potential hazard, a self-verification process for the tower span is also included: Determine whether the x-coordinate of the bottom center point of the hazard box in the pixel coordinate system is greater than the x-coordinate of the tower center point; If the x-coordinate of the bottom center point is greater than the x-coordinate of the tower center point, the location distance of the hazard is determined to be within the effective range limited by the current tower span, and the three-dimensional physical dimension acquisition of the hazard continues; otherwise, the location distance of the hazard is determined to be beyond the effective range, and the hazard is filtered out.
6. The method for filtering repetitive alarms on parallel lines based on multi-sensor parameters according to claim 1, characterized in that: Two parallel transmission lines are designated as the first transmission line and the second transmission line, respectively. Visual feature extraction includes: The images of the hidden danger areas of the first transmission line and the second transmission line, which were retained after the first filtering, are respectively input into the feature matching model to extract the image feature points in each image. The image feature points in the images of the hidden danger areas of the first transmission line and the images of the hidden danger areas of the second transmission line are matched to obtain a list of visual feature matching point pairs between the two images of the hidden danger areas. In the list of visual feature matching points, the number of visual feature matching point pairs in which the pixel coordinates of the image feature points of the first power transmission line hazard area image fall within the boundary range of the first power transmission line hazard box, and the pixel coordinates of the image feature points of the second power transmission line hazard area image fall within the boundary range of the second power transmission line hazard box, is used as the visual feature quantification value between the hazard of the first power transmission line and the hazard of the second power transmission line.
7. The method for filtering repetitive alarms on parallel lines based on multi-sensor parameters according to claim 6, characterized in that: A second layer of filtering is performed, identifying potential duplicate risks based on visual characteristics, including: For each target hazard retained after the first layer of filtering in the visualization image of the first transmission line, traverse all candidate hazard retained after the first layer of filtering in the visualization image of the second transmission line. The number of visual feature matching point pairs between the target hazard and each candidate hazard is obtained respectively; The candidate hidden danger of the second transmission line with the largest number of visual feature matching point pairs is identified as the suspected duplicate hidden danger corresponding to the target hidden danger.
8. The method for filtering repetitive alarms on parallel lines based on multi-sensor parameters according to claim 7, characterized in that: The criteria for determining duplicate alarms in the second layer of filtering are: Obtain the height and width values of the target hazard, as well as the height and width values of suspected repeating hazards; Calculate the first ratio of the height of the suspected repeating hazard to the height of the target hazard, and the second ratio of the width of the suspected repeating hazard to the width of the target hazard; If both the first ratio and the second ratio are within the preset ratio range, the target hidden danger and the suspected duplicate hidden danger are determined to be duplicate alarms of the same physical hidden danger; If any ratio is not within the preset ratio range, the target hidden danger and the suspected repeated hidden danger are determined to be non-repeating hidden dangers.
9. The method for filtering repetitive alarms on parallel lines based on multi-sensor parameters according to claim 1, characterized in that: The method also includes a process for determining the attribution of potential hazards: Extract conductor point cloud subsets corresponding to the first and second transmission lines from the multi-source data respectively; Obtain the coordinates of the reference feature point of the hidden danger in three-dimensional space, and calculate the first clearance distance from the reference feature point to the first transmission line conductor point cloud subset and the second clearance distance to the second transmission line conductor point cloud subset respectively. Calculate the ratio of the first clearance distance to the second clearance distance; If the ratio exceeds the preset attribution threshold, the hazard is determined to be a single-line threat that only threatens a single transmission line; if the ratio does not exceed the preset attribution threshold, the hazard is determined to be a cross-line threat that threatens two transmission lines simultaneously.
10. A parallel line repeat alarm filtering system based on multi-sensor parameters, used to implement the parallel line repeat alarm filtering method based on multi-sensor parameters as described in any one of claims 1-9, characterized in that, include: The module comprises a data acquisition module, an overlapping region determination module, a first filtering module, a feature extraction module, and a second filtering module, wherein: The data acquisition module is used to acquire multi-source data of two parallel transmission lines at adjacent towers. The multi-source data includes visualization images, ground point clouds, tower point clouds, conductor point clouds, and sensor parameters of the visualization equipment, which are collected by visualization devices installed on the two parallel transmission lines respectively. The overlapping area determination module is used to determine the multi-view image overlap area between the visualized images of two parallel transmission lines based on sensor parameters, ground point cloud, tower point cloud and conductor point cloud. The first filtering module is used to detect potential hazards in the visualized image to generate hazard boxes. For each hazard box, it determines whether its preset representative point is located in the overlapping area of the corresponding multi-view image. If not, the hazard is determined to be a non-repeating hazard and filtered out. The feature extraction module is used to extract the visual and spatial features of the hazards retained after the first layer of filtering. The spatial features include the three-dimensional physical dimensions of the hazards recovered based on the mapping relationship between monocular images and tower spans. The second filtering module is used to identify suspected repeating hazards based on visual features, and to determine a repeating alarm when the three-dimensional physical size deviation between the suspected repeating hazard and the target hazard is within a preset threshold range.