Power transmission line multi-mode warning system and expelling method

Abstract

The invention discloses a power transmission line multi-mode warning system and a power transmission line multi-mode expelling method, belongs to the technical field of power transmission line safety monitoring, and aims at solving the problems that power transmission line invasion target monitoring is not accurate, and the warning and expelling effect is poor. An environment image is acquired through an acquisition unit, and redundancy compensation is carried out on a fault unit. In the aspect of image splicing, a database is constructed by using structural feature points of a power transmission line, rapid projection transformation of a fixed area is realized, incremental feature matching is adopted for a dynamic area, and an environment panoramic image is generated. The method comprises the following steps: establishing a background coordinate system based on an environment panoramic image by aligning a fixed structure region, extracting multi-modal features of an intrusion target, constructing a dynamic trajectory parameter set, completing species classification and behavior recognition, constructing a multi-dimensional evaluation index system, dividing threat levels, and generating a thermodynamic diagram. And finally, according to data such as threat levels, a multi-mode grading warning system is constructed, warning equipment is dynamically adjusted, a target track is tracked, a warning effect is evaluated, and accurate and efficient invasion target expelling is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission line safety monitoring, and more particularly to a power transmission line multimodal warning system and a method for driving away power transmission line hazards. Background Art

[0002] With the rapid development of power generation in recent years, power transmission and distribution lines and electrical equipment have become increasingly widespread, extensive, and long. These lines operate in complex environments, encompassing open wilderness, urban economic zones, roads, and bridges. These facilities not only must withstand the constant onslaught of severe weather, including heavy rain, strong winds, and high temperatures, but also face diverse external damage risks, such as construction work involving large machinery within power protection zones, and bird strikes or collisions. These external damage incidents are characterized by suddenness, concealment, and frequency. Traditional warning technology systems have limitations. Repellent devices based on static physical barriers or single-frequency signals, due to their limited environmental adaptability and rigid operation modes, struggle to cope with dynamic intrusion scenarios and exhibit significant performance degradation in complex weather conditions or during long-term operation. Furthermore, manual inspections, limited by the length of transmission lines and complex terrain, fail to promptly detect potential hazards, leading to threats to power grid operation and frequent economic losses. Addressing these monitoring blind spots for external damage in complex environments and improving the timeliness and accuracy of early warning responses have become pressing technical challenges in power facility protection. Therefore, in order to overcome these limitations, the present invention proposes a multi-modal warning system and a method for driving away power line hazards. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a multimodal warning system and expulsion method for power transmission lines, which solves the safety hazards caused by external forces such as bad weather, large machinery construction operations, and bird landings or collisions due to the large number of points, wide area, long lines and complex environment of power transmission lines and equipment. It breaks through the problems of inefficient traditional danger warning and early warning and difficult manual inspections, and realizes effective monitoring and early warning of sudden and hidden external force damage.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A multi-modal warning system for a power transmission line, comprising:

[0006] By deploying acquisition units to collect environmental images, the structural feature points of the transmission lines are extracted and a standard stitching database is constructed. Incremental feature matching is used for dynamic areas to extract dynamic feature points from newly added or changed pixel areas. The environmental image is projected into a panoramic coordinate system by combining the homography matrix of the fixed structural area with the local transformation matrix of the dynamic area to generate a panoramic image of the environment.

[0007] Based on the geometric alignment of fixed structural areas in adjacent frames of panoramic images, a background coordinate system is established. Multimodal feature descriptors of intrusion targets in dynamic areas are extracted. A Kalman filter and nearest neighbor data association algorithm are used to establish a dynamic trajectory parameter set for intrusion targets. Species classification and behavior recognition of intrusion targets are performed, and a multidimensional evaluation index system is constructed. Fuzzy logic algorithms are used to classify threat levels and generate real-time threat heat maps.

[0008] Obtain the threat level, biological characteristics data and real-time positioning data of the intrusion target, build a multimodal hierarchical warning system, calculate the priority weight of the warning equipment and screen the warning equipment combination and working parameters; predict the location of the intrusion target, generate a dynamic tracking window, control the output direction of the warning device and the dynamic coupling with the intrusion target location; and track the dynamic trajectory parameters and real-time threat level of the intrusion target to evaluate the warning effect.

[0009] Specifically, the steps of collecting environmental images include:

[0010] Configure the acquisition cycle, activate the image acquisition mode of the acquisition unit; configure the acquisition angle and scanning range of the acquisition unit;

[0011] During the acquisition process, the heartbeat signal is used to continuously monitor the operating status of the acquisition unit, and the acquisition unit status parameters are obtained in real time to determine whether the acquisition unit has an abnormal operating status. If there is an abnormality, it is marked as a faulty acquisition unit, the alarm mechanism is triggered, and its coordinate information is recorded. If it is normal, it is marked as a normal acquisition unit.

[0012] When there is a faulty acquisition unit, its location coordinates are obtained, the status data of adjacent acquisition units are retrieved, and redundant compensation units are selected from adjacent normally functioning acquisition units;

[0013] Based on the acquisition angle and scanning range of the faulty acquisition unit, dynamic adjustment instructions are sent to the redundant compensation unit to adjust the acquisition angle and scanning range of the redundant compensation unit to fill the image acquisition blind spot caused by the faulty acquisition unit;

[0014] Control the normal acquisition units to work in parallel and synchronously acquire environmental images according to the set acquisition cycle; and add time stamps and spatial coordinate information to the acquired environmental images to generate an original environmental image set.

[0015] Specifically, the steps of generating a panoramic image of the environment include:

[0016] Setting a fixed structural area of the transmission line and extracting structural feature points of the fixed structural area;

[0017] According to the calibration plate image taken by the acquisition unit, the acquisition unit is calibrated, the homography matrix of the structural feature points is calculated, and the distortion correction parameters are fitted;

[0018] Based on the homography matrix, distortion correction parameters, spatial coordinates, perspective projection coordinates of each acquisition unit, and descriptors used to characterize the local gradient direction and intensity distribution of the structural feature points, a stitching standard database is constructed;

[0019] Based on the spatial coordinates of the structural feature points and the structural topological relationship, the structural feature points are uniquely encoded and a mapping table of structural feature points across acquisition units is established;

[0020] Configure a detection area for each structural feature point, dynamically adjust the detection area position based on the real-time angle parameters of the acquisition unit, locate candidate structural feature points within the detection area, perform hash index matching on the extracted descriptors and the splicing standard database, and select valid matching structural feature points according to the preset distance threshold;

[0021] The homography matrix and distortion correction parameters of the structural feature points are extracted from the stitching standard database, and the fixed structure area where the effectively matched structural feature points are located is projected and transformed.

[0022] Specifically, the steps of generating a panoramic image of the environment also include:

[0023] Segment the environmental image, identify the dynamic area and the fixed structure area, use incremental feature matching on the dynamic area, and obtain the local transformation matrix of the dynamic area;

[0024] Based on the homography matrix of the fixed structure area and the local transformation matrix of the dynamic area, the images of each acquisition unit are projected into the panoramic coordinate system to generate a preliminary panoramic image;

[0025] For the overlapping areas of adjacent acquisition unit images, an illumination compensation model is constructed to calculate the brightness gain and offset coefficients and perform brightness equalization processing;

[0026] The Laplace pyramid fusion algorithm is used to decompose the overlapping areas at multiple scales. By setting fusion weights for the fixed structure area and the dynamic area, the overlapping areas are seamlessly spliced to generate a panoramic image of the environment.

[0027] Specifically, incremental feature matching is used in the dynamic area, and the specific steps of obtaining the local transformation matrix of the dynamic area include:

[0028] A cache queue containing dynamic feature point coordinates, descriptors, and timestamps is established for each acquisition unit, and a complete set of dynamic feature points is extracted from the dynamic area of the initial frame to construct an initial cache queue;

[0029] The relative transformation matrix obtained by aligning the fixed structure area of the current frame with the previous frame is used to predict the position of the dynamic area mask of the previous frame, and the incremental mask is generated by combining the pixel difference method;

[0030] Adopting the adaptive threshold corner detection algorithm to extract dynamic feature points in the incremental mask coverage area and generate binary descriptors to construct the incremental dynamic feature point set;

[0031] Match the incremental dynamic feature point set with the dynamic feature points in the cache queue, screen candidate matching pairs by measuring similarity and setting a similarity threshold, and eliminate incorrect matching points to obtain successfully matched dynamic feature points;

[0032] The timestamp of the successfully matched dynamic feature points is updated and moved to the head of the cache queue. The unsuccessfully matched dynamic feature points are marked as new dynamic feature points and added to the cache queue. The unmatched dynamic feature points that exceed the preset time threshold are removed, and the local transformation matrix of the dynamic area is calculated based on the matching results.

[0033] Specifically, the steps for classifying threat levels and generating a real-time threat heat map include:

[0034] Based on the pre-stored homography matrix, the fixed structure areas of the adjacent frame panoramic images are geometrically aligned, and the background displacement error of the acquisition unit is compensated to establish the pixel-level mapping relationship between the fixed structure areas of the adjacent frames;

[0035] Scan the dynamic areas of the adjacent frame panoramic images in real time, extract multi-scale target features, and construct a multimodal feature descriptor;

[0036] Construct a Kalman filter model to predict the coordinates of the intrusion target position in the current frame and generate a dynamic search window centered on the predicted coordinates;

[0037] Within the search window of the dynamic area of the current frame's panoramic image, the cosine similarity of the intrusion target's multimodal feature descriptor with that of the previous frame's panoramic image is calculated to screen candidate matching intrusion targets and perform cross-frame intrusion target trajectory matching.

[0038] The confidence decay mechanism is started for unmatched trajectories, a tracker is created for the newly detected intrusion target, and the Kalman filter state is initialized to associate the intrusion target's trajectories across frames.

[0039] Specifically, the steps for classifying threat levels and generating a real-time threat heat map include:

[0040] Based on the spatial coordinate mapping relationship of the fixed structure area, the pixel displacement of the intrusion target is converted into physical distance, and the dynamic trajectory parameter set of the intrusion target is established;

[0041] The invading targets are classified into species, and the multimodal feature descriptors of adjacent frames are stacked to form a temporal feature sequence for behavioral pattern recognition, and the confidence value of the invading target behavior pattern is output;

[0042] A multi-dimensional evaluation indicator system is constructed, and risk weights are assigned to intrusion target types. Movement trends are quantified by the rate of change of the distance between the intrusion target and the transmission line. A safe distance threshold is set based on the voltage level of the transmission line, and the distance between the intrusion target and the transmission line is calculated in real time to form a multi-dimensional evaluation indicator data set.

[0043] The Gaussian membership function is defined through fuzzy logic algorithm, multi-dimensional evaluation indicators are comprehensively processed, and the threat level is quantified in combination with the preset rule base;

[0044] The threat level is mapped to geographic grid cells, and the threat density of the grid cells is calculated using the kernel density estimation algorithm. The mapping generates a real-time threat heat map with an overlaid electronic map.

[0045] Specifically, the steps to build a multimodal graded warning system include:

[0046] Receive the threat level of the intrusion target, obtain the biological characteristic data of the intrusion target according to the intrusion target type, and obtain the real-time positioning data of the intrusion target. The real-time positioning data includes spatial coordinates and motion trajectory parameters;

[0047] Based on the pre-stored policy library data, a mapping table of threat levels, biological characteristics and warning plans is established;

[0048] The spatial coordinates and motion trajectory parameters of the intrusion target are integrated to drive the servo gimbal to lock the intrusion target. The Kalman filter is used to predict the intrusion target's motion trajectory and generate a dynamic tracking window, and the initial pointing direction of the warning device is calibrated synchronously.

[0049] Based on the biological characteristics data and real-time positioning data of the intrusion target, the warning device combination and working parameters are selected from the strategy library;

[0050] The auditory sensitive frequency band and visual response threshold of the intrusion target are extracted, and the priority weight of each warning device is calculated based on the intrusion target's movement speed and the rate of change of the distance from the power transmission line.

[0051] According to the priority weight of the warning device, a warning device combination is selected from the policy library, and the working parameters of each warning device are determined to drive away the intruder target;

[0052] During the directional expulsion process, the position sensor deployed on the warning device collects the warning device deviation data in real time, and combines it with the real-time positioning data of the intrusion target to adjust the warning device pointing error.

[0053] Specifically, the steps to build a multimodal graded warning system also include:

[0054] During the warning process, the warning device sets an evaluation time period to track the dynamic trajectory parameters and real-time threat level of the intrusion target. If, within the evaluation time period, the intrusion target's movement speed is less than the configured speed threshold, or the difference between the movement direction angle and the initial direction angle is greater than the configured angle threshold, or the distance from the power transmission line is greater than the configured warning distance threshold, the warning is considered valid; otherwise, the warning is considered invalid.

[0055] When the alarm is determined to be effective, the pre-configured policy adjustment plan is called to change the alarm policy and adjust the working parameters of the alarm device according to the threat level of the current intrusion target;

[0056] When it is determined that the warning is invalid, an abnormal warning is triggered and an abnormal alarm message is sent.

[0057] A multi-modal warning and driving away method for a power transmission line comprises the following steps:

[0058] Step S1: Collect environmental images through the deployed acquisition units, activate the acquisition mode by configuring the acquisition cycle, adjust the acquisition angle and scanning range, and add timestamps and spatial coordinate information to the collected images to generate an original environmental image set;

[0059] Step S2: Process the original environmental image set, extract the structural feature points of the transmission lines, build a standard stitching database, combine the homography matrix of the fixed structure area and the local transformation matrix of the dynamic area to project the image into a unified panoramic coordinate system, and generate an environmental panoramic image through illumination compensation and image fusion;

[0060] Step S3: Based on the panoramic image of the environment, a background coordinate system is established by geometrically aligning fixed structure areas, and multimodal feature descriptors of intrusion targets in dynamic areas are extracted. A dynamic trajectory parameter set of intrusion targets is established, species classification and behavior recognition are performed, a multidimensional evaluation index system is constructed, and a fuzzy logic algorithm is used to classify threat levels and generate a real-time threat heat map.

[0061] Step S4: Based on the threat level, biological characteristic data and real-time positioning data of the intrusion target, a mapping relationship table of threat level, biological characteristics and warning schemes is established to select the combination of warning devices and working parameters;

[0062] Step S5: During the warning process of the warning device, the dynamic trajectory parameters and real-time threat level of the intrusion target are tracked, and the warning effect is determined according to the set evaluation time period to perform multi-modal warning and expulsion of the intrusion target on the power transmission line.

[0063] Beneficial effects of the present invention:

[0064] The present invention ensures the continuity and comprehensiveness of environmental image acquisition by deploying acquisition units and utilizing a redundant compensation mechanism. It improves the accuracy and efficiency of panoramic image generation by combining the structural feature point splicing of fixed structural areas with the dynamic area incremental matching technology. Based on multimodal feature extraction and Kalman filtering algorithm, it realizes the precise tracking of intrusion target trajectories and behavior recognition, and ensures the scientific nature of threat level classification through a fuzzy logic multidimensional evaluation model. It constructs a multimodal graded warning system, combines dynamic tracking with feedback adjustment mechanism, and realizes differentiated and precise expulsion of targets of different risk levels. At the same time, through warning effect evaluation and abnormal early warning, it significantly improves the intelligence, adaptability and expulsion efficiency of transmission line safety protection, reduces false alarm and missed alarm rates and interference with the surrounding ecology. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a structural diagram of a multi-modal warning system for power transmission lines according to the present invention;

[0066] Figure 2 A flowchart of the specific steps of collecting environmental images of the present invention;

[0067] Figure 3 A flowchart of specific steps for using incremental feature matching for dynamic regions in the present invention;

[0068] Figure 4 A flowchart for the threat level assessment and classification of different threat levels for the present invention;

[0069] Figure 5 The present invention provides a flow chart of a multi-modal warning and expulsion method for a power transmission line. DETAILED DESCRIPTION

[0070] Example 1:

[0071] See also Figure 1 ,This embodiment introduces a multimodal warning system for power transmission lines, including: an environmental perception module, an intelligent hub module and a warning execution module;

[0072] The environmental perception module focuses on capturing images of the scene surrounding power transmission lines. Using multiple evenly deployed acquisition units, it captures environmental images, enabling real-time visual perception of bird movements, the shape and spatial location of intruding objects, and more. It supports dynamic adjustment of acquisition angles, covering both circumferential and axial spatial coverage, meeting the needs of multi-dimensional image acquisition in complex environments. By employing multiple acquisition units in parallel, it ensures complete image coverage even when a single acquisition unit experiences an anomaly. It also features continuous or intermittent image capture, generating raw image data with timestamps and spatial coordinates, providing raw data support for identifying and locating intruding objects. The field of view of a single acquisition unit is physically limited, preventing it from fully covering the three-dimensional space surrounding a power transmission line. By deploying multiple acquisition units and enabling coordinated circumferential and axial scanning, a panoramic monitoring area can be constructed, eliminating blind spots associated with a single perspective. Furthermore, by operating in parallel, if a single acquisition unit fails due to failure, obstruction, or environmental interference, other acquisition units can take over its coverage area through a redundant compensation mechanism, ensuring continuous operation of the monitoring system and preventing the loss of critical data due to single-point failures.

[0073] See also Figure 2 Preferably, the specific steps of collecting the environment image include:

[0074] Configure the collection cycle and send commands to the collection unit based on the preset collection cycle, adjusting its operating state, switching the collection unit from its initial protected state to an exposed state and simultaneously activating image acquisition mode. When not in operation, the collection unit retracts its protective structure, effectively protecting against environmental interference such as wind, rain, dust, and mechanical collisions, extending the device's lifespan. By presetting the collection cycle and activating the collection function only when needed, this reduces inefficient power consumption while ensuring high-density monitoring during high-risk periods, balancing monitoring efficiency with device loss.

[0075] Potential intruders around power transmission lines are distributed at different heights and orientations, and a single fixed angle cannot meet multi-dimensional monitoring needs. The acquisition unit's acquisition angle and scanning range are configured, and through the coordinated adjustment of circumferential rotation and axial tilt, a scanning field covering the three-dimensional space around the transmission line is formed, enabling multi-dimensional real-time sampling of bird movement trajectories, the morphology of intruders, and their spatial location.

[0076] The acquisition unit is exposed to the outdoors for a long time and may fail due to power failure, mechanical jamming, communication interruption and other problems. During the acquisition process, the operating status of each acquisition unit is continuously monitored through the heartbeat signal, and the status parameters such as power supply voltage, motor speed, image data transmission integrity, etc. are obtained in real time. The above status parameters are analyzed in real time based on the preset abnormality judgment rules. The abnormal operation status of a single acquisition unit is judged by multiple consecutive signal interruptions or abnormal data characteristics. If there is an abnormal operation status, such as the power supply voltage is lower than the critical value, the motor speed fluctuates abnormally, or multiple consecutive frames of image data are missing, it is marked as a faulty acquisition unit, and the alarm mechanism is triggered synchronously and the coordinate information of the faulty acquisition unit is recorded. If there is no abnormal operation status, it is marked as a normal acquisition unit.

[0077] When a faulty acquisition unit stops operating, a monitoring gap will form in its original coverage area, which may lead to missed detection of intrusion targets. After obtaining the location coordinates of the faulty acquisition unit, the status data of the adjacent acquisition units of the faulty acquisition unit are retrieved, and redundant compensation units that can be used as redundant compensation are selected from the adjacent acquisition units in normal operation;

[0078] Based on the acquisition angle and scanning range of the fault acquisition unit, dynamic adjustment instructions are sent to the redundant compensation unit to adjust the acquisition angle and scanning range of the redundant compensation unit to expand the coverage in the circumferential and axial dimensions, fill the image acquisition blind spots caused by the fault acquisition unit, and ensure real-time image acquisition of scenes around the transmission line without omission.

[0079] Controls normal acquisition units to work in parallel, synchronously capturing environmental images according to a set acquisition cycle, enabling multi-angle, real-time sampling of scenes around power transmission lines. This ensures that the motion characteristics of intrusion targets from different perspectives are captured simultaneously, improving the accuracy and completeness of back-end trajectory analysis.

[0080] Add timestamps and spatial coordinate information to the collected environmental images to generate a standardized set of original environmental images for the back-end system to perform intrusion target identification and trajectory analysis.

[0081] The intelligent central module generates panoramic environmental images by performing spatiotemporal synchronization calibration, feature extraction and matching, and splicing of raw environmental image sets collected by multiple acquisition units. This provides complete, real-time visual data support for intrusion target detection and trajectory analysis around transmission lines. Based on an edge computing architecture, it analyzes the features of the panoramic environmental images to achieve species classification, behavior recognition, and spatial positioning of intrusion targets. It calculates dynamic parameters based on the target's motion trajectory and conducts threat level assessments based on multi-dimensional assessment data such as target type, motion trend, and distance from the transmission line. This assessment classifies different threat levels and generates real-time threat heat maps, laying the core data and strategic foundation for the accurate identification and dynamic intervention of external force damage risks to transmission lines.

[0082] Preferably, the specific steps of generating a panoramic image of the environment include:

[0083] In the transmission line environment, the motion characteristics of intrusion targets and fixed structures are different. The segmentation model is used to segment the dynamic area and fixed structure area of the environmental image, and identify intrusion target areas such as birds and foreign objects and fixed structure areas such as towers and transmission lines.

[0084] The fixed structures of power transmission lines are stable and can be used as reference points for image stitching. By defining the fixed structure area of the power transmission lines, we use structured light scanning or photogrammetry to obtain the spatial coordinates of the structural feature points. Then, using feature detection algorithms, we extract the corresponding structural feature points from each acquisition unit's perspective to improve the stitching accuracy and stability of the panoramic image.

[0085] There is distortion in the acquisition unit lens, and the viewing angles and positions of each acquisition unit are different. The acquisition units are calibrated simultaneously, the homography matrix of the structural feature points in the fixed structure area is calculated, and the distortion correction parameters are fitted. The Zhang calibration method is used to solve the intrinsic and extrinsic parameters of the calibration plate image taken by each acquisition unit. The homography matrix is calculated by combining the three-dimensional coordinates and two-dimensional projection coordinates of the structural feature points, and the lens distortion model is fitted to obtain the radial distortion parameters and tangential distortion parameters to eliminate the influence of lens distortion on the image.

[0086] Based on the homography matrix, distortion correction parameters, spatial coordinates of structural feature points, perspective projection coordinates of each acquisition unit, and descriptors used to characterize the local gradient direction and intensity distribution of structural feature points, a structured stitching standard database containing geometric transformation parameters is constructed, and a fast data query mechanism is established to improve the efficiency of subsequent image stitching.

[0087] There are many structural feature points in the fixed structure area of the transmission line. Based on the spatial coordinates of the structural feature points and the structural topological relationship, the structural feature points are uniquely encoded. For example, the spatial coordinates and the structural position are mapped into fixed-length codes through a hash function, and the projection coordinates of the structural feature points in different acquisition unit images are associated to establish a cross-acquisition unit structural feature point mapping table with the code as the key and the multi-view projection coordinates as the value.

[0088] Changes in the angle of the acquisition unit or environmental interference may cause the position of the feature points to shift. A detection area centered on the historical spatial coordinates is configured for each structural feature point, and the position of the detection area is dynamically adjusted in combination with the real-time angle parameters of the acquisition unit. When the image input is acquired, the candidate structural feature points are located within the detection area using the corner detection algorithm. After extracting the descriptor, hash index matching is performed with the splicing standard database, and the effectively matched structural feature points are screened according to the preset distance threshold.

[0089] For effectively matched structural feature points, the pre-stored homography matrix and distortion correction parameters are extracted from the stitching standard database, and a fast projection transformation is performed on the fixed structure area, converting it from the acquisition unit coordinate system to the panoramic coordinate system to avoid repeated calculations, improve image stitching efficiency, and meet real-time monitoring needs.

[0090] Dynamic areas change frequently. If full-area matching is performed, the calculation amount is large and the efficiency is low. Incremental feature matching is used for dynamic areas. Real-time feature points are extracted only for newly added or changed pixel areas, and local matching is performed with the dynamic area features of adjacent images to narrow the calculation range.

[0091] See also Figure 3 Specifically, the specific steps of using incremental feature matching for dynamic areas include:

[0092] A dynamic feature point cache queue containing dynamic feature point coordinates, descriptors and timestamps is established for each acquisition unit. A feature detection algorithm is used to extract a complete set of dynamic feature points from the dynamic area of the initial frame environment image to construct an initial cache queue.

[0093] The relative transformation matrix obtained by aligning the fixed structure area of the current frame environment image with the previous frame environment image is used to predict the position of the dynamic area mask of the previous frame. The pixel difference method is used to identify the newly added or changed pixel areas to generate an incremental mask.

[0094] Adopting the adaptive threshold corner detection algorithm to extract dynamic feature points in the incremental mask coverage area and generate binary descriptors to construct the incremental dynamic feature point set;

[0095] The incremental dynamic feature point set is matched with the dynamic feature points of the latest frame environment image in the cache queue first. Candidate matching pairs are screened by measuring similarity and setting a similarity threshold. At the same time, epipolar constraints are applied to eliminate incorrect matching points of candidate matching pairs that violate geometric relationships to obtain successfully matched dynamic feature points. The successfully matched dynamic feature points are then timestamped and moved to the head of the cache queue.

[0096] The dynamic feature points that are not successfully matched are marked as new dynamic feature points and added to the cache queue. The dynamic feature points that are not matched for more than a preset time threshold are removed from the cache queue. The local transformation matrix of the dynamic area is calculated based on the dynamic feature point matching results.

[0097] Based on the homography matrix of the fixed structure area and the local transformation matrix of the dynamic area, the images of each acquisition unit are projected into a unified panoramic coordinate system to generate a preliminary panoramic image, which integrates information from different perspectives and provides a complete visual data foundation for subsequent intrusion target detection and analysis.

[0098] There are illumination differences and overlapping areas between adjacent acquisition unit images, so illumination compensation is needed to balance the brightness. For the overlapping areas of adjacent acquisition unit images, an illumination compensation model is constructed. Based on the historical brightness parameters of the fixed structure area and the real-time brightness statistics of the dynamic area, the brightness gain and offset coefficient are calculated to perform brightness equalization on the overlapping areas.

[0099] The Laplace pyramid fusion algorithm is used to perform multi-scale decomposition of overlapping areas to generate image pyramids containing information of different frequencies. By setting different fusion weights for fixed structure areas and dynamic areas, seamless stitching of overlapping areas is achieved to generate a complete panoramic image of the environment.

[0100] See also Figure 4 ,Preferably, the specific steps of conducting threat level assessment, ,classifying different threat levels and generating a real-time threat heat map include:

[0101] Fixed structure regions in adjacent panoramic images remain stable. Aligning these regions using pre-stored homography matrices eliminates background displacement caused by acquisition unit jitter, providing a stable reference for intrusion target detection. Based on the coordinate invariance of fixed structure regions in adjacent panoramic images, the system geometrically aligns these regions by stitching pre-stored homography matrices from a standard database. Bidirectional optical flow is used to compensate for background displacement errors caused by minor acquisition unit jitter, establishing a pixel-level mapping relationship between fixed structure regions in adjacent frames and providing a stable background reference coordinate system for intrusion target detection.

[0102] A single feature is difficult to accurately describe the intrusion target. It is necessary to combine multimodal features such as contour, texture, and motion to construct a comprehensive intrusion target descriptor to improve recognition accuracy. The dynamic areas of the panoramic images of adjacent frames are scanned in real time. Multi-scale target features are extracted through the feature pyramid network, and the pixel-level bounding box of the intrusion target is located. The contour feature vector, texture feature vector and motion vector based on the optical flow field of adjacent frames of the intrusion target are simultaneously extracted to construct a multimodal feature descriptor that includes gradient direction histogram, local binary pattern and optical flow amplitude and phase information.

[0103] The trajectory of the intrusion target needs to be continuously tracked to analyze its behavioral trends. A Kalman filter model is constructed, and the state vector is defined as the spatial coordinates of the intrusion target and its first-order derivative, namely the velocity. The state transfer matrix adopts a uniform motion model including a time step, and the observation matrix is the unit matrix. The predicted coordinates of the intrusion target position in the current frame are predicted by the intrusion target state in the previous frame, and a dynamic search window centered on the predicted coordinates is generated.

[0104] Within the search window of the dynamic area of the current frame's panoramic image, candidate matching intrusion targets are screened whose cosine similarity with the intrusion target's multimodal feature descriptor in the previous frame's panoramic image is greater than a preset matching threshold, and the nearest neighbor data association algorithm is used to complete cross-frame intrusion target trajectory matching;

[0105] For the previous frame trajectory that does not match the candidate intrusion target, the trajectory confidence decay mechanism is activated. If there is no match for multiple consecutive frames, the trajectory is terminated. For newly detected intrusion targets without historical trajectories within the search window, a new tracker is created and the Kalman filter state is initialized, thereby realizing cross-frame trajectory association of intrusion targets in dynamic areas and ensuring the continuity and uniqueness of the intrusion target's motion trajectory.

[0106] Based on the spatial coordinate mapping relationship of the fixed structure area, the pixel displacement of the intrusion target is converted into physical distance. The finite difference method is used to calculate the intrusion target's motion speed, acceleration and motion direction angle relative to the transmission line, and the dynamic trajectory parameter set of the intrusion target is established.

[0107] Different species and behaviors pose different degrees of threat to transmission lines. The multimodal feature descriptors of the intruding targets are input into the deep convolutional neural network for species classification, and the probability distribution of the intruding targets belonging to categories such as birds, rodents, and mechanical operating equipment is output through the fully connected layer. The multimodal feature descriptors of adjacent frames are stacked to form a temporal feature sequence, which is input into the long short-term memory network for behavioral pattern recognition, and the confidence value of the intruding target behavior belonging to categories such as approaching transmission lines, staying, climbing, and turning back is output.

[0108] A single indicator cannot fully assess the threat of intrusion targets. Therefore, a multidimensional evaluation indicator system is constructed, which includes the intrusion target type, movement trend, and distance from the transmission line. The risk weight is assigned to the intrusion target type based on the probability of different intrusion target types causing transmission line failures based on historical statistics. The movement trend is quantified by the rate of change of the distance between the intrusion target and the transmission line. The safety distance threshold is set based on the voltage level of the transmission line, and the Euclidean distance between the current location of the intrusion target and the transmission line is calculated in real time, forming a standardized multidimensional evaluation indicator data set.

[0109] Fuzzy logic algorithm is used to comprehensively process multidimensional evaluation indicators, and Gaussian membership functions are defined for the risk weight of intrusion target type, distance change rate, and actual distance respectively, and fuzzy sets are divided into low risk, medium risk, and high risk levels; complex multi-source data are converted into clear threat levels, providing a clear decision-making basis for risk warning and disposal.

[0110] A fuzzy rule library containing preset rules is established. For example, if the intrusion target is a mechanical operating equipment and the distance is less than the safety distance threshold and the distance change rate is negative, the threat level is high; the fuzzy inference results are defuzzified using the center of gravity method to obtain a quantitative threat level value of 0-100, and the threat level is divided into low risk, medium risk, and high risk based on threshold comparison.

[0111] Based on the geographic information data of power transmission lines, the monitoring area is divided into uniform spatial grids, and each grid is assigned unique geographic coordinates. The threat level value of each intrusion target is mapped to the corresponding grid according to the spatial coordinates. The threat density of each grid cell is calculated using a kernel density estimation algorithm. The threat density is mapped to different gradient colors using a color mapping table. A real-time threat heat map is generated and superimposed on the electronic map of the power transmission lines. Different color gradients are used to visually display the risk distribution and evolution trends. The color gradient intuitively presents risk areas and evolution trends, helping operation and maintenance personnel quickly locate high-risk points and formulate targeted protection strategies.

[0112] The warning execution module builds a multimodal graded warning system based on the threat level assessment results and dynamic strategy instructions output by the intelligent central module. Specifically, it receives the risk level signals sent by the intelligent central module, including low risk, medium risk, high risk, and the biological characteristics data of the intruder target, and dynamically calls the corresponding warning combination plan from the pre-stored strategy library. For low-risk scenarios, it triggers the wide-band ultrasonic repellent device and the periodic laser flashing module, and realizes non-contact repelling by covering the biological auditory sensitive area with the ultrasonic frequency band and forming a visual interference band with the laser flashing; for medium-risk scenarios, it superimposes high-decibel buzzer alarms and directional laser scanning, and realizes the coordinated action of sound and light repelling. To enhance the warning effect; for high-risk scenarios, start full-power light flash warning, voice alarm and multi-matrix laser barrier, combine with servo motor to drive the laser emission unit to adjust the circumferential and axial angles, so that the laser beam path is dynamically matched with the motion trajectory of the intruding target to form a physical isolation barrier; during the execution of the strategy, the motion parameters of the intruding target are collected in real time, including: the change in flight direction, the attenuation rate of the travel speed, and the warning parameters are dynamically adjusted through the feedback control algorithm, including: ultrasonic frequency switching, laser intensity adjustment, and alarm tone changes. If insufficient response from the intruding target is detected, the warning level will be automatically increased to achieve precise intervention and dynamic adaptation to the risk of external force damage to the transmission line.

[0113] Preferably, the specific steps of constructing a multimodal graded warning system include:

[0114] The threat level reflects the potential risk level of the intrusion target. The biological characteristic data is related to the type of intrusion target and determines the adaptability of the warning method. The intelligent central module receives the threat level output, including: low risk, medium risk, and high risk. According to the type of intrusion target, the biological characteristic data of the intrusion target is obtained, including the auditory sensitive frequency band of the intrusion target and the visual response threshold of the intrusion target; and the real-time positioning data of the intrusion target is obtained, including the spatial coordinates of the intrusion target and the motion trajectory parameters.

[0115] Based on pre-stored policy database data, a mapping table is established between threat level, biological characteristics, and warning plans. The policy database stores the combination of multi-modal warning devices, operating parameters, and coordination logic. This forms a standardized policy retrieval mechanism, which retrieves the appropriate warning plan based on input data, improving response efficiency and expulsion accuracy.

[0116] The tracking and positioning unit is driven by the real-time positioning data of the intrusion target. By fusing the spatial coordinates and motion trajectory parameters of the intrusion target, the servo pan / tilt system deployed on the transmission line tower is controlled to drive the wide-angle camera to lock the intrusion target. The Kalman filter algorithm is used to predict the intrusion target position at the next moment, and a dynamic tracking window containing the predicted coordinates and error range is generated. The initial angle calibration of the warning device is synchronously driven to align the initial pointing direction of the warning device with the current position of the intrusion target, ensuring that the output direction of the warning device is dynamically coupled with the real-time position of the intrusion target.

[0117] Different intruders have varying sensitivities to stimuli like sound and light. Based on the threat level, biological characteristics of the intruder, and real-time location data, the strategy library is used to select appropriate warning device combinations and operating parameters. The target's auditory sensitivity frequency band and visual response threshold are extracted, and combined with dynamic parameters such as the target's movement speed and the rate of change of distance from the power line, the priority weight of each warning device is calculated. For example, for intruders sensitive to sound, the weight of ultrasonic repellent devices and buzzer alarms is prioritized; for intruders with strong visual perception, the weight of laser warning devices is increased.

[0118] Based on the calculated priority weights, a combination of warning devices is selected from the strategy library, and the operating parameters of each warning device are determined. For example, an ultrasonic repellent device is combined with a laser warning device for directional warning. The sweep frequency signal of the ultrasonic unit is set according to the auditory sensitivity frequency band of the intruding target, and is emitted in a direction toward the center of the tracking window through an array speaker. At the same time, the output power of the laser warning device is modulated so that the laser beam forms a periodically flashing visual guide band within the tracking window. For situations where the repellent effect needs to be enhanced, a directional buzzer alarm and a laser scanning repellent device are added. The buzzer emits a warning sound at a specific frequency, and the direction of sound wave propagation is consistent with the tangent direction of the intruding target's motion trajectory. The laser scanning repellent device drives a reflector to scan a certain range. The scanning speed is proportional to the speed of the intruding target, forming a moving light wall in front of the intruding target.

[0119] The movement of the intrusion target or the device's own errors may cause the warning direction to deviate. During the directional expulsion process, the position sensor deployed on the warning device collects the warning device deviation data in real time. Combined with the real-time positioning data of the intrusion target, the PID control algorithm is used to dynamically adjust the servo motor speed and direction to control the warning device error.

[0120] During the warning device's alarm process, the dynamic trajectory parameters and real-time threat level of the intrusion target are tracked, the warning effect is evaluated, and the effectiveness of the warning is determined based on objective data, providing a clear decision-making basis for subsequent strategy adjustments or manual intervention to avoid subjective misjudgment.

[0121] Set an evaluation time period. If, during the evaluation time period, the speed of the intrusion target is less than the configured speed threshold, or the difference between the movement direction angle and the initial direction angle is greater than the configured angle threshold, or the distance from the transmission line is greater than the configured warning distance threshold, the alarm is considered valid; otherwise, the alarm is considered invalid.

[0122] If the warning is deemed effective, the pre-configured policy adjustment scheme is invoked to modify the warning strategy based on the current threat level of the intruder. If the threat level decreases, the policy adjustment scheme adjusts the warning device's operating parameters, such as ultrasonic emission power, laser flashing frequency, and alarm volume, to those corresponding to a low threat level. If the threat level remains high, the policy adjustment scheme adjusts operating parameters, such as laser scanning range and alarm tone frequency, to those corresponding to a high threat level. This adaptive adjustment of the warning strategy ensures effective repelling while reducing energy consumption and environmental impact, improving the economic efficiency and sustainability of operations.

[0123] If the warning is deemed ineffective, an abnormality alert is triggered, sending an abnormality alarm message to the management terminal, including target information, real-time motion status parameters, and implemented warning measures. A manual assistance work order is automatically generated according to a pre-set process and pushed to the operator's terminal and the monitoring center. Operators can remotely modify the alarm device configuration parameters to adjust the alarm strategy or conduct on-site manual intervention based on the work order information to ensure transmission line safety. If the warning is ineffective and the target continues to threaten transmission line safety, an abnormality alert must be triggered immediately. Based on the target information, motion status, and implemented measures, manual intervention is requested.

[0124] Example 2

[0125] See also Figure 5 This embodiment introduces a multi-modal warning and expulsion method for a power transmission line, comprising the following steps:

[0126] Step S1: Use the deployed acquisition units to collect environmental images, activate the acquisition mode by configuring the acquisition cycle, adjust the acquisition angle and scanning range, monitor the operating status of the acquisition units in real time, perform redundancy compensation for faulty acquisition units, and add timestamps and spatial coordinate information to the acquired images to generate the original environmental image set;

[0127] Step S2: Process the original environmental image set, extract the structural feature points of the transmission lines, build a standard stitching database, use the homography matrix and distortion correction parameters to project the fixed structure area, use incremental feature matching to process the dynamic area, combine the homography matrix of the fixed structure area and the local transformation matrix of the dynamic area to project the image into a unified panoramic coordinate system, and generate an environmental panoramic image through illumination compensation and image fusion;

[0128] Step S3: Based on the panoramic image of the environment, a background coordinate system is established by geometrically aligning fixed structure areas. Multimodal feature descriptors of intrusion targets in dynamic areas are extracted. Kalman filtering and data association algorithms are used to establish a dynamic trajectory parameter set for intrusion targets. Species classification and behavior recognition are performed, a multidimensional evaluation index system is constructed, and a fuzzy logic algorithm is used to classify threat levels and generate a real-time threat heat map.

[0129] Step S4: Based on the threat level, biological characteristics data, and real-time positioning data of the intrusion target, a mapping relationship table between threat level, biological characteristics, and warning schemes is established, the priority weight of the warning device is calculated, and the appropriate warning device combination and operating parameters are selected; the target is locked by the servo pan / tilt system, and the target position is predicted by Kalman filtering to generate a dynamic tracking window, and the warning device direction is calibrated;

[0130] Step S5: During the warning process of the warning device, the dynamic trajectory parameters and real-time threat level of the intrusion target are tracked, and the warning effect is determined according to the set evaluation time period. If it is effective, the strategy adjustment plan is called according to the threat level to change the warning strategy. If it is not effective, an abnormal warning is triggered and an abnormal alarm information is sent to realize multi-modal warning and expulsion of transmission line intrusion targets.

[0131] Preferably, the specific steps of generating a panoramic image of the environment include:

[0132] Setting a fixed structural area of the transmission line and extracting structural feature points of the fixed structural area;

[0133] According to the calibration plate image taken by the acquisition unit, the acquisition unit is calibrated, the homography matrix of the structural feature points is calculated, and the distortion correction parameters are fitted;

[0134] Based on the homography matrix, distortion correction parameters, spatial coordinates, perspective projection coordinates of each acquisition unit, and descriptors used to characterize the local gradient direction and intensity distribution of the structural feature points, a stitching standard database is constructed;

[0135] Based on the spatial coordinates of the structural feature points and the structural topological relationship, the structural feature points are uniquely encoded and a mapping table of structural feature points across acquisition units is established;

[0136] Configure a detection area for each structural feature point, dynamically adjust the detection area position based on the real-time angle parameters of the acquisition unit, locate candidate structural feature points within the detection area, perform hash index matching on the extracted descriptors and the splicing standard database, and select valid matching structural feature points according to the preset distance threshold;

[0137] Extract the homography matrix and distortion correction parameters of the structural feature points from the stitching standard database, and perform projection transformation on the fixed structure area where the effectively matched structural feature points are located;

[0138] Segment the environmental image, identify the dynamic area and the fixed structure area, use incremental feature matching on the dynamic area, and obtain the local transformation matrix of the dynamic area;

[0139] Based on the homography matrix of the fixed structure area and the local transformation matrix of the dynamic area, the images of each acquisition unit are projected into the panoramic coordinate system to generate a preliminary panoramic image;

[0140] For the overlapping areas of adjacent acquisition unit images, an illumination compensation model is constructed to calculate the brightness gain and offset coefficients and perform brightness equalization processing;

[0141] The Laplace pyramid fusion algorithm is used to perform multi-scale decomposition of overlapping areas. By setting fusion weights for fixed structure areas and dynamic areas, the overlapping areas are seamlessly spliced to generate a panoramic image of the environment.

[0142] Preferably, incremental feature matching is used in the dynamic region, and the specific steps of obtaining the local transformation matrix of the dynamic region include:

[0143] A cache queue containing dynamic feature point coordinates, descriptors, and timestamps is established for each acquisition unit, and a complete set of dynamic feature points is extracted from the dynamic area of the initial frame to construct an initial cache queue;

[0144] The relative transformation matrix obtained by aligning the fixed structure area of the current frame with the previous frame is used to predict the position of the dynamic area mask of the previous frame, and the incremental mask is generated by combining the pixel difference method;

[0145] Adopting the adaptive threshold corner detection algorithm to extract dynamic feature points in the incremental mask coverage area and generate binary descriptors to construct the incremental dynamic feature point set;

[0146] Match the incremental dynamic feature point set with the dynamic feature points in the cache queue, screen candidate matching pairs by measuring similarity and setting a similarity threshold, and eliminate incorrect matching points to obtain successfully matched dynamic feature points;

[0147] The timestamp of the successfully matched dynamic feature points is updated and moved to the head of the cache queue. The unsuccessfully matched dynamic feature points are marked as new dynamic feature points and added to the cache queue. The unmatched dynamic feature points that exceed the preset time threshold are removed, and the local transformation matrix of the dynamic area is calculated based on the matching results.

[0148] Preferably, the specific steps of classifying threat levels and generating a real-time threat heat map include:

[0149] Based on the pre-stored homography matrix, the fixed structure areas of the adjacent frame panoramic images are geometrically aligned, and the background displacement error of the acquisition unit is compensated to establish the pixel-level mapping relationship between the fixed structure areas of the adjacent frames;

[0150] Scan the dynamic areas of the adjacent frame panoramic images in real time, extract multi-scale target features, and construct a multimodal feature descriptor;

[0151] Construct a Kalman filter model to predict the coordinates of the intrusion target position in the current frame and generate a dynamic search window centered on the predicted coordinates;

[0152] Within the search window of the dynamic area of the current frame's panoramic image, the cosine similarity of the intrusion target's multimodal feature descriptor with that of the previous frame's panoramic image is calculated to screen candidate matching intrusion targets and perform cross-frame intrusion target trajectory matching.

[0153] The confidence decay mechanism is activated for unmatched trajectories, a tracker is created for the newly detected intrusion target, and the Kalman filter state is initialized to associate the intrusion target's trajectories across frames.

[0154] Based on the spatial coordinate mapping relationship of the fixed structure area, the pixel displacement of the intrusion target is converted into physical distance, and the dynamic trajectory parameter set of the intrusion target is established;

[0155] The invading targets are classified into species, and the multimodal feature descriptors of adjacent frames are stacked to form a temporal feature sequence for behavioral pattern recognition, and the confidence value of the invading target behavior pattern is output;

[0156] A multi-dimensional evaluation indicator system is constructed, and risk weights are assigned to intrusion target types. Movement trends are quantified by the rate of change of the distance between the intrusion target and the transmission line. A safe distance threshold is set based on the voltage level of the transmission line, and the distance between the intrusion target and the transmission line is calculated in real time to form a multi-dimensional evaluation indicator data set.

[0157] The Gaussian membership function is defined through fuzzy logic algorithm, multi-dimensional evaluation indicators are comprehensively processed, and the threat level is quantified in combination with the preset rule base;

[0158] The threat level is mapped to geographic grid cells, and the threat density of the grid cells is calculated using the kernel density estimation algorithm. The mapping generates a real-time threat heat map with an overlaid electronic map.

[0159] Working principle and its effect:

[0160] This invention deploys multiple acquisition units, activates acquisition modes at preset intervals, and dynamically adjusts angles to capture images of the surrounding environment in three-dimensional space, covering the perimeter of power transmission lines. Heartbeat monitoring and redundancy compensation mechanisms ensure seamless acquisition even when a single acquisition unit fails, guaranteeing data continuity. A standard stitching database is constructed using structural feature points in fixed structural areas of power transmission lines. A homography matrix and distortion correction parameters are combined to rapidly project and transform fixed areas. Incremental feature matching is used in dynamic areas, processing only newly added pixels. This effectively generates distortion-free, seamlessly stitched panoramic images of the environment, improving image accuracy and processing speed.

[0161] Based on panoramic environmental images, a stable background coordinate system is established through geometric alignment of fixed structures. A feature pyramid network is used to extract multimodal features such as the outline, texture, and motion of the intruding target. Combined with a Kalman filter and a nearest neighbor data association algorithm, this system enables continuous tracking of trajectories across frames, accurately calculating dynamic parameters such as the target's speed and direction. Deep learning models are used to classify the target species and identify their behavior. A multidimensional evaluation index system is constructed, encompassing target type, motion trend, and safe distance. Fuzzy logic algorithms are used to quantify threat levels and generate heat maps, visually presenting risk distribution and providing a scientific basis for graded warnings.

[0162] The warning execution module establishes a mapping table to dynamically select warning device combinations based on the threat level, target biological characteristics, and real-time positioning data. For example, ultrasound is prioritized for sound-sensitive targets, while laser warnings are enhanced for visually sensitive targets. The target position is dynamically locked using a servo pan / tilt and Kalman filter, a tracking window is generated, and the laser beam direction is calibrated to ensure real-time coupling between the warning direction and the target. During the expulsion process, target motion parameters are collected in real time, device errors are adjusted using a PID control algorithm, and the warning effect is evaluated based on preset thresholds. If effective, the parameters are adaptively adjusted to reduce energy consumption. If ineffective, an abnormal warning is triggered to request manual intervention.

[0163] The present invention realizes all-round monitoring of the environment around the transmission line and high-precision stitching of environmental panoramic images, thereby improving the accuracy of target detection; through multimodal feature fusion and trajectory tracking, the type and behavior trend of intruding targets can be accurately identified, and the error rate of threat level assessment can be reduced; the multimodal graded warning and dynamic adjustment mechanism makes the expulsion strategy highly matched with the target characteristics and risk level, significantly improving the expulsion efficiency, while reducing interference with non-threatening targets, and taking into account safety and eco-friendliness; the closed-loop feedback and abnormal warning mechanism shortens the emergency response time, enhances the reliability and adaptability of the system, and provides an intelligent and refined solution for the safety protection of transmission lines.

[0164] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A multi-modal warning system for power transmission lines, characterized in that: include: By deploying acquisition units to collect environmental images, the structural feature points of the transmission lines are extracted and a standard stitching database is constructed. Incremental feature matching is used for dynamic areas to extract dynamic feature points from newly added or changed pixel areas. The environmental image is projected into a panoramic coordinate system by combining the homography matrix of the fixed structural area with the local transformation matrix of the dynamic area to generate a panoramic image of the environment. Based on the geometric alignment of fixed structural areas in adjacent frames of panoramic images, a background coordinate system is established. Multimodal feature descriptors of intrusion targets in dynamic areas are extracted. A Kalman filter and nearest neighbor data association algorithm are used to establish a dynamic trajectory parameter set for intrusion targets. Species classification and behavior recognition of intrusion targets are performed, and a multidimensional evaluation index system is constructed. Fuzzy logic algorithms are used to classify threat levels and generate real-time threat heat maps. Obtain the threat level, biological characteristics data and real-time positioning data of the intrusion target, build a multimodal hierarchical warning system, calculate the priority weight of the warning equipment and screen the warning equipment combination and working parameters; predict the location of the intrusion target, generate a dynamic tracking window, control the output direction of the warning device and the dynamic coupling with the intrusion target location; and track the dynamic trajectory parameters and real-time threat level of the intrusion target to evaluate the warning effect.

2. A multi-modal warning system for power transmission lines according to claim 1, characterized in that: The specific steps of generating the environment panoramic image include: Setting a fixed structural area of the transmission line and extracting structural feature points of the fixed structural area; According to the calibration plate image taken by the acquisition unit, the acquisition unit is calibrated, the homography matrix of the structural feature points is calculated, and the distortion correction parameters are fitted; Based on the homography matrix, distortion correction parameters, spatial coordinates, perspective projection coordinates of each acquisition unit, and descriptors used to characterize the local gradient direction and intensity distribution of the structural feature points, a stitching standard database is constructed; Based on the spatial coordinates of the structural feature points and the structural topological relationship, the structural feature points are uniquely encoded and a mapping table of structural feature points across acquisition units is established; Configure a detection area for each structural feature point, dynamically adjust the detection area position based on the real-time angle parameters of the acquisition unit, locate candidate structural feature points within the detection area, perform hash index matching on the extracted descriptors and the splicing standard database, and select valid matching structural feature points according to the preset distance threshold; The homography matrix and distortion correction parameters of the structural feature points are extracted from the stitching standard database, and the fixed structure area where the effectively matched structural feature points are located is projected and transformed.

3. A multi-modal warning system for power transmission lines according to claim 2, characterized in that: The specific steps of generating the panoramic image of the environment also include: Segment the environmental image, identify the dynamic area and the fixed structure area, use incremental feature matching on the dynamic area, and obtain the local transformation matrix of the dynamic area; Based on the homography matrix of the fixed structure area and the local transformation matrix of the dynamic area, the images of each acquisition unit are projected into the panoramic coordinate system to generate a preliminary panoramic image; For the overlapping areas of adjacent acquisition unit images, an illumination compensation model is constructed to calculate the brightness gain and offset coefficients and perform brightness equalization processing; The Laplace pyramid fusion algorithm is used to perform multi-scale decomposition of overlapping areas. By setting fusion weights for fixed structure areas and dynamic areas, the overlapping areas are seamlessly spliced to generate a panoramic image of the environment.

4. A multi-modal warning system for power transmission lines according to claim 3, characterized in that: The dynamic region adopts incremental feature matching, and the specific steps of obtaining the local transformation matrix of the dynamic region include: A cache queue containing dynamic feature point coordinates, descriptors, and timestamps is established for each acquisition unit, and a complete set of dynamic feature points is extracted from the dynamic area of the initial frame to construct an initial cache queue; The relative transformation matrix obtained by aligning the fixed structure area of the current frame with the previous frame is used to predict the position of the dynamic area mask of the previous frame, and the incremental mask is generated by combining the pixel difference method; Adopting the adaptive threshold corner detection algorithm to extract dynamic feature points in the incremental mask coverage area and generate binary descriptors to construct the incremental dynamic feature point set; Match the incremental dynamic feature point set with the dynamic feature points in the cache queue, screen candidate matching pairs by measuring similarity and setting a similarity threshold, and eliminate incorrect matching points to obtain successfully matched dynamic feature points; The timestamp of the successfully matched dynamic feature points is updated and moved to the head of the cache queue. The unsuccessfully matched dynamic feature points are marked as new dynamic feature points and added to the cache queue. The unmatched dynamic feature points that exceed the preset time threshold are removed, and the local transformation matrix of the dynamic area is calculated based on the matching results.

5. The multi-modal warning system for power transmission lines according to claim 1, characterized in that: The specific steps of classifying threat levels and generating a real-time threat heat map include: Based on the pre-stored homography matrix, the fixed structure areas of the adjacent frame panoramic images are geometrically aligned, and the background displacement error of the acquisition unit is compensated to establish the pixel-level mapping relationship between the fixed structure areas of the adjacent frames; Scan the dynamic areas of the adjacent frame panoramic images in real time, extract multi-scale target features, and construct a multimodal feature descriptor; Construct a Kalman filter model to predict the coordinates of the intrusion target position in the current frame and generate a dynamic search window centered on the predicted coordinates; Within the search window of the dynamic area of the current frame's panoramic image, the cosine similarity of the intrusion target's multimodal feature descriptor with that of the previous frame's panoramic image is calculated to screen candidate matching intrusion targets and perform cross-frame intrusion target trajectory matching. The confidence decay mechanism is started for unmatched trajectories, a tracker is created for the newly detected intrusion target, and the Kalman filter state is initialized to associate the intrusion target's trajectories across frames.

6. A multi-modal warning system for power transmission lines according to claim 5, characterized in that: The specific steps of classifying threat levels and generating a real-time threat heat map also include: Based on the spatial coordinate mapping relationship of the fixed structure area, the pixel displacement of the intrusion target is converted into physical distance, and the dynamic trajectory parameter set of the intrusion target is established; The invading targets are classified into species, and the multimodal feature descriptors of adjacent frames are stacked to form a temporal feature sequence for behavioral pattern recognition, and the confidence value of the invading target behavior pattern is output; A multi-dimensional evaluation indicator system is constructed, and risk weights are assigned to intrusion target types. Movement trends are quantified by the rate of change of the distance between the intrusion target and the transmission line. A safe distance threshold is set based on the voltage level of the transmission line, and the distance between the intrusion target and the transmission line is calculated in real time to form a multi-dimensional evaluation indicator data set. The Gaussian membership function is defined through fuzzy logic algorithm, multi-dimensional evaluation indicators are comprehensively processed, and the threat level is quantified in combination with the preset rule base; The threat level is mapped to geographic grid cells, and the threat density of the grid cells is calculated using the kernel density estimation algorithm. The mapping generates a real-time threat heat map with an overlaid electronic map.

7. The multi-modal warning system for power transmission lines according to claim 1, characterized in that: The specific steps of collecting the environment image include: Configure the acquisition cycle and activate the image acquisition mode of the acquisition unit; configure the acquisition angle and scanning range of the acquisition unit; During the acquisition process, the operating status of the acquisition unit is continuously monitored through the heartbeat signal, and the state parameters of the acquisition unit are obtained in real time to determine whether the acquisition unit has an abnormal operating state. If there is an abnormality, it is marked as a faulty acquisition unit, an alarm mechanism is triggered and its coordinate information is recorded. If it is normal, it is marked as a normal acquisition unit; When there is a faulty acquisition unit, its location coordinates are obtained, the status data of adjacent acquisition units are retrieved, and redundant compensation units are selected from adjacent normally operating acquisition units; Based on the acquisition angle and scanning range of the faulty acquisition unit, dynamic adjustment instructions are sent to the redundant compensation unit to adjust the acquisition angle and scanning range of the redundant compensation unit to fill the image acquisition blind spot caused by the faulty acquisition unit; Control the normal acquisition units to work in parallel and synchronously acquire environmental images according to the set acquisition cycle; and add time stamps and spatial coordinate information to the acquired environmental images to generate an original environmental image set.

8. The multi-modal warning system for power transmission lines according to claim 1, characterized in that: The specific steps of constructing the multimodal graded warning system include: Receive the threat level of the intrusion target, obtain biological characteristic data of the intrusion target according to the type of the intrusion target, and simultaneously obtain real-time positioning data of the intrusion target, wherein the real-time positioning data includes spatial coordinates and motion trajectory parameters; Based on the pre-stored policy library data, a mapping table of threat levels, biological characteristics and warning plans is established; The spatial coordinates and motion trajectory parameters of the intrusion target are integrated to drive the servo gimbal to lock the intrusion target. The Kalman filter is used to predict the intrusion target's motion trajectory and generate a dynamic tracking window, and the initial pointing direction of the warning device is calibrated synchronously. Based on the biological characteristics data and real-time positioning data of the intrusion target, the warning device combination and working parameters are selected from the strategy library; The auditory sensitive frequency band and visual response threshold of the intrusion target are extracted, and the priority weight of each warning device is calculated based on the intrusion target's movement speed and the rate of change of the distance from the power transmission line. According to the priority weight of the warning device, a warning device combination is selected from the policy library, and the working parameters of each warning device are determined to drive away the intruder target; During the directional expulsion process, the position sensor deployed on the warning device collects the warning device deviation data in real time, and combines it with the real-time positioning data of the intrusion target to adjust the warning device pointing error.

9. A multi-modal warning system for power transmission lines according to claim 8, characterized in that: The specific steps of constructing the multimodal graded warning system also include: During the warning process, the warning device sets an evaluation time period to track the dynamic trajectory parameters and real-time threat level of the intrusion target. If, within the evaluation time period, the intrusion target's movement speed is less than the configured speed threshold, or the difference between the movement direction angle and the initial direction angle is greater than the configured angle threshold, or the distance from the power transmission line is greater than the configured warning distance threshold, the warning is considered valid; otherwise, the warning is considered invalid. When the alarm is determined to be effective, the pre-configured policy adjustment plan is called to change the alarm policy and adjust the working parameters of the alarm device according to the threat level of the current intrusion target; When it is determined that the warning is invalid, an abnormal warning is triggered and an abnormal alarm message is sent.

10. A multi-modal warning and driving away method for a power transmission line, which is implemented based on a multi-modal warning system for a power transmission line according to any one of claims 1 to 9, characterized in that: The following steps are involved: Step S1: Collect environmental images through the deployed acquisition units, activate the acquisition mode by configuring the acquisition cycle, adjust the acquisition angle and scanning range, and add timestamps and spatial coordinate information to the collected images to generate an original environmental image set; Step S2: Process the original environmental image set, extract the structural feature points of the transmission lines, build a standard stitching database, combine the homography matrix of the fixed structure area and the local transformation matrix of the dynamic area to project the image into a unified panoramic coordinate system, and generate an environmental panoramic image through illumination compensation and image fusion; Step S3: Based on the panoramic image of the environment, a background coordinate system is established by geometrically aligning fixed structure areas, and multimodal feature descriptors of intrusion targets in dynamic areas are extracted. A dynamic trajectory parameter set of intrusion targets is established, species classification and behavior recognition are performed, a multidimensional evaluation index system is constructed, and a fuzzy logic algorithm is used to classify threat levels and generate a real-time threat heat map. Step S4: Based on the threat level, biological characteristic data and real-time positioning data of the intrusion target, a mapping relationship table of threat level, biological characteristics and warning schemes is established to select the combination of warning devices and working parameters; Step S5: During the warning process of the warning device, the dynamic trajectory parameters and real-time threat level of the intrusion target are tracked, and the warning effect is determined according to the set evaluation time period to perform multi-modal warning and expulsion of the intrusion target on the power transmission line.

Images