Unmanned aerial vehicle edge power transmission channel dynamic obstacle perception fusion early warning method and system

Abstract

The application relates to the technical field of unmanned aerial vehicle obstacle avoidance, and specifically discloses a dynamic obstacle sensing fusion early warning method and system for an unmanned aerial vehicle edge-end power transmission channel, which comprises the following steps: collecting multi-modal data of the power transmission channel in real time through a plurality of sensors carried by the unmanned aerial vehicle; processing the multi-modal data to determine dynamic obstacles in the power transmission channel and types of the dynamic obstacles, and predicting future trajectories of the dynamic obstacles; and calculating a corrected dynamic risk level based on the types of the dynamic obstacles, the current distance from the power transmission channel to a conductor and the future trajectories. According to the risk level of dynamic evaluation, the application can adaptively switch between different multi-modal data fusion architectures, and feedforwardly manage on-board computing resources, so that the system can concentrate computing power to realize accurate sensing in a high-risk scene, and save resources in a low-risk scene, thereby intelligently balancing the contradiction between sensing accuracy and on-board computing resource consumption.

Description

Technical Field

[0001] This invention relates to the field of obstacle avoidance technology for unmanned aerial vehicles (UAVs), and in particular to a method and system for dynamic obstacle perception and fusion early warning of power transmission channels at the edge of a UAV. Background Technology

[0002] In the application of drones for autonomous inspection of power transmission channels, to ensure flight safety, the drones must be able to perceive and avoid various dynamic obstacles within the channel in real time, such as construction vehicles, flying birds, or other drones. Existing technologies typically employ multi-sensor fusion to identify obstacles and combine it with path planning algorithms for obstacle avoidance. However, in practical operation, existing technologies face two irreconcilable and conflicting technical problems.

[0003] The first conflict is that, in order to accurately identify and track high-speed or small obstacles, the system needs to run complex feature layer fusion algorithms, which consumes a lot of computing resources; while in simple scenarios, such high-precision perception is a waste of computing power. Existing systems usually adopt fixed perception strategies and cannot dynamically adjust their perception accuracy and resource consumption according to actual risks, resulting in the system either being unable to see clearly at critical moments or being too laborious at normal times, making it difficult to strike a balance between safety and economy.

[0004] The second conflict arises because advanced obstacle avoidance technologies tend to employ algorithms such as Model Predictive Control (MPC) or Quick Random Tree Search (RRT), which generate a globally optimal or suboptimal smooth flight path based on predictions of the obstacle's future trajectory. However, these algorithms require computation time for planning, and their effectiveness heavily relies on the accuracy of trajectory prediction. When an obstacle undergoes unpredictable maneuvers, the optimal flight path being executed by the UAV may instantly transform into a dangerous collision path. In this situation, the planning algorithm cannot instantly generate a new safe path, leaving the UAV trapped in a dangerous decision-making "window" where the planning has failed but the response has not kept pace. This significantly increases the risk of collision. Summary of the Invention

[0005] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the objective of this invention is to propose a dynamic obstacle perception and fusion early warning method and system for UAV edge power transmission channels, in order to optimize the obstacle avoidance response speed and resource consumption of UAVs.

[0006] To achieve the above objectives, a first aspect of the present invention proposes a dynamic obstacle perception and fusion early warning method for UAV edge power transmission channels, comprising the following steps:

[0007] S1. Real-time collection of multimodal data of power transmission channels through multiple sensors carried by drones;

[0008] S2. By processing the multimodal data, determine the dynamic obstacles and their types within the power transmission channel, and predict the future trajectory of the dynamic obstacles;

[0009] S3. Calculate the corrected dynamic risk level based on the type of the dynamic obstacle, its current distance from the conductor in the power transmission channel, and its future trajectory;

[0010] S4. In response to the revised dynamic risk level, perform the following steps:

[0011] (a) Based on the modified dynamic risk level, select a corresponding fusion strategy from the fusion architecture that includes at least a feature layer fusion strategy and a decision layer fusion strategy to perform fusion processing on the multimodal data;

[0012] (b) Based on the future trajectory of the dynamic obstacle and the three-dimensional geographic information system (GIS) security boundary of the power transmission channel, a three-dimensional dynamic obstacle avoidance route is generated online in real time, and the UAV is controlled to fly along the route.

[0013] (c) Based on the expected computing power requirements of the fusion strategy selected in step (a), the computing resources of the UAV onboard terminal are scheduled in a feedforward manner;

[0014] S5. Based on the revised dynamic risk level, generate an early warning signal.

[0015] To achieve the above objectives, a second aspect of the present invention provides a dynamic obstacle perception and fusion early warning system for UAV edge power transmission channels, comprising:

[0016] The data acquisition module is used to collect multimodal data of the power transmission channel in real time through multiple sensors carried by the drone;

[0017] The dynamic sensing module is used to process the multimodal data, determine the dynamic obstacles and their types within the power transmission channel, and predict the future trajectory of the dynamic obstacles.

[0018] The risk assessment module is used to calculate the corrected dynamic risk level based on the type of the dynamic obstacle, its current distance from the conductor in the power transmission channel, and its future trajectory.

[0019] An adaptive control module, configured to perform the following operations in response to the corrected dynamic risk level output by the risk assessment module:

[0020] Based on the revised dynamic risk level, select a corresponding fusion strategy from the fusion architecture that includes at least a feature layer fusion strategy and a decision layer fusion strategy;

[0021] Based on the future trajectory of the dynamic obstacles and the three-dimensional geographic information system security boundary of the power transmission channel, a three-dimensional dynamic obstacle avoidance route is generated online in real time, and instructions for controlling the flight of the UAV are generated.

[0022] Furthermore, based on the expected computing power requirements of the selected fusion strategy, instructions are generated for feedforward scheduling of UAV onboard computing resources.

[0023] The early warning generation module is used to generate early warning signals based on the corrected dynamic risk level.

[0024] To achieve the above objectives, a third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the above-described method for dynamic obstacle perception fusion and early warning of UAV edge power transmission channels.

[0025] The UAV edge power transmission channel dynamic obstacle perception fusion early warning method and system of the present invention adaptively switches between different multimodal data fusion architectures according to the dynamically assessed risk level, and manages airborne computing resources in a collaborative and feedforward manner. This enables the system to concentrate computing power to achieve accurate perception in high-risk scenarios and save resources in low-risk scenarios, thereby intelligently balancing the contradiction between perception accuracy and airborne computing power overhead.

[0026] Secondly, under normal circumstances, the present invention executes a smooth global planning route; however, once a sudden change in the obstacle trajectory is detected, causing the planning to fail, the system can immediately interrupt the current route and instantly activate the emergency response layer to execute the fastest evasive maneuver, ensuring ultimate safety when dealing with highly dynamic and uncertain threats; finally, by introducing a self-learning and iterative optimization mechanism for predicted failure samples, the present invention also endows the system with the ability to continuously evolve, significantly improving its overall robustness and intelligence level when dealing with complex dynamic environments. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the dynamic obstacle perception and fusion early warning method for UAV edge power transmission channels provided by the present invention.

[0028] Figure 2 This is a schematic diagram of the superposition result of lidar point cloud data and infrared imaging data after spatial registration in the dynamic obstacle perception fusion early warning method for UAV edge power transmission channels provided by the present invention.

[0029] Figure 3 This is a comparison of the recognition accuracy before and after using multimodal cross-validation logic in the dynamic obstacle perception and fusion early warning method for UAV edge power transmission channels provided by this invention.

[0030] Figure 4 This is a schematic diagram comparing the Kalman filter predicted trajectory with the actual trajectory in the dynamic obstacle perception fusion early warning method for UAV edge power transmission channels provided by this invention;

[0031] Figure 5 This is a schematic diagram of the relationship between risk level and intrusion time in the dynamic obstacle perception fusion early warning method for UAV edge power transmission channels provided by the present invention;

[0032] Figure 6 This is a simulation diagram of a three-dimensional dynamic obstacle avoidance route in the dynamic obstacle perception and fusion early warning method for UAV edge power transmission channels provided by the present invention;

[0033] Figure 7 This is a schematic diagram of the APF / DWA obstacle avoidance emergency response path in the UAV edge power transmission channel dynamic obstacle perception fusion early warning method provided by the present invention;

[0034] Figure 8 This is a schematic diagram illustrating the implementation of the UAV edge power transmission channel dynamic obstacle perception fusion early warning system provided by the present invention;

[0035] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0036] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0037] The following description, with reference to the accompanying drawings, describes a method, system, and electronic device for dynamic obstacle perception and fusion early warning of unmanned aerial vehicle (UAV) edge power transmission channels according to embodiments of the present invention.

[0038] Example 1:

[0039] Figure 1 This is a flowchart illustrating the dynamic obstacle perception, fusion, and early warning method for power transmission channels using unmanned aerial vehicles (UAVs) provided in this embodiment. This method can run on a UAV platform equipped with an onboard computing unit and multiple types of sensors to achieve accurate perception, risk assessment, and early warning of dynamic targets within the power transmission channel environment. The method specifically includes the following steps:

[0040] Step S1: Real-time acquisition of multimodal data from the power transmission channel using multiple sensors mounted on the UAV. In this embodiment, the UAV is specifically a multi-rotor industrial UAV, possessing excellent hovering performance and maneuverability, suitable for close-range inspection tasks in complex power transmission channel environments. The onboard computing unit can be an embedded high-performance computing platform, such as an edge computing device containing a graphics processing unit (GPU), which provides computing power support for subsequent complex data processing and algorithm execution. The multi-sensor system is rigidly mounted on the UAV body, either directly or via a vibration-damping gimbal, to ensure the stability of data acquisition.

[0041] For example, a multi-sensor system is a carefully designed and calibrated combination of heterogeneous sensors, including at least a visible light camera, an infrared camera, and a lidar. This combination of three sensors can complement each other to adapt to the complex, all-weather, multi-scenario environment of power transmission channels.

[0042] Visible light cameras, such as a high-resolution industrial zoom camera, are primarily used to acquire high-definition texture and color information of the power transmission channel environment. The visible light image data they generate plays an irreplaceable role in identifying obstacles with obvious visual characteristics, such as construction vehicles or personnel clothing of specific colors.

[0043] An infrared camera, specifically a long-wave infrared thermal imager, images objects by detecting the thermal radiation from their surfaces. The generated infrared temperature data is not limited by lighting conditions and can effectively detect dynamic obstacles with thermal signals, such as running engines, moving people, or animals, at night or in inclement weather.

[0044] LiDAR, such as a multi-line mechanical or solid-state lidar, actively emits laser beams and receives reflected signals. It uses the Time of Flight (ToF) method to accurately measure the three-dimensional spatial coordinates of various points in the environment, thereby generating high-precision lidar point cloud data. This data plays a decisive role in directly obtaining the size, shape, and spatial location of obstacles.

[0045] Optionally, during the data acquisition phase, to ensure the accuracy of subsequent multimodal data fusion, the system needs to perform strict time synchronization and spatial registration of data from different sensors. Time synchronization can be achieved through hardware triggering or software synchronization based on the Network Time Protocol (NTP), ensuring that data frames acquired by different sensors at the same time can be correlated. Spatial registration is completed during the system calibration phase. By calculating and storing the external parameter transformation matrices of each sensor coordinate system relative to the UAV body coordinate system, all acquired data can ultimately be unified under the same coordinate system, typically the UAV body coordinate system or the global geographic coordinate system, laying the foundation for subsequent data-level or feature-level fusion. The real-time acquired multimodal data is stored in the memory buffer of the onboard computing unit and is available for subsequent processing steps in the form of streaming data.

[0046] like Figure 2 The image shows the superimposed result of spatial registration of lidar point cloud data and infrared imaging data. Figure 2 The background of the image uses a color thermal map to simulate the thermal radiation information captured by infrared cameras during nighttime inspections of actual power transmission channels. The colors, from blue to red, correspond to low-temperature to high-temperature regions. The red area in the center of the image represents a target heat source A with a significant thermal signal, which may be personnel or operating machinery. The black contour lines superimposed on it represent the height distribution of the registered lidar point cloud, reflecting the three-dimensional spatial morphology of obstacles in that area. Figure 2 The black cross in the center marks obstacle B. The dense and undulating contour lines here indicate that the target has a solid outline, such as a construction vehicle or floating object. Its high spatial overlap with heat source A demonstrates the high consistency between the lidar dynamic candidate box and the infrared candidate box after spatial registration. Furthermore, the noise texture of the contour lines simulates the uncertainties of actual sensors.

[0047] Step S2: By processing the multimodal data, determine the dynamic obstacles and their types within the power transmission channel, and predict the future trajectories of the dynamic obstacles. This step can be further broken down into several sub-steps, such as target identification, type confirmation, and trajectory prediction.

[0048] For example, the process of determining dynamic obstacles and their types specifically includes first performing preliminary processing on the individual modal data to generate their respective dynamic candidate targets.

[0049] For LiDAR point cloud data, the system first reduces data density through voxel downsampling. Then, it uses ground point filtering algorithms, such as Random Sample Consensus (RANSAC), to filter out ground point clouds. Next, it applies clustering algorithms, such as Density-Based Noise Spatial Clustering (DBSCAN), to the remaining non-ground point clouds, clustering spatially adjacent point clouds into different clusters. By analyzing the geometric features of each cluster, such as its size and number of points, and comparing it with the point cloud clusters of the previous frame, the system can identify dynamic point cloud clusters that have undergone displacement and generate a three-dimensional dynamic candidate bounding box for each.

[0050] For infrared temperature data, the system first sets an ambient temperature baseline, and then uses image segmentation algorithms, such as adaptive thresholding, to extract regions whose temperatures are significantly higher or lower than the ambient baseline. These regions are likely to correspond to heat-generating engines or living organisms. Morphological processing, such as erosion and dilation, is then applied to these regions to eliminate noise, ultimately generating a two-dimensional dynamic infrared candidate box for each connected thermal anomaly region.

[0051] For visible light images, the system can use a deep learning-based target detection model, such as the YOLO series model. This model has been pre-trained on a dataset containing common obstacles in various power transmission channels and can directly identify the categories of targets such as "people", "vehicles", and "tower cranes" in the input image and output two-dimensional visible light dynamic candidate boxes of their positions.

[0052] Optionally, after generating dynamic candidate boxes in multiple modalities, to improve the accuracy and robustness of recognition, the system will perform a multimodal candidate box cross-validation step based on preset cross-validation logic. This step aims to utilize the redundancy and complementarity of information from different sensors to confirm the authenticity of the target and accurately determine its type. The cross-validation logic is a pre-defined set of rules based on the physical characteristics of different types of dynamic obstacles in different modalities.

[0053] For example, for the category of "large construction machinery," it typically possesses a clear three-dimensional outline, significant metallic or vehicle-body visual features, and thermal signals from engine operation. Therefore, its cross-validation logic can be set as follows: when the spatial intersection-union ratio (IoU) of a lidar dynamic candidate box, a visible light dynamic candidate box, and an infrared dynamic candidate box in a unified coordinate system is greater than a certain preset threshold (e.g., 0.6), that is, when the three have a high degree of overlap, the target is identified as "large construction machinery."

[0054] Conversely, for the "flying birds" type, there might only be a weak, unstable point cloud cluster in the lidar, but clear features in visible and infrared images. Therefore, the cross-validation logic can be set as follows: when a visible light dynamic candidate box and an infrared dynamic candidate box highly overlap, and a small lidar dynamic candidate box exists near the overlapping area, the target can be confirmed.

[0055] Furthermore, for non-living, heat-free obstacles such as "suspended insulator strings" or "floating advertising banners," the cross-validation logic could be set to confirm an obstacle only if there is a high degree of overlap between the LiDAR dynamic candidate box and the visible light dynamic candidate box. Through this cross-validation based on prior knowledge, the system can effectively filter out false targets generated by noise or false detections from a single sensor, thereby greatly improving the reliability of dynamic obstacle recognition.

[0056] like Figure 3 The diagram shows the comparison of recognition accuracy before and after using multimodal cross-validation logic in different obstacle type recognition tasks. Figure 3 The horizontal axis lists five typical dynamic obstacles: construction vehicles, flying birds, inspection personnel, insulator strings, and floating banners. These are all common dynamic or semi-dynamic obstacle sources in power transmission channel environments. The vertical axis represents the system's accuracy in identifying these targets. The blue bars represent the performance of single-mode processing relying solely on lidar for identification. Its overall performance is generally limited, especially for flying birds and inspection personnel, where the accuracy is only 35% and 50% respectively, reflecting its insufficient ability to handle high-speed or irregularly shaped targets.

[0057] and Figure 3 The red bars represent the performance of the fusion strategy used in this step, which employs a three-modal approach combining infrared images, visible light images, and lidar point clouds for joint identification and confirms the results through cross-validation logic. This strategy significantly outperforms the single-modal approach across all target types, achieving a maximum accuracy of 96% and a minimum of 85%. Particularly for flying birds and floating banners, the accuracy improvement exceeds 50%, clearly demonstrating the significantly enhanced ability of the fusion perception system to distinguish targets with blurred shapes or weak thermal features.

[0058] After determining the presence and type of dynamic obstacles, the system needs to further predict their future trajectories in order to conduct a risk assessment.

[0059] For example, the process of predicting the future trajectory of a dynamic obstacle begins by calling the most suitable motion model from a pre-set motion model library based on the identified type of dynamic obstacle. This motion model library is the product of long-term operation and experience accumulation of the system, and stores a variety of classic motion models. For example, the motion model library includes uniform linear motion models, which are suitable for describing vehicles or other distant aircraft traveling long distances stably in open areas; random walk models or Brownian motion models, which are suitable for describing pedestrians or animals moving erratically within a small area; and parabolic motion models, which can be used to describe objects thrown or falling in the area of ​​power transmission channels. The system selects an initial motion model based on the identified obstacle type, such as prioritizing uniform linear motion models for "vehicles" and random walk models for "pedestrians".

[0060] Optionally, after selecting a suitable motion model, the system applies the motion model to a Kalman filter module to optimally estimate the state of the dynamic obstacle and predict its future trajectory. The Kalman filter module is a powerful regressive state estimation algorithm, well-suited for processing noisy sensor measurement data. In this embodiment, the state vector maintained by the Kalman filter module is a high-dimensional vector, which at least includes the obstacle's position, velocity, and acceleration in three-dimensional space.

[0061] For example, a nine-dimensional state vector:

[0062]

[0063] in, : Represents the state vector maintained by the Kalman filter module, used to describe the motion state of dynamic obstacles; : These represent the x-axis, y-axis, and z-axis position coordinates of the dynamic obstacle in the three-dimensional spatial coordinate system, respectively; : These represent the velocity components of a dynamic obstacle in the x-axis, y-axis, and z-axis directions in the three-dimensional coordinate system, respectively; : These represent the acceleration components of a dynamic obstacle in the x-axis, y-axis, and z-axis directions in the three-dimensional spatial coordinate system, respectively.

[0064] Specifically, the Kalman filter operates in two alternating steps: prediction and update. In the prediction step, the system uses the selected motion model (e.g., a uniform linear motion model corresponding to a state transition matrix) to predict the obstacle's state at the next moment. In the update step, the system uses the obstacle's position observed through multi-sensor fusion at the current moment as a measurement to correct the state obtained in the prediction step, thus obtaining the optimal state estimate for the current moment. By continuously executing these two steps, the Kalman filter module can not only smooth the trajectory jitter caused by sensor noise but also, based on the estimated velocity and acceleration, extrapolate a high-confidence future trajectory, which is typically represented as a series of three-dimensional spatial coordinates at future moments.

[0065] like Figure 4 It demonstrates the comparison between the actual path of a dynamic obstacle's motion trajectory, the noise observation point, and the Kalman filter predicted path.

[0066] Figure 4 The solid blue line represents the actual trajectory of the obstacle during its movement. Its shape is a gently undulating curve, reflecting the natural movement of the target in the power transmission channel under the influence of airflow disturbance and inertia. The red scatter dots represent the measurement data observed by the sensors, which are distributed on both sides of the actual trajectory. The local deviation is about 0.3 meters, reflecting the random noise and time delay errors present in multi-sensor measurements. The green dashed line represents the result of predicting the obstacle's position using the Kalman filter algorithm. The curve trend almost coincides with the solid blue line, with only a very small deviation at the turning point, indicating that the algorithm can effectively suppress noise interference and accurately track the target's dynamic characteristics in continuous time.

[0067] Step S3: After completing the perception and prediction, the system calculates the corrected dynamic risk level based on the type of dynamic obstacle, the current distance to the conductor in the power transmission channel, and the future trajectory. This step aims to quantify the abstract perception results into a specific risk indicator that can be used by the decision-making system.

[0068] For example, the process of calculating the revised dynamic risk level first involves calculating an initial value for the dynamic risk level, denoted as . This initial value primarily reflects the static risk posed by obstacles due to their inherent properties and current location. Its calculation comprehensively considers two core factors: the obstacle's type weight, denoted as... And the current safe distance between the obstacle and the conductors in the transmission channel, denoted as Among them, type weight This is a dimensionless coefficient value preset based on the potential threat level of different obstacle types to the power transmission system. For example, large tower cranes or construction machinery, due to their enormous size and potential direct contact with power lines, can have their weight set to the highest value of 1.0; moving personnel, due to the uncertainty of their behavior, can have their weight set to a medium value of 0.7; while smaller birds, posing less direct threat, can have their weight set to a lower value of 0.3. The current safe distance... This refers to the minimum Euclidean distance between the 3D bounding box boundary of an obstacle and the 3D model of the nearest power transmission line at the current moment. The 3D model of the power transmission line is usually pre-existing in the UAV's geographic information system in the form of a catenary equation.

[0069] Based on the above analysis, the initial value of the dynamic risk level is... The calculation formula can be expressed as:

[0070]

[0071] in, It is a very small positive number used to prevent the denominator from being zero when the distance is zero.

[0072] This formula intuitively reflects that the more dangerous the type of obstacle and the closer it is to the conductor, the higher the initial risk.

[0073] Optionally, considering only the risk at the current location is insufficient; the system must also consider the future movement of obstacles. Therefore, after calculating the initial dynamic risk level, the system determines whether the predicted future trajectory will intrude into a pre-defined transmission channel safety zone. This safety zone is a three-dimensional virtual electronic fence established around the transmission line, with its radius set according to the voltage level and safety regulations. The system performs collision detection on the sequence of future trajectory points predicted by the Kalman filter against the geometric model of this safety zone. If a trajectory point at a future moment is detected to be located inside the safety zone, the system determines that the future trajectory poses an intrusion risk and calculates the time difference from the current moment to the first intrusion. This time difference is defined as the predicted intrusion time, denoted as . If the predicted future trajectory does not collide with the safe zone throughout the entire prediction time domain, then it can be... Set it to an infinite or sufficiently large constant value.

[0074] Finally, the system is based on the initial value of the dynamic risk level. and predicting intrusion time To calculate the final, revised dynamic risk level, denoted as... According to the core idea of ​​this embodiment, the final risk should be directly proportional to the initial risk and inversely proportional to the intrusion time. This is because a shorter intrusion time implies a more imminent danger, and therefore the risk level should naturally be higher. Therefore, for An example calculation formula is as follows:

[0075]

[0076] in, It is an adjustable urgency coefficient used to amplify the influence of the time factor; It is a very small positive number, used to prevent the denominator from being zero when the intrusion time is extremely short.

[0077] With this modified formula, an obstacle that is currently far away but is moving at high speed toward the guide line may be rated as high-risk due to its extremely short predicted intrusion time, which greatly improves the foresight of risk assessment.

[0078] like Figure 5 This demonstrates the quantitative relationship between the modified risk level and the predicted intrusion time in the dynamic risk assessment method. The curve is presented as a red broken line. It can be seen that when the predicted intrusion time is extremely short, such as about 1 second, the risk level is close to 1.2, indicating that the obstacle is about to rapidly enter the safety boundary of the power transmission channel. At this time, the system should immediately switch to the feature layer fusion strategy and pre-allocate airborne computing resources to ensure accurate identification of high-speed threats. However, when the intrusion time gradually increases to more than 5 seconds, the curve drops rapidly to below 1, indicating that such obstacles will not pose an imminent threat in the short term. The system can adopt a decision layer fusion strategy with lower computational cost, thereby effectively saving computing power and extending the drone's endurance.

[0079] Step S4: After calculating the corrected dynamic risk level, the system will execute subsequent response steps to dynamically adjust the fusion strategy for processing multimodal data and the UAV's flight strategy in response to the corrected dynamic risk level, including the following:

[0080] (a) Based on the modified dynamic risk level, select a corresponding fusion strategy from the fusion architecture that includes at least a feature layer fusion strategy and a decision layer fusion strategy to perform fusion processing on the multimodal data;

[0081] (b) Based on the future trajectory of the dynamic obstacle and the three-dimensional geographic information system (GIS) security boundary of the power transmission channel, a three-dimensional dynamic obstacle avoidance route is generated online in real time, and the UAV is controlled to fly along the route.

[0082] (c) Based on the expected computing power requirements of the fusion strategy selected in step (a), feedforward scheduling of the computing resources of the UAV onboard unit.

[0083] Step S5: Finally, based on the revised dynamic risk level, an early warning signal is generated. For example, when When a certain high-level threshold is exceeded, the system can send an emergency alarm message to the remote ground control station and trigger an emergency obstacle avoidance procedure.

[0084] In summary, the method disclosed in this embodiment systematically integrates multimodal data acquisition, robust target recognition based on cross-validation, trajectory prediction based on adaptive models, and dynamic risk quantification that comprehensively considers the current state and future trends. This constructs a complete and efficient framework for dynamic obstacle perception and risk assessment in UAV inspection, providing reliable, accurate, and forward-looking information input for subsequent intelligent decision-making and autonomous obstacle avoidance.

[0085] Example 2:

[0086] This second embodiment further refines step S4 based on the first embodiment. The core of this embodiment lies in constructing an online closed-loop control system that integrates obstacle avoidance planning and emergency response. This system receives the corrected dynamic risk level calculated in the first embodiment. As the core input, and based on this, its sensing and flight behaviors are dynamically and collaboratively adjusted to maximize the system's energy efficiency and robustness while ensuring ultimate safety. This embodiment specifically includes the following steps:

[0087] First, when the risk assessment module outputs the revised dynamic risk level Then, the first step is to dynamically adjust the fusion strategy used to process multimodal data.

[0088] For example, the core idea of ​​this step is to make risk-based decisions, intelligently balancing the system's need for sensing accuracy with its consumption of computing resources. A second preset risk threshold is pre-defined within the system, denoted as... This threshold is used to distinguish between routine risks and emergency high-risk risks. When the revised dynamic risk level... Higher than the second preset risk threshold When the system determines that the current risk is an urgent and high-risk situation, the accuracy requirement for acquiring obstacle state information, such as position, velocity, and attitude, is at its highest. Therefore, the system will select a feature layer fusion strategy to fuse the multimodal data.

[0089] Feature layer fusion, also known as intermediate layer fusion, is a deep fusion method performed in the early stages of data processing. Under this strategy, raw data from visible light cameras, infrared cameras, and LiDAR, or low-level features extracted after preliminary preprocessing—such as pixel-level color gradients and SIFT feature points in visible light images, temperature gradient features in infrared images, and geometric normals and curvatures of LiDAR point clouds—are fed into a unified deep neural network model. This network model, through its complex internal convolution, pooling, and attention mechanisms, learns how to optimally combine and correlate these heterogeneous low-level features in the feature space, thereby jointly inferring the precise state of obstacles. Because feature layer fusion utilizes the most original and abundant information, avoiding information loss that may occur in individual modal processing, it typically obtains the highest accuracy obstacle state information. Its advantages are particularly evident when distinguishing targets with similar shapes but different materials or temperatures, or when performing target detection in adverse weather conditions.

[0090] Optionally, when the revised dynamic risk level Less than or equal to the second preset risk threshold When the system determines that the current risk is a routine risk or that there is no significant threat in the environment, the system's primary goal is to conserve valuable onboard computing resources to extend flight time or support other inspection tasks. Therefore, the system will select a decision-level fusion strategy with lower computational overhead for fusion processing.

[0091] Decision-level fusion, also known as high-level fusion, is a fusion method performed at the end of the data processing pipeline. In this strategy, the data stream from each sensor passes through an independent, complete processing pipeline, each generating a preliminary decision result regarding the target state. For example, the visible light processing pipeline outputs a target detection box and its category, the lidar processing pipeline outputs a 3D point cloud clustering box, and the infrared processing pipeline outputs a heat source target box. Then, a lightweight fusion center module fuses these high-level decision results from different information sources. The fusion method can be simple, such as weighted voting based on spatial overlap; it can also be more complex, such as treating each decision result as different observations of the same target state and then using an independent Kalman filter or Bayesian inference network to fuse these observations to obtain a more reliable final decision. Because the processing pipelines can run in parallel, and the computational cost of the fusion center is much smaller than that of the deep network used in feature-level fusion, the overall computational overhead of the decision-level fusion strategy is significantly reduced.

[0092] For example, in order to further improve the adaptability of the fusion strategy, especially the tracking robustness when dealing with high-speed dynamic obstacles, this embodiment also adaptively adjusts the fusion weights of the lidar point cloud data and the visible light image according to the speed v of the dynamic obstacle during the fusion process.

[0093] Specifically, the system estimates the obstacle's velocity v in real time, which can be directly output by the Kalman filter module in Example 1. Simultaneously, the system runs an optical flow analysis module for the visible light image sequence, using, for example, the Lucas-Kanade optical flow algorithm or the Horn-Schunck optical flow algorithm.

[0094] Optical flow analysis can calculate the motion vector field of pixels in an image, thus obtaining a motion feature independent of target recognition and purely based on image changes. When the obstacle's velocity *v* is higher, it means the target's apparent motion in the image is faster. In this case, target detection models relying solely on single-frame features may experience performance degradation due to motion blur or drastic pose changes. The motion features provided by optical flow analysis are precisely most sensitive to this high-speed motion. Therefore, the system dynamically increases the fusion weight of optical flow-based motion features in visible light images. In feature layer fusion, this means assigning higher channel weights to the optical flow feature map at the feature stitching layer of the neural network; while in decision layer fusion, it means giving higher confidence to the visible light observation results, which have been corrected by optical flow analysis, during final state estimation. In this way, the system dynamically correlates the physical motion state of the obstacle with its representation in sensor data, effectively enhancing the robustness of tracking high-speed targets.

[0095] Optionally, to avoid system lag or even crashes due to sudden increases in computing power before the system decides to switch to the high-overhead feature layer fusion strategy, this embodiment also introduces a feedforward, proactive resource management mechanism. Specifically, before dynamically adjusting the fusion strategy, the system pre-evaluates the expected computing power requirements of the selected fusion strategy, especially the feature layer fusion strategy, within a future time window (e.g., the next second). This expected computing power requirement can be obtained through offline model performance profiling, i.e., by running the algorithm once on the target hardware platform in advance, recording its average CPU utilization, GPU memory utilization, and computing unit utilization, and storing these indicators in a lookup table. When the system decides to enable feature layer fusion, it retrieves the corresponding computing power requirement from this lookup table.

[0096] For example, after obtaining the expected computing power requirement, the system compares it with the current computing power load L on the UAV's onboard unit. The current computing power load L is obtained by monitoring the system load information reported by the operating system kernel in real time. If the predicted total load, i.e., the sum of the current computing power load and the expected computing power requirement, exceeds a preset system load threshold (such as 85% CPU utilization), it means that the system is about to enter an overload state. At this time, the system will not passively wait for the overload to occur, but will proactively reduce or suspend the execution priority of non-urgent tasks in a feedforward manner to release computing power resources in advance and make room for the upcoming high-intensity perception tasks.

[0097] Non-urgent tasks typically refer to background tasks that do not have high real-time requirements and can be delayed or executed at a reduced frequency. For example, a panoramic image stitching task for power transmission channels requires a large amount of memory and CPU for image feature matching and stitching, and can be temporarily suspended; another example is a 3D detailed modeling task for power equipment, which requires continuous processing of high-density point cloud data, and its data processing frequency can be temporarily reduced or the task can be paused.

[0098] Through this feedforward resource scheduling, this embodiment ensures that the core perception algorithm can obtain sufficient computing resources when facing high risks and requiring the most accurate perception capabilities, thereby guaranteeing the stability of the entire system and the performance of critical tasks.

[0099] Next, this embodiment further describes the points that need attention when generating a three-dimensional dynamic obstacle avoidance flight path in real time and controlling the UAV to fly along the path, which are also described here using the corrected dynamic risk level. It serves as input and is responsible for generating safe and efficient flight control commands.

[0100] For example, the system has a preset first risk threshold, denoted as... This threshold is typically lower than or equal to the aforementioned second preset risk threshold. When the revised dynamic risk level Higher than the first preset risk threshold When the obstacle poses a substantial threat, the system needs to initiate proactive obstacle avoidance planning. At this point, the system will execute a real-time online generation of a dynamic 3D obstacle avoidance path. The core of this step is to transform the obstacle avoidance problem into a constrained optimization problem and solve it using advanced planning algorithms in a very short time.

[0101] Optionally, the input to this step is multi-source and precise. It includes the future trajectory of the dynamic obstacle predicted by Example 1, which is a series of four-dimensional coordinate points with timestamps; the three-dimensional geographic information system (GIS) security boundary of the power transmission channel preloaded in the system, which is a precise three-dimensional geometric model that defines the no-fly zone, typically represented as a columnar or polygonal columnar envelope around each conductor and tower; and the kinematic constraints of the UAV itself, including parameters such as the UAV's maximum flight speed, maximum acceleration, and maximum angular velocity, ensuring that the generated flight path is physically executable by the UAV.

[0102] For example, based on the above input, the system solves for the flight path in real time using either the Fast Search Random Tree (RRT) algorithm or the Model Predictive Control (MPC) algorithm. If the RRT algorithm is chosen, the system, starting from the UAV's current state, rapidly grows a path tree capable of exploring collision-free safe zones within a configuration space that considers kinematic constraints, through random sampling and node expansion. In particular, its optimized variant, the RRT algorithm, can further optimize a feasible path after finding one, aiming to find a near-optimal path within the constraints of computational time.

[0103] If the MPC algorithm is chosen, the system will establish a mathematical model within a finite future time domain, including the UAV dynamics model, obstacle prediction trajectory, safety boundary constraints, and optimization objectives such as energy consumption or path length. The optimal control input sequence for a short future time period will then be solved using numerical optimization methods. The advantage of MPC lies in its ability to explicitly handle various constraints and generate very smooth flight trajectories.

[0104] Regardless of the algorithm used, the final output is a three-dimensional dynamic obstacle avoidance flight path composed of a series of dense landmarks. The system then controls the UAV to fly along this path. Throughout the flight, the path ensures that the distance between the UAV and the power transmission line within the transmission channel is always greater than a preset minimum safe flight distance. This is achieved by incorporating the GIS safety boundary as a hard constraint into the planning algorithm.

[0105] like Figure 6 The core simulation process of online generation of three-dimensional dynamic obstacle avoidance routes was demonstrated. Figure 6 The red dashed line represents the future trajectory of the dynamic obstacle predicted by the system through the Kalman filter. It exhibits a spiraling trend in space, simulating irregularly moving targets such as flying birds or drones.

[0106] The black solid line represents the three-dimensional shape of the transmission conductor modeled in the geographic information system. Its drooping structure reflects the typical catenary distribution characteristics, truly reflecting the physical characteristics of the transmission line under high-voltage operation.

[0107] The solid green line represents the 3D flight path generated in real time during obstacle avoidance calculations. This path winds its way from the aircraft's starting point to the target point, with its altitude varying near obstacles. This indicates that the system has detected a potential threat and is actively increasing its altitude to ensure that the minimum safe flight distance from the guide wire remains above a set threshold. The path also exhibits a slight curvature in the horizontal direction to avoid potential overlap between the obstacle trajectory prediction area and the guide wire.

[0108] Moreover, this path is not a static planning result, but is dynamically generated based on the future trajectory of obstacles, 3D GIS boundaries, and UAV dynamics model. After the risk level is determined to be medium to high risk, it is generated through planning algorithms (such as RRT or MPC), which has the characteristics of real-time and responding to emergencies.

[0109] However, the effectiveness of the aforementioned obstacle avoidance scheme heavily relies on the accuracy of predicting the future trajectory of obstacles. In the real world, dynamic obstacles, especially intelligent life forms or human-controlled machinery, exhibit highly unpredictable behavior. When an obstacle undergoes an unforeseen sudden maneuver, the originally planned optimal route may instantly become invalid. To address this technical problem, this embodiment creatively introduces a two-layer control architecture, namely, an emergency response layer with a faster reaction speed overlayed on top of the aforementioned conventional processing methods.

[0110] For example, during the UAV's flight along a three-dimensional dynamic obstacle avoidance path, the system continuously monitors the real-time trajectory of dynamic obstacles at a higher frequency. This real-time trajectory is the observed position at the current moment, obtained by rapidly processing the latest frame of sensor data, with extremely low latency. The system calculates in real time the trajectory deviation between the real-time trajectory and the future trajectory on which the path was based. This deviation can be simply defined as the Euclidean distance between the real-time observed position and the predicted position at the same time t.

[0111] Optionally, the system internally sets a preset abrupt deviation threshold. This threshold needs to balance sensitivity and stability; it can be a fixed empirical value or an adaptive value related to the obstacle type or speed. When the trajectory deviation increases sharply within a short period and exceeds this abrupt deviation threshold, the system can determine that the underlying motion model can no longer describe the obstacle's current behavior, the prediction has completely failed, and therefore, the 3D dynamic obstacle avoidance route generated based on the erroneous prediction has also become invalid.

[0112] In this emergency, the system will trigger a circuit breaker-takeover mechanism: First, the system will immediately stop executing the 3D dynamic obstacle avoidance flight path generated by the RRT or MPC algorithm, meaning the UAV's flight control system will no longer follow the original landmark sequence. Immediately afterwards, the system will instantly activate an emergency response layer based on the Artificial Potential Field (APF) method or the Dynamic Window (DWA) method. This emergency response layer is a controller with extremely lightweight computation and millisecond-level response speed.

[0113] For example, if an artificial potential field method is used, the emergency response layer generates a local repulsive field around the drone based on the real-time trajectory of the dynamic obstacle, i.e., its latest position. This repulsive field is centered on the obstacle, and its strength increases exponentially with decreasing distance. Simultaneously, the system may define a virtual gravitational point away from the obstacle. The resultant force acting on the drone will drive it to perform the most direct and rapid emergency obstacle avoidance maneuvers, such as sudden stops, lateral or upward accelerations.

[0114] If the dynamic window method is adopted, the emergency response layer will sample a series of speed commands that can be reached in the next instant in the speed space of the drone, combined with its kinematic constraints, and evaluate the short-term trajectory generated by each command, mainly assessing the collision risk with the real-time obstacle position.

[0115] Ultimately, the speed command that minimizes risk and allows for the fastest escape from danger is selected and sent to the flight controller. Whether it's APF or DWA, the goal is not global optimization, but instantaneous safety.

[0116] Guided by the emergency response layer, the UAV performs emergency avoidance maneuvers until the real-time distance between the UAV and the dynamic obstacle returns to above a preset safety threshold, indicating that the emergency danger has been eliminated. Finally, after the emergency obstacle avoidance maneuver is completed, control is returned from the emergency response layer to the planning layer. The system will then trigger the RRT algorithm or MPC algorithm again to regenerate a new, safe, and subsequent three-dimensional dynamic obstacle avoidance path based on the updated and realistic UAV and obstacle states.

[0117] like Figure 7 This embodiment demonstrates the emergency response mechanism triggered in case of sudden changes in obstacle trajectory or failure of the original planned path during the execution of a dynamic obstacle avoidance route by the UAV, particularly the obstacle avoidance path adjustment principle based on the artificial potential field method or the dynamic window method.

[0118] Figure 7The vector field formed by the black arrows represents the synthetic obstacle avoidance direction perceived by the aircraft at any position in the scene. The attractive force towards the target point ensures that the overall flight target remains unchanged, while the red circular obstacle area generates a strong repulsive force on the surrounding area, causing the drone to spontaneously move away from the collision risk. The blue asterisk marks the current flight point, and the green dot represents the target position set by the system. The synthetic vector that passes through the obstacle area in the middle of the path is clearly curved and deflected, reflecting the strategy change process mentioned above in response to sudden risks in a short period of time.

[0119] Figure 7 This invention clearly demonstrates that when dealing with sudden deviations from the trajectory, the system immediately severs the original flight path control and activates a lightweight emergency obstacle avoidance mechanism. It does not rely on complex trajectory prediction but dynamically generates local safe speed commands based on the real-time position of obstacles, guiding the drone to escape the danger zone in a very short time. Figure 7 The spatial distribution characteristics of the mid-potential field vector show that, while maintaining the global heading unchanged, the system uses the field strength gradient to guide the UAV around obstacles and restores the original navigation logic in a suitable area, thus balancing flight safety and path efficiency, demonstrating its robustness and real-time response capability.

[0120] In summary, this embodiment not only resolves the contradiction between perception accuracy and computing power consumption, but also solves the contradiction between optimal planning and real-time response through an innovative two-layer control architecture, thereby raising the autonomous safety and intelligence level of UAVs in complex dynamic environments to a new level.

[0121] Example 3:

[0122] This embodiment, based on the online real-time processing method disclosed in Embodiment 1 or Embodiment 2, further provides an offline technical solution for self-learning and iterative optimization. The core objective of this solution is to systematically improve the performance of its core algorithm modules by deeply analyzing and mining cases of perception and prediction failures in historical inspection tasks. This enables the UAV system to continuously evolve, exhibiting higher prediction accuracy and decision reliability in future inspection tasks. The method in this embodiment is typically executed on a ground server or cloud computing platform after the UAV completes its inspection task and returns.

[0123] Specifically, the method in this embodiment mainly includes three steps: review and analysis of historical data, screening and identification of high-value negative samples, and targeted model optimization.

[0124] First, a necessary prerequisite before performing offline optimization is that complete data recording was performed during the online inspection task. In the real-time operation of the UAV in Example 1, the onboard computing unit not only performed perception and early warning tasks, but also recorded key intermediate process data and final results in a structured log.

[0125] For example, for each dynamic obstacle identified and continuously tracked by the system, the logging system will record a data packet containing multi-dimensional information, using its unique identifier as the primary key. This data packet should include at least:

[0126] (1) The type of obstacle to be identified, such as a vehicle or a pedestrian, and the observation status at each moment during the entire tracking period. This observation status can be the location information directly obtained by multi-sensor fusion.

[0127] (2) The future trajectory prediction sequence output by the Kalman filter module of Example 1 at each time step;

[0128] (3) and the type of motion model selected by the system when making predictions, such as uniform linear model or random walk model;

[0129] (4) In addition, environmental parameters such as light intensity and weather conditions can be recorded.

[0130] These meticulously recorded log data form the basis for subsequent offline analysis and learning.

[0131] After the inspection task is completed, the system initiates an offline optimization process. The first step of the process is to compare the predicted trajectory of the dynamic obstacle with its actual observed trajectory obtained through a post-event high-precision positioning module, and calculate the trajectory prediction error. This step aims to quantify the actual performance of the online prediction model.

[0132] Optionally, the post-event high-precision positioning module is not a physical entity, but rather a trajectory reconstruction algorithm that operates offline, boasting higher computational complexity and accuracy. During online operation, to ensure real-time performance, the system may only use a Kalman filter for state estimation. However, in offline mode, without real-time constraints, the system can utilize all observation data recorded throughout the tracking period and employ more advanced smoothing algorithms, such as the Raeden-Don-Stribell RTS smoother, to recalculate the historical trajectory of the obstacle. The RTS smoother can comprehensively utilize all past and future observations to estimate the current state; therefore, the trajectory it generates is generally considered to be the closest to the actual observed trajectory on the ground in the mission.

[0133] For example, trajectory prediction error can be calculated using several industry-recognized evaluation metrics. One commonly used metric is the Average Displacement Error (ADE), which calculates the average Euclidean distance between the predicted and actual trajectory points over the prediction time domain. Another metric is the Final Displacement Error (FDE), which focuses only on the Euclidean distance between the predicted and actual points at the last moment of the prediction time domain; this metric better reflects the model's ability to predict long-term trends. Regardless of the metric used, the goal is to obtain a quantitative value that objectively measures the success or failure of a single prediction.

[0134] The second step in the process is to screen high-value negative samples based on the trajectory prediction error. Specifically, the system will determine that when the trajectory prediction error exceeds a preset error threshold, the corresponding dynamic obstacle sample, environmental parameters, and the selected motion model will be jointly identified as high-value negative samples.

[0135] For example, the preset error threshold is not a fixed value, but can be a dynamic, scenario-dependent parameter. For instance, for obstacles far from power transmission lines, the system can tolerate larger prediction errors, so its error threshold can be set higher. However, for obstacles already within the warning zone and very close to the power lines, any small prediction deviation could lead to catastrophic consequences, so its error threshold should be set extremely low. In this way, the system can more effectively target prediction failures that pose a substantial threat at critical moments.

[0136] Optionally, these failed prediction samples are defined as high-value negative samples because they precisely expose the capability boundaries and knowledge blind spots of the current prediction model. Negative samples indicate that the model performs poorly on such samples, while high value lies in the rich information they contain. A high-value negative sample data package contains not only trajectory data, but more importantly, its accompanying complete "context," namely the aforementioned recorded environmental parameters, obstacle types, and the motion model selected by the system at the time. This contextual information is crucial for diagnosing the causes of failure.

[0137] For example, the system may find that a large number of negative samples appear in the specific scenario of "turning on a narrow road", and that the system has invariably made the wrong choice of the uniform straight line model at that time. This discovery directly reveals a major flaw in the current model selection logic.

[0138] The third step in the process involves offline training of the trajectory prediction model, which is used to predict future trajectories, using a high-value negative sample set to achieve iterative evolution of the model. This optimization can be applied to the model selection logic or to the model's internal parameters.

[0139] For example, an important optimization approach is to focus on optimizing the selection logic of each motion model in the motion model library. In Implementation 1, the selection of motion models might be based on a simple set of rules. Through offline optimization, the system can train a smarter model selector using all the high-value negative samples collected. For example, a classification model, such as a decision tree or a small neural network, can be built. The input features of this model are contextual information from the high-value negative samples, such as obstacle type, current speed, angular velocity, curvature of the area, etc., while its training labels are no longer the incorrectly selected motion model at the time, but the best motion model that should have been selected in that scenario, derived through post-analysis. For example, for the negative sample of "turning on a narrow road," its correct label should be the constant rev and speed model CTRV. By training on a large number of such negative samples, this new model selector will learn more refined decision rules; for example, when it detects that the vehicle's speed is not zero and the angular velocity is greater than a certain threshold, it will automatically select the CTRV model instead of the uniform straight-line model. Once training is complete, this optimized model selector will be deployed to the drone's online system to replace the original simple rules, thereby enabling more accurate model selection in future tasks.

[0140] Alternatively, another optimization approach is to update the state transition parameters of the Kalman filter module. The process noise covariance matrix in the Kalman filter, denoted by Q, represents the system's estimate of the uncertainty of the motion model itself. When the system detects that a certain type of motion model (e.g., a uniform linear motion model) repeatedly exhibits large prediction errors in specific scenarios (e.g., densely populated pedestrian areas), this indicates that the model is unreliable in that scenario, and its inherent uncertainty is underestimated. Therefore, the optimization module can specifically adjust the value of the process noise covariance matrix Q associated with this model.

[0141] Specifically, the system can systematically increase the terms in the Q-matrix related to velocity or acceleration changes. This increases process noise, essentially telling the Kalman filter, "Don't rely too much on the predictions of your motion model; depend more on new sensor observations." This allows the filter to track the target's actual maneuvers more quickly in similar situations, exhibiting better adaptability. This fine-tuning of model parameters is an effective way to learn from data and improve the model's physical assumptions.

[0142] In summary, the offline optimization method disclosed in this embodiment proactively learns from high-value negative samples that failed to predict, enabling targeted optimization of its internal motion model selection logic or model physical parameters, thereby achieving continuous iteration and improvement of cognitive capabilities. This self-learning and self-improvement mechanism greatly enhances the long-term adaptability and robustness of this embodiment in the face of constantly changing and complex dynamic environments, making the entire system ultimately behave as a continuously evolving intelligent inspection entity.

[0143] Example 4:

[0144] like Figure 8 As shown, corresponding to the above method embodiments, the present invention also proposes a dynamic obstacle perception and fusion early warning system for UAV edge power transmission channels, comprising:

[0145] The data acquisition module is used to collect multimodal data of the power transmission channel in real time through multiple sensors carried by the drone;

[0146] The dynamic sensing module is used to process the multimodal data, determine the dynamic obstacles and their types within the power transmission channel, and predict the future trajectory of the dynamic obstacles.

[0147] The risk assessment module is used to calculate the corrected dynamic risk level based on the type of the dynamic obstacle, its current distance from the conductor in the power transmission channel, and its future trajectory.

[0148] An adaptive control module, configured to perform the following operations in response to the corrected dynamic risk level output by the risk assessment module:

[0149] Based on the revised dynamic risk level, select a corresponding fusion strategy from the fusion architecture that includes at least a feature layer fusion strategy and a decision layer fusion strategy;

[0150] Based on the future trajectory of the dynamic obstacles and the three-dimensional geographic information system security boundary of the power transmission channel, a three-dimensional dynamic obstacle avoidance route is generated online in real time, and instructions for controlling the flight of the UAV are generated.

[0151] Furthermore, based on the expected computing power requirements of the selected fusion strategy, instructions are generated for feedforward scheduling of UAV onboard computing resources.

[0152] The early warning generation module is used to generate early warning signals based on the corrected dynamic risk level.

[0153] First, the system adaptively switches between different multimodal data fusion architectures based on dynamically assessed risk levels and collaboratively and feedforwardly manages onboard computing resources. This enables the system to concentrate computing power for accurate perception in high-risk scenarios and conserve resources in low-risk scenarios, thus intelligently balancing the contradiction between perception accuracy and onboard computing power overhead. Second, under normal circumstances, the system executes a smooth global planning route. However, once a sudden change in obstacle trajectory is detected, causing the planning to fail, the system can immediately interrupt the current route and instantly activate the emergency response layer to execute the fastest evasive maneuver, ensuring ultimate safety when dealing with highly dynamic and uncertain threats. Finally, by introducing a self-learning and iterative optimization mechanism for predicted failure samples, the system is also endowed with the ability to continuously evolve, significantly improving its overall robustness and intelligence level when dealing with complex dynamic environments.

[0154] Example 5:

[0155] Corresponding to the above embodiments, the present invention also proposes an electronic device.

[0156] like Figure 9 The diagram shows a structural schematic of an electronic device according to the present invention. The electronic device 100 includes a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, for example, via a bus 102. Optionally, the electronic device 100 may further include a transceiver 104. It should be noted that in practical applications, the transceiver 104 is not limited to one unit, and the structure of this electronic device 100 does not constitute a limitation on the embodiments of the present invention.

[0157] Processor 101 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 101 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0158] Bus 102 may include a pathway for transmitting information between the aforementioned components. Bus 102 may be a PCI bus or an EISA bus, etc. Bus 102 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0159] The memory 103 stores a computer program corresponding to the dynamic obstacle perception and fusion early warning method for a UAV edge power transmission channel according to the above embodiments of the present invention. This computer program is controlled and executed by the processor 101. The processor 101 executes the computer program stored in the memory 103 to implement the content shown in the aforementioned method embodiments.

[0160] Among them, electronic devices 100 include, but are not limited to: mobile terminals such as laptops and PADs (tablet computers) and fixed terminals such as desktop computers. Figure 9 The electronic device 100 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0161] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for dynamic obstacle perception fusion early warning of UAV edge power transmission channel, characterized in that, Includes the following steps: S1. Real-time collection of multimodal data of power transmission channels through multiple sensors carried by drones; S2. By processing the multimodal data, determine the dynamic obstacles and their types within the power transmission channel, and predict the future trajectory of the dynamic obstacles; S3. Calculate the corrected dynamic risk level based on the type of the dynamic obstacle, its current distance from the conductor in the power transmission channel, and its future trajectory; S4. In response to the revised dynamic risk level, perform the following steps: (a) Based on the modified dynamic risk level, select a corresponding fusion strategy from the fusion architecture that includes at least a feature layer fusion strategy and a decision layer fusion strategy to perform fusion processing on the multimodal data; (b) Based on the future trajectory of the dynamic obstacle and the three-dimensional geographic information system (GIS) security boundary of the power transmission channel, a three-dimensional dynamic obstacle avoidance route is generated online in real time, and the UAV is controlled to fly along the route. (c) Based on the expected computing power requirements of the fusion strategy selected in step (a), the computing resources of the UAV onboard terminal are scheduled in a feedforward manner; S5. Based on the revised dynamic risk level, generate an early warning signal.

2. The method of claim 1, wherein, Identify dynamic obstacles and their types, including: Based on lidar point cloud data, infrared temperature data, and visible light images, dynamic candidate boxes for lidar, infrared, and visible light are generated respectively. Based on the preset cross-validation logic, cross-validation of multimodal candidate boxes is performed on the types of the dynamic obstacles; The cross-validation logic determines the intersection or union relationship of the lidar dynamic candidate box, infrared dynamic candidate box, and visible light dynamic candidate box based on the different types of dynamic obstacles.

3. The method according to claim 1 or 2, characterized in that, The prediction of the future trajectory of the dynamic obstacle includes: Based on the type of the dynamic obstacle, a motion model that is compatible with that type is called from a preset motion model library; the motion model library includes uniform linear motion models, random walk models, and parabolic motion models. The adapted motion model is applied to the Kalman filter module to predict the future trajectory of the dynamic obstacle; the Kalman filter module is used to estimate the state vector containing position, velocity and acceleration.

4. The method of claim 1, wherein, The calculated and corrected dynamic risk level includes: Based on the type weight of the dynamic obstacle and the current safe distance from the guide wire, calculate the initial value of the dynamic risk level; Determine whether the future trajectory will intrude into the preset power transmission channel safety zone; if so, calculate the predicted intrusion time. The corrected dynamic risk level is calculated based on the initial value of the dynamic risk level and the predicted intrusion time. The revised dynamic risk level is directly proportional to the initial value of the dynamic risk level and inversely proportional to the predicted intrusion time.

5. The method of claim 2, wherein, When selecting a corresponding fusion strategy to perform fusion processing on the multimodal data, the process further includes dynamically adjusting the selected fusion strategy, specifically including the following steps: When the corrected dynamic risk level is higher than the second preset risk threshold, the risk is determined to be an emergency high-risk risk. Then, the feature layer fusion strategy is selected to fuse the multimodal data to obtain the obstacle status information with the highest accuracy. When the corrected dynamic risk level is lower than or equal to the second preset risk threshold, the risk is determined to be a normal risk. In this case, a decision-level fusion strategy with lower computational overhead is selected for fusion processing to save computing resources. Furthermore, during the fusion process, the fusion weights of the lidar point cloud data and the visible light image are adaptively adjusted based on the velocity v of the dynamic obstacle.

6. The method of claim 1, wherein, The process of generating a three-dimensional dynamic obstacle avoidance flight path in real time and controlling the UAV to fly along that path specifically includes the following steps: When the corrected dynamic risk level is higher than the first preset risk threshold, the future trajectory of the dynamic obstacle, the three-dimensional geographic information system (GIS) safety boundary of the power transmission channel, and the kinematic constraints of the UAV itself are taken as input. A three-dimensional dynamic obstacle avoidance route is generated online in real time through the fast search random tree (RRT) algorithm or the model predictive control (MPC) algorithm, and the UAV is controlled to fly along the three-dimensional dynamic obstacle avoidance route. The three-dimensional dynamic obstacle avoidance flight path ensures that the distance between the UAV and the power transmission line in the power transmission channel is always greater than the preset minimum safe flight distance while avoiding the dynamic obstacles.

7. The method of claim 5, wherein, Before dynamically adjusting the selected fusion strategy, the expected computing power requirements of the selected fusion strategy in the next time window should be assessed in advance; The expected computing power requirement is compared with the current computing power load L of the UAV's onboard terminal. If the predicted total load will exceed the preset load threshold, the execution priority of non-urgent tasks is proactively reduced or suspended in a feedforward manner to release computing power resources in advance.

8. The method of claim 3, wherein, The method further includes: After the inspection task is completed, the predicted trajectory of the dynamic obstacle is compared with its actual observed trajectory obtained by the high-precision positioning module afterward, and the trajectory prediction error is calculated. When the trajectory prediction error is greater than the preset error threshold, the corresponding dynamic obstacle sample, environmental parameters and the selected motion model are identified as high-value negative samples. The trajectory prediction model for predicting future trajectories is trained offline using the set of high-value negative samples, with a focus on optimizing the selection logic of each motion model in the motion model library and / or updating the state transition parameters of the Kalman filter module.

9. The method of claim 6, wherein, The method further includes: During the flight of the UAV along the three-dimensional dynamic obstacle avoidance route, the real-time trajectory of the dynamic obstacle is continuously monitored, and the trajectory deviation between the real-time trajectory and the future trajectory on which the route was generated is calculated. When the trajectory deviation exceeds a preset abrupt deviation threshold, it is determined that the three-dimensional dynamic obstacle avoidance route has failed, and the following steps are executed: (1) Immediately stop the execution of the three-dimensional dynamic obstacle avoidance route generated by the RRT algorithm or MPC algorithm. (2) And instantly activate an emergency response layer based on the artificial potential field method (APF) or the dynamic window method (DWA); (3) The emergency response layer generates a local repulsive field or an instantaneous safe speed command based on the real-time trajectory of the dynamic obstacle to guide the UAV to perform emergency obstacle avoidance maneuvers until the real-time distance between the UAV and the dynamic obstacle recovers to above the safe threshold: (4) After the emergency obstacle avoidance maneuver is completed, the RRT algorithm or MPC algorithm is triggered again to regenerate the three-dimensional dynamic obstacle avoidance route based on the updated obstacle state.

10. An unmanned aerial vehicle edge power transmission channel dynamic obstacle perception fusion early warning system, characterized in that, include: The data acquisition module is used to collect multimodal data of the power transmission channel in real time through multiple sensors carried by the drone; The dynamic sensing module is used to process the multimodal data, determine the dynamic obstacles and their types within the power transmission channel, and predict the future trajectory of the dynamic obstacles. The risk assessment module is used to calculate the corrected dynamic risk level based on the type of the dynamic obstacle, its current distance from the conductor in the power transmission channel, and its future trajectory. An adaptive control module, configured to perform the following operations in response to the corrected dynamic risk level output by the risk assessment module: Based on the revised dynamic risk level, select a corresponding fusion strategy from the fusion architecture that includes at least a feature layer fusion strategy and a decision layer fusion strategy; Based on the future trajectory of the dynamic obstacles and the three-dimensional geographic information system security boundary of the power transmission channel, a three-dimensional dynamic obstacle avoidance route is generated online in real time, and instructions for controlling the flight of the UAV are generated. Furthermore, based on the expected computing power requirements of the selected fusion strategy, instructions are generated for feedforward scheduling of UAV onboard computing resources. The early warning generation module is used to generate early warning signals based on the corrected dynamic risk level.

Images