A method and system for three-dimensional perception and obstacle avoidance of power cables for low-altitude aircraft

Abstract

This invention discloses a three-dimensional perception and obstacle avoidance method and system for power cables of low-altitude aircraft, belonging to the field of visual perception technology for low-altitude aircraft. The method includes: acquiring images of the foreground scene and pose data; jointly detecting power cables and towers to obtain detection results representing the distance relationship between the power cables and towers in the image; performing topological continuity constraint processing and temporal fusion processing combining detection results from adjacent frames; associating multiple frames of observation results for the same power cable, back-projecting the associated power cable observation results into three-dimensional rays, performing multi-frame three-dimensional solving to obtain a three-dimensional representation of the power cable, and constructing and updating a three-dimensional power cable map containing topological relationships; calculating the safe distance between the flight path and the power cable, and outputting alarm information and detour path suggestions. This invention solves the technical problem that existing two-dimensional cable detection results are difficult to directly support obstacle avoidance processing for low-altitude aircraft.

Description

Technical Field

[0001] This invention belongs to the field of visual perception technology for low-altitude aircraft, specifically relating to a method and system for three-dimensional perception and obstacle avoidance of power cables for low-altitude aircraft. Background Technology

[0002] Low-altitude aircraft are easily affected by ground obstacles and small aerial obstacles during low-altitude flight missions such as inspection, plant protection, delivery, and manned flights. Among these, power cables are particularly difficult to perceive stably in low-altitude scenarios due to their thin diameter, long spatial extension, low image contrast, and complex backgrounds. Especially under conditions of long-distance observation, weak texture backgrounds, complex urban backgrounds, or high-speed flight, power cables may only appear as thin, elongated lines in images, making it difficult for low-altitude aircraft to obtain stable and accurate cable perception results in a timely manner.

[0003] Existing obstacle perception methods commonly include target detection based on visual images, image segmentation, lidar ranging, millimeter-wave radar detection, and multi-sensor fusion perception. For slender targets like power cables, simply using two-dimensional target detection boxes or two-dimensional segmentation results typically only provides the cable's position in the image plane, making it difficult to directly obtain its position, orientation, and connectivity in three-dimensional space. Since obstacle avoidance and path planning for aircraft rely more heavily on the spatial distance relationship between obstacles and the flight path, simply outputting two-dimensional image detection results is insufficient to directly support safe distance calculations and obstacle avoidance decisions for low-altitude aircraft.

[0004] Meanwhile, existing single-frame image detection methods typically do not fully utilize the temporal consistency between consecutive video frames, making them susceptible to single-frame noise, missed detections, false detections, lighting changes, or background interference, resulting in jitter in cable detection results over time. For slender cable targets, if the detection results show breaks, isolated points, or incorrect connections in the image, it will affect the cross-frame association of the same cable and the stability of 3D reconstruction. Furthermore, existing methods usually lack the utilization of the association with cable suspension structures such as poles and towers, making it difficult to provide effective constraints on the endpoints, suspension points, and topological connections of power cables during the 3D reconstruction process.

[0005] Therefore, how to stably upscale the power cable detection results in images to three-dimensional space in low-altitude aircraft scenarios, and further construct a three-dimensional power cable map that can be used for safety distance calculation and obstacle avoidance information output, has become a technical problem that needs to be solved. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for three-dimensional perception and obstacle avoidance of power cables for low-altitude aircraft, in order to solve the technical problem mentioned in the background art that the existing technology is difficult to obtain stable three-dimensional cable spatial information based on two-dimensional cable detection results, and thus it is difficult to support the calculation of safe distance and the output of obstacle avoidance information for low-altitude aircraft.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] In a first aspect, the present invention provides a method for three-dimensional perception and obstacle avoidance of power cables for low-altitude aircraft, comprising: acquiring an image of the scene in front of the low-altitude aircraft and the pose data corresponding to the low-altitude aircraft;

[0009] Based on the scene image, joint detection of power cables and towers is performed to obtain detection results characterizing the distance relationship between the power cables and towers in the image;

[0010] The detection results are subjected to topological continuity constraint processing, and temporal fusion processing is performed in combination with the detection results in adjacent frames to obtain stable detection results;

[0011] Multiple frames of observation results for the same power cable are correlated, and the correlated power cable observation results are back-projected into a three-dimensional ray based on the pose data. The three-dimensional representation of the power cable is obtained by solving the three-dimensional ray in multiple frames. The three-dimensional representation of the power cable is anchored or constrained by the tower detection results.

[0012] A three-dimensional power cable map containing topological relationships is constructed and updated based on the three-dimensional representation of the power cables.

[0013] Based on the three-dimensional power cable map, the safe distance between the flight path and the power cable is calculated, and alarm information and detour route suggestions are output.

[0014] Furthermore, joint detection of power cables and poles is performed based on the scene image to obtain detection results characterizing the distance relationship between the power cables and poles in the image. This includes: performing distance map regression processing on the scene image using a detection model to output distance map results corresponding to the power cables and poles; determining the pixel regions of the power cables and the pixel regions of the poles based on the distance map results to obtain the joint detection results of the power cables and poles. Characterizing the distance relationship between pixels and cable / pole targets using distance map regression is more suitable for the joint detection requirements of slender cable targets and pole targets.

[0015] Furthermore, the detection results are subjected to topological continuity constraint processing, and temporal fusion processing is performed in conjunction with the detection results in adjacent frames to obtain stable detection results. This includes: applying topological continuity constraints to the detection results to maintain the continuity of power cables, thereby suppressing power cable breaks and isolated regions; aligning the detection results in adjacent frames based on the motion relationship between adjacent frames, and fusing the aligned multi-frame detection results to obtain the stable detection results. Through topological continuity constraints and temporal fusion, the stability of cable detection results in both spatial structure and temporal dimensions can be improved.

[0016] Furthermore, the observation results of multiple frames for the same power cable are correlated, including: preliminary matching of the observation results based on the spatial continuity and directional consistency of the power cable in adjacent frames; correction of the preliminary matching results by combining the temporal tracking results in adjacent frames; and, when multiple candidate correlation results exist, disambiguation of the candidate correlation results by combining the tower detection results to determine the observation results of the same power cable across multiple frames. This correlation method can reduce the risk of false correlations that arise when relying solely on a single clue for cross-frame matching.

[0017] Furthermore, based on the pose data, the associated power cable observation results are back-projected into three-dimensional rays. A three-dimensional representation of the power cable is obtained by performing multi-frame three-dimensional solving on these rays. This includes: back-projecting the associated power cable observation results into corresponding three-dimensional rays based on the pose data; performing triangulation on multiple frames of three-dimensional rays corresponding to the same power cable when the observation baselines between adjacent frames meet preset conditions to obtain three-dimensional points or three-dimensional line segments of the power cable; and using the three-dimensional position of the towers corresponding to the tower detection results to constrain at least one of the following: endpoint positioning of the power cable, correction of the three-dimensional solution results, and topology consistency verification. By introducing tower detection results as anchoring or constraint information, the stability and spatial consistency of the three-dimensional representation of the power cable can be improved.

[0018] Furthermore, constructing and updating a 3D power cable map containing topological relationships based on the 3D representation of the power cables includes: converting the 3D representation of the power cables into power cable nodes and power cable segments connecting the power cable nodes; establishing or updating topological edges based on the spatial distance, directional angle, and confidence level between adjacent power cable nodes; and determining the target connection object based on at least one of the following when multiple candidate connection objects exist: closer distance, better directional match, and higher confidence level. Expressing the 3D spatial distribution and connection relationships of power cables through nodes, segments, and topological edges provides a structured spatial basis for subsequent obstacle avoidance information output.

[0019] Furthermore, the safe distance between the flight path and the power cables is calculated based on the three-dimensional power cable map, and alarm information and detour path suggestions are output. This includes: determining the safe distance based on the spatial relationship between the flight path and the three-dimensional power cable map; generating graded alarm information based on the correspondence between the safe distance and preset safety conditions; generating detour path suggestions based on the three-dimensional power cable map; and outputting the graded alarm information and the detour path suggestions to the flight control system and / or the pilot display interface. By calculating the safe distance based on the three-dimensional power cable map, the cable perception results can be transformed into spatial risk information that can be used for obstacle avoidance.

[0020] Secondly, based on the same inventive concept, the present invention also provides a three-dimensional perception and obstacle avoidance system for power cables of low-altitude aircraft, comprising:

[0021] The data acquisition module is used to acquire images of the scene in front of the low-altitude aircraft and the corresponding pose data of the low-altitude aircraft.

[0022] A joint detection module, connected to the data acquisition module, is used to perform joint detection of power cables and towers based on the scene image, and obtain detection results characterizing the distance relationship between the power cables and towers in the image;

[0023] A stabilization processing module, connected to the joint detection module, is used to perform topological continuity constraint processing on the detection results and perform temporal fusion processing on the detection results in adjacent frames to obtain stable detection results.

[0024] The three-dimensional solving module is connected to the data acquisition module, the joint detection module, and the stabilization processing module, respectively. It is used to associate multiple frames of observation results of the same power cable, and back-project the associated power cable observation results into three-dimensional rays based on the pose data. The three-dimensional representation of the power cable is obtained by multi-frame three-dimensional solving of the three-dimensional rays. The three-dimensional representation of the power cable is anchored or constrained by the tower detection results.

[0025] The topology map module, connected to the 3D solution module, is used to construct and update a 3D power cable map containing topological relationships based on the 3D representation of the power cables.

[0026] The obstacle avoidance output module, connected to the topology map module, is used to calculate the safe distance between the flight path and the power cable based on the three-dimensional power cable map, and output alarm information and detour path suggestions.

[0027] Thirdly, based on the same inventive concept, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the three-dimensional perception and obstacle avoidance method for power cables of low-altitude aircraft as described in the first aspect.

[0028] Fourthly, based on the same inventive concept, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the three-dimensional perception and obstacle avoidance method for power cables of low-altitude aircraft as described in the first aspect.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] This invention continuously processes the scene image in front of the low-altitude aircraft, the aircraft's pose data, the joint detection results of power cables and towers, the stabilization detection results, and the three-dimensional power cable map, so that the power cable detection results can be upgraded from a two-dimensional image plane to a three-dimensional space. This data can then be used for safety distance calculation, graded alarms, and bypass path suggestions, thereby solving the problem that existing two-dimensional cable detection results are difficult to directly support obstacle avoidance processing for low-altitude aircraft.

[0031] This invention transforms power cables and towers into distance map regression targets, enabling the detection model to output continuous distance response results. Compared to ordinary two-dimensional bounding box representations, this method is more suitable for representing long-distance, low-contrast, and slender power cable targets. It helps preserve the linear extension features of power cables in the image and provides stable two-dimensional observation input for subsequent topological continuity constraints and 3D reconstruction.

[0032] This invention applies topological continuity constraints to the power cable detection results and performs temporal fusion with the detection results of adjacent frames. This reduces breaks, isolated regions, and single-frame jitter in the power cable detection results, making the detection results of the same power cable maintain more stable continuity in the image plane and time dimension, thereby providing a stable data foundation for the multi-frame observation association of the same power cable.

[0033] This invention combines multiple frames of observation results of the same power cable with the pose data of a low-altitude aircraft, backprojects the two-dimensional power cable observation results into three-dimensional rays, and performs three-dimensional solutions on the multiple frames of three-dimensional rays to obtain the position and orientation of the power cable in three-dimensional space. At the same time, it combines the tower detection results to anchor or constrain the three-dimensional representation of the power cable, and can use the tower as a spatial reference for the endpoints or suspension points of the power cable, reducing ambiguity and drift in the multi-frame three-dimensional solution.

[0034] This invention organizes power cables into a three-dimensional power cable map composed of power cable nodes, power cable segments, and topological edges, enabling the representation of the spatial location, extension direction, and connection relationships of power cables within the map. Compared to methods that only output two-dimensional detection results or two-dimensional segmentation masks, this three-dimensional power cable map provides a spatial basis for calculating safe distances along flight paths and supports obstacle avoidance information output based on cable topology.

[0035] This invention, through dynamic region of interest selection, hierarchical feature tracking, inertial measurement unit pre-integration assisted sampling, and multi-level positioning mode switching, can maintain continuous pose data output under conditions of low texture background, high-speed flight, or short-term lack of visual features, so that the temporal fusion of detection results of adjacent frames, 3D reconstruction of power cables, and incremental update of 3D power cable maps have a continuous spatial reference.

[0036] This invention, by employing a shared backbone network and a dual-detection-head structure, can simultaneously output power cable distance maps and tower distance maps based on a single feature extraction, reducing redundant computations caused by building separate detection models. Combining image block segmentation, batch inference, and distance map stitching, this invention can adapt to deployment environments with limited airborne computing resources and obtain complete scene-specific power cable and tower detection results even with high-resolution image input. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of the system structure according to an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0040] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0041] like Figures 1-3 As shown:

[0042] First embodiment:

[0043] like Figure 1 As shown in the figure, this embodiment provides a three-dimensional perception and obstacle avoidance method for power cables of low-altitude aircraft, including the following steps:

[0044] S100: Acquire an image of the scene in front of the low-altitude aircraft and the corresponding pose data of the low-altitude aircraft.

[0045] When performing 3D perception and obstacle avoidance tasks for power cables, low-altitude aircraft acquire images of the scene in front of the aircraft and the corresponding aircraft pose data, which serve as input data for subsequent cable and tower detection and 3D cable map construction.

[0046] Specifically, the low-altitude aircraft continuously acquires scene images along its flight path using onboard image acquisition equipment. These scene images include power cables, poles, and the background environment to be detected. The acquired raw images can be high-resolution images, such as a sequence of images with a resolution of 4096×3000 and a frame rate of 6 FPS. To adapt to the input size of the subsequent detection model, the raw images are preprocessed. Preprocessing includes zero-padding the raw images to form square images of a predetermined size; for example, the 4096×3000 raw image is zero-padding to form an image input area of ​​3072px, and this image input area is divided into multiple image blocks. As one specific implementation, the image input area can be divided into 12 image blocks of 1024×1024 pixels, so that each image block can serve as the input image block for subsequent joint detection of cables and poles.

[0047] Simultaneously with acquiring the scene image, the pose data corresponding to the low-altitude aircraft is also acquired. This pose data includes data characterizing the spatial position and attitude state of the low-altitude aircraft, which can be provided by visual-inertial odometry (VIO) results or obtained by combining angular velocity and linear acceleration data collected by an inertial measurement unit (IMU). The IMU data is used throughout the image patch sampling, feature tracking, state fusion, and 3D measurement processes. During the image acquisition and preprocessing stage, the IMU data is used to assist in determining the image patch sampling center, ensuring that the sampled image patch preferentially covers the field of view containing cable targets, and providing motion priors for the temporal fusion of detection results from subsequent adjacent frames and 3D ray backprojection.

[0048] After the above processing, an image patch set corresponding to the flight time and aircraft pose data are obtained. The image patch set is used for subsequent joint detection of power cables and towers, and the aircraft pose data is used to upscale the cable detection results in the two-dimensional image plane to three-dimensional space in subsequent processing, and to provide a spatial reference for the construction of a three-dimensional power cable map.

[0049] S200, based on the scene image, perform joint detection of power cables and towers to obtain detection results characterizing the distance relationship between the power cables and towers in the image; including:

[0050] The scene image is processed by distance map regression using a detection model, and the distance map results corresponding to power cables and towers are output.

[0051] Based on the distance map results, the pixel regions of power cables and towers are determined, and the joint detection results of the power cables and towers are obtained.

[0052] Specifically, the preprocessed scene image or image patch is input into the detection model, which then performs distance map regression processing on the scene image or image patch. The detection model employs a shared backbone network and a dual-detection-head structure. The shared backbone network is used for feature extraction from the input image, and the two detection heads are used to output distance map results corresponding to power cables and distance map results corresponding to poles, respectively. The shared backbone network can be a lightweight convolutional network, such as MobileNetV4-Conv-M (mobile convolutional neural network) or ConvNeXt-Tiny (lightweight convolutional neural network), to simultaneously detect power cables and poles under conditions of limited onboard computing resources.

[0053] The distance map results are used to represent the normalized distance values ​​of image pixels relative to the target location. The distance value corresponding to the target area is close to 0, and the distance value corresponding to pixels farther away from the target area gradually increases. For power cables, the cable distance map output by the detection model is used to characterize the distance relationship between each pixel in the image and the nearest power cable position; for poles, the pole distance map output by the detection model is used to characterize the distance relationship between each pixel in the image and the nearest pole position. By converting both power cables and poles into distance map regression targets, the structural mismatch caused by directly using a fixed-width bounding box to represent slender cables can be avoided, making the detection results more suitable for representing power cable targets with long distances, low widths, and slender shapes.

[0054] In one specific implementation, the annotation data for power cables is manually generated polyline annotation data, and the annotation data for poles and towers is generated using bounding box annotation data. Distance transformations are applied to the annotation data for both power cables and poles to generate corresponding distance map supervision labels. For power cable and pole / tower areas in a common scene, pixels within the target's neighborhood can be mapped to normalized distance values, and distance map labels are generated using small-scale pooling. This allows the detection model to output continuous distance response results for thin cables and pole / tower targets after training.

[0055] After obtaining the distance map results, the pixel regions of power cables are determined based on the power cable distance map, and the pixel regions of poles are determined based on the pole distance map. Specifically, pixels in the distance map results that meet the target distance conditions can be identified as candidate pixels for the corresponding target, and connectivity analysis or region filtering can be performed on the candidate pixels to obtain the pixel regions of power cables and poles. The pixel regions of power cables are used to characterize the distribution position of power cables in the image plane, and the pixel regions of poles are used to characterize the distribution position of poles in the image plane.

[0056] Through the above processing, a joint detection result for power cables and towers is obtained. This joint detection result includes the pixel regions of the power cables and towers, as well as the distance relationship between the power cables and towers in the image, represented by the corresponding distance map results. This joint detection result serves as input data for subsequent topological continuity constraint processing, inter-frame temporal fusion processing, and 3D reconstruction of the power cables.

[0057] In one implementation, the detection model used to output power cable distance map results and tower distance map results is trained using training samples with distance map supervision labels.

[0058] Specifically, sample images containing power cables and poles are acquired, and the power cables and poles in the sample images are labeled separately. For power cables, a manual polyline annotation method is used to generate cable annotation results, which are used to characterize the centerline or linear extension position of the power cables in the sample images. For poles, a bounding box annotation method is used to generate pole annotation results, which are used to characterize the target area of ​​the poles in the sample images.

[0059] After obtaining the cable and pole labeling results, distance transformations are performed on both results to generate corresponding distance map supervision labels. For power cables, the target area is determined based on the cable polyline labeling results, and the Euclidean distance from each pixel in the sample image to the nearest target area is calculated. For poles, the target area is determined based on the pole bounding box labeling results, and the Euclidean distance from each pixel in the sample image to the nearest target area is calculated. During the distance transformation, pixels within the vicinity of the target area are considered as target neighbor pixels; for example, pixels within 5px of the target area are considered as the nearest neighbor area. A distance response is generated based on the Euclidean distance from each pixel to the target area.

[0060] After generating the distance response, the distance response is truncated and normalized to obtain distance map supervision labels with a value range of [0, 1]. Specifically, the upper limit of distance is set to 128px. Distance values ​​exceeding the upper limit are truncated according to the upper limit, and the truncated distance values ​​are normalized to the range of [0, 1]. The normalized distance value corresponding to the target area is 0, and the normalized distance value corresponding to pixels far from the target area increases with increasing distance. Further, the normalized distance map is subjected to 16×16 minimum pooling to form distance map supervision labels corresponding to the output scale of the detection model. Thus, distance map supervision labels for power cables and towers are obtained respectively.

[0061] When training the detection model, sample images or image patches segmented from sample images are used as training input, and power cable distance map supervision labels and pole distance map supervision labels are used as training supervision information to jointly train the detection model. The training loss of the detection model includes power cable distance map regression loss, pole distance map regression loss, and a topological continuity loss term to maintain the topological continuity of power cables. The power cable distance map regression loss is used to constrain the difference between the cable distance map output by the detection model and the power cable distance map supervision labels, the pole distance map regression loss is used to constrain the difference between the pole distance map output by the detection model and the pole distance map supervision labels, and the topological continuity loss term is used to constrain the cable detection results to maintain topological continuity.

[0062] In one specific implementation, the detection model is initialized using ImageNet pre-trained parameters and trained on the sample images. During training, data augmentation processing is performed on the sample images, including color dithering and horizontal flipping, to increase the variation range of the training samples. The training optimizer uses the Adam optimizer and can employ the SMAC3 / BOHB hyperparameter search method for training parameter search. Five-fold cross-validation is used during training evaluation, and model parameters for the corresponding training rounds are selected and saved based on the power cable detection quality indicators. Through the above training process, a detection model capable of simultaneously outputting power cable distance maps and tower distance maps is obtained.

[0063] The detection model employs a shared backbone network and a dual-detection-head structure to simultaneously perform power cable distance map regression and pole distance map regression on the same input image block.

[0064] Specifically, the scene image or image patch to be detected is input into a shared backbone network, which extracts features from the input image to obtain a shared feature map. This shared feature map is simultaneously input to both the cable detection head and the tower detection head, allowing the cable detection task and the tower detection task to share image features, thereby avoiding redundant computation caused by building separate detection models. The shared backbone network can be a lightweight convolutional network pre-trained on ImageNet, such as ConvNeXt-Tiny or MobileNetV4-Conv-M. ConvNeXt-Tiny can output a feature map with a downsampling ratio of 16x and 384 channels, while MobileNetV4-Conv-M can output a feature map with a downsampling ratio of 32x. By employing a lightweight shared backbone network, the detection model can complete the joint detection of power cables and towers under conditions of limited onboard computing resources.

[0065] The cable detection head is used to output a power cable distance map based on the shared feature map. The cable detection head includes a 1×1 convolutional layer and a clamped-ReLU (truncated linear rectifier unit) activation layer. The 1×1 convolutional layer maps the shared feature map to the distance response result of the corresponding power cable, and the clamped-ReLU activation layer restricts the distance response result to the [0, 1] value range. In the cable distance map, the region with a distance value of 0 corresponds to the region where the power cable is located, and the region with a distance value greater than 0 represents the normalized distance relationship from the corresponding pixel to the nearest power cable position.

[0066] The tower detection head is used to output tower distance map results based on the shared feature map. The tower detection head includes a 1×1 convolutional layer and a clamped-ReLU activation layer. The 1×1 convolutional layer maps the shared feature map to the distance response results of the corresponding towers, and the clamped-ReLU activation layer restricts the distance response results to the [0, 1] value range. In the tower distance map, regions with a distance value of 0 correspond to the regions where towers are located, and regions with a distance value greater than 0 represent the normalized distance relationship from the corresponding pixel to the nearest tower location.

[0067] Through the shared backbone network and dual-detection-head structure described above, the detection model can output cable distance map results and tower distance map results based on a single feature extraction. The cable distance map results are used to determine the pixel regions of power cables, and the tower distance map results are used to determine the pixel regions of towers. Together, they constitute the joint detection results of power cables and towers, and serve as inputs for subsequent topology continuity constraint processing, inter-frame temporal fusion processing, and 3D reconstruction processing of power cables.

[0068] In one implementation, during the inference phase, high-resolution scene images acquired by the low-altitude aircraft are subjected to image preprocessing, image block segmentation, batch inference, and distance map stitching to obtain the detection results of power cables and towers corresponding to the complete scene.

[0069] Specifically, the raw scene images acquired by the low-altitude aircraft can be image sequences with a resolution of 4096×3000 and a frame rate of 6 FPS. The raw scene images are de-mosaiced, and then zero-padding is applied to the de-mosaiced images to form a square input region of a predetermined size. As a specific implementation, the de-mosaiced raw scene images are zero-paddinged to an input region of 3072px, and this input region is divided into 12 image blocks of 1024×1024 pixels. These image blocks serve as the inference input for the detection model, adapting to the input size of the detection model and reducing the memory usage required for a single inference iteration.

[0070] After image block segmentation, multiple image blocks are input into the graphics processor for batch inference. In one specific implementation, every four image blocks constitute an inference batch, and TRT FP16 (TensorRT-based 16-bit floating-point inference) is used to accelerate inference for these image blocks. Through batch inference, the detection model outputs cable distance maps and tower distance maps for each image block.

[0071] After obtaining the distance map results corresponding to each image block, the cable distance map results and the tower distance map results are stitched together according to the segmentation position of each image block in the original scene image to obtain the cable distance map and tower distance map corresponding to the complete scene. In cases where there are overlapping areas between adjacent image blocks, the distance map responses within the overlapping areas can be merged based on the spatial position of the corresponding image blocks, ensuring that the stitched distance map results maintain a correspondence with the target position in the original scene image.

[0072] After obtaining the cable distance map and pole distance map corresponding to the complete scene, thresholding is performed on the cable distance map and pole distance map to obtain the power cable segmentation mask and pole segmentation mask, respectively. The power cable segmentation mask is used to represent the pixel region corresponding to the power cable in the original scene image, and the pole segmentation mask is used to represent the pixel region corresponding to the pole in the original scene image. Furthermore, the power cable segmentation mask and pole segmentation mask can be upsampled to the resolution corresponding to the original scene image, so that the detection result is consistent with the original scene image in pixel coordinates.

[0073] Through the above reasoning and deployment process, power cable detection results and pole detection results corresponding to the original scene image can be obtained under high-resolution scene image input conditions. These power cable detection results and pole detection results can continue to serve as input data for topological continuity constraints, inter-frame temporal fusion, multi-frame 3D solving, and 3D power cable map construction, thus ensuring consistency between the training output format, inference output format, and subsequent 3D perception processing chain of the detection model.

[0074] S300, the detection results are subjected to topological continuity constraint processing, and temporal fusion processing is performed in conjunction with the detection results in adjacent frames to obtain stable detection results; including:

[0075] The detection results are subjected to topological continuity constraints to maintain the continuity of power cables, thereby suppressing power cable breaks and isolated areas.

[0076] The detection results in adjacent frames are aligned based on the motion relationship between adjacent frames, and the aligned multi-frame detection results are fused to obtain the stabilized detection result.

[0077] Specifically, after obtaining the pixel regions of power cables, towers, and the corresponding distance map results, topological continuity constraint processing is applied to the pixel regions of the power cables to maintain a continuous linear structure in the image plane. This topological continuity constraint processing constrains the topological relationship between the cable regions and the background regions in the power cable detection results, ensuring that pixel regions belonging to the same power cable remain connected and suppressing cable breaks, isolated regions, and local false detections caused by long distances, low contrast, or complex backgrounds.

[0078] During the training of the detection model, a composite loss function is used to constrain the distance map regression results of power cables and towers. The composite loss function is expressed as follows:

[0079] ;

[0080] in, Represents the composite loss function. This represents the regression loss of the power cable distance map. This represents the regression loss of the tower distance plot. This represents the MALIS topological continuity loss term. MALIS stands for Maximum Affinity Learning of Image Segmentation, which is used to constrain the topological continuity of power cable detection results. This represents the weighting coefficient of the topology continuity loss term in the composite loss function. The power cable distance map regression loss and tower distance map regression loss are used to constrain the difference between the distance map regression results and the distance map supervision labels. The topology continuity loss term is used to impose continuity constraints on the power cable detection results at the topology level, keeping the background areas separated by cables separated and reducing cable breaks and isolated points. The detection model trained by the composite loss function can balance pixel-level distance response and cable topology continuity when outputting power cable distance map results.

[0081] After obtaining the detection results of adjacent frames, a temporal fusion process is performed on the detection results of the adjacent frames based on the motion relationship between them. Specifically, the distance map results in the previous frame and the current frame are aligned between frames so that the power cable detection results in adjacent frames are mapped to the same image space. Subsequently, the aligned multi-frame distance map results are fused to obtain the fused distance map result. The temporal fusion process can use DIS (Dense Inverse Search) optical flow to determine the dense optical flow relationship between adjacent frames, and the distance map results in adjacent frames are deformed and aligned according to the dense optical flow relationship. Then, the deformed and aligned distance map results of the previous frame are fused with the distance map results of the current frame to suppress instantaneous noise, missed detections, and jitter in single-frame detection.

[0082] In one specific implementation, the temporal fusion processing and image patch inference are executed in parallel. The distance map result from the previous frame is deformed to the image space corresponding to the current frame according to the optical flow field, and then weighted and fused with the distance map result from the current frame to obtain the final segmentation mask or stabilized distance map result. Thus, the detection results of power cables and towers remain smooth and consistent in the temporal dimension, reducing temporal fluctuations caused by single-frame inference, and providing stable input for subsequent multi-frame observation association and 3D solution of the same power cable.

[0083] After topological continuity constraint processing and temporal fusion processing, stable detection results are obtained. The stable detection results include power cable detection results that maintain linear continuity and their corresponding tower detection results. These stable detection results are used to correlate multiple frames of observation results for the same power cable and, in conjunction with low-altitude aircraft pose data, to elevate the power cable detection results to three-dimensional space.

[0084] S400: Correlate multiple frames of observation results for the same power cable, and back-project the correlated power cable observation results into a three-dimensional ray based on the pose data. Obtain the three-dimensional representation of the power cable by performing multi-frame three-dimensional solution on the three-dimensional ray. In this case, the three-dimensional representation of the power cable is anchored or constrained by combining the tower detection results.

[0085] Correlating multiple frames of observations of the same power cable, including:

[0086] Based on the spatial continuity and directional consistency of power cables in adjacent frames, preliminary matching of observation results across multiple frames is performed.

[0087] The initial matching results are corrected by combining the temporal tracking results in adjacent frames;

[0088] When multiple candidate association results exist, the ambiguity of the candidate association results is resolved by combining the tower detection results, so as to determine the multi-frame observation results of the same power cable.

[0089] Based on the pose data, the associated power cable observation results are back-projected into a three-dimensional ray. A three-dimensional representation of the power cable is obtained by performing multi-frame three-dimensional solving on the three-dimensional ray, including:

[0090] Based on the pose data, the associated power cable observation results are back-projected into the corresponding three-dimensional ray;

[0091] When the observation baseline between adjacent frames meets the preset conditions, triangulation is performed on the three-dimensional rays corresponding to the same power cable in multiple frames to obtain the three-dimensional points or three-dimensional line segments of the power cable.

[0092] Using the three-dimensional position of the tower corresponding to the tower detection results, constraints are applied to at least one of the following: endpoint positioning of the power cable, correction of the three-dimensional solution results, and topology consistency verification.

[0093] Specifically, after obtaining the stabilization detection results, the observation results belonging to the same power cable in adjacent frames are first associated. The stabilization detection results include the power cable detection results and pole detection results in adjacent frames. The power cable detection results are used to provide the pixel distribution of the power cable in the image plane, and the pole detection results are used to provide the anchoring information corresponding to the power cable endpoints or suspension points. For the power cable detection results in adjacent frames, preliminary matching is performed based on the spatial continuity and directional consistency of the power cable pixel region in the image plane, and cable observation results that are close in location and direction are preferentially identified as candidate association results.

[0094] After initial matching, the candidate association results are corrected by combining the temporal tracking results in adjacent frames. Specifically, the motion relationship between adjacent frames is used to predict the projection position of the power cable in adjacent frames, and cross-frame mismatches are filtered according to the correspondence between the predicted projection position and the detection result of the current frame. Thus, the pixel trajectory of the same power cable can remain consistent in multiple consecutive frames, reducing misassociations caused by single clue matching. For image regions with bifurcations, intersections, or multiple cables that are close together, when there are multiple candidate association results, ambiguity resolution is performed by combining the tower detection results. Specifically, the tower position corresponding to the tower detection result is used as the anchoring basis for the power cable endpoint or suspension point, so that the cable observation results connected to the same tower are preferentially associated as the same power cable, thereby determining the multi-frame observation results of the same power cable.

[0095] After determining the observation results of the same power cable across multiple frames, the observation results are back-projected into three-dimensional space based on the pose data of the low-altitude aircraft. The pose data includes the spatial position and attitude information of the low-altitude aircraft corresponding to each frame of the scene image. For any power cable observation result in any frame, the observation result is back-projected into a corresponding three-dimensional ray based on the pose data corresponding to that frame and the pixel position of the power cable in the image plane. The three-dimensional ray is used to represent the possible position and direction of the power cable pixel observation result in three-dimensional space.

[0096] When the observation baseline between adjacent frames meets preset conditions, triangulation is performed on multiple frames of three-dimensional rays corresponding to the same power cable to obtain three-dimensional points or three-dimensional line segments of the power cable. The observation baseline is used to characterize the spatial interval between the acquisition positions of adjacent frames; when the observation baseline meets preset conditions, the three-dimensional rays in different frames can provide disparity constraints for three-dimensional solution. By performing multi-frame three-dimensional solution on multiple frames of three-dimensional rays corresponding to the same power cable, the position and orientation of the power cable in three-dimensional space are obtained, forming a three-dimensional representation of the power cable.

[0097] During the 3D solution process, the 3D representation of the power cable is anchored or constrained based on the tower detection results. Specifically, the 3D position of the tower corresponding to the tower detection results is used to constrain at least one of the following: endpoint positioning of the power cable, correction of the 3D solution results, and topology consistency verification. For power cable observation results close to the tower, the 3D position of the tower is used as the anchor position of the power cable endpoint or suspension point to screen candidate cable segments with the tower as the endpoint, and the endpoint of the power cable is constrained to fall near the corresponding position of the tower during the 3D solution. For the obtained 3D points or 3D line segments of the power cable, the connection relationship between the tower and the power cable is further used to perform topology consistency verification to correct cross-frame association errors or reconstruction drift caused by single-thread association.

[0098] After the above processing, a three-dimensional representation of the power cables is obtained. This three-dimensional representation includes three-dimensional points and / or three-dimensional line segments of the power cables, and characterizes the position and orientation of the power cables in three-dimensional space. This three-dimensional representation of the power cables serves as the foundational data for subsequently constructing and updating a three-dimensional power cable map containing topological relationships.

[0099] S500, constructing and updating a three-dimensional power cable map containing topological relationships based on the three-dimensional representation of the power cables; including:

[0100] The three-dimensional representation of the power cable is converted into power cable nodes and power cable segments connecting the power cable nodes;

[0101] Establish or update topology edges based on the spatial distance, directional angle, and confidence level between adjacent power cable nodes;

[0102] When multiple candidate connection objects exist, the target connection object is determined based on at least one of the following: closer distance, better direction matching, and higher confidence level.

[0103] Based on the three-dimensional representation of power cables, a three-dimensional power cable map containing topological relationships is constructed and updated, so that the position, orientation and connection relationship of power cables in three-dimensional space can be expressed in the form of a graph structure.

[0104] Specifically, after obtaining the three-dimensional points or three-dimensional line segments of the power cable, the three-dimensional representation of the power cable is converted into power cable nodes and power cable segments connecting the power cable nodes. The power cable nodes are used to represent the endpoints, suspension points, key points, or segment connection points of the power cable in three-dimensional space, and the power cable segments are used to represent the spatial extension relationship between adjacent power cable nodes. Thus, the three-dimensional representation of the power cable is organized into graph structure data containing nodes and line segments. Each node in the graph structure data has a three-dimensional position attribute, and may further have direction attributes and confidence attributes. Each line segment is used to represent the power cable connection relationship between corresponding nodes.

[0105] When constructing a 3D power cable map containing topological relationships, topological edges are established or updated based on the spatial distance, directional angle, and confidence level between adjacent power cable nodes. Specifically, for a power cable node to be added to the 3D power cable map, the spatial distance between the node and existing power cable nodes is first calculated. When the spatial distance meets the proximity condition, the corresponding existing power cable node is identified as a candidate connection object. Subsequently, the directional angle between the cable direction corresponding to the node to be added and the cable direction corresponding to the candidate connection object is calculated, and it is determined whether the two belong to a power cable connection relationship with the same direction based on the directional angle. When both the spatial distance and directional angle meet the connection condition, topological edges are established or updated based on the confidence level of the corresponding detection results or 3D solution results, so that the topological edges represent the connection relationship between adjacent power cable nodes.

[0106] When multiple candidate connection objects exist, the target connection object is determined based on at least one of the following: closer distance, better directional matching, and higher confidence level. Specifically, candidate connection objects that are spatially closer to the power cable node to be added are preferentially selected; when multiple candidate connection objects with similar spatial distances exist, candidate connection objects with smaller directional angles and higher directional consistency are further selected; when spatial distance and directional consistency still cannot uniquely determine the target connection object, the target connection object is determined by combining the confidence level corresponding to the candidate connection object. Through the above method, power cable nodes that satisfy the connection relationship are established as topological edges, and existing topological edges are updated.

[0107] During incremental map maintenance, the newly acquired 3D representation of power cables in each frame is integrated into the 3D power cable map. For new detection results that can be associated with existing power cable nodes or segments, the existing node positions and confidence levels are updated with weights based on measurement uncertainty. For new detection results that cannot be associated with existing nodes or segments, they are added to the 3D power cable map as new power cable nodes or segments. For power cable nodes or topological edges that have not been observed for a long time and whose confidence levels have decreased, their confidence levels in the map are reduced to suppress false detection results from polluting the 3D power cable map.

[0108] After the above processing, a 3D power cable map containing topological relationships is obtained. This 3D power cable map represents the distribution and connection relationships of power cables in 3D space using power cable nodes, power cable segments, and topological edges, and is incrementally updated with new 3D representations of power cables. This 3D power cable map is used to subsequently calculate the safe distance between the flight path and the power cables, and to generate corresponding alarm information and detour path suggestions.

[0109] S600 calculates the safe distance between the flight path and the power cables based on the three-dimensional power cable map, and outputs alarm information and detour route suggestions; including:

[0110] The safety distance is determined based on the spatial relationship between the flight path and the three-dimensional power cable map;

[0111] Based on the correspondence between the safety distance and the preset safety conditions, hierarchical alarm information is generated;

[0112] Based on the three-dimensional power cable map, a bypass route suggestion is generated, and the graded alarm information and the bypass route suggestion are output to the flight control system and / or the pilot display interface.

[0113] Specifically, after obtaining a 3D power cable map containing topological relationships, the planned or currently predicted flight path of the low-altitude aircraft is acquired, and the flight path and the 3D power cable map are placed in the same spatial coordinate system. The 3D power cable map includes power cable nodes, power cable segments, and topological edges representing the connection relationships of the power cables. The flight path includes the spatial trajectory of the low-altitude aircraft within a preset range ahead. By calculating the spatial relationship between the flight path and the cable spatial objects represented by the power cable nodes, power cable segments, or topological edges in the 3D power cable map, the safe distance between the flight path and the power cables is determined.

[0114] When calculating the safety distance, power cable nodes or segments located within a preset range ahead of the flight path in the three-dimensional power cable map are taken as the objects to be evaluated, and the minimum spatial distance between the flight path and the objects to be evaluated is calculated. The minimum spatial distance is used to characterize the closest approach between the low-altitude aircraft and the power cable when moving along the flight path. For power cables composed of multiple nodes and segments, the spatial distance between the flight path and the corresponding power cable segment can be calculated separately, and the minimum value among them is taken as the safety distance.

[0115] After determining the safety distance, a tiered alarm information is generated based on the correspondence between the safety distance and preset safety conditions. The preset safety conditions are used to characterize the risk level corresponding to different safety distance ranges; when the safety distance is lower than the corresponding safety condition, an alarm information of the corresponding level is generated. The alarm information is used to indicate the degree of spatial risk between the flight path and power cables, and the alarm information may include at least one of the following: early warning level alarm information, emergency level alarm information, and mandatory intervention level alarm prompt information.

[0116] Furthermore, bypass path suggestions are generated based on the three-dimensional power cable map. These bypass path suggestions are generated according to the spatial location, extension direction, and topological connection relationship of the power cables in the three-dimensional power cable map, ensuring that the suggested path avoids power cable spatial areas with insufficient safety clearance. When generating bypass path suggestions, power cable nodes or power cable segments within a preset range ahead of the flight path are used as constraints. While maintaining the required safety clearance with the power cables, the path adjustment direction or candidate bypass path for bypassing the corresponding power cables is determined.

[0117] After generating the tiered alarm information and the bypass path suggestions, these are output to the flight control system and / or the pilot's display interface. The tiered alarm information indicates the safety risk level between the current flight path and the power cables, while the bypass path suggestions provide a reference for flight path adjustments. The tiered alarm information and bypass path suggestions are not directly limited to flight control commands; the flight control system or pilot's display interface makes subsequent decisions or displays based on the received information. This forms a closed loop for obstacle avoidance information processing, from a 3D power cable map to safe distance calculation, tiered alarms, and bypass path suggestion output.

[0118] In one implementation, when performing three-dimensional perception and obstacle avoidance tasks for power cables, the low-altitude aircraft enhances the robustness of the visual-inertial positioning process to maintain continuous pose output in situations with low texture, high-speed motion, or insufficient local visual features.

[0119] Specifically, dynamic region of interest (ROI) selection is performed on the high-resolution scene image. The scene image is divided into 8×8 candidate ROIs, and a comprehensive texture score is calculated for each candidate ROI. The comprehensive texture score is jointly determined by the Laplacian variance, the FAST feature point density, and the spatial distribution uniformity of the feature points, where FAST stands for Features from Accelerated Segment Test. The comprehensive texture score can be expressed as:

[0120] ;

[0121] in, This represents the overall texture score of the candidate region of interest. This represents the Laplace variance within the candidate region of interest. This represents the density of FAST feature points within the candidate region of interest. This indicates the uniformity of the spatial distribution of feature points within the candidate region of interest. Based on the comprehensive texture score, a target region of interest is selected from the candidate regions of interest for feature tracking, allowing the visual-inertial localization process to preferentially utilize image regions with richer texture information and more uniform feature distribution.

[0122] When updating the region of interest (ROI) between adjacent frames, a smoothing factor is introduced to suppress abrupt changes in the ROI location. The smoothing factor can be expressed as:

[0123] ;

[0124] in, Represents the smoothing factor. This represents the distance between the center of the current candidate region of interest and the center of the selected region of interest in the previous frame. This represents an exponential function. The smoothing factor smooths the scores of candidate regions of interest, ensuring spatial continuity of the target regions of interest in adjacent frames and reducing jumps in regions of interest caused by local image noise or instantaneous texture changes.

[0125] After determining the target region of interest (ROI), hierarchical tracking of image feature points is performed. For feature points located within the ROI, the predicted position from the inertial measurement unit (IMU) is used as the initial value, and KLT (Kanade-Lucas-Tomasi) tracking is performed on a high-resolution sub-image to obtain high-precision feature point trajectories. KLT feature tracking detects corner features and iteratively solves for the feature point displacement between adjacent frames based on the assumption of invariant grayscale values ​​of neighboring pixels. For feature points located outside the ROI, continuous tracking is maintained on the downsampled full image, and they participate in pose estimation with a weight of 0.6 to maintain the continuity of global motion constraints. When the number of trackable feature points falls below 40, new feature points are added within the ROI to maintain the effective number of features required for visual-inertial localization.

[0126] Furthermore, pre-integration is performed using inertial measurement unit (IMU) data to assist in image patch sampling and inter-frame motion estimation. The IMU continuously acquires angular velocity and linear acceleration data of the low-altitude aircraft and predicts the rotation and translation between adjacent frames based on this data. The predicted inter-frame rotation and translation are used to determine the image region where the cable target may appear in the next frame and adaptively adjust the sampling center of the cable detection image patch. When the predicted image displacement exceeds 50% of the image size, a 50% overlap sampling mode is triggered, creating an overlapping region between adjacent image patches to reduce the risk of power cable targets falling into sampling blind zones during high-speed motion.

[0127] When the visual-inertial positioning state changes, the positioning mode is automatically switched based on the number of visual features, flight speed, and inertial measurement unit (IMU) integration time. The positioning process includes a standard visual-inertial positioning mode, a high-speed compensation mode, an IMU-dominated mode, and a Global Navigation Satellite System (GNSS) correction mode. The standard visual-inertial positioning mode is used to output pose data when there are sufficient visual features and the flight state is stable; the high-speed compensation mode is used to enhance inter-frame tracking by combining IMU prediction results when the flight speed is high or the image motion is large; the IMU-dominated mode is used to maintain short-term pose output by using IMU pre-integration results when there are insufficient visual features in multiple consecutive frames; and the GNSS correction mode is used to introduce GNSS data to correct cumulative drift when visual-inertial positioning continues to degrade.

[0128] In one specific implementation, the standard visual-inertial positioning mode is used when the number of visual features is sufficient and the flight speed is low. For example, when the number of trackable feature points is greater than or equal to 60 and the linear velocity is less than 3 m / s, the standard visual-inertial positioning mode is adopted, and positioning processing is performed using a 640×480, 30Hz image input. The high-speed compensation mode is used when the flight speed is high. For example, when the linear velocity is greater than 3 m / s, the image input is switched to 320×240, 120Hz, and the search initialization is assisted by the inertial measurement unit pre-integration result to improve the inter-frame tracking continuity under high-speed motion. The inertial measurement unit-dominated mode is used when the number of visual features is insufficient for a short period of time. For example, when the number of trackable feature points is less than 15 and there are insufficient features for 5 consecutive frames, the system switches to inertial measurement unit integration positioning, and the visual observation results are only used as occasional correction information. The effective duration of the inertial measurement unit-dominated mode does not exceed 2 seconds. The global navigation satellite system correction mode is used when the visual-inertial positioning is continuously degrading. For example, when the integration time of the inertial measurement unit exceeds 2 seconds, the barometric altimeter is used to correct the absolute position data of the global navigation satellite system.

[0129] Through the aforementioned dynamic region of interest selection, hierarchical feature tracking, inertial measurement unit pre-integration assisted sampling, and multi-level positioning mode switching processing, continuous pose data output can be maintained even when the low-altitude aircraft is in a low-texture background, moving at high speed, or with insufficient visual features for a short period. This pose data is then used for temporal fusion of detection results from adjacent frames, 3D ray backprojection of power cable observation results, and incremental updates of the 3D power cable map.

[0130] In one implementation, without changing the main line of three-dimensional perception and obstacle avoidance processing of power cables, the detection model, temporal fusion method, three-dimensional reconstruction method, map representation method or positioning method can be replaced to adapt to different computing resources, sensor configurations or flight environments.

[0131] Specifically, the shared backbone network in the detection model can be replaced by other lightweight feature extraction networks. For example, the shared backbone network can be a lightweight network such as EfficientNet-B0 (efficient convolutional neural network) or RepVGG (re-parameterized visual geometry network) to balance detection accuracy and inference speed on different airborne computing platforms. For the implementation using MobileNetV4-Conv-M as the shared backbone network, MobileNetV3-Large can also be used as an alternative backbone network to adapt to different computing performance and model deployment requirements. Regardless of the backbone network used, the detection model maintains the shared feature extraction and dual-detection-head output structure, and outputs power cable distance map results and tower distance map results respectively.

[0132] In terms of temporal fusion, besides using DIS (Dense Inverse Search) optical flow to deform, align, and fuse the distance map results of adjacent frames, RAFT (Recurrent All-Pairs Field Transforms) optical flow or PWC-Net (Image Pyramid, Deformation, and Cost Volume Network) optical flow can also be used to obtain the motion correspondence between adjacent frames. For application scenarios with limited computational resources, a feature-point-based sparse optical flow tracking method can also be used to determine the local motion relationship between adjacent frames, and the cable detection results in adjacent frames can be aligned and fused based on the local motion relationship. All of the above alternative methods are used to improve the temporal consistency between the detection results of adjacent frames and reduce jitter, missed detections, or false detections in single-frame detection results.

[0133] In the 3D reconstruction of power cables, besides backprojecting and triangulating multiple frames of observations of the same power cable based on aircraft pose data, joint calibration can be performed by combining the depth output of a monocular depth estimation network with visual-inertial positioning (VIO) results to obtain the 3D spatial position corresponding to the power cable observation results. For example, the depth results output by Depth-Anything (a general monocular depth estimation network) can be used as auxiliary depth information and combined with VIO pose data to determine the 3D representation of the power cable. For flight platforms equipped with LiDAR, LiDAR ranging results can also be used as an auxiliary depth source and spatially correlated with the pixel regions of the power cable obtained by visual detection to assist in obtaining the 3D representation of the power cable. All of the above alternative 3D reconstruction methods use the power cable detection results as input and are used to obtain the position or extension direction of the power cable in 3D space.

[0134] In terms of map representation, besides using a 3D power cable topology map containing power cable nodes, power cable segments, and topological edges, voxel occupancy maps or cable parametric models can also be used to represent the 3D spatial distribution of power cables. For example, a voxel occupancy map can be used to represent the occupancy status of the spatial area where power cables are located, and a cable parametric model can be used to express the spatial extension shape of power cables using curve parameters or line segment parameters. Compared to 3D power cable topology maps, voxel occupancy maps and cable parametric models have relatively weaker expressive capabilities for cable connection relationships. Therefore, when continuous maintenance of cable endpoints, cable connection relationships, and tower anchor point relationships is required, a 3D power cable map containing topological relationships should be preferred.

[0135] Regarding positioning methods, in addition to visual-inertial positioning (VIS) for obtaining low-altitude aircraft attitude data, laser inertial odometry (LAO) or Global Navigation Satellite System (GNSS) / real-time dynamic differential positioning (JDS / PD) can also be used. LAO obtains attitude data by fusing lidar measurement data and inertial measurement unit (INS) data; JDS / PD obtains attitude data through satellite positioning data and differential correction information. In flight environments with urban obstructions, cable-related blockages, or unstable satellite signals, VIS or LAO can provide more continuous local attitude output; in open areas, JDS / PD can provide absolute position correction.

[0136] Through the above alternative implementation methods, while keeping the processing chain of "joint detection of power cables and towers, stabilization of detection results, improvement of two-dimensional observation results to three-dimensional representation, construction of three-dimensional power cable maps, and output of obstacle avoidance information" unchanged, the specific models, sensors, or map representation forms can be adjusted according to different hardware platforms and application environments.

[0137] Through the above implementation methods, the power cable detection results can be upgraded from a two-dimensional image plane to a three-dimensional space, and further formed into a three-dimensional power cable map containing topological relationships. This enables low-altitude aircraft to perform safety distance calculations, graded alarms, and bypass path suggestions based on the three-dimensional spatial location and connection relationship of power cables.

[0138] Specifically, by uniformly converting power cables and towers into distance map regression targets, the detection model can output continuous distance response results for distant, weakly textured, and slender power cable targets, reducing the problem that ordinary bounding box detection methods cannot accurately represent linear targets. For scenes with distances of hundreds of meters, weakly textured backgrounds, or power cables with small widths in the image, the distance map regression results can better preserve the linear extension features of power cables in the image, providing stable two-dimensional observation input for subsequent topological continuity constraints and 3D reconstruction.

[0139] By introducing topological continuity constraints into the training of the detection model and the processing of detection results, the number of breaks, isolated points, and erroneous connections in power cable detection results can be reduced, making the detection results more consistent with the physical form of continuous power cable extension. Further temporal fusion of detection results from adjacent frames can reduce instantaneous noise, missed detections, and jitter in single-frame detection, ensuring consistency of power cable observation results across consecutive frames in the temporal dimension. This provides a more stable data foundation for the correlation of multi-frame observations of the same power cable.

[0140] By combining multi-frame observations of the same power cable with low-altitude aircraft pose data, the two-dimensional cable detection results are back-projected into three-dimensional rays. Then, by solving the multi-frame three-dimensional rays in three dimensions, the position and orientation of the power cable in three-dimensional space can be obtained. Further, by combining tower detection results with anchoring or constraint of the three-dimensional representation of the power cable, towers can be used as spatial references for the cable endpoints or suspension points, reducing ambiguity and drift in the multi-frame three-dimensional solution and improving the consistency between the three-dimensional representation of the power cable and the actual cable spatial structure.

[0141] By organizing the three-dimensional representation of power cables into a 3D power cable map consisting of power cable nodes, power cable segments, and topological edges, the spatial location, extension direction, and connection relationship of power cables can be simultaneously expressed in the map. Compared to processing methods that only output two-dimensional detection boxes or two-dimensional segmentation masks, the 3D power cable map can provide a spatial basis for calculating safe distances for flight paths and can support subsequent obstacle avoidance information output according to the cable topology.

[0142] By employing dynamic region of interest selection, hierarchical feature tracking, inertial measurement unit pre-integration assisted sampling, and multi-level positioning mode switching, continuous pose data output can be maintained even under conditions of low-texture background, high-speed flight, or short-term lack of visual features. This enables the temporal fusion of detection results from adjacent frames, 3D reconstruction of power cables, and incremental updates of 3D power cable maps to have a continuous spatial reference.

[0143] By employing a shared backbone network and a dual-detection-head structure, the system can simultaneously output power cable distance maps and tower distance maps based on a single feature extraction, reducing the computational and storage overhead caused by repeated feature extraction. Combined with image block segmentation, batch inference, and distance map stitching, it can adapt to deployment environments with limited airborne computing resources and obtain complete scene-specific power cable and tower detection results even with high-resolution image input.

[0144] Therefore, low-altitude aircraft can calculate the safe distance between the flight path and the power cables based on a three-dimensional power cable map containing topological relationships, and generate graded alarm information and bypass path suggestions, so that the power cable perception results can be transformed from simple image detection results into three-dimensional spatial information that can be used for obstacle avoidance.

[0145] Compared with existing cable detection methods that only output two-dimensional detection boxes or two-dimensional segmentation masks, this invention uniformly converts power cables and towers into distance map regression targets. Furthermore, it combines topological continuity constraints, inter-frame temporal fusion, multi-frame three-dimensional solution, and the construction of a three-dimensional power cable map containing topological relationships. This allows the power cable detection results to be elevated from a two-dimensional image plane to a three-dimensional space, thereby providing a spatial basis for calculating safe distances, graded alarms, and bypass path suggestions for low-altitude aircraft.

[0146] By employing distance map regression, this invention can output continuous distance response results for distant, weakly textured, and slender power cable targets, mitigating the problem that ordinary bounding box detection methods struggle to accurately represent slender linear targets. In scenarios where power cables are far away, have weak background texture, or are narrow in width within the image, the distance map regression results can effectively preserve the linear extension characteristics of the power cables in the image, providing stable two-dimensional observation input for subsequent topological continuity constraints and 3D reconstruction.

[0147] By introducing the MALIS topology continuity loss term, this invention can enhance the topology continuity of power cable detection results, reduce cable breaks, isolated points and erroneous connections, and make the detection results more consistent with the physical form of continuous extension of real power cables, thereby improving the stability of subsequent multi-frame observation association and 3D reconstruction.

[0148] By combining multi-frame observations of the same power cable with low-altitude aircraft pose data, and using tower detection results to anchor or constrain the 3D representation of the power cable, this invention reduces ambiguity and drift in multi-frame 3D solutions, and improves the consistency between the 3D representation of the power cable and the actual cable spatial structure. By organizing the 3D representation of the power cable into a 3D power cable map composed of power cable nodes, power cable segments, and topological edges, this invention can simultaneously express the spatial location, extension direction, and connection relationships of the power cable in the map, providing a reliable 3D spatial foundation for subsequent obstacle avoidance information output.

[0149] By employing dynamic region of interest selection, hierarchical feature tracking, inertial measurement unit pre-integration assisted sampling, and multi-level positioning mode switching, this invention can maintain continuous pose output under conditions of low-texture background, high-speed flight, or short-term lack of visual features, thereby improving the positioning robustness and safety redundancy of low-altitude aircraft in cable scenarios.

[0150] By employing a shared backbone network and a dual-detection-head structure, this invention can simultaneously output power cable distance map results and tower distance map results based on a single feature extraction, reducing redundant calculations caused by building separate independent detection models. Combined with image block segmentation, batch inference, and distance map stitching processing, this invention can adapt to the computing power, memory, and power consumption limitations of airborne embedded platforms, and obtain power cable and tower detection results corresponding to the complete scene under high-resolution image input conditions.

[0151] Second embodiment:

[0152] like Figure 2 As shown, this embodiment provides a three-dimensional perception and obstacle avoidance system for power cables of low-altitude aircraft. This system is used to execute the aforementioned three-dimensional perception and obstacle avoidance method for power cables of low-altitude aircraft. The system includes a data acquisition module, a joint detection module, a stabilization processing module, a three-dimensional solution module, a topology map module, and an obstacle avoidance output module. The modules are connected in the order of data flow: image acquisition, target detection, detection result stabilization, three-dimensional solution, topology map construction, and obstacle avoidance information output.

[0153] The data acquisition module is used to acquire images of the scene in front of the low-altitude aircraft and the corresponding pose data of the aircraft. The scene image is acquired by the onboard image acquisition equipment during the flight of the low-altitude aircraft, and includes power cables, towers, and the background environment. The data acquisition module is also used to preprocess the acquired raw scene image. This preprocessing includes at least one of de-mosaicing, zero-filling, and image block segmentation, so that the preprocessed scene image or image blocks can be used as input for subsequent joint detection. The pose data is used to characterize the spatial position and attitude state of the low-altitude aircraft at the time of acquisition of the corresponding image frame, and is used for subsequent temporal fusion of detection results, 3D ray back projection, and 3D power cable map construction.

[0154] The joint detection module is connected to the data acquisition module and is used to perform joint detection of power cables and poles based on the scene image to obtain detection results characterizing the distance relationship between the power cables and poles in the image. Specifically, the joint detection module inputs the scene image or image blocks segmented from the scene image into the detection model. The detection model performs distance map regression processing on the input image and outputs distance map results corresponding to the power cables and the poles, respectively. The distance map results characterize the normalized distance relationship from each pixel in the image to the corresponding target location, with regions having a distance value of 0 corresponding to the target location. The joint detection module determines the pixel region of the power cables based on the power cable distance map results and the pixel region of the poles based on the pole distance map results, thereby obtaining the joint detection results of the power cables and poles.

[0155] The stabilization module is connected to the joint detection module and performs topological continuity constraint processing on the detection results. It also performs temporal fusion processing on the detection results from adjacent frames to obtain a stabilized detection result. Specifically, the stabilization module applies topological continuity constraints to the power cable detection results to maintain the continuity of the power cables, ensuring that pixel regions belonging to the same power cable maintain a continuous linear structure in the image plane and suppressing power cable breaks, isolated regions, and local false detections. The stabilization module also aligns the distance map results in adjacent frames based on the motion relationship between them and fuses the aligned multi-frame distance map results to reduce instantaneous noise, missed detections, and jitter in single-frame detection, thus obtaining a stabilized detection result.

[0156] The 3D solving module is connected to the data acquisition module, the joint detection module, and the stabilization processing module, respectively. It is used to correlate multiple frames of observation results for the same power cable and back-project the correlated power cable observation results into 3D rays based on the pose data. A 3D representation of the power cable is obtained by performing multi-frame 3D solving on these 3D rays, wherein the 3D representation of the power cable is anchored or constrained by the tower detection results. Specifically, the 3D solving module performs preliminary matching of multi-frame observation results based on the spatial continuity and directional consistency of the power cable in adjacent frames, and corrects the preliminary matching results by combining the temporal tracking results in adjacent frames. When multiple candidate correlation results exist, the 3D solving module performs ambiguity resolution by combining the tower detection results to determine the multi-frame observation results for the same power cable. Subsequently, the 3D solving module back-projects the correlated power cable observation results into corresponding 3D rays based on the pose data corresponding to each frame. When the observation baseline between adjacent frames meets preset conditions, triangulation is performed on the multi-frame 3D rays corresponding to the same power cable to obtain the 3D points or 3D line segments of the power cable. The 3D solution module also uses the 3D position of the tower corresponding to the tower detection results to constrain at least one of the following: endpoint positioning of power cables, correction of 3D solution results, and topology consistency verification.

[0157] The topology map module is connected to the 3D solving module and is used to construct and update a 3D power cable map containing topological relationships based on the 3D representation of the power cables. Specifically, the topology map module converts the 3D representation of the power cables into power cable nodes and power cable segments connecting the power cable nodes, and establishes or updates topological edges based on the spatial distance, directional angle, and confidence level between adjacent power cable nodes. The power cable nodes are used to represent the endpoints, suspension points, key points, or segment connection points of the power cables in 3D space; the power cable segments are used to represent the spatial extension relationship between adjacent power cable nodes; and the topological edges are used to represent the connection relationship between power cable nodes. When multiple candidate connection objects exist, the topology map module determines the target connection object based on at least one of the following: closer distance, better directional match, and higher confidence level, and updates the 3D power cable map based on the determined target connection object.

[0158] The obstacle avoidance output module is connected to the topology map module and is used to calculate the safe distance between the flight path and the power cables based on the 3D power cable map, and output alarm information and detour path suggestions. Specifically, the obstacle avoidance output module places the flight path and the 3D power cable map in the same spatial coordinate system, and determines the safe distance according to the spatial relationship between the flight path and the 3D power cable map. The obstacle avoidance output module generates graded alarm information based on the correspondence between the safe distance and preset safety conditions, and generates detour path suggestions based on the 3D power cable map. The graded alarm information and the detour path suggestions are output to the flight control system and / or the pilot display interface to provide obstacle avoidance information output based on the 3D power cable map.

[0159] In the system provided in this embodiment, the specific processing procedures performed by each module can be referred to the corresponding steps in the first embodiment mentioned above, and will not be repeated here. Through the above system, the images of the scene ahead and the pose data collected by the low-altitude aircraft can be sequentially converted into joint detection results of power cables and towers, stabilization detection results, three-dimensional representation of power cables, and a three-dimensional power cable map containing topological relationships. Based on the three-dimensional power cable map, safe distances, alarm information, and detour path suggestions are output.

[0160] Third embodiment:

[0161] like Figure 3 As shown, this embodiment provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method described in the first embodiment.

[0162] Specifically, the electronic device can be installed in the onboard computing platform of the low-altitude aircraft, or it can communicate with the image acquisition device, inertial measurement unit, positioning device, flight control system, and / or pilot display interface of the low-altitude aircraft. The image acquisition device is used to acquire images of the scene in front of the low-altitude aircraft, the inertial measurement unit and / or positioning device is used to provide the corresponding pose data of the low-altitude aircraft, and the processor is used to perform joint detection of power cables and towers, stabilization processing of detection results, correlation of multi-frame observation results of power cables, 3D solution, 3D power cable map construction, and obstacle avoidance information output based on the image of the scene in front and the pose data.

[0163] When the processor executes the computer program, it acquires an image of the scene in front of the low-altitude aircraft and the pose data corresponding to the low-altitude aircraft; it performs joint detection on power cables and towers based on the scene image to obtain detection results characterizing the distance relationship between the power cables and towers in the image; it performs topological continuity constraint processing on the detection results and performs temporal fusion processing with the detection results in adjacent frames to obtain stable detection results; it correlates the observation results of the same power cable in multiple frames and backprojects the correlated power cable observation results into a three-dimensional ray based on the pose data, and obtains the three-dimensional representation of the power cable by solving the three-dimensional ray in multiple frames, wherein the three-dimensional representation of the power cable is anchored or constrained in combination with the tower detection results; it constructs and updates a three-dimensional power cable map containing topological relationships based on the three-dimensional representation of the power cable; it calculates the safe distance between the flight path and the power cable based on the three-dimensional power cable map, and outputs alarm information and detour path suggestions.

[0164] In one embodiment, the processor may include at least one of a central processing unit, a graphics processing unit, an embedded computing chip, a neural network processing unit, or a field-programmable gate array. The processor can be used to perform processing such as image preprocessing, image block segmentation, distance map regression inference, distance map stitching, temporal fusion, 3D ray backprojection, 3D power cable map updating, and safety distance calculation. The memory is used to store at least one of computer programs, detection model parameters, temporary image data, distance map results, pose data, 3D power cable map data, and obstacle avoidance output data.

[0165] In one embodiment, the electronic device may further include a communication interface for receiving scene images from an image acquisition device, receiving pose-related data from an inertial measurement unit and / or a positioning device, and outputting the alarm information and the bypass path suggestion to the flight control system and / or the pilot's display interface. When the electronic device interacts with the flight control system of the low-altitude aircraft through the communication interface, the alarm information and bypass path suggestion are output as obstacle avoidance information and are not directly limited to flight control commands.

[0166] In this embodiment, the electronic device executes a computer program stored in the memory via a processor, enabling the aforementioned three-dimensional perception and obstacle avoidance method for low-altitude aircraft power cables to be implemented on an airborne computing platform or a computing platform connected to the low-altitude aircraft. The specific processing steps and technical effects performed by the electronic device can be found in the corresponding content of the first embodiment, and will not be repeated here.

[0167] Fourth embodiment:

[0168] This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first embodiment.

[0169] The computer-readable storage medium can be a tangible medium capable of storing computer programs, including but not limited to read-only memory, random access memory, flash memory, hard disk, solid-state drive, memory card, optical disk, or other media capable of storing program code. The computer program may include program instructions for implementing image acquisition and preprocessing, joint detection of cables and towers, stabilization processing of detection results, correlation of multi-frame observation results, three-dimensional solution, construction of three-dimensional power cable maps, and output of obstacle avoidance information.

[0170] In one embodiment, the computer-readable storage medium may also store at least one of the following: detection model parameters, distance map regression inference program, temporal fusion program, 3D ray back projection program, 3D power cable map maintenance program, and safety distance calculation program. When the processor reads and executes the program instructions in the computer-readable storage medium, it can generate a 3D representation of the power cables based on the scene image and pose data in front of the low-altitude aircraft, and further generate a 3D power cable map containing topological relationships, alarm information, and bypass path suggestions.

[0171] The specific processing procedures and their technical effects implemented by the computer program stored in the computer-readable storage medium can be referred to the corresponding contents in the foregoing method embodiments, system embodiments and electronic device embodiments, and will not be repeated here.

[0172] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented, in whole or in part, as a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium, such as a solid-state drive (SSD).

[0173] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for three-dimensional perception and obstacle avoidance of power cables for low-altitude aircraft, characterized in that, include: Acquire an image of the scene in front of the low-altitude aircraft and the corresponding pose data of the low-altitude aircraft. Based on the scene image, joint detection of power cables and towers is performed to obtain detection results characterizing the distance relationship between the power cables and towers in the image; The detection results are subjected to topological continuity constraint processing, and temporal fusion processing is performed in combination with the detection results in adjacent frames to obtain stable detection results; Multiple frames of observation results for the same power cable are correlated, and the correlated power cable observation results are back-projected into a three-dimensional ray based on the pose data. The three-dimensional representation of the power cable is obtained by solving the three-dimensional ray in multiple frames. The three-dimensional representation of the power cable is anchored or constrained by the tower detection results. A three-dimensional power cable map containing topological relationships is constructed and updated based on the three-dimensional representation of the power cables. Based on the three-dimensional power cable map, the safe distance between the flight path and the power cable is calculated, and alarm information and detour route suggestions are output.

2. The method according to claim 1, characterized in that, Based on the scene image, joint detection of power cables and towers is performed to obtain detection results characterizing the distance relationship between the power cables and towers in the image, including: The scene image is processed by distance map regression using a detection model, and the distance map results corresponding to power cables and towers are output. Based on the distance map results, the pixel regions of power cables and towers are determined, and the joint detection results of the power cables and towers are obtained.

3. The method according to claim 1, characterized in that, The detection results are subjected to topological continuity constraint processing, and temporal fusion processing is performed on the detection results in adjacent frames to obtain stable detection results, including: The detection results are subjected to topological continuity constraints to maintain the continuity of power cables, thereby suppressing power cable breaks and isolated areas. The detection results in adjacent frames are aligned based on the motion relationship between adjacent frames, and the aligned multi-frame detection results are fused to obtain the stabilized detection result.

4. The method according to claim 1, characterized in that, Correlating multiple frames of observations of the same power cable, including: Based on the spatial continuity and directional consistency of power cables in adjacent frames, preliminary matching of observation results across multiple frames is performed. The initial matching results are corrected by combining the temporal tracking results in adjacent frames; When multiple candidate association results exist, the ambiguity of the candidate association results is resolved by combining the tower detection results, so as to determine the multi-frame observation results of the same power cable.

5. The method according to claim 1, characterized in that, Based on the pose data, the associated power cable observation results are back-projected into a three-dimensional ray. A three-dimensional representation of the power cable is obtained by performing multi-frame three-dimensional solving on the three-dimensional ray, including: Based on the pose data, the associated power cable observation results are back-projected into the corresponding three-dimensional ray; When the observation baseline between adjacent frames meets the preset conditions, triangulation is performed on the three-dimensional rays corresponding to the same power cable in multiple frames to obtain the three-dimensional points or three-dimensional line segments of the power cable. Using the three-dimensional position of the tower corresponding to the tower detection results, constraints are applied to at least one of the following: endpoint positioning of the power cable, correction of the three-dimensional solution results, and topology consistency verification.

6. The method according to claim 1, characterized in that, Constructing and updating a 3D power cable map containing topological relationships based on the 3D representation of the power cables includes: The three-dimensional representation of the power cable is converted into power cable nodes and power cable segments connecting the power cable nodes; Establish or update topology edges based on the spatial distance, directional angle, and confidence level between adjacent power cable nodes; When multiple candidate connection objects exist, the target connection object is determined based on at least one of the following: closer distance, better direction matching, and higher confidence level.

7. The method according to claim 1, characterized in that, Based on the aforementioned 3D power cable map, the safe distance between the flight path and the power cables is calculated, and alarm information and detour route suggestions are output, including: The safety distance is determined based on the spatial relationship between the flight path and the three-dimensional power cable map; Based on the correspondence between the safety distance and the preset safety conditions, hierarchical alarm information is generated; Based on the three-dimensional power cable map, a bypass route suggestion is generated, and the graded alarm information and the bypass route suggestion are output to the flight control system and / or the pilot display interface.

8. A three-dimensional perception and obstacle avoidance system for power cables of low-altitude aircraft, characterized in that, include: The data acquisition module is used to acquire images of the scene in front of the low-altitude aircraft and the corresponding pose data of the low-altitude aircraft. A joint detection module, connected to the data acquisition module, is used to perform joint detection of power cables and towers based on the scene image, and obtain detection results characterizing the distance relationship between the power cables and towers in the image; A stabilization processing module, connected to the joint detection module, is used to perform topological continuity constraint processing on the detection results and perform temporal fusion processing on the detection results in adjacent frames to obtain stable detection results. The three-dimensional solving module is connected to the data acquisition module, the joint detection module, and the stabilization processing module, respectively. It is used to associate multiple frames of observation results of the same power cable, and back-project the associated power cable observation results into three-dimensional rays based on the pose data. The three-dimensional representation of the power cable is obtained by solving the three-dimensional ray in multiple frames. The three-dimensional representation of the power cable is anchored or constrained by the tower detection results. The topology map module, connected to the 3D solution module, is used to construct and update a 3D power cable map containing topological relationships based on the 3D representation of the power cables. The obstacle avoidance output module, connected to the topology map module, is used to calculate the safe distance between the flight path and the power cable based on the three-dimensional power cable map, and output alarm information and detour path suggestions.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images