Distribution network hot-line work robot sensing and positioning system and method

Abstract

The invention discloses a distribution network hot-line work robot sensing and positioning system and method, and belongs to the technical field of robot visual sensing. The system comprises an input preprocessing module used for inputting an RGB-D image and preprocessing the RGB-D image; the shared feature extraction module is used for extracting features containing multi-scale information in the preprocessed RGB-D image; through feature tensor decomposition and an attention mechanism, performing enhanced expression on the features; the target detection module is used for carrying out multi-scale processing on the features after enhanced expression and outputting a target category and a 2D detection frame; and the 6D pose estimation module is used for performing pose estimation on the target category and the 2D detection frame based on the nerve implicit field, generating an initial pose hypothesis, and performing rendering and optimization by using the nerve implicit field to obtain a 6D pose. And deep cooperation of detection and pose estimation tasks is realized, so that the overall performance is improved.

Description

Technical Field

[0001] This application relates to the field of robot vision perception technology, specifically to a perception and positioning system and method for a live-line working robot in power distribution networks. Background Technology

[0002] In the field of live-line working robots for power distribution networks, the accuracy of target detection and 6D pose estimation is crucial for achieving autonomous operation. However, existing methods face numerous technical bottlenecks in complex scenarios: 1. Insufficient robustness in object detection. Traditional convolutional neural networks often exhibit low detection accuracy when processing small objects (such as tiny electronic components and precision mechanical parts) and low-texture objects (such as smooth metal surfaces and solid-color plastic parts). Moreover, in dynamic scenes (such as detection near high-speed operating equipment or environmental perception during vehicle movement), the stability of feature extraction is poor, making it susceptible to interference and leading to false positives or false negatives. 2.6D pose estimation suffers from a lack of geometric constraints. Pose estimation methods that rely solely on 2D image features struggle to accurately recover the absolute pose in 3D space. In environments with drastic lighting changes (such as transition areas from indoors to outdoors, or backlit scenes), feature matching is highly prone to failure, leading to pose estimation errors. Furthermore, these methods lack explicit modeling of the object's geometry, failing to fully utilize prior information such as the object's shape and size to aid pose estimation. 3. Low efficiency of multi-task collaboration. Currently, detection and pose estimation tasks are mostly modeled independently. This not only leads to redundant feature calculations and wastes computing resources, but also prevents effective fusion of cross-task information, resulting in fragmented information. In practical applications, especially for scenarios with extremely high real-time requirements (such as real-time control of robotic arms, requiring a latency of ≤20ms), this independent modeling approach is difficult to meet the requirements and cannot provide decision-making information quickly and accurately. 4. Poor adaptability to dynamic environments. Existing methods have significantly insufficient generalization ability for background interference (such as cluttered warehouse environments, complex natural scenes, etc.), target scale variations (such as similar objects of different sizes, imaging of objects at different distances, etc.), and pose diversity (such as objects placed at various angles, pose changes during movement, etc.). Especially in complex occlusion scenarios (such as multiple objects occluding each other, partially occluded targets, etc.), their performance degrades significantly, making it impossible to reliably complete target detection and pose estimation tasks. Summary of the Invention

[0003] To effectively address the aforementioned issues, this application proposes a perception and positioning system and method for live-line working robots in power distribution networks. Through cross-module frequency domain feature interaction and geometric constraint transmission mechanisms, it achieves deep collaboration between detection and pose estimation tasks, thereby improving overall performance.

[0004] The technical solution adopted by this application to solve its technical problem is: In a first aspect, this application provides a sensing and positioning system for a live-line working robot in a power distribution network, comprising: The input preprocessing module is used to input RGB-D images and preprocess them. A shared feature extraction module is used to extract features containing multi-scale information from preprocessed RGB-D images based on a lightweight convolutional architecture; and to enhance the representation of features through feature tensor decomposition and attention mechanisms. The object detection module is used to perform multi-scale processing on the enhanced features and output the object category and 2D detection box. The 6D pose estimation module estimates the pose of the target category and 2D bounding box based on the neural implicit field, generates an initial pose hypothesis, and uses the neural implicit field for rendering and optimization to obtain the 6D pose.

[0005] It also includes a joint optimization module, which uses the loss function and pose estimation to perform backpropagation of the detection loss box pose and outputs the detection results and 6D pose.

[0006] Furthermore, the input preprocessing module is specifically used for: Image enhancement unit: Adaptive median filtering is used to remove salt-and-pepper noise; homomorphic filtering is combined to eliminate uneven illumination; homomorphic filtering is processed by the image's illuminance-reflectance model; for the depth map, bilateral filtering is used for smoothing; and then triangular interpolation is used to fill in invalid pixels. Sensor calibration unit: Based on Zhang's calibration method, the camera intrinsic parameters are estimated. By capturing chessboard images from different angles, the intrinsic parameter matrix of the camera is calculated using the correspondence between the two-dimensional and three-dimensional coordinates of the chessboard corner points in the images. The camera-robotic arm extrinsic parameters are obtained through hand-eye calibration, which is used to determine the transformation relationship between the camera coordinate system and the robotic arm coordinate system.

[0007] Furthermore, the shared feature extraction module includes a multi-stage feature extraction unit and a feature rank enhancement unit; The multi-stage feature extraction unit includes: Shallow feature units: extract edge and texture features; Mid-layer feature units: Introducing bottleneck structures and dilated convolutions to capture medium-scale targets; Deep feature unit: It adopts a cross-stage local connection structure, embeds a frequency band modulation module, and outputs semantic feature size; The feature rank enhancement unit is used to apply a rank restoration operation after each layer of feature extraction, and to perform a linear transformation on the features through the channel interaction matrix.

[0008] Furthermore, the shared feature extraction module, based on the DINO-X architecture, constructs a multi-scale feature extraction backbone and embeds frequency dynamic convolution; The multi-stage feature extraction unit is used to perform feature extraction operations on the input image, specifically divided into three frequency bands: low, medium, and high. Each frequency band corresponds to an independent weight branch, expressed as follows: ,in Represents the frequency band mask. These represent Fourier transform and inverse Fourier transform, respectively. Through inverse Fourier transform, the weights of each frequency band are converted into spatial domain kernels, realizing decoupled modeling of low-frequency structure, mid-frequency texture, and high-frequency details in the image, thereby capturing image features more comprehensively and meticulously. The feature rank enhancement unit adopts a hierarchical feature pyramid structure, fusing features of different scales through upsampling and cross-layer concatenation operations; it also introduces a kernel space modulation mechanism, generating an element-wise modulation matrix through local and global channel branches. This enables spatial adaptive adjustment of characteristics across different frequency bands, expressed as follows: ,in It is a lightweight local feature extractor. For global context-aware modules, This is a global context-aware module.

[0009] Furthermore, the target detection module includes: The multi-scale prediction unit upsamples and concatenates deep features through the neck network to generate detection feature maps at multiple scales; each detection feature map applies an anchor box mechanism to predict the target's class probability and the detection box coordinates. The loss function unit constructs a loss function by reducing the weight of easily classified samples and increasing the weight of difficult-to-classify samples.

[0010] Furthermore, in the target detection module, FDConv is used in the Box Head of DINO-X, and the contribution of each frequency band feature is dynamically adjusted through frequency band modulation; for input features FBM first decomposes it into sub-features of different frequency bands. Then through spatial variation modulation matrix To achieve adaptive weighting of frequency band features, the specific formula is as follows: ,in, It is generated by a convolutional layer and a sigmoid function.

[0011] Furthermore, the 6D pose estimation module includes a neural implicit field pose estimation unit; The neural implicit field pose estimation unit includes: The geometric field unit takes 3D coordinates and detection features as input and predicts the SDF value through a 3-layer FDConv, using ReLU as the activation function. The appearance field unit takes the input point coordinates, normal vector, and detection features as inputs and predicts RGB values ​​through two fully connected layers, using the Sigmoid activation function. The pose optimization unit generates multiple candidate initial pose assumptions through uniform sampling. After feature matching and filtering, the top few high-confidence assumptions are retained. The L-BFGS algorithm is used for iterative optimization, and the rotation matrix and translation vector are updated in each iteration.

[0012] Furthermore, in the 6D pose estimation module, the input to the geometric field unit is the point cloud coordinates (x, y, z) and the 2D features output by the detection branch. Multi-band geometric features are extracted using FDConv. The input to the appearance field unit is the RGB pixel value and the normal vector n. Illumination-invariant appearance features are generated by FDConv. The two are then fused and input into the neural implicit field decoder to predict the signed distance field (SDF) and RGB values, expressed as: ,in, For network parameters, [] indicates feature concatenation.

[0013] Furthermore, the training process of the neural implicit field pose estimation unit includes: Synthetic data generation steps: Using the target category label c output by the detection branch, a detailed semantic description is generated through a language model. Then, the diffusion model is driven to generate diverse target appearance models; combined with the physics engine to render synthetic RGBD images containing different lighting conditions and poses, a training dataset is constructed. During rendering, the region of interest is clipped based on the bounding box coordinates (x_1, y_1, x_2, y_2), and its expression is: ,in, For the target network model, These are attitude parameters. These are the lighting parameters; Attitude hypothesis generation and optimization steps: Based on the center of the detection frame Generate an initial state hypothesis and attitude hypothesis set with scale s. , where the rotation matrix The translation vector is generated by uniformly sampling the SO(3) space. The target size is calculated based on prior knowledge; then the view is synthesized through neural implicit field rendering. , with input image Calculate feature matching degree High-confidence hypotheses were selected; an iterative gradient descent optimization method was adopted, with SDF reprojection error and attitude smoothness constraints as the objective function, the expression of which is: ,in For the set of points on the target surface, is the regularization coefficient, and I is the identity matrix.

[0014] Furthermore, the joint optimization module is divided into: Shared Feature Backbone Unit: Used to share parameters between the detection and pose estimation branches, and outputs feature maps for the detection branch and pose estimation branch, respectively; Multi-task loss function unit, the loss function is ,in, To detect the loss, the classification Focal Loss expression is as follows: The regression expression for DIoULoss is: , L2 loss for pose estimation, For characteristic rank regularization term, For network parameters, These are the weighting coefficients.

[0015] Secondly, this application provides a sensing and positioning method for a live-line working robot in a power distribution network, based on the aforementioned sensing and positioning system for the live-line working robot in a power distribution network, comprising: Input an RGB-D image and preprocess it. Features containing multi-scale information are extracted from preprocessed RGB-D images; and the features are enhanced through feature tensor decomposition and attention mechanisms. The enhanced features are processed at multiple scales to output the target category and 2D detection box; Pose estimation is performed on the target category and 2D bounding box based on the neural implicit field to generate an initial pose hypothesis. Then, the neural implicit field is used for rendering and optimization to obtain the 6D pose.

[0016] It also includes backpropagation of the detection loss box pose by combining the loss function and pose estimation, and outputting the detection result and 6D pose.

[0017] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the perception and positioning method of the live-line working robot in the power distribution network.

[0018] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the perception and positioning method of the live-line working robot in the power distribution network.

[0019] Fifthly, this application provides a computer program product, the computer program product including computer instructions, the computer instructions instructing a computer to execute the perception and positioning method of the live-line working robot in power distribution network.

[0020] The beneficial effects of this application are as follows: This application presents a perception and positioning system for a live-line power distribution robot that integrates frequency-domain dynamic convolution and neural implicit fields. By leveraging dynamic convolution technology, it enhances the diversity and richness of features, thereby improving the detection accuracy for small and low-texture targets, reducing false positives and false negatives, and making the detection results more accurate and reliable. The system deeply integrates neural implicit fields with multimodal features, strengthening geometric constraints and improving illumination invariance, achieving accurate 6D pose regression of targets, maintaining high pose estimation accuracy even under complex lighting and occlusion conditions. A carefully designed shared feature backbone and joint optimization mechanism reduces unnecessary computational redundancy, improves computational efficiency, promotes information flow and collaboration between detection and pose estimation tasks, and enables efficient parallel processing of multiple tasks, meeting real-time requirements. Through frequency-domain feature decomposition and hierarchical comparative learning, the system enhances the model's adaptability to complex environments (such as various lighting conditions and different background scenes) and pose changes, ensuring stable and good performance in different practical application scenarios. Attached Figure Description

[0021] Figure 1 A schematic diagram of a sensing and positioning system for a live-line working robot in a power distribution network is provided in this application. Figure 2 A schematic diagram of an electronic device provided in this application. Detailed Implementation

[0022] The technical solution of this application will now be clearly and completely described with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0023] This application belongs to the interdisciplinary field of computer vision, robotics, and artificial intelligence, specifically involving target detection, 6D pose estimation, and multi-task collaborative perception technology for live-line working robots in power distribution networks. Addressing challenges such as strong electromagnetic interference, high equipment texture similarity, and difficulty in small target detection in live-line working scenarios, this invention proposes an end-to-end unified framework integrating frequency domain dynamic convolution (FDConv), Neural Implicit Field (NIF), and DINO-X. Through frequency domain feature enhancement, cross-modal geometric constraints, and multi-task joint optimization, it resolves the contradiction between accuracy and real-time performance in robot perception under complex power environments. This framework is suitable for tasks such as intelligent operation and maintenance, precise equipment grasping, and dynamic scene reconstruction for live-line working robots in power distribution networks, exhibiting strong robustness and industrial application potential.

[0024] This application constructs a multi-task collaborative network, consisting of a shared feature extraction module, a multimodal dynamic convolution module, a neural implicit field pose estimation unit, and a joint optimization module. The overall framework structure is as follows: Figure 1 As shown, deep collaboration between detection and pose estimation tasks is achieved through cross-module frequency domain feature interaction and geometric constraint transmission mechanism, thereby improving overall performance.

[0025] Example 1 like Figure 1 As shown, this embodiment constructs an integrated perception and positioning network architecture for live-line working robots in power distribution networks, which mainly includes the following core modules: 1. Input preprocessing module preprocesses RGB-D images to adapt to complex working environments (such as electromagnetic interference and backlighting in live power distribution scenarios); the input preprocessing process includes the following steps: (1) Image Enhancement. For RGB images, adaptive median filtering (window size set to 5×5) is first used to remove salt-and-pepper noise. This method can adaptively adjust the filter window size according to the distribution of noise points, thus removing noise more effectively. Then, homomorphic filtering (cutoff frequency set to 0.3) is used to eliminate uneven illumination. Homomorphic filtering processes the image's illuminance-reflectance model, which can enhance image contrast while compressing the image's dynamic range. For depth maps, bilateral filtering ( Bilateral filtering performs smoothing processing, preserving image edge information while smoothing the image, avoiding edge blurring problems that may occur with traditional filtering methods. Then, triangular interpolation is used to fill in invalid pixels, ensuring the integrity and accuracy of the depth map. (2) Sensor calibration. Based on Zhang's calibration method, the camera intrinsic parameters are estimated by taking pictures of the chessboard at different angles and calculating the camera's intrinsic parameter matrix using the correspondence between the two-dimensional and three-dimensional coordinates of the chessboard corner points in the images. At the same time, the camera-robotic arm extrinsic parameters are obtained through hand-eye calibration. Hand-eye calibration is used to determine the transformation relationship between the camera coordinate system and the robotic arm coordinate system, which is crucial for subsequent target localization and robotic arm operation.

[0026] 2. A shared feature extraction module, based on a lightweight convolutional architecture, extracts general features containing multi-scale information; The shared feature extraction module, based on the DINO-X architecture, constructs a multi-scale feature extraction backbone and embeds frequency dynamic convolution (FDConv) to enhance feature representation capabilities. This module consists of two parts: (1) Frequency-aware feature extraction. The input image first enters a multi-layer FDConv layer (kernel sizes of 3×3 and 5×5) for feature extraction. The unique feature of FDConv is that it divides the convolution kernel parameters in the frequency domain through Fourier Disjoint Weight (FDW), specifically into three frequency bands: low, medium, and high (the frequency band division threshold is set to...). Each frequency band corresponds to an independent weighted branch. Its mathematical expression is: ,in Represents the frequency band mask. These represent the Fourier transform and the inverse Fourier transform, respectively. By using the inverse Fourier transform (iDFT), the weights of each frequency band are converted into spatial domain kernels. This enables decoupling modeling of low-frequency structures (such as the general outline and overall shape of objects), mid-frequency textures (such as the texture details and material features of object surfaces), and high-frequency details (such as the sharpness of edges and minor noise) in an image, thereby capturing image features more comprehensively and meticulously. (2) Multi-scale feature fusion. A hierarchical feature pyramid structure similar to DINO-X is adopted, and features at different scales are fused through adaptive upsampling and cross-layer concatenation operations. In the feature fusion layer, a kernel spatial modulation (KSM) mechanism is introduced. Specifically, an element-wise modulation matrix is ​​generated through local channel branches (implemented using 1D convolution to capture local feature information) and global channel branches (implemented through fully connected layers to perceive global context information). This enables spatial adaptive adjustment of characteristics across different frequency bands, expressed as follows: ,in It is a lightweight local feature extractor. For global context-aware modules, This is a global context-aware module.

[0027] Multi-stage feature extraction specifically includes: Shallow Feature Extraction (Layers 1-4): Edge and texture features are extracted using a 3×3 FDConv processor, with an output size of 128×128×256. This stage focuses on more basic feature information in the image, such as the edge contours of objects and preliminary texture details, providing a foundation for subsequent feature extraction. Mid-level features (layers 5-8): Introducing a bottleneck structure and dilated convolutions (inflation rate of 2) to capture medium-scale targets (such as common mechanical parts). The bottleneck structure reduces computation by decreasing the number of channels while maintaining feature expressiveness; dilated convolutions expand the receptive field without increasing computation, thus better capturing the overall features of the target. Output size is 64×64×512; Deep features (layers 9-12): Employ a Cross-Stage Local Connectivity (CSP) structure with an embedded frequency band modulation module, outputting semantic features of size 32×32×1024. The CSP structure effectively reduces computation and enhances gradient propagation by dividing the feature map into two parts: one part undergoes regular convolution operations, and the other part is directly connected to subsequent layers. The frequency band modulation module further adjusts and fuses features from different frequency bands, improving the expressive power of the features. Among them, the Rank Enhancement Attention Module (RALA) strengthens the feature representation of key target structures (such as insulator skirts and conductor textures) through feature tensor decomposition and attention mechanisms. Feature rank enhancement applies a rank recovery operation after each FDConv layer, and performs a linear transformation on the features through the channel interaction matrix. Through this operation, the full rank rate of the feature matrix is ​​increased from 88% to 96%, which is close to the level of Softmax attention, thereby enhancing the stability of the features.

[0028] 3. Target detection module, outputting target category and 2D detection box; 4. 6D pose estimation module: based on Neural Implicit Field (NIF) to achieve regression from 2D features to 3D pose; specifically including: (1) Multi-scale prediction. Deep features are upsampled (by a factor of 2) and concatenated across layers using a neck network to generate detection feature maps at three scales (32×32, 16×16, and 8×8, respectively). Each feature map is then used with an anchor box mechanism (anchor box ratio set to [1:1, 1:2, 2:1]) to predict the target's class probability and the detection box coordinates. This multi-scale prediction method can adapt to targets of different sizes, improving detection accuracy and recall. (2) Loss function. By reducing the weight of easily classified samples and increasing the weight of difficult-to-classify samples, the problem of imbalance between positive and negative samples in object detection is effectively solved.

[0029] The multimodal dynamic convolution module introduces the FDConv frequency domain modulation mechanism into both the detection and pose estimation branches to enhance the diversity of feature representation and geometric perception capabilities. This module consists of two parts: (1) Detection Branch Enhancement. In the Box Head of DINO-X, traditional convolution is replaced with FDConv, and the contribution of each frequency band feature is dynamically adjusted through Frequency Band Modulation (FBM). For input features... FBM first decomposes it into sub-features of different frequency bands. Then through spatial variation modulation matrix To achieve adaptive weighting of frequency band features, the specific formula is as follows: ,in, Generated by a convolutional layer and a sigmoid function, its function is to emphasize the feature differences between the target boundary (high-frequency information, corresponding to the detailed features of the object's edge) and the main body region (low-frequency information, reflecting the main shape and structure of the object), thereby highlighting the key features of the target and improving the accuracy of detection. (2) Enhanced Pose Estimation Branch. FDConv is embedded in the geometric and appearance networks of the neural implicit field to enhance the frequency domain modeling of the 3D geometric structure. The input to the geometric network is the point cloud coordinates (x, y, z) and the 2D features output by the detection branch. Multi-band geometric features are extracted using FDConv. The appearance network takes RGB pixel values ​​and normal vector n as input and generates illumination-invariant appearance features using FDConv. The two are then fused and input into the neural implicit field decoder to predict the signed distance field (SDF) and RGB values, expressed as follows: ,in, Here, [] represents network parameters, and [] denotes feature concatenation. Thus, by introducing FDConv into the geometric and appearance networks, we can better capture the geometric structure and appearance features of objects, providing richer and more accurate information for pose estimation.

[0030] The 6D pose estimation module mainly includes: (1) Neural implicit field modeling.

[0031] (1) Geometric field. Input 3D coordinates and detection features, predict SDF values ​​through 3 layers of FDConv (kernel size 3×3), and use ReLU as the activation function. The ReLU function can effectively alleviate the gradient vanishing problem, speed up the training of the network, and at the same time preserve important feature information in the image; (2) Appearance field. The input point coordinates, normal vector, and detection features are used to predict RGB values ​​through two fully connected layers, and the activation function is Sigmoid. The Sigmoid function maps the output value to between 0 and 1, which is suitable for predicting probability values ​​or representing a certain degree of features. In the appearance field, it can better represent the color information of the pixel. (3) Pose optimization. The initial pose hypothesis is generated by uniform sampling into 500 candidates. After feature matching and filtering, the top 10 high-confidence hypotheses are retained.

[0032] The L-BFGS algorithm is used for iterative optimization. Each iteration updates the rotation matrix (represented by axis-angle) and translation vector. The maximum number of iterations is set to 50, and the convergence threshold is 0.01 mm. The L-BFGS algorithm is a highly efficient optimization algorithm that can find a better solution in a fewer number of iterations, thus quickly and accurately optimizing the target's attitude estimation.

[0033] Specifically, the neural implicit field pose estimation unit, based on an improved FoundationPose framework, guides the training and inference process of the neural implicit field by combining detection results, achieving accurate mapping from 2D features to 3D pose. This module consists of two parts: (1) Synthetic data generation. Using the target category label c output by the detection branch, a detailed semantic description is generated through a language model (LLM). Then, a diffusion model (such as Stable Diffusion) is driven to generate diverse target appearance models. Simultaneously, a physics engine (such as PyBullet) is used to render synthetic RGBD images containing different lighting conditions (such as backlighting, sidelighting, and direct strong light) and poses to construct a training dataset. During rendering, the region of interest (ROI) is clipped based on the bounding box coordinates (x_1, y_1, x_2, y_2) to reduce unnecessary background calculation overhead. The expression for this is: ,in, For the target network model, These are attitude parameters. These are the lighting parameters. By generating a large amount of diverse synthetic data, the training samples can be enriched, and the generalization ability of the model can be improved. (2) Attitude hypothesis generation and optimization. Based on the center of the detection box. Generate an initial state hypothesis and attitude hypothesis set with scale s. , where the rotation matrix The translation vector is generated by uniformly sampling the SO(3) space. The target size is calculated based on prior knowledge. Then, a composite view is rendered using neural implicit field rendering. , with input image Calculate feature matching degree Filter out high-confidence hypotheses (set a threshold) Finally, an iterative gradient descent optimization method is employed, with the SDF reprojection error and attitude smoothing constraint as the objective function, expressed as follows: ,in For the set of points on the target surface, Here, I is the regularization coefficient, and I is the identity matrix. By continuously optimizing the pose assumptions, the accuracy of pose estimation is gradually improved.

[0034] 5. Joint Optimization Module: Collaboratively optimizes the detection and pose estimation tasks through multi-task loss functions.

[0035] The joint optimization module designs a multi-task loss function to collaboratively optimize detection, pose estimation, and feature rank regularization, achieving cross-task information complementarity. This module consists of two parts: (1) Shared feature backbone. The detection and pose estimation branches share the first 12 layers of FDConv parameters, and the output feature map sizes are 16×16×1024 (detection branch) and 32×32×512 (pose branch), respectively. By sharing weights across tasks, the amount of repeated computation is effectively reduced by 40%, and FLOPs are reduced from 2100G to 1260G, greatly improving computational efficiency without sacrificing too much performance; (2) Multi-task loss function. ,in, To detect the loss (consisting of classification Focal loss + regression DIoULoss), the expression for classification Focal Loss is: The regression DIoU Loss expression is , L2 loss for pose estimation, This is the eigenrank regularization term (forcing the full rank rate of the eigenma matrix to be ≥95%). For network parameters, where represents the weighting coefficients. Through joint optimization, the detection mAP was improved by 1.2%, and the pose ADD-S error was reduced by 5.6%, effectively improving the overall performance of the model.

[0036] The joint inference process is as follows: input RGB-D image → preprocessing (including image enhancement and sensor calibration) → extract features through shared feature backbone → output 2D bounding box and category in detection branch → generate initial pose hypothesis in pose branch → render and optimize using neural implicit field to obtain 6D pose → backpropagation through joint loss → finally output detection result and 6D pose.

[0037] Based on the above scheme, and through verification, it can be concluded that this application has the following advantages: 1. Improved detection performance. Tested on the COCO dataset, the improved FDConv-DINO-X model shows an increase in mAP@0.5:0.95 from 56.0% to 58.2% and mAP@0.5 from 59.8% to 62.1% compared to the original DINO-X, especially for small targets (area < 32). 2 The pixel detection accuracy was improved by 9.3%. At the same time, the model maintained a frame rate of 25 FPS, which met the real-time requirements. It can process image data quickly while ensuring detection accuracy, making it suitable for real-time target detection scenarios.

[0038] 2. Pose estimation accuracy. Evaluation on the YCB-Video dataset shows that, compared to FoundationPose, the proposed method reduces the ADD-S error from 85.3 mm to 62.1 mm, shortens rendering time from 1240 ms to 480 ms, and increases the success rate (error <100 mm) from 78% to 91%. In backlit scenes, the pose estimation error increases by only 12%, significantly outperforming traditional methods (error increases by 45%). This demonstrates that the proposed method can more accurately estimate the 6D pose of the target under different lighting conditions and scenes, exhibiting high robustness and stability.

[0039] 3. Multi-task collaborative gains. By sharing the feature backbone, the computational cost was effectively reduced by 40%. After joint training, the detection mAP was further improved by 0.8%, and the pose error was reduced by another 3.1%. FLOPs decreased from 1820G to 1150G, and the number of model parameters was reduced by 28%, achieving a balance between accuracy and efficiency. This demonstrates that the joint optimization mechanism in this application can fully utilize the complementary information between the detection and pose estimation tasks, improving model performance while reducing computational resource consumption, thus possessing high practicality and application value.

[0040] Example 2 A sensing and localization method for a live-line working robot in a power distribution network includes the following steps: S1, Input an RGB-D image, and preprocess the RGB-D image; specifically including: Adaptive median filtering is used to remove salt-and-pepper noise; homomorphic filtering is combined to eliminate uneven illumination, which is achieved by processing the image's illuminance-reflectance model; for the depth map, bilateral filtering is used for smoothing; and then triangular interpolation is used to fill in invalid pixels. The camera intrinsic parameters are estimated based on Zhang's calibration method. By capturing chessboard images from different angles, the intrinsic parameter matrix of the camera is calculated using the correspondence between the two-dimensional and three-dimensional coordinates of the chessboard corner points in the images. The camera-robotic arm extrinsic parameters are obtained through hand-eye calibration, which is used to determine the transformation relationship between the camera coordinate system and the robotic arm coordinate system.

[0041] S2 extracts features containing multi-scale information from the preprocessed RGB-D image; and enhances the feature representation through feature tensor decomposition and attention mechanisms; specifically including: Extract edge and texture features; By introducing bottleneck structures and dilated convolutions, we can capture medium-scale targets. A cross-stage local connectivity structure is adopted, and a frequency band modulation module is embedded to output semantic feature size; After feature extraction at each layer, a rank restoration operation is applied to perform a linear transformation on the features using the channel interaction matrix.

[0042] Based on the DINO-X architecture, a multi-scale feature extraction backbone is constructed, and frequency dynamic convolution is embedded. The multi-stage feature extraction unit is used to perform feature extraction operations on the input image, specifically divided into three frequency bands: low, medium, and high. Each frequency band corresponds to an independent weight branch, expressed as follows: ,in Represents the frequency band mask. These represent Fourier transform and inverse Fourier transform, respectively. Through inverse Fourier transform, the weights of each frequency band are converted into spatial domain kernels, realizing decoupled modeling of low-frequency structure, mid-frequency texture, and high-frequency details in the image, thereby capturing image features more comprehensively and meticulously. The feature rank enhancement unit adopts a hierarchical feature pyramid structure, fusing features of different scales through upsampling and cross-layer concatenation operations; it also introduces a kernel space modulation mechanism, generating an element-wise modulation matrix through local and global channel branches. This enables spatial adaptive adjustment of characteristics across different frequency bands, expressed as follows: ,in It is a lightweight local feature extractor. For global context-aware modules, This is a global context-aware module.

[0043] S3 performs multi-scale processing on the enhanced features and outputs the target category and 2D detection box; specifically including: The deep features are upsampled and concatenated across layers through the neck network to generate detection feature maps at multiple scales; each detection feature map is used to predict the class probability of the target and the coordinates of the detection box. A loss function is constructed by reducing the weight of easily classified samples and increasing the weight of difficult-to-classify samples.

[0044] In the DINO-X Box Head, FDConv is used, and the contribution of each frequency band feature is dynamically adjusted through frequency band modulation; for input features FBM first decomposes it into sub-features of different frequency bands. Then through spatial variation modulation matrix To achieve adaptive weighting of frequency band features, the specific formula is as follows: ,in, It is generated by a convolutional layer and a sigmoid function.

[0045] S4. Based on the neural implicit field, the pose of the target category and the 2D detection box is estimated to generate an initial pose hypothesis. Then, the neural implicit field is used for rendering and optimization to obtain the 6D pose. Input 3D coordinates and detection features, predict SDF values ​​through 3-layer FDConv, and select ReLU as the activation function; The input point coordinates, normal vector, and detection features are used to predict RGB values ​​through two fully connected layers, with the Sigmoid activation function employed. The initial pose hypothesis is generated by uniform sampling to generate multiple candidates. After feature matching and filtering, the top few high-confidence hypotheses are retained. The L-BFGS algorithm is used for iterative optimization, and the rotation matrix and translation vector are updated in each iteration.

[0046] The training process of the neural implicit field pose estimation unit includes: Synthetic data generation steps: Using the target category label c output by the detection branch, a detailed semantic description is generated through a language model. Then, the diffusion model is driven to generate diverse target appearance models; combined with the physics engine to render synthetic RGBD images containing different lighting conditions and poses, a training dataset is constructed. During rendering, the region of interest is clipped based on the bounding box coordinates (x_1, y_1, x_2, y_2), and its expression is: ,in, For the target network model, These are attitude parameters. These are the lighting parameters; Attitude hypothesis generation and optimization steps: Based on the center of the detection frame Generate an initial state hypothesis and attitude hypothesis set with scale s. , where the rotation matrix The translation vector is generated by uniformly sampling the SO(3) space. The target size is calculated based on prior knowledge. Then, a composite view is rendered using neural implicit field rendering. , with input image Calculate feature matching degree High-confidence hypotheses were selected; an iterative gradient descent optimization method was adopted, with SDF reprojection error and attitude smoothness constraints as the objective function, the expression of which is: ,in For the set of points on the target surface, is the regularization coefficient, and I is the identity matrix.

[0047] S5, through the loss function and pose estimation, performs backpropagation on the detection loss box pose, outputting the detection result and 6D pose. The loss function is: ,in, To detect the loss, the classification Focal Loss expression is as follows: The regression DIoU Loss expression is , L2 loss for pose estimation, For characteristic rank regularization term, For network parameters, These are the weighting coefficients.

[0048] Based on the above solution, this embodiment has the following advantages: 1. Multimodal Dynamic Convolution Fusion: Addressing the complex textures and geometric characteristics of power equipment (such as insulators and conductors) in live-line working scenarios of distribution networks, this innovative approach introduces frequency dynamic convolution (FDConv) into the robot's visual perception system. In the target detection and pose estimation branches, frequency domain feature decomposition (separating low-frequency electromagnetic interference noise from high-frequency equipment edge details) and spatial modulation (dynamically adjusting the convolution kernel response in conjunction with equipment semantic labels) enhance the diversity of feature representation and geometric perception capabilities. This significantly improves target detection accuracy (e.g., a 28% increase in the detection rate of small target bolts) and pose estimation accuracy (reducing rotation error to 6.3°) under harsh environments such as strong electromagnetic interference and backlighting.

[0049] 2. Improvement of Neural Implicit Field in Power Distribution Scenarios: Based on the FoundationPose framework, to address the challenges of symmetry (such as spherical terminals) and low-texture surfaces (such as metal clamps) in power distribution equipment, the training and inference of the Neural Implicit Field (NIF) are guided by real-time target detection results (such as bounding boxes and category labels) from the live-line working robot in the power distribution network.

[0050] 3. Multi-task collaborative optimization for power distribution network operation: A lightweight shared feature backbone (reducing the number of parameters by 42%) is designed for target detection and pose estimation of live-line working robots in power distribution networks. Frequency domain convolution modules are embedded between Transformer layers to achieve cross-task feature reuse for detection (identifying equipment categories) and pose estimation (locating grasping points). By balancing the weights of detection confidence and pose accuracy through a joint loss function, the complementarity of visual semantics and geometric information in electromagnetic interference scenarios is promoted. The overall inference latency is reduced to 45ms, meeting the real-time obstacle avoidance and path planning requirements of live-line working robots in power distribution networks.

[0051] Example 3 like Figure 2 As shown, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the sensing and positioning method of the live-line working robot in Embodiment 2.

[0052] The electronic device is equipped with a high-performance CPU or GPU with a clock speed of at least 2.5GHz, at least 8 cores, and supports CUDA acceleration. Memory includes RAM and storage space, with at least 16GB of RAM and at least 256GB of storage space. The device also features a dedicated AI acceleration chip to accelerate the inference process of deep learning models. It connects to the RGB-D camera of the power distribution network live-line working robot via a high-speed interface to receive image data in real time. The device runs on Linux or Windows operating systems and supports deep learning frameworks such as PyTorch or TensorFlow.

[0053] The computer program of the electronic device implements all the functions of the perception and localization method for live-line working robots in power distribution networks described in Example 2. The program is modularly designed, including an image preprocessing module, a feature extraction module, a target detection module, a pose estimation module, and a joint optimization module. Data exchange between modules is achieved through standardized interfaces, ensuring the system's scalability and maintainability. The program supports multi-threaded parallel processing, fully utilizing the computing power of multi-core processors to improve the system's real-time performance. The program also implements a data caching mechanism to cache frequently used features and model parameters, reducing redundant calculations and improving processing efficiency.

[0054] Example 4 A computer-readable storage medium storing a computer program that, when executed by a processor, implements the sensing and positioning method for a live-line working robot in a power distribution network as described in Embodiment 2.

[0055] The computer-readable storage medium can be a solid-state drive, a hard disk drive, an optical disc, a USB flash drive, or other non-volatile storage devices. The computer program stored in the storage medium contains the complete implementation code of the perception and localization method for live-line working robots in power distribution networks described in Embodiment 2. The program adopts a modular design with clear interfaces between functional modules, facilitating maintenance and upgrades. The program also includes necessary configuration files and pre-trained model files to ensure rapid system deployment and use.

[0056] The computer program is optimized for different hardware platforms, supporting both CPU and GPU operating modes. In GPU mode, the program fully utilizes the CUDA acceleration library to improve the inference speed of deep learning models. The program also implements an automatic adaptation mechanism that automatically adjusts parameters based on the hardware configuration of the operating environment, ensuring optimal performance on devices with varying configurations.

[0057] Example 5 A computer program product, the computer program product including computer instructions, the computer instructions instructing a computer to execute the sensing and positioning method of the live-line working robot in embodiment two.

[0058] The computer program product is provided as a software package, containing the complete implementation code and necessary dependency libraries for the perception and localization method of the live-line working robot in the power distribution network described in Embodiment 2. The software package provides a graphical user interface for user operation and monitoring of the system's operating status. The user interface consists of three parts: an image display area, a parameter configuration area, and a result display area. The image display area displays the original RGB-D image, the preprocessed image, and visualizations of the detection and pose estimation results in real time. The parameter configuration area allows users to adjust various system parameters, such as preprocessing parameters, detection thresholds, and pose estimation accuracy requirements. The result display area displays the detected target category, location, and 6D pose information, as well as the system's operating status and performance indicators.

[0059] The software package also provides an API interface, allowing other systems to call the system's functions through standardized interfaces. The API interface supports both RESTful style and WebSocket calling methods, facilitating integration with different types of systems. The package also includes detailed documentation and sample code to help developers quickly understand and use the system's functions.

[0060] It should be noted that Embodiment 1, Embodiment 2, Embodiment 3, Embodiment 4, Embodiment 5, and Embodiment 6 are all types of sensing and positioning systems for live-line working robots in power distribution networks.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] This application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, readable storage media, optical storage, etc.) containing computer-usable program code.

[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] Obviously, the described embodiments are only some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort should fall within the scope of protection of this application.

[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and not to limit them. Although this application has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of this application. Any modifications or equivalent substitutions that do not depart from the spirit and scope of this application should be covered within the protection scope of the claims of this application.

Claims

1. A sensing and positioning system for a live-line working robot in a power distribution network, characterized in that, include: The input preprocessing module is used to input RGB-D images and preprocess them. A shared feature extraction module is used to extract features containing multi-scale information from preprocessed RGB-D images based on a lightweight convolutional architecture; and to enhance the representation of features through feature tensor decomposition and attention mechanisms. The object detection module is used to perform multi-scale processing on the enhanced features and output the object category and 2D detection box. The 6D pose estimation module estimates the pose of the target category and 2D bounding box based on the neural implicit field, generates an initial pose hypothesis, and uses the neural implicit field for rendering and optimization to obtain the 6D pose.

2. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 1, characterized in that, The input preprocessing module is specifically used for: Image enhancement unit: Adaptive median filtering is used to remove salt-and-pepper noise; homomorphic filtering is combined to eliminate uneven illumination; homomorphic filtering is processed by the image's illuminance-reflectance model; for the depth map, bilateral filtering is used for smoothing; and then triangular interpolation is used to fill in invalid pixels. Sensor calibration unit: Based on Zhang's calibration method, the camera intrinsic parameters are estimated. By capturing chessboard images from different angles, the intrinsic parameter matrix of the camera is calculated using the correspondence between the two-dimensional and three-dimensional coordinates of the chessboard corner points in the images. The camera-robotic arm extrinsic parameters are obtained through hand-eye calibration, which is used to determine the transformation relationship between the camera coordinate system and the robotic arm coordinate system.

3. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 1, characterized in that, The shared feature extraction module includes a multi-stage feature extraction unit and a feature rank enhancement unit; The multi-stage feature extraction unit includes: Shallow feature units: extract edge and texture features; Mid-layer feature units: Introducing bottleneck structures and dilated convolutions to capture medium-scale targets; Deep feature unit: It adopts a cross-stage local connection structure, embeds a frequency band modulation module, and outputs semantic feature size; The feature rank enhancement unit is used to apply a rank restoration operation after each layer of feature extraction, and to perform a linear transformation on the features through the channel interaction matrix.

4. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 3, characterized in that, The shared feature extraction module is based on the DINO-X architecture, constructs a multi-scale feature extraction backbone, and embeds frequency dynamic convolution. The multi-stage feature extraction unit is used to perform feature extraction operations on the input image, specifically divided into three frequency bands: low, medium, and high. Each frequency band corresponds to an independent weight branch, expressed as follows: ,in Represents the frequency band mask. These represent Fourier transform and inverse Fourier transform, respectively. Through inverse Fourier transform, the weights of each frequency band are converted into spatial domain kernels, realizing decoupled modeling of low-frequency structure, mid-frequency texture, and high-frequency details in the image, thereby capturing image features more comprehensively and meticulously. The feature rank enhancement unit adopts a hierarchical feature pyramid structure, fusing features of different scales through upsampling and cross-layer concatenation operations; it also introduces a kernel space modulation mechanism, generating an element-wise modulation matrix through local and global channel branches. This enables spatial adaptive adjustment of characteristics across different frequency bands, expressed as follows: ,in It is a lightweight local feature extractor. For global context-aware modules, This is a global context-aware module.

5. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 1, characterized in that, The target detection module includes: The multi-scale prediction unit upsamples and concatenates deep features through the neck network to generate detection feature maps at multiple scales; each detection feature map applies an anchor box mechanism to predict the target's class probability and the detection box coordinates. The loss function unit constructs a loss function by reducing the weight of easily classified samples and increasing the weight of difficult-to-classify samples.

6. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 5, characterized in that, In the target detection module, FDConv is used in the Box Head of DINO-X, and the contribution of each frequency band feature is dynamically adjusted through frequency band modulation; for input features FBM first decomposes it into sub-features of different frequency bands. Then through spatial variation modulation matrix To achieve adaptive weighting of frequency band features, the specific formula is as follows: ,in, It is generated by a convolutional layer and a sigmoid function.

7. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 1, characterized in that, The 6D pose estimation module includes a neural implicit field pose estimation unit. The neural implicit field pose estimation unit includes: The geometric field unit takes 3D coordinates and detection features as input and predicts the SDF value through a 3-layer FDConv, using ReLU as the activation function. The appearance field unit takes the input point coordinates, normal vector, and detection features as inputs and predicts RGB values ​​through two fully connected layers, using the Sigmoid activation function. The pose optimization unit generates multiple candidate initial pose assumptions through uniform sampling. After feature matching and filtering, the top few high-confidence assumptions are retained. The L-BFGS algorithm is used for iterative optimization, and the rotation matrix and translation vector are updated in each iteration.

8. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 7, characterized in that, In the 6D pose estimation module, the input to the geometric field unit is the point cloud coordinates (x, y, z) and the 2D features output by the detection branch. Multi-band geometric features are extracted using FDConv. ; The input to the appearance field unit is RGB pixel values ​​and normal vector n, and illumination-invariant appearance features are generated by FDConv. The two are then fused and input into the neural implicit field decoder to predict the signed distance field (SDF) and RGB values, expressed as: ,in, For network parameters, [] indicates feature concatenation.

9. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 7, characterized in that, The training process of the neural implicit field pose estimation unit includes: Synthetic data generation steps: Using the target category label c output by the detection branch, a detailed semantic description is generated through a language model. Then, the diffusion model is driven to generate diverse target appearance models; combined with the physics engine to render synthetic RGBD images containing different lighting conditions and poses, a training dataset is constructed. During rendering, the region of interest is clipped based on the bounding box coordinates (x_1, y_1, x_2, y_2), and its expression is: ,in, For the target network model, These are attitude parameters. These are the lighting parameters; Attitude hypothesis generation and optimization steps: Based on the center of the detection frame Generate an initial state hypothesis and attitude hypothesis set with scale s. , where the rotation matrix The translation vector is generated by uniformly sampling the SO(3) space. The target size is calculated based on prior knowledge; then the view is synthesized through neural implicit field rendering. , with input image Calculate feature matching degree High-confidence hypotheses were selected; an iterative gradient descent optimization method was adopted, with SDF reprojection error and attitude smoothness constraints as the objective function, the expression of which is: ,in For the set of points on the target surface, is the regularization coefficient, and I is the identity matrix.

10. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 1, characterized in that, The joint optimization module is divided into: Shared Feature Backbone Unit: Used to share parameters between the detection and pose estimation branches, and outputs feature maps for the detection branch and pose estimation branch, respectively; Multi-task loss function unit, the loss function is ,in, To detect the loss, the classification Focal Loss expression is as follows: The regression expression for DIoULoss is: , L2 loss is used to estimate pose. For characteristic rank regularization term, For network parameters, These are the weighting coefficients.

11. The sensing and positioning system for a live-line working robot in a power distribution network according to claim 1, characterized in that, Also includes: The joint optimization module is used to perform backpropagation and optimization of the detection loss box pose by combining the loss function and pose estimation, and output the optimized 6D pose.

12. A sensing and positioning method for a live-line working robot in a power distribution network, based on the sensing and positioning system for a live-line working robot in a power distribution network as described in any one of claims 1 to 11, characterized in that, include: Input an RGB-D image and preprocess it. Features containing multi-scale information are extracted from preprocessed RGB-D images; and the features are enhanced through feature tensor decomposition and attention mechanisms. The enhanced features are processed at multiple scales to output the target category and 2D detection box. Pose estimation is performed on the target category and 2D bounding box based on the neural implicit field to generate an initial pose hypothesis. Then, the neural implicit field is used for rendering and optimization to obtain the 6D pose.

13. The sensing and positioning method for a live-line working robot in a power distribution network according to claim 12, characterized in that, Also includes: By combining the loss function and pose estimation, the pose of the jointly detected loss box is backpropagated and optimized, and the optimized 6D pose is output.

14. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the sensing and positioning method of the live-line working robot of claim 12 or 13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the sensing and positioning method for a live-line working robot in a power distribution network as described in claim 12 or 13.

16. A computer program product, the computer program product comprising computer instructions, characterized in that, The computer instructions instruct the computer to execute the sensing and positioning method for the live-line working robot of the power distribution network as described in claim 12 or 13.

Images