Diffusion strategy-based power aerial work manipulator control method and related equipment

Abstract

This invention relates to the fields of intelligent robot control and automation of power operations, specifically to a control method and related equipment for a robotic arm used in high-altitude power operations based on a diffusion strategy. This method involves real-time acquisition of multimodal perception data, followed by single-modal feature extraction and fusion using a Transformer module based on cross-attention to generate a conditional vector. This conditional vector is then input into a pre-constructed conditional diffusion model, generating a robotic arm joint control sequence under preset operational safety and feasibility constraints. After execution by a servo driver, the sequence is fed back to a feature fusion module by collecting the robot's state and environmental data, forming a closed-loop control. This method achieves precise and safe control of the robotic arm in high-altitude power operations, improving both operational efficiency and safety.

Description

Technical Field

[0001] This invention relates to the fields of intelligent robot control and power operation automation technology, specifically to a control method and related equipment for a power high-altitude operation robotic arm based on a diffusion strategy. Background Technology

[0002] With the development of automation in power system operation and maintenance, power aerial work robots will gradually replace manual labor in completing high-risk live-line work tasks. Power aerial work scenarios are characterized by strong electromagnetic interference, confined spaces at high altitudes, dynamic wind deviation disturbances, non-standardized work objects, and stringent live-line safety constraints, which place extremely high demands on the positioning accuracy, compliant control capabilities, dynamic obstacle avoidance response speed, and environmental generalization capabilities of the robotic arm.

[0003] In existing mainstream robotic arm control solutions, traditional teaching schemes rely on manually pre-calibrated fixed points and work trajectories, which can only reproduce standardized work processes and cannot adapt to dynamic working conditions such as dynamic conductor offset, work vehicle base sway, and non-standardized work objects. Their generalization ability for non-preset scenarios is severely insufficient. The accompanying PID control, based on a rigid adjustment mechanism for position errors, cannot achieve precise and compliant closed-loop control of end-effector contact force. In contact work scenarios involving fittings, conductors, and insulators, this can easily cause damage to the work object or collisions with live conductors, failing to meet the positioning accuracy and compliant contact requirements of live-line work. While deep reinforcement learning-based control schemes rely on massive online trial-and-error iterations for policy convergence, resulting in extremely low sample utilization efficiency, high-voltage live-line work is a high-risk scenario with confined working spaces and inviolable safety red lines, making large-scale online trial-and-error impossible. Furthermore, during policy training and inference, it is impossible to achieve full-process closed-loop control of hard constraints such as live-line safety distances and joint movement limits, posing a risk of probabilistic breaches of safety red lines and failing to meet the core requirement of absolute safety in live-line work. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a control method and related equipment for a power high-altitude operation robotic arm based on a diffusion strategy, which addresses the shortcomings of the prior art and solves the technical problem of insufficient control accuracy of traditional control methods in complex operation scenarios.

[0005] The objective of this invention is achieved through the following technical solutions: In a first aspect, the present invention provides a control method for a power high-altitude operation robotic arm based on a diffusion strategy, comprising: Real-time acquisition of multimodal perception data of high-altitude power operation scenarios, including environmental perception data, robotic arm body status data, and operation task data; A multimodal feature fusion module is used to extract single-modal features from multimodal perception data. After feature fusion based on the single-modal feature vectors, a conditional vector is obtained. The multimodal feature fusion module is a Transformer module based on cross-attention. The condition vector is input into a pre-built condition diffusion model. Under the preset operational safety and feasibility domain, a joint control sequence for controlling the robot arm's movements is generated through forward diffusion constraints and reverse denoising. The joint control commands are sent to the servo driver of the robotic arm to control the robotic arm to perform the corresponding operation. The robot arm collects body state data and environmental perception data after performing actions and feeds them back to the multimodal feature fusion module to achieve closed-loop control.

[0006] As a further improvement of the present invention, after acquiring the multimodal sensing data of the high-altitude power operation scenario, the method further includes preprocessing the multimodal sensing data: The environmental perception data is subjected to distortion correction, noise filtering and target recognition processing to extract point cloud features of the work object and obstacles; The Kalman filter method is used to filter and denoise the state data of the robotic arm body and perform gravity compensation processing to extract the motion state features and contact state features of the robotic arm body. The processed environmental perception data, robotic arm body state data, and task data are timestamped and normalized to form a synchronized multimodal feature vector.

[0007] As a further improvement of the present invention, the multimodal feature fusion module includes: The preprocessed multimodal feature vectors are encoded by separate feature encoding branches to obtain feature vectors for each modality. The feature encoding branch corresponding to the environmental perception data is a residual convolutional neural network, and the feature encoding branch corresponding to the robotic arm body state data and the task data is a multilayer perceptron. Using the Transformer module, cross-modal correlation is performed on the feature vectors of each modality, and the key feature weights of the task target, obstacles, and contact state are strengthened to obtain global features; The global features are conditionally encoded to obtain a condition vector that matches the dimension of the conditional diffusion model.

[0008] As a further improvement of the present invention, the conditional diffusion model includes: A predefined operational safety feasible domain is defined, which includes at least one of the following: live safety distance constraint, joint movement limit constraint, and contact force threshold constraint. During the forward diffusion constraint process, a robotic arm trajectory sequence sample is generated based on the condition vector. For each step of the forward diffusion, a safety domain truncation and projection operation is performed on the robotic arm trajectory sequence sample to ensure that the robotic arm trajectory sequence sample is always within the safe and feasible domain of the operation. The robotic arm trajectory sequence sample includes the joint state data of the robotic arm at different times. In the reverse denoising generation process, the condition vector noise is processed based on a dual-branch coupled denoising network to generate a joint control sequence that satisfies the operational safety and feasibility domain.

[0009] As a further improvement of the present invention, the dual-branch coupled denoising network includes: The pose control branch is used to optimize the joint angle and end pose control accuracy of the robotic arm based on the conditional vector and noisy joint state data using the first U-Net unit. The force compliance branch is used to optimize the end contact force and joint torque control accuracy of the robotic arm based on the condition vector and noisy joint state data using the second U-Net unit. The pose control branch and force compliance branch are coupled through the Transformer module to generate a global denoising prediction result, which in turn yields the joint control sequence of the robotic arm.

[0010] As a further improvement of the present invention, the conditional diffusion model is trained using a multi-constraint coupling loss function, which is:

[0011] In the formula, For multi-constraint coupling loss function, For the first Step noise prediction mean square error loss, For the loss of end-effector pose accuracy, As the first weight, For contact force compliance loss, For the loss of safe distance from energized surfaces, For joint limitation loss, As the second weight, As the third weight, As the fourth weight, The sampling period.

[0012] As a further improvement of the present invention, the conditional diffusion model further includes a projection correction step: The conditional diffusion model is inferred in real time by using a skip-step accelerated sampling algorithm with safety projection. The number of sampling steps is compressed and real-time safety checks are performed on the generated intermediate trajectory in each sampling step. If the trajectory exceeds the safe and feasible domain of the operation, projection correction is performed.

[0013] Secondly, the present invention provides a power aerial work platform robotic arm control system based on a diffusion strategy, comprising: The data acquisition module is used to acquire multimodal perception data of the high-altitude power operation scenario in real time. The multimodal perception data includes environmental perception data, robotic arm body status data, and operation task data. The multimodal feature fusion module is used to extract single-modal features from multimodal perception data, and to obtain a conditional vector after feature fusion based on the single-modal feature vectors; the multimodal feature fusion module is a Transformer module based on cross-attention; The instruction generation module is used to input the condition vector into a pre-built condition diffusion model, and generate a joint control sequence for controlling the robot arm's movements through forward diffusion constraints and reverse denoising, under the condition of satisfying preset safety constraints. The control execution module is used to send the joint control commands generated by the command generation module to the servo driver of the robotic arm, so as to control the robotic arm to perform the corresponding operation. The feedback module is used to collect the body state data and environmental perception data after the robotic arm performs the action and feed them back to the multimodal feature fusion module to realize closed-loop control.

[0014] Thirdly, the present invention provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed as described above, a diffusion-strategy-based control method for a power aerial work platform.

[0015] Fourthly, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the above-described diffusion-strategy-based control method for a power aerial work platform.

[0016] The beneficial effects of this invention are as follows: This invention provides a control method for a robotic arm for high-altitude power operations based on a diffusion strategy. It acquires multimodal perception data of the high-altitude power operation scenario in real time, extracts single-modal features from the multimodal perception data using a multimodal feature fusion module, and fuses the single-modal feature vectors to obtain a conditional vector, effectively integrating comprehensive operation scenario information. Simultaneously, the multimodal perception data employs a Transformer module based on cross-attention, significantly improving the comprehensiveness and accuracy of feature representation compared to traditional single-modal feature extraction methods. The conditional vector is input into a pre-constructed conditional diffusion model. Under the preset operational safety and feasibility domain, a joint control sequence for controlling the robotic arm's movements is generated through forward diffusion constraints and inverse denoising, ensuring the robotic arm's operation... While ensuring operational safety, this invention generates more precise joint control sequences through the inverse denoising process of the conditional diffusion model, thus overcoming the shortcomings of traditional control methods in terms of insufficient control accuracy in complex operation scenarios. Joint control commands are sent to the servo driver of the robotic arm to control the robotic arm to execute corresponding operation actions, achieving precise execution of operation commands. The invention collects the robotic arm's body state data and environmental perception data after the execution of actions and feeds them back to the multimodal feature fusion module to achieve closed-loop control. Through a dynamic feedback mechanism, the control strategy is continuously optimized, improving the control adaptability and operational stability of the robotic arm in high-altitude power operation scenarios. The synergistic effect between the various steps of this invention achieves closed-loop optimization of the entire process from multimodal perception to precise control of the robotic arm in high-altitude power operation scenarios, significantly improving operational safety, control accuracy, and system adaptability. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the power high-altitude operation robotic arm control method in an embodiment of the present invention; Figure 2 This is a flowchart of the data acquisition and preprocessing process in an embodiment of the present invention; Figure 3 This is a flowchart of the perceptual feature fusion and conditional coding process in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the forward diffusion process of charged safety hard constraints in an embodiment of the present invention. Figure 5 This is a flowchart of the reverse denoising model training process in an embodiment of the present invention; Figure 6This is a flowchart of the online real-time inference process in an embodiment of the present invention; Figure 7 This is an internal structural diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives and technical solutions of this invention clearer and easier to understand, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0020] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. The described embodiments are only some embodiments of the present invention, and not all embodiments.

[0021] Example 1 In existing technologies, the control of robotic arms for high-altitude power operations mostly relies on single-modal perception data for decision-making. This dependence on vision or single sensor data fails to comprehensively capture the complex environmental information of the work scene, the changes in the robotic arm's own state, and the dynamic requirements of the task. This leads to problems such as errors in object identification and the omission of obstacles, similar to the drawbacks of traditional manual live-line work, which relies on human observation and suffers from blind spots and inaccurate position control. Furthermore, traditional control methods lack effective multimodal feature fusion techniques. Since each modal data point is independent, cross-modal information cannot be collaboratively utilized, resulting in insufficient accuracy and robustness of control decisions. Furthermore, existing control models often fail to fully consider the special safety requirements of high-altitude power operations, lacking effective control over key constraints such as live safety distances and joint movement limits, which can easily lead to safety hazards such as electric shock and robotic arm jamming. Moreover, most control methods do not have a complete closed-loop control mechanism, making it impossible to adjust the control strategy in real time according to the actual performance of the robotic arm. This results in difficulty in ensuring operational accuracy, failing to meet the needs of refined operations such as wire splicing and bolt tightening in high-altitude power operations, and also making it difficult to adapt to operational scenarios under complex weather conditions. Operational efficiency and safety need to be further improved.

[0022] Therefore, this embodiment provides a control method for a power high-altitude operation robotic arm based on a diffusion strategy, such as... Figure 1As shown, the multimodal sensing data is processed into condition vectors by the multimodal feature fusion module. After processing the condition vectors using the condition diffusion model, a joint control sequence for controlling the robotic arm's movements is generated. After the movements are completed based on the joint control commands, the body state data and environmental sensing data after the movements are executed are fed back to the multimodal feature fusion module, ultimately achieving closed-loop control. The control method in this embodiment is particularly suitable for high-precision, high-compliance, and high-safety control of robotic arms in high-altitude operation scenarios such as live-line disconnection, insulator replacement, equipment inspection, and foreign object removal on 10kV AC / DC transmission and distribution lines. The specific implementation method of the power high-altitude operation robotic arm control method based on the diffusion strategy is as follows.

[0023] S1: Real-time acquisition of multimodal perception data for high-altitude power operations.

[0024] The multimodal perception data includes environmental perception data, robotic arm body status data, and task data.

[0025] Furthermore, environmental perception data is collected using a binocular depth camera. RGB-D images of the work scene are acquired, and based on these images, the pose information of the work objects (such as wires, insulators, and fittings) and the point cloud information of obstacles in the work scene are further extracted to achieve spatial perception of the work environment.

[0026] The robotic arm's physical status data is acquired through a variety of distributed sensors. These include joint encoders that collect angle and angular velocity data for each joint; joint torque sensors that collect joint torque data; end effector six-dimensional force sensors that collect end effector contact force / torque data; and a base IMU that collects data on the aerial work platform's base sway and attitude deviation. This multi-sensor approach enables comprehensive perception of the robotic arm's own motion, contact status, and the status of the work platform.

[0027] The task data is collected based on the task instruction system. The system acquires the target parameters of the current task, such as the conductor connection posture for live-line disconnection and connection operations, the target location for equipment inspection operations, and the disassembly and assembly posture for insulator replacement operations, to achieve accurate perception of task requirements.

[0028] By using environmental perception data, robotic arm status data, and task data, the system fully reflects the environmental changes at the work site, the robotic arm's own status, and the task requirements, thereby enabling multi-dimensional and high-frequency perception of high-altitude power operations.

[0029] This embodiment, after acquiring multimodal sensing data, also preprocesses the multimodal sensing data, using differentiated processing methods to eliminate noise interference in various types of data, unify data formats, and extract effective features. For example... Figure 2 As shown, the preprocessing steps include: The environmental perception data is processed by distortion correction, noise filtering, and target recognition. First, distortion correction is used to eliminate lens distortion errors of the binocular depth camera. Then, noise filtering is used to remove random noise from the image. Finally, target recognition methods such as target detection and instance segmentation are used to extract normalized point cloud features of the work object and obstacles, thereby realizing feature extraction and optimization of the environmental perception data.

[0030] The Kalman filter method is used to filter and denoise the state data of the robotic arm body and perform gravity compensation processing. The Kalman filter eliminates the vibration noise caused by the base shaking, and the gravity compensation eliminates the influence of gravity on the data collected by the end effector six-dimensional force sensor and joint torque sensor, so as to accurately extract the motion state characteristics and contact state characteristics of the robotic arm body.

[0031] The processed environmental perception data, robotic arm body status data, and task data are timestamped and normalized. Timestamping ensures the synchronization of the three types of multimodal data in the time dimension, avoiding feature misalignment caused by acquisition delay. Normalization maps all data to a unified numerical range, eliminating the impact of data dimension differences on subsequent feature fusion and model training, and finally forming a synchronized multimodal feature vector.

[0032] S2: Use the multimodal feature fusion module to extract single-modal features from the multimodal sensing data, and then perform feature fusion based on the single-modal feature vectors to obtain the conditional vector.

[0033] This embodiment uses a multimodal feature fusion module to extract features and perform cross-modal fusion on the preprocessed multimodal sensing data, mapping the multi-source heterogeneous sensing data to a unified feature space and generating a conditional vector that adapts to the input of the subsequent conditional diffusion model. The multimodal feature fusion module is a Transformer module based on cross-attention, which has powerful cross-modal feature association and key feature enhancement capabilities.

[0034] In some embodiments of this application, the multimodal feature fusion module processing flow is as follows: Figure 3 As shown, specifically: Single-modal feature encoding: A separate feature encoding branch is set for the preprocessed multimodal feature vector for each modality. For example, in this embodiment, considering the high-dimensional spatial features of the point cloud features of the environmental perception data, a residual convolutional neural network (ResNet-18) is used as its corresponding feature encoding branch. Deep spatial features of the environmental perception data are extracted through convolution and residual connection operations. Considering the one-dimensional numerical features of the robotic arm body state data and the task data, a multilayer perceptron (MLP) is used as its corresponding feature encoding branch. Deep numerical features of the two types of data are extracted through nonlinear transformation of fully connected layers, ultimately obtaining independent high-dimensional feature vectors for each modality. Cross-modal feature fusion: Utilizing the Transformer module based on cross-attention, the feature vectors of each modality are taken as input. The internal correlation of single-modal features is realized through the self-attention mechanism, and the cross-modal correlation between different modal features is realized through the cross-attention mechanism. At the same time, during the attention calculation process, higher attention weights are set for key features such as the task target, obstacles, and contact state to enhance key features. Finally, a global feature that integrates information from the environment, the robotic arm itself, and the task is obtained. Conditional encoding: Conditional encoding processes the global features by performing dimensional transformation and feature mapping, converting the global features into low-dimensional conditional vectors that match the input dimensions of the conditional diffusion model. These conditional vectors serve as the generation conditions for the conditional diffusion model and can accurately represent the full-scene information of current high-altitude power operations.

[0035] S3: Input the condition vector into the pre-built condition diffusion model. Under the preset operational safety and feasibility domain, generate the joint control sequence for controlling the robot arm's movements through forward diffusion constraints and reverse denoising.

[0036] The conditional diffusion model is a conditional denoising diffusion probability model customized for high-altitude live-line working scenarios. This model uses conditional vectors generated by multimodal fusion as generation conditions and robotic arm joint control sequences as generation targets. Through forward diffusion safety constraints and inverse denoising precise generation, it achieves safe, accurate, and compliant generation of control commands. In some embodiments of this application, the specific implementation process of the conditional diffusion model is as follows: First, predefine the safe and feasible domain of the operation. Operational safety feasible area The only permissible space for the control trajectory of the robotic arm is the set of all robotic arm control trajectories that satisfy the safety constraints for live-line work, which includes at least the live-line safety distance constraint. Joint movement limitation (i.e., the maximum joint angle) and minimum value and contact force threshold constraints .

[0037] Among them, the operational safety feasible domain It is a closed set, defined as: for a sequence of robotic arm control trajectories ( for Joint angle vector at time step The robotic arm state at every moment on the trajectory satisfies the following three sets of inequality constraints. Operational safety feasible area The expression is:

[0038] in, (Electrified safety distance constraints): ; (Joint rigid restraint): ; (Contact force compliance constraint): .

[0039] In the formula, This is the minimum Euclidean distance between the entire linkage of the robotic arm and the charged body. The minimum bounding box generated for each subset of charged body point clouds using the Directed Bounding Box (OBB) algorithm contains complete spatial feature parameters such as the spatial vertex coordinates and normal vectors of the bounding box; The minimum safe distance for live working at the corresponding voltage level is specified for 10kV AC / DC transmission and distribution lines. The value should be no less than 0.4m; for other voltage levels, follow GB / T 18857 "Technical Guidelines for Live Working on Distribution Lines"; The actual contact force at the end of the robotic arm is obtained by subtracting the dynamic compensation forces such as gravity, Coriolis force, and centrifugal force calculated by the rigid body dynamics algorithm from the measured value of the six-dimensional force sensor.

[0040] Secondly, such as Figure 4 As shown, the forward diffusion constraint process is performed as follows: Based on the condition vector, a robotic arm trajectory sequence sample is generated. This robotic arm trajectory sequence sample includes the joint state data of the robotic arm at different times. For each step of the forward diffusion, the robotic arm trajectory sequence sample generated is subjected to safety domain truncation and projection operations. Specifically, after Gaussian noise addition at each step, the indicator function is used to determine whether the noise-added sample is within the operational safety feasible region. If the sample exceeds the operational safety feasible region, it is immediately projected to the nearest boundary of the operational safety feasible region. This ensures that the robotic arm trajectory sequence sample is always within the operational safety feasible region throughout the entire forward diffusion process, thereby preventing the generation of illegal trajectories from the source of diffusion and enabling the model to learn only compliant and safe trajectories.

[0041] The conditional probability formula for forward diffusion is:

[0042] in, As an indicator function, when the robotic arm trajectory sequence samples In the safe and feasible domain The value is 1 if the time is within the range, and 0 otherwise.

[0043] By performing truncation projection on samples that exceed the safe domain in the above manner, the non-compliant samples are projected to the nearest boundary of the safe and feasible domain, ensuring that the samples diffused at each step are compliant, and preventing the model from learning non-compliant trajectories from the root of data generation.

[0044] Then, as Figure 5 As shown, the reverse denoising generation process is performed: based on a dual-branch coupled denoising network, the noisy condition vector is processed for noise reduction and trajectory reconstruction to generate a joint control sequence that meets the operational safety and feasibility domain. In this embodiment, the dual-branch coupled denoising network is designed to address the dual requirements of positioning accuracy and contact compliance in high-altitude power operations. It includes a pose control branch, a force compliance branch, and a feature coupling module. Specifically, the pose control branch uses the first U-Net unit to perform denoising processing based on the condition vector and noisy joint state data, with the optimization targets being the joint angle of the robotic arm and the end-effector pose control accuracy. The core optimization focuses on the positioning accuracy and obstacle avoidance safety of the trajectory. The first U-Net unit in this embodiment has a structure of 4 layers of downsampling and 4 layers of upsampling.

[0045] The force compliance branch uses a lightweight second U-Net unit with 2 layers of downsampling and 2 layers of upsampling to denoise the conditional vector and noisy joint state data. The core optimization goal is to improve the compliance of the operation contact process, with the end contact force and joint torque control accuracy of the robotic arm as the optimization objective.

[0046] Furthermore, the pose control branch and the force compliance branch are coupled through the Transformer module. The pose control branch provides trajectory position constraints for the force compliance branch, while the force compliance branch provides contact compliance constraints for the pose control branch. Finally, they are fused to generate a global denoised prediction result. Based on this result, a noise-free robotic arm joint control sequence that meets the operational safety feasible domain was reconstructed.

[0047] Global denoising prediction results for:

[0048] In the formula, Adaptive weights for pose denoising prediction results. The pose denoising prediction result is obtained from the pose control branch. For the adaptive weights of the force-sensing denoising prediction results, The force-sensing denoising prediction result is obtained from the force-sensing compliance branch. The adaptive weights satisfy: ; Adaptive weight switching is achieved based on the contact state detection results of the end-effector six-dimensional force sensor: during the non-contact phase Prioritize positioning accuracy and obstacle avoidance safety; during the contact phase Adaptive boost, with a maximum value of 0.8, prioritizes contact flexibility to avoid damage to the workpiece.

[0049] The general diffusion model, which only uses noise prediction as its loss objective, cannot adapt to the multi-dimensional and highly constrained requirements of live-line work. A fixed-weight loss function cannot match the optimization priorities of different diffusion step sizes. Therefore, this embodiment constructs a multi-constraint coupled loss function with adaptive diffusion step sizes based on the loss function of the general diffusion model, achieving differentiated optimization objectives at different denoising stages. During the training of the conditional diffusion model, the multi-constraint coupled loss function is used for model optimization to ensure that the model training is always within safe constraints, while matching the optimization priorities of different diffusion stages. The calculation formula for the multi-constraint coupled loss function is as follows:

[0050] In the formula, For multi-constraint coupling loss function, For the first Step noise prediction mean square error loss, To account for the loss of end-effector pose accuracy, the positioning error is constrained to be ≤ , As the first weight, To compensate for contact force compliance loss, constrain contact force ≤ , To account for the safety distance loss while charged, the constraint trajectory always satisfies ; To limit joint movement loss, constrain joint parameters ≤ ; As the second weight, As the third weight, As the fourth weight, The sampling period.

[0051] Each weight is adaptively adjusted according to the inverse denoising step size, wherein the first weight Progressing with denoising step size ( from Decreasing to 0) linearly increasing, the focus in the later stage of noise reduction is to optimize positioning accuracy, the second weight Adaptive enhancement during the contact phase; third weight. Maintaining the highest value of 10.0 during the full denoising stage is the highest priority mandatory constraint; the fourth weight Maintain a fixed value of 5.0 during the full noise reduction stage.

[0052] In the early stage of diffusion ( The stage of restoring data from pure noise to noisy data is centered on safety constraints. , Weighting accounts for over 80%; later stage of diffusion ( (The stage of restoring noisy data to precise control commands) focuses on accuracy and compliance constraints. , The weights are gradually increased to achieve differentiated optimization at different stages.

[0053] To meet the real-time requirements of high-altitude power operations, the conditional diffusion model also includes a projection correction step in the online inference phase: the conditional diffusion model is inferred in real time through a step-by-step accelerated sampling algorithm with safe projection, compressing the 1000 sampling steps of the traditional diffusion model to 20-50 steps, achieving millisecond-level instruction generation; at the same time, real-time safety verification is performed on the generated intermediate trajectory in each sampling step. If the state of the intermediate trajectory at any time exceeds the safe and feasible region of the operation, projection correction is immediately performed to pull the trajectory back to the nearest boundary of the safe and feasible region of the operation, ensuring that the trajectory is compliant throughout the sampling process. The final generated joint control sequence has both real-time performance and safety.

[0054] Specifically, such as Figure 6 As shown, real-time sensing data acquisition involves synchronously acquiring multimodal sensing data at a preset control frequency (≥60Hz), completing preprocessing and feature fusion, and generating a real-time condition vector. To address the real-time and absolute safety requirements of high-altitude power operations, a safety-priority customized modification of the general denoising diffusion implicit model (DDIM) is implemented. A step-by-step accelerated sampling algorithm with full-process safety projection is constructed, compressing the sampling steps from 1000 to 20-50. Based on the real-time condition vector, the optimal control trajectory and joint control commands for the robotic arm are generated within 10ms. In each sampling step, a safety check is performed on the generated intermediate trajectory. If the state of the trajectory at any time exceeds... Immediately perform projection correction to pull the trajectory back into the safe and feasible region, ensuring compliance throughout the sampling process. The sampling iteration formula is:

[0055]

[0056]

[0057] in: , The noise predicted by the denoising network. The number of jump steps. The safe and feasible region projection function projects the violation samples to the nearest boundary of the safe and feasible region.

[0058] By performing a truncation projection operation on samples that exceed the safe region in the above manner, the non-compliant samples are projected to the nearest boundary of the safe and feasible region, ensuring that the samples diffused at each step are compliant, and preventing the model from learning the non-compliant trajectory from the root of data generation; the modified forward diffusion conditional probability formula is accompanied by a projection operation to ensure that the samples are always constrained within the safe and feasible region.

[0059] S4: Send joint control commands to the servo driver of the robotic arm to control the robotic arm to perform the corresponding operation.

[0060] The joint control sequence generated by the conditional diffusion model is parsed into control commands for each joint of the robotic arm. These joint control commands are then sent to the servo driver of the robotic arm. The servo driver drives the motors of each joint of the robotic arm to move according to the control commands, controlling the robotic arm to perform the corresponding work actions according to the generated continuous and smooth trajectory, thus realizing the conversion from decision commands to mechanical actions.

[0061] In some embodiments of this application, the frequency of sending joint control commands is consistent with the frequency of previous data acquisition, not lower than 60Hz, to ensure the continuity and smoothness of the robotic arm's movements. After receiving the control commands, the servo driver uses a closed-loop servo control algorithm to achieve precise driving of the joint motor, so that the actual position and contact force of the robotic arm end are consistent with the preset value of the control commands, thereby improving the accuracy of the operation.

[0062] By matching the command transmission frequency with the data acquisition frequency, the continuity of the robotic arm's movements is ensured, avoiding motion jams or trajectory interruptions. The closed-loop servo control algorithm enables precise driving of the joint motors, improving the accuracy of the robotic arm's operation and ensuring that control commands are executed accurately, achieving seamless integration of decision-making and execution.

[0063] S5: Collects the body state data and environmental perception data after the robotic arm performs the action and feeds them back to the multimodal feature fusion module to achieve closed-loop control.

[0064] In some embodiments of this application, during the execution of a task by the robotic arm, the system collects the robotic arm's body state data (including joint angles, angular velocities, end contact forces, etc.) and environmental perception data (including changes in the pose of the work object, changes in the position of obstacles, etc.) in real time at a preset frequency. This feedback data, together with the newly collected task data, is input into the multimodal feature fusion module. The steps S2 to S4 above are repeated to generate a new joint control sequence that adapts to the changes in the current task state, thereby realizing the dynamic adjustment and optimization of the robotic arm control commands.

[0065] Through a fully closed-loop control system, the system can perceive the execution effect of the robotic arm's operation and the dynamic changes in the operation scenario in real time, and adjust the control commands in a timely manner. This effectively counteracts the effects of dynamic disturbances such as wire deflection, base swaying, and changes in the position and posture of the work object in high-altitude power operation scenarios, improves the robotic arm's adaptability to dynamic working conditions and its robustness in operation, and ensures that the operation process is always accurate, safe, and smooth.

[0066] The power high-altitude operation robotic arm control method in this embodiment achieves full-dimensional perception of the operation scene through real-time acquisition and preprocessing of multimodal sensing data. It achieves efficient fusion of multi-source heterogeneous data through a multimodal feature fusion module based on cross-attention Transformer. A customized conditional diffusion model enables safe, precise, and compliant joint control sequence generation. A fully closed-loop control system achieves real-time adaptation to dynamic working conditions. By employing forward diffusion safety domain constraints, safety priority design of multi-constraint coupling loss functions, and projection correction in the inference process, the model-generated trajectory fully meets the safety constraints of live-line operation, eliminating the generation of illegal trajectories from the source and significantly reducing the safety risks of live-line operation, thus meeting the core requirement of absolute safety in live-line operation. A dual-branch coupled denoising network simultaneously optimizes the positioning accuracy and contact force compliance control of the robotic arm's end effector, resolving the contradiction between accuracy and compliance that traditional rigid adjustment methods cannot balance, effectively avoiding damage to the work object or collisions with live conductors, and improving operation quality. The generative control architecture based on diffusion strategy can dynamically generate a continuous and smooth control sequence adapted to the current working conditions based on real-time acquired multimodal data. It does not require manual pre-calibration of the entire process trajectory and can effectively adapt to dynamic working conditions such as dynamic conductor offset, base sway, and non-standardized work objects.

[0067] Example 2 This embodiment provides a power aerial work platform robotic arm control system based on a diffusion strategy, including: The data acquisition module is used to acquire multimodal perception data of the high-altitude power operation scenario in real time. The multimodal perception data includes environmental perception data, robotic arm body status data, and operation task data. The multimodal feature fusion module is used to extract single-modal features from multimodal perception data. After feature fusion based on the single-modal feature vectors, a conditional vector is obtained. The multimodal feature fusion module is a Transformer module based on cross-attention. The instruction generation module is used to input the condition vector into the pre-built condition diffusion model. Under the condition of satisfying the preset safety constraints, it generates the joint control sequence for controlling the robot arm's movements through forward diffusion constraints and reverse denoising. The control execution module is used to send the joint control commands generated by the command generation module to the servo driver of the robotic arm, so as to control the robotic arm to perform the corresponding operation. The feedback module is used to collect the body state data and environmental perception data after the robotic arm performs the action and feed them back to the multimodal feature fusion module to realize closed-loop control.

[0068] Specific limitations regarding the diffusion-strategy-based power aerial work platform control system can be found in the above-described limitations of the diffusion-strategy-based power aerial work platform control method; the corresponding technical effects are equivalent and will not be repeated here. Each module in the aforementioned diffusion-strategy-based power aerial work platform control system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, allowing the processor to call and execute the corresponding operations of each module.

[0069] Figure 7 An internal structural diagram of a computer device is shown in one embodiment. This computer device may specifically be a terminal or a server. Figure 7 As shown, the computer device includes a processor, memory, network interface, display, camera, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a diffusion-strategy-based control method for a power-driven aerial work platform robotic arm. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0070] As will be understood by those skilled in the art, computer equipment Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. Specific computing devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0071] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0072] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0073] In summary, the present application provides a diffusion-strategy-based control method, system, computer equipment, and storage medium for a robotic arm used in high-altitude power operations. First, addressing the perception requirements of high-altitude power operations, it simultaneously collects three types of multimodal data: environmental perception, robotic arm status, and operational tasks. Preprocessing is performed based on different data characteristics to provide comprehensive and synchronized foundational data for subsequent control strategy generation. Then, a multimodal feature fusion module based on cross-attention Transformer is constructed to extract single-modal features from each modality, strengthening the weights of key information such as the operational target, obstacles, and contact status, providing accurate input conditions for the diffusion model. Finally, this method uses the conditional vector generated by multimodal fusion as input and the robotic arm joint control sequence as the generation target to construct a conditional diffusion model adapted to high-altitude live-line power operations, achieving accurate and safe generation of robotic arm control commands. During the forward diffusion process of the model, a safe and feasible region for live-line work is predefined, including live-line safety distance, joint hard limits, and contact force compliance constraints. After each step of adding noise, a safety region truncation and projection operation is performed to ensure that noisy samples are always constrained within the safe and feasible region, allowing the model to learn only compliant and safe trajectories from the source of diffusion. In the reverse denoising generation process, a pose-force perception dual-branch coupled denoising network is constructed to solve the core contradiction of traditional solutions that cannot balance accuracy and compliance.

[0074] Furthermore, this method constructs a multi-constraint coupled loss function with an adaptive diffusion step size for the conditional diffusion model. Based on the noise prediction loss, it incorporates constraints such as pose accuracy, contact force compliance, electrical safety, and joint limit. The weights of each constraint are adaptively adjusted with the inverse denoising step size, ensuring that the model training is always within safe constraints. At the same time, it matches the optimization focus of different diffusion stages, achieving coordinated optimization of safety, accuracy, and compliance.

[0075] Real-time acquisition and preprocessing of multimodal perception data enables full-dimensional perception of the work scenario; a multimodal feature fusion module based on cross-attention Transformer enables efficient fusion of multi-source heterogeneous data; a customized conditional diffusion model enables safe, accurate, and compliant joint control sequence generation; and a fully closed-loop control system enables real-time adaptation to dynamic working conditions.

[0076] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0077] The above-described embodiments are merely preferred embodiments of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A control method for a power aerial work platform robotic arm based on a diffusion strategy, characterized in that, include: Real-time acquisition of multimodal perception data of high-altitude power operation scenarios, including environmental perception data, robotic arm body status data, and operation task data; A multimodal feature fusion module is used to extract single-modal features from multimodal perception data. After feature fusion based on the single-modal feature vectors, a conditional vector is obtained. The multimodal feature fusion module is a Transformer module based on cross-attention. The condition vector is input into a pre-built condition diffusion model. Under the preset operational safety and feasibility domain, a joint control sequence for controlling the robot arm's movements is generated through forward diffusion constraints and reverse denoising. The joint control commands are sent to the servo driver of the robotic arm to control the robotic arm to perform the corresponding work actions; The robot arm collects body state data and environmental perception data after performing actions and feeds them back to the multimodal feature fusion module to achieve closed-loop control.

2. The control method for a power aerial work platform robotic arm based on a diffusion strategy according to claim 1, characterized in that, After acquiring multimodal sensing data of the power high-altitude operation scenario, the process also includes preprocessing the multimodal sensing data: The environmental perception data is subjected to distortion correction, noise filtering and target recognition processing to extract point cloud features of the work object and obstacles; The Kalman filter method is used to filter and denoise the state data of the robotic arm body and perform gravity compensation processing to extract the motion state features and contact state features of the robotic arm body. The processed environmental perception data, robotic arm body state data, and task data are timestamped and normalized to form a synchronized multimodal feature vector.

3. The control method for a power aerial work platform robotic arm based on a diffusion strategy according to claim 2, characterized in that, The multimodal feature fusion module includes: The preprocessed multimodal feature vectors are encoded by separate feature encoding branches to obtain feature vectors for each modality. The feature encoding branch corresponding to the environmental perception data is a residual convolutional neural network, and the feature encoding branch corresponding to the robotic arm body state data and the task data is a multilayer perceptron. Using the Transformer module, cross-modal correlation is performed on the feature vectors of each modality, and the key feature weights of the task target, obstacles, and contact state are strengthened to obtain global features; The global features are conditionally encoded to obtain a condition vector that matches the dimension of the conditional diffusion model.

4. The control method for a power aerial work platform robotic arm based on a diffusion strategy according to claim 1, characterized in that, The conditional diffusion model includes: A predefined operational safety feasible domain is defined, which includes at least one of the following: live safety distance constraint, joint movement limit constraint, and contact force threshold constraint. During the forward diffusion constraint process, a robotic arm trajectory sequence sample is generated based on the condition vector. For each step of the forward diffusion, a safety domain truncation and projection operation is performed on the robotic arm trajectory sequence sample to ensure that the robotic arm trajectory sequence sample is always within the safe and feasible domain of the operation. The robotic arm trajectory sequence sample includes the joint state data of the robotic arm at different times. In the reverse denoising generation process, the condition vector noise is processed based on a dual-branch coupled denoising network to generate a joint control sequence that satisfies the operational safety and feasibility domain.

5. The control method for a power aerial work platform robotic arm based on a diffusion strategy according to claim 4, characterized in that, The dual-branch coupled denoising network includes: The pose control branch is used to optimize the joint angle and end pose control accuracy of the robotic arm based on the conditional vector and noisy joint state data using the first U-Net unit. The force compliance branch is used to optimize the end contact force and joint torque control accuracy of the robotic arm based on the condition vector and noisy joint state data using the second U-Net unit. The pose control branch and force compliance branch are coupled through the Transformer module to generate a global denoising prediction result, which in turn yields the joint control sequence of the robotic arm.

6. The control method for a power aerial work platform robotic arm based on a diffusion strategy according to claim 4, characterized in that, The conditional diffusion model is trained using a multi-constraint coupling loss function, which is as follows: In the formula, For multi-constraint coupling loss function, For the first Step noise prediction mean square error loss, For the loss of end-effector pose accuracy, As the first weight, For contact force compliance loss, For the loss of safe distance from energized surfaces, For joint limitation loss, As the second weight, As the third weight, As the fourth weight, The sampling period.

7. The control method for a power aerial work platform robotic arm based on a diffusion strategy according to claim 6, characterized in that, The conditional diffusion model also includes a projection correction step: The conditional diffusion model is inferred in real time by using a skip-step accelerated sampling algorithm with safety projection. The number of sampling steps is compressed and real-time safety checks are performed on the generated intermediate trajectory in each sampling step. If the trajectory exceeds the safe and feasible domain of the operation, projection correction is performed.

8. A control system for a power aerial work platform robotic arm based on a diffusion strategy, characterized in that, include: The data acquisition module is used to acquire multimodal perception data of the high-altitude power operation scenario in real time. The multimodal perception data includes environmental perception data, robotic arm body status data, and operation task data. The multimodal feature fusion module is used to extract single-modal features from multimodal perception data, and to obtain a conditional vector after feature fusion based on the single-modal feature vectors; the multimodal feature fusion module is a Transformer module based on cross-attention; The instruction generation module is used to input the condition vector into a pre-built condition diffusion model, and generate a joint control sequence for controlling the robot arm's movements through forward diffusion constraints and reverse denoising, under the condition of satisfying preset safety constraints. The control execution module is used to send the joint control commands generated by the command generation module to the servo driver of the robotic arm, so as to control the robotic arm to perform the corresponding operation. The feedback module is used to collect the body state data and environmental perception data after the robotic arm performs the action and feed them back to the multimodal feature fusion module to achieve closed-loop control.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1 to 7, for control of a power aerial work platform based on a diffusion strategy.

10. A computer device, characterized in that, include: Processor and computer-readable storage media; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the power aerial work robot control method based on a diffusion strategy as described in any one of claims 1 to 7.

Images