A quadruped robot mechanical arm end force feedback teleoperation control system and method

Abstract

The present disclosure belongs to the technical field of robot control, and specifically provides a four-legged robot mechanical arm end force feedback teleoperation control system and method. In view of the key challenges of control errors caused by communication time delay and soft obstacles difficult to identify in a dynamic environment, the system introduces a predictive contact response mechanism, pre-models and judges the end contact state of the mechanical arm, realizes feedforward control and buffer adjustment for communication time delay. At the same time, a dynamic semantic map is constructed combined with multi-source sensing information, and the soft obstacle inference and path optimization are carried out by fusing the end force sense change trend, so as to significantly improve the identification and avoidance ability of the system in a complex and invisible obstacle environment. The present application has good foresight, adaptability and high redundancy safety characteristics, and is suitable for multi-task inspection operation in industrial sites such as thermal power plants, and is particularly suitable for remote human-machine collaborative operation scenes in narrow spaces.

Description

Technical Field

[0001] This disclosure relates to the field of robot control technology, and in particular to a force feedback teleoperation control system and method for the end effector of a quadruped robot arm. Background Technology

[0002] Currently, industrial sites such as power plants, petrochemical plants, and thermal power plants increasingly rely on robots for remote inspection operations in hazardous environments. Quadruped robots, due to their excellent terrain adaptability, have become an important tool for inspection in complex scenarios. However, existing quadruped robots equipped with robotic arms for remote operation generally suffer from the following problems:

[0003] Most existing systems use a fixed threshold method for end-effector contact detection, meaning that a contact state is determined when the force or torque detected by the sensor exceeds a set value. However, during dynamic operations, due to robotic arm vibration, elastic deformation of flexible structures, or sensor noise, misjudgments or omissions are prone to occur, leading to the failure of operation commands or the unintended triggering of safety mechanisms. Traditional teleoperation systems often rely on direct mapping relationships, failing to determine the consistency between operational intentions and actual feedback. This results in the system's inability to respond or correct in a timely manner when accidental or unexpected contact occurs, affecting the accuracy of human-machine collaboration and the user experience. In remote environments, communication links suffer from unpredictable delays, especially in long-distance teleoperation tasks across network segments and regions. This delay severely affects the synchronization of feedback and control commands, easily leading to system oscillations, end-effector impacts, or even mission failure. Current obstacle avoidance mechanisms, often based on visual point clouds or depth maps, can only identify visible obstacles and cannot effectively perceive soft, transparent, or partially occluded objects, making collisions likely in complex spaces. Furthermore, visual mapping and path reconstruction are delayed and lack timeliness. Existing path planning is mostly fully autonomous, neglecting the coordination mechanism between operator input and system feedback. It cannot flexibly switch between active and passive control strategies according to task needs, nor can it dynamically adjust the trajectory during execution to cope with sudden obstacles or unexpected interference.

[0004] Therefore, there is an urgent need to build an intelligent system that integrates "force feedback teleoperation + robotic arm collaborative obstacle avoidance + body navigation obstacle avoidance + multimodal interaction" to improve the safety and accuracy of remote inspection operations. Summary of the Invention

[0005] This disclosure aims to solve at least one of the technical problems existing in the prior art, and proposes a force feedback teleoperation control system and method for the end effector of a quadruped robot arm.

[0006] In a first aspect, this disclosure provides a force feedback teleoperation control system for the end effector of a quadruped robot arm, comprising:

[0007] Binocular vision perception module, spatial relationship calculation module, communication and time synchronization module, teleoperation control module, force feedback judgment module, obstacle avoidance and path planning module, task execution and feedback module:

[0008] The binocular vision perception module is used to construct visual point clouds and depth maps to identify the spatial pose and surface state of target objects.

[0009] The spatial relationship calculation module calculates the relative pose relationship between the target posture and the end effector state information of the robotic arm based on the target posture and the end effector state information, and generates the trajectory parameters required for the operation task;

[0010] The communication and time synchronization module is based on a high-frequency data transmission mechanism and a buffer synchronization mechanism to achieve low-latency and stable communication between the remote operation terminal and the robot terminal.

[0011] The teleoperation control module combines operator input and task requirements to perform position or impedance control, and automatically switches to pre-contact mode when the robot detects that the end effector is close to the target, predicting contact risks and adjusting the control strategy in advance.

[0012] The force feedback judgment module monitors the force sensor data at the robot's end in real time to form force information, and judges whether it has contacted the target or whether abnormal force changes have occurred based on a multi-threshold force recognition strategy.

[0013] The obstacle avoidance and path planning module combines visual point cloud and force information to construct a dynamic occupancy map, and adopts a two-stage path planning and force dynamic compensation mechanism to realize sudden obstacle response and path replanning.

[0014] The task execution and feedback module controls the robotic arm to execute specific operation instructions, collects status and abnormal information and feeds it back to the operator, forming a task closed loop.

[0015] Preferably, the pre-contact mode includes:

[0016] When the distance between the end effector of the robotic arm and the target is less than a preset threshold, the control system automatically enters the pre-contact mode.

[0017] A contact probability estimation model is constructed using the terminal velocity, target depth variation trend, and force sensor disturbance signal.

[0018] When the probability of contact exceeds the threshold, the control system automatically reduces the control stiffness, limits the propulsion speed, and activates the compliant control strategy.

[0019] Preferably, the force feedback judgment module includes:

[0020] An active contact event recognition mechanism determines whether contact has occurred based on the rate of change of force and direction.

[0021] The predictive feedback mechanism generates tactile feedback proactively before the operator perceives it by fitting the force trend.

[0022] The force channel calibration mechanism corrects the operation path based on tactile information when sensor malfunctions or control drift occurs.

[0023] Preferably, the obstacle avoidance and path planning module includes:

[0024] The primary path planning stage uses RRT to quickly search for collision-free paths, while the secondary path optimization stage introduces the CHOMP algorithm to optimize path smoothness, obstacle avoidance distance, and attitude continuity.

[0025] When the force feedback judgment module detects an abnormal change in the force at the robot's end and the vision does not update in time, a local path reconstruction mechanism is triggered.

[0026] A soft obstacle volume reasoning mechanism is constructed. By using the continuous and slowly increasing force trend and visual occlusion judgment, a virtual obstacle avoidance shadow volume is established to bypass invisible obstacles.

[0027] Preferably, the communication and time synchronization module specifically includes:

[0028] The underlying data transmission adopts a TCP / UDP dual-protocol support mechanism: large amounts of information such as visual images and point clouds use UDP channels to improve the transmission rate; all communication nodes use ROS2's QoS policy to manage communication quality and ensure the priority transmission of key data for teleoperation control and force feedback response.

[0029] A local NTP server runs on the main control computing unit of the control system. The camera, IMU, and force sensor are all connected to this time source via LAN. The local system clock is periodically calibrated to achieve millisecond-level time synchronization accuracy. In the vision module, a local timestamp is added when the image frame is acquired, and it is aligned with the robotic arm status data through the ROS2 time synchronization mechanism.

[0030] Preferably, the teleoperation control module includes real-time probability modeling of the contact state, and the contact probability estimation model is as follows:

[0031]

[0032] Where v is the velocity of the terminal velocity along the target direction. This represents the rate of change of the relative distance between the endpoint and the target. This indicates the amplitude of the micro-perturbation in the force sensor. , , These are weighting coefficients. Use the Sigmoid activation function;

[0033] when When the value exceeds the set threshold, the controller will actively adjust the control parameters of the robotic arm, including reducing the end effector stiffness and switching to compliant control; limiting the maximum end effector propulsion rate to avoid impact; and activating the micro-force feedback channel for the operator to sense that contact is about to occur.

[0034] Preferably, the teleoperation control module further includes a predictive end-effector force feedback adjustment mechanism, specifically comprising:

[0035] Based on the contact probability estimation results in the teleoperation control module Combined with real-time six-dimensional force perturbation trend Based on the end-effector motion status, determine whether the system is currently in a "high-risk contact" state. Once the preset criteria are met: The control system will actively adjust the force feedback signal in the following ways:

[0036] Adjust the feedback channel bandwidth and sampling frequency to prioritize the transmission of critical disturbance information; dynamically increase the force feedback gain coefficient. This allows the operator to feel the approaching resistance even without actual contact;

[0037] Limit the rate of change of feedback, smooth transient jump signals, and avoid false alarms or oscillations.

[0038] Preferably, the obstacle avoidance and path planning module includes a dynamic obstacle avoidance adjustment mechanism:

[0039] Real-time monitoring of the rate of change of force at the end Path tracking deviation Based on the environmental point cloud update frequency, a triggering factor is constructed:

[0040]

[0041] when If there are interference obstacles or sudden environmental changes outside the path, local trajectory replanning is immediately performed. A new path segment with the minimum cost is searched at the current state point and spliced ​​onto the original trajectory to achieve a seamless transition.

[0042] Preferably, the obstacle avoidance and path planning module further includes a soft obstacle volume inference mechanism:

[0043] Maintain a force time series within the control period. Calculate the average force growth rate over a continuous period of time. If all three of the following conditions are met:

[0044] (1) The force rises slowly ;

[0045] (2) The propulsion speed is zero;

[0046] (3) The missing rate of depth data in the path-direction visual image is >90%;

[0047] At this time, the control system detects continuous but slowly rising end force, accompanied by difficulty in advancing along the path. Combined with the lack of visual occlusion information, it automatically determines that the current path may be occupied by soft obstacles.

[0048] The present invention also provides a force feedback teleoperation control method for the end effector of a quadruped robot arm, the method being used in the force feedback teleoperation control system of the end effector of the quadruped robot arm, the method comprising:

[0049] Task assignment and inspection point navigation;

[0050] Upon reaching the designated inspection area, the robotic arm activates the binocular vision perception module to collect image data of the current work area and obtains the scene's 3D point cloud information through binocular parallax calculation.

[0051] Once the target pose is determined, the system activates the obstacle avoidance and path planning module; during trajectory execution, when the distance between the robotic arm end and the target surface is less than 30mm, it automatically switches to pre-contact mode;

[0052] During the contact process, the operator issues remote operation control commands through the master control terminal; the master-slave mapping module maps the input from the master terminal to the motion of the slave terminal, and at the same time integrates the force feedback from the sensor to compare the consistency of the master and slave states;

[0053] Once the current task is completed, the system controller automatically switches to the reset path, controls the robotic arm to retract to the initial safe position, updates the task list, and enters the task flow for the next inspection target point.

[0054] Beneficial effects:

[0055] This invention aims to address the problems of insufficient contact accuracy, large control delay, and poor obstacle avoidance robustness in existing remote teleoperation technologies. To address the key challenges of control errors caused by communication delays and the difficulty in identifying soft obstacles in dynamic environments, a comprehensive control system and method integrating sensing, computation, control, feedback, and obstacle avoidance functions is proposed. This system introduces a predictive contact response mechanism, pre-modeling and judging the contact state of the robotic arm's end effector to achieve feedforward control and buffering adjustment oriented towards communication delays. Simultaneously, a dynamic semantic map is constructed by combining multi-source sensor information, and the trend of end-effector force changes is integrated for soft obstacle reasoning and path optimization, thereby significantly improving the system's ability to identify and avoid complex, invisible obstacle environments. This invention possesses good foresight, adaptability, and high redundancy safety characteristics, making it suitable for multi-task inspection operations in industrial sites such as thermal power plants, and particularly suitable for remote human-machine collaborative operation scenarios in confined spaces. Attached Figure Description

[0056] Figure 1 This is a diagram illustrating the architecture of a force feedback teleoperation control system for the end effector of a quadruped robot arm, provided in an embodiment of this disclosure. Detailed Implementation

[0057] To enable those skilled in the art to better understand the technical solutions of this disclosure, the disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] like Figure 1 As shown, a force feedback teleoperation control system for the end effector of a quadruped robot arm includes:

[0059] Binocular vision perception module, spatial relationship calculation module, communication and time synchronization module, teleoperation control module, force feedback judgment module, obstacle avoidance and path planning module, task execution and feedback module:

[0060] (1) The binocular vision perception module is used to construct visual point clouds and depth maps, and to identify the spatial pose and surface state of target objects. The binocular vision perception module is the core component of this system to realize three-dimensional environment perception and target localization. It is responsible for acquiring dense depth maps, spatial point clouds, and the spatial pose of the target. Its core work includes image acquisition, camera calibration, stereo matching, depth post-processing, and target localization. It also improves the accuracy and stability of spatial perception in complex dynamic scenes by fusing visual and multi-source state information. This module provides basic three-dimensional environmental information and target geometric description for subsequent spatial relationship calculation, teleoperation control, and obstacle avoidance decision-making. Specifically, it includes the following:

[0061] 1.1 Hardware Layout

[0062] This system employs two industrial-grade low-distortion cameras to form a fixed-baseline binocular system, mounted in a relatively stable position on the quadruped robot arm to ensure that the camera's field of view covers the interactive area. The baseline distance B of the binocular system is optimized based on the target operation distance, typically ranging from 60 to 120 mm, balancing depth accuracy and field of view.

[0063] Hardware synchronization is employed to ensure that both cameras acquire images simultaneously, avoiding matching errors caused by time differences. Simultaneously, the camera's exposure time is automatically adjusted based on ambient lighting conditions to guarantee appropriate brightness and contrast, thereby improving image quality.

[0064] 1.2 Camera Calibration

[0065] The intrinsic parameters of each camera are calibrated using a calibration board, and the intrinsic parameter matrix K of each camera is obtained:

[0066]

[0067] in and These are the focal lengths of the camera in the x and y directions, respectively. and These are the coordinates of the principal point of the image in the x and y directions, respectively. Intrinsic parameter calibration determines the internal geometry and optical properties of the camera, providing fundamental parameters for subsequent image correction and 3D reconstruction.

[0068] By simultaneously photographing the calibration board, the relative positional relationship between the left and right cameras is obtained, i.e., the extrinsic parameters (rotation matrix R and translation vector t). The extrinsic parameter calibration determines the geometric transformation relationship between the left and right images, enabling the mapping of points in the left image to the right image, or unifying points in the left and right images into a common coordinate system.

[0069] The calibrated intrinsic and extrinsic parameters are used to correct the left and right images, eliminating camera distortion and aligning the rows of the left and right images, thus simplifying the subsequent stereo matching process. The corrected images satisfy the epipolar constraint, meaning that corresponding points in the left and right images lie on the same horizontal line, greatly reducing the search range for stereo matching.

[0070] 1.3 Stereo Matching and Depth Calculation

[0071] Traditional industrial binocular vision systems often employ regular stereo matching methods such as SGBM and BM, which are prone to depth holes or disparity errors in areas lacking texture. Deep learning methods, on the other hand, possess stronger feature extraction and contextual understanding capabilities. This system introduces the DispNet stereo matching model based on an end-to-end convolutional neural network to generate high-precision dense disparity maps, replacing traditional cost aggregation + semi-global matching stereo algorithms based on prior assumptions. The model is fine-tuned and trained on image sets of industrial scenes (such as glossy metal equipment and monotonously grayscale walls) to enhance matching robustness in areas with weak texture and repetitive patterns. The system maintains depth estimation stability under different lighting and material conditions, improving overall mapping accuracy by more than 15%, meeting the spatial error tolerance range (<5mm) required for precise robotic arm operations (such as button pressing and cable plugging / unplugging).

[0072] The disparity of each pixel is calculated using a stereo matching algorithm. ,in These are the x-coordinates of corresponding points in the left and right cameras, respectively. Parallax reflects the difference in position of an object in the left and right images and is inversely proportional to the object's depth. Combining the parallax map and camera parameters, the depth Z of each pixel can be recovered using the following triangulation formula:

[0073]

[0074] Where f is the camera focal length, B is the binocular baseline (the physical distance between the two cameras), and d is the disparity value of the corresponding point. Furthermore, the pixels (u,v) in the image can be back-projected onto a three-dimensional coordinate system to obtain their spatial coordinates (X,Y,Z):

[0075]

[0076] The network is fine-tuned and trained on industrial environment image sets to improve its adaptability to special materials such as reflective metals and transparent glass. It is also deployed on edge computing devices (such as Jetson Xavier) through lightweight modifications, achieving an inference speed of up to 20 FPS, which meets the real-time requirements of remote operation scenarios.

[0077] 1.4 Post-processing of depth information

[0078] To improve the spatial continuity and geometric consistency of depth estimation, two post-processing methods, voxel filtering and RANSAC plane fitting, are employed for global sparsification and local structure optimization, respectively. Voxel filtering divides the 3D space into multiple voxel grids, downsamples the points within each voxel grid, and retains the center point or average value point within the grid, thereby reducing the number of point clouds and removing noise points.

[0079]

[0080] in This represents the i-th point within the voxel, and N is the number of points within that voxel.

[0081] For planar regions in the scene, the RANSAC algorithm is used to fit the plane equation, and then the points located on the plane are projected onto the plane to improve the depth accuracy of the planar region.

[0082] The YOLOv12 object detection network is integrated to locate interactive objects (such as buttons, surfaces, and controllers) in images. The object detection network can quickly and accurately identify target objects in an image and provide their bounding boxes. Combined with depth information, the specific position and pose of the target object in 3D space can be calculated. For example, for points within the bounding box, the distance between the target object and the camera can be obtained by calculating their average depth value; by analyzing the orientation and size of the bounding box, the pose of the target object can be estimated.

[0083] 1.5 Data Fusion and Output

[0084] In traditional systems, the vision module operates independently, lacking an understanding of the robot's own motion state, making it prone to image jitter and depth jumps when the robot jumps or turns. This module introduces IMU and six-dimensional force sensor data into the binocular vision system, using an extended Kalman filter (EKF) framework for time alignment and state estimation. The point cloud and target pose output by the vision module are jointly estimated with the robot's motion state inferred by the IMU, improving perception stability under dynamic conditions. Force information is used to determine contact occurrence and perform visual occlusion compensation.

[0085] The dense depth map, 3D point cloud, target candidate bounding box and its spatial position output by the binocular vision module are fused with the robotic arm's state information (such as joint angles and end effector pose) and input into the spatial relationship calculation module. By fusing visual information and robotic arm state information, the spatial positional relationship between the target object and the robotic arm's end effector can be calculated more accurately, providing a more precise basis for the robotic arm's motion control.

[0086] (2) The spatial relationship calculation module calculates the relative pose relationship between the target posture and the end state information of the robotic arm based on the target posture and the end state information of the robotic arm, and generates the trajectory parameters required for the operation task.

[0087] The spatial relationship calculation module aims to accurately infer the three-dimensional spatial positional relationship between the target object and the end effector of the robotic arm, providing pose reference and path basis for teleoperation control. Based on a multi-source information fusion strategy, this module combines binocular vision perception results, the current posture state of the robotic arm, and coordinate system transformation relationships to complete the entire calculation process from image target localization to end-effector spatial constraints.

[0088] First, the module receives the coordinates of the 3D target point provided by the binocular vision module. This refers to spatial coordinates, defined in the camera coordinate system. Using the known camera-robotic arm calibration relationship (i.e., the transformation matrix from the camera coordinate system to the robotic arm coordinate system), the coordinate transformation of the target point can be achieved. This process is based on the rigid body transformation formula:

[0089]

[0090] in, Let R be the homogeneous coordinates of the target point; R is the rotation matrix and t is the translation vector, which are usually obtained from extrinsic parameter calibration. This transformation allows the system to represent the target point in the robot arm's base system, thus enabling direct comparison with the current pose of the robot arm's end effector.

[0091] Secondly, in order to estimate the spatial relationship between the end effector and the target point, the system needs to acquire the position and attitude information of the robotic arm end effector in the base coordinate system in real time, denoted as and The forward kinematics modeling of the robotic arm employs the Denavit-Hartenberg (DH) parametric method, defining the coordinate system and transformation matrix for each joint, and constructing a homogeneous transformation chain from the base to the end effector. Position and orientation information are calculated using the robotic arm's forward kinematics model.

[0092]

[0093] Where n is the number of robotic arm joints. Let be the transformation matrix of the i-th joint. The parameter is the joint angle. The homogeneous transformation matrix from the end effector to the base can be obtained through forward kinematics. Thus, the end pose can be extracted. With direction.

[0094] Next, by comparing the spatial relative positions of the target point and the end effector... The system can determine whether the current target is within the end effector's operating range and design a pose adjustment strategy based on this vector. Furthermore, the module can also solve for the end effector posture that the robotic arm should achieve to reach the target point, i.e., inverse kinematics solution.

[0095]

[0096] in, This represents the function for solving inverse kinematics. This is the solution set for the joint angles of the robotic arm. During the solution process, the module needs to consider the feasibility of the solution, obstacle avoidance constraints, and control stability, employing techniques such as numerical optimization.

[0097] The spatial relationship calculation module also handles the relative positioning and time-varying updates of dynamic obstacles. It estimates the robot's motion trend by combining IMU data with continuous visual input to detect changes in the relative positions of targets or obstacles, and continuously updates the module. Vector and pose error form a real-time feedback mechanism, improving the stability and accuracy of control response.

[0098] In summary, the spatial relationship calculation module, through key technologies such as multi-source information fusion, coordinate transformation, and forward and inverse kinematics modeling, completes the spatial mapping from visual perception to actuator control, providing a high-precision geometric basis and motion control input for remote operation control and safe obstacle avoidance.

[0099] (3) The communication and time synchronization module is based on the high-frequency data transmission mechanism and the buffer synchronization mechanism to achieve low-latency and stable communication between the remote operation terminal and the robot terminal.

[0100] The communication and time synchronization module is responsible for coordinating data interaction and time consistency among multiple heterogeneous devices such as the vision system, force feedback unit, robotic arm controller, remote operation terminal, and IMU. It ensures the real-time, integrity, and synchronization of information transmission between modules, providing support for the stable operation and high-precision control of the entire control system.

[0101] The communication architecture employs a ROS2-based distributed communication system, utilizing the DDS (Data Distribution Service) middleware to achieve efficient data publishing and subscription between multiple modules. To reduce development complexity and improve compatibility, the underlying data transmission adopts a dual-protocol support mechanism of TCP / UDP: large volumes of information such as visual images and point clouds use UDP channels to improve transmission rates, while critical control signals and status feedback are transmitted reliably via TCP channels. All communication nodes use ROS2's QoS policy to manage communication quality, ensuring priority transmission of critical data such as remote operation control and force feedback responses.

[0102] To achieve time consistency within the control system, the Chrony + NTP time synchronization scheme is selected as the primary solution. A local NTP server runs on the main control computing unit, and all slave devices (including cameras, IMUs, force sensors, etc.) connect to this time source via LAN, periodically calibrating the local system clock to achieve millisecond-level time synchronization accuracy. In the vision module, local timestamps are added during image frame acquisition, and the data is aligned with the robotic arm's status data through the ROS2 time synchronization mechanism. When there are out-of-sync devices (such as some sensors that do not support system time alignment), the system uses a linear time interpolation method for time alignment compensation.

[0103] For data integration, the communication module embeds a time-synchronization buffer queue scheduler, which combines data from different sources according to timestamps and pushes them uniformly to the data fusion module. This scheduler performs sliding matching based on time windows, eliminating hysteresis frames with large errors to ensure that all sensor information entering the fusion calculation is within the same time period, supporting the synchronization and consistency of subsequent path planning and force feedback control.

[0104] (4) The teleoperation control module combines operator input and task requirements to perform position or impedance control, and automatically switches to pre-contact mode when the robot detects that the end effector is close to the target, predicting contact risks and adjusting the control strategy in advance. The teleoperation control module is the core component of the entire system to realize remote human-machine interaction control. It is mainly responsible for converting the operator's instructions into the motion behavior of the end effector in real time. This module not only needs to correctly understand the operator's control intention, but also needs to combine multi-source perception information in complex environments to dynamically adjust the motion path and end effector behavior of the robot arm to ensure operation accuracy, safety and response speed. Teleoperation control not only faces the requirement of operation accuracy, but also must overcome key difficulties such as network communication delay, environmental uncertainty and target position disturbance. Therefore, based on the traditional teleoperation method, this module proposes a closed-loop control strategy that integrates visual-force feedback and designs a predictive contact response mechanism to significantly improve the adaptability and stability of the teleoperation system in actual complex task scenarios. Specifically, it includes the following points:

[0105] 4.1 Teleoperation Input Analysis

[0106] In terms of teleoperation input parsing, the system is compatible with various mainstream human-machine interface input methods, including VR controllers, teach pendants, and graphical user interfaces. Before entering the control system, all input commands are first standardized into six-dimensional pose information of the target end effector through an instruction abstraction and conversion mechanism, including the target position. Target attitude and expected speed This forms a unified format for control command flow.

[0107]

[0108] Based on these standardized instructions and combined with real-time sensing results, the system drives the robotic arm's end effector to move towards the target. Meanwhile, to ensure the accuracy of remote teleoperation responses, all input instructions are accompanied by a timestamp. It also aligns with visual, force, and other sensor data through a time synchronization module to ensure the consistency of system perception and control in terms of time.

[0109] 4.2 End-point control mechanism for multimodal fusion

[0110] Compared to traditional teleoperation systems that rely solely on vision or preset paths, this module combines visual target posture recognition with real-time force-sensor contact monitoring. It adjusts control commands in real time to achieve a smooth transition from "approach-touch-hold," significantly reducing the probability of accidental touches and the risk of target damage. It includes the following three stages:

[0111] Coarse alignment control stage: The system initially obtains the position of the target object (such as a button or cable interface) in three-dimensional space through the binocular vision module. with posture Then, the initial path is solved using inverse kinematics:

[0112]

[0113] During this phase, the robotic arm moves quickly to the vicinity of the target, avoiding obstacles but not directly contacting the target.

[0114] Precision alignment control phase: When the distance between the end point and the target is less than 50mm from the set threshold, the system initiates visual closed-loop correction based on the target detection results of consecutive frames and the point cloud projection error.

[0115]

[0116] This error serves as the controller compensation input, enabling fine alignment adjustments between the end-effector position and the target, compensating for errors caused by the robot's own movement, vibration, or slight changes in the target's pose.

[0117] Contact control phase: When the system approaches the target and is about to make contact, the system activates the "soft contact mode," using a six-dimensional force sensor to monitor the contact force between the end effector and the environment. and with the set contact threshold In contrast, when the system detects that the end-effector force is gradually approaching a preset threshold, the controller dynamically reduces the end-effector propulsion speed and simultaneously controls the end-effector stiffness through an impedance control strategy to achieve compliant contact and prevent target damage. If the force exceeds the limit, propulsion is immediately interrupted, and an abnormal status is reported. The controller configuration is as follows:

[0118]

[0119] Compared to traditional teleoperation systems that rely solely on vision or fixed paths for contact operations, this module achieves precise and controllable contact judgment and response through a fusion of visual and force feedback. Its working principle is based on a hybrid control model, using visually estimated target positions for feedforward path control, while simultaneously utilizing information from force sensors as a feedback channel to adjust the control output. This ensures safe and reliable interaction even when position errors accumulate or the target surface structure is unclear.

[0120] 4.3 Predictive Force Control Response Mechanism under Network Latency

[0121] In remote teleoperation scenarios, due to unavoidable network communication delays, system responses typically lag behind the operator's actual control intentions. This is especially true when the robotic arm's end effector approaches a target surface and requires delicate actions (such as pressing, inserting, or grasping), which can easily lead to overshoot, collisions, or target loss. To address this, this system proposes a predictive contact response mechanism to reduce the risk of misoperation caused by control delays and improve operational stability and fault tolerance.

[0122] By modeling and predicting the impending contact state in advance, and proactively adjusting the control strategy before contact actually occurs, potential impacts or overshoots can be buffered. In specific implementation, when the system detects that the distance between the robotic arm's end effector and the target is less than 30mm (an adjustable threshold), the controller automatically switches to "pre-contact mode." In this mode, the system no longer relies entirely on operator commands to trigger deceleration, but instead introduces multi-source signals (visual distance, end effector speed, and six-dimensional force sensor disturbance signals) as input to perform real-time probabilistic modeling of the contact state.

[0123] The model for estimating the contact probability is as follows:

[0124]

[0125] Where v is the velocity of the terminal velocity along the target direction. This represents the rate of change of the relative distance between the endpoint and the target. This indicates the amplitude of the micro-perturbation in the force sensor. , , These are weighting coefficients. This is the Sigmoid activation function.

[0126] when When the force exceeds a set threshold, the controller will proactively adjust the robotic arm's control parameters, including reducing end-effector stiffness and switching to compliant control; limiting the maximum end-effector propulsion rate to avoid impact; and activating a micro-force feedback channel for the operator to sense impending contact. This contact risk buffering is completed before the operator explicitly issues a command, significantly reducing operational errors caused by communication delays. This mechanism possesses a degree of foresight and adaptability, enabling the system to maintain high operational stability even under conditions of low communication quality.

[0127] (5) The force feedback judgment module monitors the force sensor data at the robot's end in real time, forming force information. Based on a multi-threshold force recognition strategy, it determines whether the robot is in contact with the target and whether abnormal force changes have occurred. The end-effector force feedback judgment module is an important component of this system in ensuring the safety and accuracy of interaction during remote teleoperation tasks. Its core task is to identify in real time whether the end of the robotic arm is in physical contact with the target or obstacle, and trigger corresponding feedback adjustment and control strategies accordingly. Based on the three-dimensional target positioning information provided by the binocular vision perception module and the main control commands output by the teleoperation control module, this module is responsible for judging the authenticity and rationality of the interaction state, and is a key link in realizing closed-loop control of human-machine interaction. Specifically, it includes the following points:

[0128] 5.1 Dynamic Modeling Judgment

[0129] Traditional end-point contact detection relies heavily on a single force threshold strategy, which is difficult to adapt to the complex disturbance environment in dynamic teleoperation processes, such as: unrecognized light touches, false alarms, and buffer misjudgments, which seriously affect the accuracy and security of remote control.

[0130] This module constructs a dynamic contact determination model that integrates multi-source sensing data. It fuses multi-dimensional information based on visual spatial data provided by a six-dimensional force / torque sensor, an IMU inertial unit, and a binocular vision sensing module, and combines this with the robotic arm's end effector's own motion trend to comprehensively determine the contact state. The system collects the following data streams in each control cycle: real-time mechanical information output from the end effector force / torque sensor. Acceleration changes fed back by the IMU The binocular vision perception module provides the depth distance between the end effector and the target object. The teleoperation control module provides operator input actions (direction, speed). The system normalizes this information and inputs it into the contact judgment model, which then uses the following logic to determine whether a contact state has been entered:

[0131]

[0132] in, For dynamically changing thresholds, To accommodate visual distance tolerance, the system automatically adjusts parameters under different task modes to improve adaptability. It obtains contact status labels (contact / non-contact) and further identifies contact types (effective operation, accidental collision, hovering, etc.), providing a basis for realistic contact judgments for the obstacle avoidance and path planning modules, and providing force control adjustment signals for the remote operation control module.

[0133] 5.2 Contact Intent Identification and Consistency Assessment

[0134] Existing systems can only determine whether contact has occurred, but cannot distinguish the intent of the operation. For example, they cannot differentiate between "active pressing" and "accidental collision with an obstacle," or identify a "no-operation" state (such as no contact). This can lead to false feedback and misjudgment of actions.

[0135] This module introduces an operation behavior mapping mechanism, constructing "position-force" change curves for typical operation processes through extensive sampling. For example, pressing a button results in a continuous increase in displacement and a rapid, stable increase in force; accidentally touching an obstacle results in minimal displacement and a sudden spike in force; no operation results in a large displacement and approximately zero force. These curves are converted into standard maps, forming a template library. During real-time operation, the system calculates the distance or similarity between the current data trajectory and the template, uses a classifier for intent matching, and determines the current behavior type. The system can automatically identify various typical operation intentions, effectively distinguishing between proactive behavior and abnormal contact, improving the intelligent understanding capability of the teleoperation system, and enhancing operational accuracy and fault tolerance.

[0136] 5.3 Predictive force feedback adjustment

[0137] During remote operation, the "direction of the master controller command" and the "direction of the end effector force" often do not match. This is especially common in cases of communication delay, operation drift, and misoperation, which may lead to false feedback or system oscillation.

[0138] The system monitors the master-slave intent in real time during operation. Figure 1 Cosine similarity is calculated using the following formula:

[0139]

[0140] like If the force is below a set threshold, it indicates that the current direction of force is significantly inconsistent with the intended direction of operation. The system determines this as "unintended contact" or "false triggering" and activates the feedback suppression mechanism or safety shutdown. This significantly reduces "false feedback" during teleoperation, improves control accuracy and feedback consistency, and enhances the human-machine interaction experience.

[0141] 5.4 Predictive end-effector feedback regulation mechanism

[0142] In remote control scenarios, communication network latency can prevent the operator's stop command from being transmitted in a timely manner, easily leading to "force overshoot" or false triggering at the moment of actual contact. To address this, a predictive end-effector force feedback adjustment mechanism is designed in this module. This mechanism provides the operator with an early "tactile cue" by modeling the contact probability and modulating the feedback channel after the controller issues the action command but before the actual contact occurs.

[0143] This mechanism is based on the contact probability estimation results in the teleoperation control module. Combined with real-time six-dimensional force perturbation trend Based on the end-effector motion status, determine whether the system is currently in a "high-risk contact" state. Once the preset criteria are met ( The system will actively adjust the force feedback signal in the following ways:

[0144] Adjust the feedback channel bandwidth and sampling frequency to prioritize the transmission of critical disturbance information; dynamically increase the force feedback gain coefficient. This allows operators to sense "approaching resistance" even without actual contact; it limits the rate of change in feedback, smooths transient jumps, and avoids false alarms or oscillations. This strategy constructs a control-sensing bidirectional predictive closed-loop system: the remote operation control module is responsible for anti-collision and shock avoidance in advance along the control path, while the force feedback judgment module is responsible for buffering false sensing delays in advance along the sensing path. Working together, they significantly improve the consistency and stability of the system's operation in complex and unstable communication environments while ensuring the safety of remote operation.

[0145] (6) The obstacle avoidance and path planning module combines visual point cloud and force information to construct a dynamic occupancy map, and adopts a two-stage path planning and force dynamic compensation mechanism to realize sudden obstacle response and path replanning. The obstacle avoidance and path planning module aims to ensure the spatial safety of the robotic arm's end effector movement during remote operation, avoiding collisions with obstacles, objects being operated on, or human-machine interaction areas in the environment. The main function of this module is to dynamically construct a workspace model based on real-time perception data and operation commands, and generate a safe and feasible end effector trajectory on this basis. This module is a key link in realizing the "autonomous assistance + human-machine integration" control strategy, especially in narrow spaces, multi-object interference, or frequent changes in operation paths, ensuring the flexibility and safety of the system is crucial. Specifically, it includes the following points:

[0146] 6.1 Data Fusion and Environmental Mapping

[0147] To achieve dynamic obstacle avoidance capabilities, the system first integrates data streams from multiple sensors, including 3D point clouds or depth maps of obstacles provided by the binocular vision module, target pose sequences provided by the end-effector trajectory planning module, force anomaly points provided by the force feedback module (for dynamically adjusting path tolerance), and external map information synchronized by the communication module (such as global environment models and historical trajectories). By performing time synchronization and spatial registration on the above data, the system constructs a real-time updated 3D semantic occupancy grid map, spatially encoding obstacles, contact points, and operational targets within the region of interest. The occupancy probability is determined by a weighted average of perception confidence and historical contact information.

[0148] 6.2 Path Planning Process Design

[0149] Based on real-time mapping results, a two-stage path planning strategy is adopted to complete safe obstacle avoidance and path reconstruction, ensuring the accessibility, safety, and compliance of the trajectory.

[0150] Coarse path search: Utilizing Rapidly-exploring Random Tree (RRT), a collision-free path is searched in 3D space from the current end-effector pose to the target pose. The initial path only considers obstacle constraints and does not involve dynamics.

[0151] Fine-tuned trajectory optimization: Based on the initial path, the system introduces the CHOMP optimization method with multiple constraints to further optimize path smoothness, end-effector attitude continuity, and motion comfort. The objective function comprehensively considers the following factors:

[0152] Obstacle avoidance distance term: Maximize the Euclidean distance between the path and the obstacle point cloud;

[0153] Force safety term: Minimizes potential force mutations along the path;

[0154] Robotic arm constraints: Consider joint limits, speed, and acceleration limits;

[0155] Task tracking item: Minimize deviation from the original operation trajectory within a controllable range.

[0156] 6.3 Force Feedback-Guided Dynamic Obstacle Avoidance Adjustment Mechanism

[0157] Conventional visual obstacle avoidance systems can only identify visible obstacles. They are slow to react to sudden human figures, soft objects, or occluded areas during operation, especially in confined spaces. The system path often relies on global planning and cannot be adjusted in real time, easily leading to forced advancement, jamming, or accidental collisions of the robotic arm. To address the problems of slow response to "sudden interference objects" and inconsistency with operator intentions in traditional path planning systems, this module designs a path replanning mechanism based on force feedback guidance. The innovation of this mechanism lies in: by monitoring the force change trend and path tracking error at the robotic arm's end effector in real time, it determines whether there are unmapped obstacles on the path. Once the determination is successful, local path replanning is immediately triggered, achieving dynamic obstacle avoidance.

[0158] The system monitors the rate of change of force at the end in real time. Path tracking deviation Based on the environmental point cloud update frequency, a triggering factor is constructed:

[0159]

[0160] when If there are interference obstacles or sudden environmental changes outside the path, local trajectory replanning is immediately performed. A new path segment with the minimum cost is searched at the current state point and spliced ​​onto the original trajectory to achieve a seamless transition.

[0161] It has a higher responsiveness to sudden contact, avoiding delays and misjudgments; and ensures alignment with the operator's intentions. Figure 1 Consistency is ensured to prevent the system from forcibly modifying the trajectory; when point cloud updates are insufficient, force sensing is used to compensate for obstacle avoidance, enhancing redundancy and safety.

[0162] 6.4 Active Soft Obstacle Reasoning Mechanism

[0163] This module also introduces a "soft obstacle volume reasoning mechanism" based on force state recognition and historical trajectory inversion, which is used to establish an obstacle avoidance "volume shadow zone" for non-rigid, invisible but perceptible objects (such as curtains, cushions, and human hands).

[0164] The system maintains a force time series within the control cycle. Calculate the average force growth rate over a continuous period of time. If the following conditions are met:

[0165] The force rises slowly ;

[0166] The propulsion speed approaches zero;

[0167] The missing rate of depth data in the path-direction visual image is >90%.

[0168] At this point, the system detects a continuous but slowly increasing force at the end of the path, accompanied by difficulties in path progression. Combined with a lack of visual occlusion information, it automatically determines that the current path may be occupied by soft obstacles. The system automatically generates a local "virtual volume" region (such as a sphere, ellipsoid, or cube), setting the occupancy probability of this region to 1. In the next path optimization, this region is forcibly avoided, achieving avoidance of invisible soft objects.

[0169] (7) The task execution and feedback module controls the robotic arm to execute specific operation instructions, collects status and abnormal information and feeds it back to the operator, forming a task closed loop. The task execution and feedback module is located at the end of the system's execution chain, directly controlling the robotic arm's action output. At the same time, it links with sensors, controllers and human-machine interface to build a complete operation closed loop, which is the core supporting part for realizing system controllability, safety and interaction efficiency.

[0170] This module first receives task commands from the teleoperation control module. Task types include: end-effector pose points, trajectory sequence, end-effector operation commands (such as gripping, releasing, and insertion / removal), and control parameters (such as compliance settings and force threshold limits). After parsing, the task commands are converted into low-level control commands and transmitted in real-time to the robotic arm controller and end-effector via an industrial bus (such as CAN or EtherCAT), ensuring high-speed and stable command transmission. This module employs a non-blocking command management mechanism, supporting parallel queuing of multiple tasks and dynamic insertion, cancellation, or adjustment, improving task scheduling flexibility and system response speed.

[0171] During task execution, the system simultaneously collects the robotic arm's motion status, end-effector data, and controller feedback information to evaluate the effectiveness of the current operation. This includes joint angles and end-effector posture, six-dimensional force-torque values, task completion status indicators, path deviation information, and real-time force control offset. This data is not only used internally by the system to determine the success of the operation but also provided to the operator in real-time through a visual interface, offering clear task execution status prompts and enhancing operational transparency and trust.

[0172] When the system detects task anomalies during execution, such as the end effector failing to complete the operation according to the predetermined trajectory, abnormal force control feedback (e.g., touching an unexpected object), unsuccessful target recognition, or the robotic arm becoming physically unreachable, an anomaly warning mechanism will be automatically triggered. The module's built-in anomaly recognition rule base can categorize and process typical faults, automatically interrupting the current task or switching to protection mode, while simultaneously pushing the anomaly type, occurrence time, and relevant data to the human-machine interface. Operators can then choose manual intervention measures such as "retry," "cancel," or "switch task mode" based on system prompts, achieving human-machine collaborative decision-making.

[0173] To enhance the intelligent linkage capabilities of the entire system, this module also supports transmitting execution results back to upstream modules for strategy updates by the path planning module or teleoperation control module, forming a true closed-loop control logic of task execution, status feedback, and strategy optimization. The system can also record successful and failed samples during multiple executions, providing a data foundation for future operation learning and model fine-tuning.

[0174] This invention also provides a method for force feedback teleoperation control of the end effector of a quadruped robot arm, which is used in the force feedback teleoperation control system of the quadruped robot arm. The specific implementation process is as follows:

[0175] Task Issuance and Inspection Point Navigation: The host task control center sends inspection instructions to the robot dog through the task scheduling module. The instructions include the spatial location or semantic tags of multiple task points (such as switch buttons, thermometers, pressure gauges, etc.). The robot dog autonomously navigates to the vicinity of the task point based on its built-in SLAM system, maintaining a stable standing posture to provide a basis for the robotic arm's operation.

[0176] Target recognition and localization: Upon reaching the designated inspection area, the robotic arm activates its binocular vision perception module to acquire image data of the current work area and obtain 3D point cloud information of the scene through binocular parallax calculation. Subsequently, the semantic recognition and target localization module identifies target objects (such as red buttons, circular thermometers, etc.) based on deep learning methods, extracts the target's spatial pose information (position + attitude), and calculates its surface normal vector and spatial topological relationships by combining point cloud depth estimation. This step provides crucial input for path planning and operation strategy formulation.

[0177] Path Generation and Obstacle Avoidance Planning: Once the target pose is determined, the system initiates the obstacle avoidance and path planning modules. In the perception layer, the system integrates obstacle point clouds from the vision module and contact or collision history information provided by the force feedback module to construct a semantic occupancy grid map. The path planning process consists of two stages: the first stage uses a fast search algorithm based on RRT to generate a coarse, collision-free path from the current end position to the target point; the second stage uses the CHOMP optimizer to fine-tune the path, comprehensively considering factors such as obstacle avoidance, posture continuity, target trajectory accuracy, motion comfort, and joint constraints, outputting a smooth, safe, and controllable operational trajectory. When the path planning module identifies potential soft obstacles or insufficient point cloud occlusion, it also proactively activates a "soft obstacle inference mechanism," utilizing force sensing trends to construct a virtual obstacle avoidance shadow zone, enabling path detours for non-rigid bodies.

[0178] End-effector control and contact prediction adjustment: During trajectory execution, when the robotic arm's end-effector approaches the target surface (distance <30mm), the system activates a predictive contact control mechanism. This mechanism integrates end-effector velocity, target surface orientation, and force sensor perturbation signals, and performs real-time estimation based on a contact probability model.

[0179]

[0180] Where v is the velocity of the terminal velocity along the target direction. This represents the rate of change of the relative distance between the endpoint and the target. This indicates the amplitude of the micro-perturbation in the force sensor. , , These are weighting coefficients. The activation function is Sigmoid. If the contact probability exceeds the threshold, the controller actively reduces the end effector stiffness, limits the propulsion rate, and completes the contact buffer ahead of time, thereby effectively avoiding collisions or overshoot caused by time delay.

[0181] Remote operation command-driven and human-machine interface control: During the operation, the operator issues remote operation control commands through the master control terminal. The master-slave mapping module maps the master input to the slave motion, while simultaneously integrating sensor force feedback to compare the consistency of master and slave states. If the system detects that the actual force does not match the operator's intention, it initiates force feedback-driven control adjustment, dynamically modifying the end-effector path or posture to achieve closed-loop control in a human-machine collaborative manner, improving operational flexibility and safety. If abnormal force occurs during remote operation (such as being unable to push, jamming, sudden change, etc.), the system immediately triggers local path replanning or automatic withdrawal commands to ensure the safety of both the equipment and the target.

[0182] Task Execution and Feedback: After the robotic arm completes the designated operation (such as pressing a button or the temperature probe making stable contact with the target), the end effector uses state machine logic to determine whether the task is completed and feeds back the operation status, task success or failure, end effector force information, etc., to the control terminal. If the task fails, the system will provide detailed error codes (such as path failure, unstable contact, abnormal force threshold, etc.) for the operator to confirm and re-initiate the command or switch the control mode.

[0183] Task Loop and Reset: Upon completion of the current task, the system controller automatically switches to the reset path, controlling the robotic arm to retract to its initial safe position. Simultaneously, the task list is updated, and the task flow for the next inspection target point begins. This process is executed cyclically throughout the entire inspection cycle, supporting the continuous completion of multiple targets and batch tasks.

[0184] Compared with the prior art, the present invention has the following beneficial effects:

[0185] 1. Significantly improve the robustness and safety of end-effector interaction judgment: By introducing a multi-source sensing fusion contact modeling mechanism, combined with force-position relationship modeling and operation intention recognition methods, the system can accurately judge the physical contact state between the end of the robotic arm and the environment, distinguish between "effective contact" and "false contact interference". Compared with the traditional fixed threshold judgment method, it has higher anti-interference ability and discrimination accuracy, and effectively reduces the false triggering rate.

[0186] 2. Achieving predictive response and buffer protection in remote force control scenarios: To address the control lag problem caused by network latency, the system designs a predictive contact response mechanism. By combining force change trends, visual depth, and end-effector dynamic states to construct a contact probability model, the system switches to compliant control in advance and adjusts stiffness and speed limits to reduce collision risks and ensure operational stability in high-latency scenarios.

[0187] 3. Enhance obstacle avoidance and path adaptation capabilities in dynamic environments: Based on point cloud mapping, force feedback change and trajectory deviation joint judgment mechanism, the system realizes dynamic identification and path replanning of "non-visual visible objects" such as sudden soft obstacles and occluded areas. It has the ability to infer the volume of soft obstacles and effectively avoids the path misjudgment problem caused by perception blind spots or delays in traditional obstacle avoidance systems.

[0188] 4. Supports autonomous assistance and human-machine integrated path generation strategies: In terms of path planning, a phased coarse planning + fine trajectory optimization process is adopted. Combined with force control safety items and original task tracking constraints, a path generation mechanism that balances task achievement and compliant safety is realized, meeting the adaptability and flexibility requirements of high-complexity scenarios.

[0189] Enhance the system's decision-making closed-loop capability in variable operational tasks: Through the task execution and feedback module, the system supports unified collection and feedback of execution status (success / failure), real-time force feedback, and abnormal status, which helps to form a closed-loop collaboration mechanism between the operator and the system, providing support for subsequent human-machine consensus decision-making and improving the overall task execution efficiency and safety controllability.

[0190] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. Various modifications and improvements can be made by those skilled in the art without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. A force feedback teleoperation control system for the end effector of a quadruped robot arm, characterized in that, include: Binocular vision perception module, spatial relationship calculation module, communication and time synchronization module, teleoperation control module, force feedback judgment module, obstacle avoidance and path planning module, task execution and feedback module: The binocular vision perception module is used to construct visual point clouds and depth maps to identify the spatial pose and surface state of target objects. The spatial relationship calculation module calculates the relative pose relationship between the target posture and the end effector state information of the robotic arm based on the target posture and the end effector state information, and generates the trajectory parameters required for the operation task; The communication and time synchronization module is based on a high-frequency data transmission mechanism and a buffer synchronization mechanism to achieve low-latency and stable communication between the remote operation terminal and the robot terminal. The teleoperation control module combines operator input and task requirements to perform position or impedance control, and automatically switches to pre-contact mode when the robot detects that the end effector is close to the target, predicting contact risks and adjusting the control strategy in advance. The force feedback judgment module monitors the force sensor data at the robot's end in real time to form force information, and judges whether it has contacted the target or whether abnormal force changes have occurred based on a multi-threshold force recognition strategy. The obstacle avoidance and path planning module combines visual point cloud and force information to construct a dynamic occupancy map, and adopts a two-stage path planning and force dynamic compensation mechanism to realize sudden obstacle response and path replanning. The task execution and feedback module controls the robotic arm to execute specific operation instructions, collects status and abnormal information and feeds it back to the operator, forming a task closed loop.

2. The force feedback teleoperation control system for the end effector of a quadruped robot arm according to claim 1, characterized in that, The pre-contact mode includes: When the distance between the end effector of the robotic arm and the target is less than a preset threshold, the control system automatically enters the pre-contact mode. A contact probability estimation model is constructed using the terminal velocity, target depth variation trend, and force sensor disturbance signal. When the probability of contact exceeds the threshold, the control system automatically reduces the control stiffness, limits the propulsion speed, and activates the compliant control strategy.

3. The force feedback teleoperation control system for the end effector of a quadruped robot arm according to claim 1, characterized in that, The force feedback judgment module includes: An active contact event recognition mechanism determines whether contact has occurred based on the rate of change of force and direction. The predictive feedback mechanism generates tactile feedback proactively before the operator perceives it by fitting the force trend. The force channel calibration mechanism corrects the operation path based on tactile information when sensor malfunctions or control drift occurs.

4. The force feedback teleoperation control system for the end effector of a quadruped robot arm according to claim 1, characterized in that, The obstacle avoidance and path planning module includes: The primary path planning stage uses RRT to quickly search for collision-free paths, while the secondary path optimization stage introduces the CHOMP algorithm to optimize path smoothness, obstacle avoidance distance, and attitude continuity. When the force feedback judgment module detects an abnormal change in the force at the robot's end and the vision does not update in time, a local path reconstruction mechanism is triggered. A soft obstacle volume reasoning mechanism is constructed. By using the continuous and slowly increasing force trend and visual occlusion judgment, a virtual obstacle avoidance shadow volume is established to bypass invisible obstacles.

5. The force feedback teleoperation control system for the end effector of a quadruped robot arm according to claim 1, characterized in that, The communication and time synchronization module specifically includes: The underlying data transmission adopts a TCP / UDP dual-protocol support mechanism: large amounts of information such as visual images and point clouds use UDP channels to improve the transmission rate; all communication nodes use ROS2's QoS policy to manage communication quality and ensure the priority transmission of key data for teleoperation control and force feedback response. A local NTP server runs on the main control computing unit of the control system. The camera, IMU, and force sensor are all connected to the time source via LAN. The local system clock is periodically calibrated to achieve millisecond-level time synchronization accuracy. In the vision module, a local timestamp is added when the image frame is acquired, and it is aligned with the robotic arm status data through the ROS2 time synchronization mechanism.

6. The force feedback teleoperation control system for the end effector of a quadruped robot arm according to claim 1, characterized in that, The teleoperation control module includes real-time probability modeling of the contact state, and the contact probability estimation model is as follows: ； Where v is the velocity of the terminal velocity along the target direction. This represents the rate of change of the relative distance between the endpoint and the target. This indicates the amplitude of the micro-perturbation in the force sensor. , , These are weighting coefficients. Use the Sigmoid activation function; when When the value exceeds the set threshold, the controller will actively adjust the control parameters of the robotic arm, including reducing the end effector stiffness and switching to compliant control; limiting the maximum end effector propulsion rate to avoid impact; and activating the micro-force feedback channel for the operator to sense that contact is about to occur.

7. The force feedback teleoperation control system for the end effector of a quadruped robot arm according to claim 6, characterized in that, The teleoperation control module also includes a predictive end-effector force feedback adjustment mechanism, specifically comprising: Based on the contact probability estimation results in the teleoperation control module Combined with the rate of change of end force Based on the end-effector motion status, determine whether the system is currently in a "high-risk contact" state. Once the preset criteria are met: The control system will actively adjust the force feedback signal in the following ways: Adjust the feedback channel bandwidth and sampling frequency to prioritize the transmission of critical disturbance information; dynamically increase the force feedback gain coefficient. This allows the operator to feel the approaching resistance even without actual contact; Limit the rate of change of feedback, smooth transient jump signals, and avoid false alarms or oscillations.

8. The force feedback teleoperation control system for the end effector of a quadruped robot arm according to claim 1, characterized in that, The obstacle avoidance and path planning module includes a dynamic obstacle avoidance adjustment mechanism: Real-time monitoring of the rate of change of force at the end Path tracking deviation Combined with the environmental point cloud update frequency f map Construct triggering factors: ； when , The threshold changes dynamically. If there are interference obstacles or sudden environmental changes outside the path, local trajectory replanning is immediately performed. A new path segment with the minimum cost is searched at the current state point and spliced ​​onto the original trajectory to achieve a seamless transition.

9. The force feedback teleoperation control system for the end effector of a quadruped robot arm according to claim 1, characterized in that, The obstacle avoidance and path planning module also includes a soft obstacle volume inference mechanism: Maintain a force time series within the control period. Calculate the average force growth rate over a continuous period of time. If all three of the following conditions are met: (1) The force rises slowly ; (2) The propulsion speed is zero; (3) The missing rate of depth data in the path-direction visual image is >90%; To accommodate visual distance tolerance, and given that the control system detects continuous but slowly rising end force, accompanied by difficulties in path advancement, and considering the lack of visual occlusion information, it automatically determines that the current path may be occupied by soft obstacles.

10. A method for force feedback teleoperation control of the end effector of a quadruped robot arm, characterized in that, The method is used in the force feedback teleoperation control system of the end effector of any of the quadruped robot manipulators according to claims 1 to 9, and the method includes: Task assignment and inspection point navigation; Upon reaching the designated inspection area, the robotic arm activates the binocular vision perception module to collect image data of the current work area and obtains the scene's 3D point cloud information through binocular parallax calculation. Once the target pose is determined, the system activates the obstacle avoidance and path planning module; during trajectory execution, when the distance between the robotic arm end and the target surface is less than 30mm, it automatically switches to pre-contact mode; During the contact process, the operator issues remote operation control commands through the master control terminal; the master-slave mapping module maps the input from the master terminal to the motion of the slave terminal, and at the same time integrates the force feedback from the sensor to compare the consistency of the master and slave states; Once the current task is completed, the system controller automatically switches to the reset path, controls the robotic arm to retract to the initial safe position, updates the task list, and enters the task flow for the next inspection target point.

Images