Four-foot robot mechanical arm tail end force feedback teleoperation control system and method

Abstract

The invention belongs to the technical field of robot control, particularly provides a force feedback teleoperation control system and method for the tail end of a mechanical arm of a quadruped robot, and aims at the key challenges that control errors are caused by communication time delay and soft obstacles are difficult to recognize in a dynamic environment. Modeling and judgment are conducted on the contact state of the tail end of the mechanical arm in advance, and feedforward control and buffer adjustment oriented to communication time delay are achieved. And meanwhile, a dynamic semantic map is constructed in combination with multi-source sensing information, and soft obstacle reasoning and path optimization are performed by fusing a tail end force sense change trend, so that the recognition and avoidance capabilities of the system in a complex and invisible obstacle environment are remarkably improved. The system has good perspectiveness, self-adaptability and high redundancy safety characteristics, is suitable for multi-task inspection operation of industrial sites such as a thermal power plant, and is especially suitable for a remote man-machine cooperative operation scene in a narrow space.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of robot control, in particular to a four-legged robot mechanical arm end force feedback teleoperation control system and method. BACKGROUND

[0002] Currently, industrial sites such as power, petrochemical, thermal power plants and the like increasingly rely on robots to perform remote inspection operations in dangerous environments. Four-legged robots, due to their good terrain adaptability, have become an important carrier for complex scene inspection. However, when the existing four-legged robots carry mechanical arms for remote teleoperation, the following problems generally exist: Most of the existing systems use fixed threshold method to judge the end contact, that is, when the force or torque detected by the sensor exceeds the set value, it is judged as a contact state. However, in the dynamic operation process, due to the vibration of the mechanical arm, the elastic deformation of the flexible structure or the influence of sensor noise, false positives or false negatives are easily generated, resulting in invalidation of operation instructions or false triggering of safety mechanisms. Traditional teleoperation systems rely on direct mapping relationship and cannot judge the consistency between operation intention and actual feedback, which leads to the system being unable to respond or correct in time when false contact or unexpected contact occurs, thereby affecting the accuracy of human-machine collaboration and operation experience. In the remote environment, the communication link has unpredictable delay, especially for long-distance teleoperation tasks across network segments and regions. This delay seriously affects the synchronicity of force feedback and control instructions, and easily leads to system oscillation, end impact and even task failure. Current obstacle avoidance mechanisms are mostly based on visual point cloud or depth map, which can only identify explicit obstacles and cannot effectively perceive soft, transparent or partially occluded objects, which is easy to collide in complex space. In addition, visual mapping and path reconstruction have a delay and lack of timeliness. The existing path planning is mostly in full autonomous mode, ignoring the collaborative mechanism of operator input and system feedback, and cannot flexibly switch between active and passive control strategies according to task needs, nor can it dynamically adjust the trajectory to respond to unexpected obstacles or disturbances during execution.

[0003] Therefore, it is urgent to build an intelligent system integrating "force feedback teleoperation + mechanical arm collaborative obstacle avoidance + ontology navigation obstacle avoidance + multi-modal interaction" to improve the safety and precision of remote inspection operations. SUMMARY

[0004] The present disclosure aims to at least solve one of the technical problems existing in the prior art, and proposes a four-legged robot mechanical arm end force feedback teleoperation control system and method.

[0005] In a first aspect, the present disclosure provides a four-legged robot mechanical arm end force feedback teleoperation control system, comprising: The binocular vision perception module, the spatial relationship solving module, the communication and time synchronization module, the teleoperation control module, the force feedback judgment module, the obstacle avoidance and path planning module, and the task execution and feedback module. The binocular vision perception module is configured to construct a visual point cloud and a depth map, and identify a spatial pose and a surface state of a target object. The spatial relationship solving module is configured to calculate a relative pose relationship between a target pose and an end state of a robot arm based on the target pose and the end state of the robot arm, and generate trajectory parameters required for an operation task. The communication and time synchronization module is configured to realize low-delay stable communication between a remote operation end and a robot end based on a high-frequency data transmission mechanism and a buffer synchronization mechanism. The teleoperation control module is configured to combine operator input and task requirements to perform position or impedance control, and automatically switch to a pre-contact mode when the robot detects that the end is close to a target, predict contact risks and adjust control strategies in advance. The force feedback judgment module is configured to monitor robot end force sensor data in real time to form force sensation information, and judge whether the target is contacted and whether abnormal force changes occur based on a multi-threshold force recognition strategy. The obstacle avoidance and path planning module is configured to construct a dynamic occupancy map in combination with the visual point cloud and the force sensation information, and realize burst obstacle response and path re-planning by adopting a two-stage path planning and force dynamic compensation mechanism. The task execution and feedback module is configured to control the robot arm to execute specific operation instructions, collect state and abnormal information, and feed back to the operator to form a task closed loop.

[0006] Preferably, the pre-contact mode comprises: When the distance between the end of the robot arm and the target is less than a preset threshold, the control system automatically enters the pre-contact mode. A contact probability estimation model is constructed using end velocity, target depth change trend, and force sensor disturbance signal. When the contact probability exceeds a threshold, the control system automatically reduces control stiffness, limits propulsion speed, and activates a compliant control strategy.

[0007] Preferably, the force feedback judgment module comprises: An active contact event recognition mechanism judges whether a contact occurs based on force change rate and direction. A predictive feedback mechanism generates haptic feedback in advance before the operator perceives by fitting force trends. A force channel calibration mechanism corrects the operation path based on haptic information when sensor abnormalities or control drifts occur.

[0008] Preferably, the obstacle avoidance and path planning module comprises: The primary path planning adopts RRT to quickly search for a collision-free path, and the CHOMP algorithm is introduced in the secondary path optimization stage to optimize path smoothness, obstacle avoidance distance, and attitude continuity. When the force feedback judgment module detects an abnormal mutation in the force on the robot end and the vision is not updated in time, the local path reconstruction mechanism is triggered. A soft obstacle volume inference mechanism is constructed to establish a virtual obstacle avoidance shadow volume to bypass invisible obstacles through continuous and slow rising force trends and visual occlusion judgments.

[0009] Preferably, the communication and time synchronization module specifically includes: The underlying data transmission adopts a TCP / UDP dual protocol support mechanism: UDP channels are used for large data volume information such as visual images and point clouds to improve transmission speed; all communication nodes use the QoS strategy of ROS2 to manage communication quality, ensuring priority transmission of key data for teleoperation control and force feedback response; A local NTP server is run on the main control unit of the control system, and the camera, IMU, and force sensor are connected to this time source through LAN, periodically calibrating the local system clock to achieve millisecond-level time synchronization accuracy; in the vision module, the image frame is captured with a local timestamp, and the time synchronization mechanism of ROS2 is used to align the data with the state of the robot arm.

[0010] Preferably, the teleoperation control module includes real-time probability modeling of the contact state, and the contact probability estimation model is as follows:

[0011] where v is the speed of the end along the target direction, represents the relative distance change rate between the end and the target, represents the perturbation amplitude in the force sensor, , , is a weighting coefficient, is a Sigmoid activation function; When is higher than the set threshold, the controller will actively adjust the control parameters of the robot arm, including reducing the end stiffness, switching to a compliant control state; limiting the maximum propulsion rate of the end to avoid impact; activating the micro-force feedback channel for the operator to perceive that contact is about to occur.

[0012] Preferably, the teleoperation control module further includes a predictive end force feedback adjustment mechanism, specifically including: Based on the contact probability estimation results in the teleoperation control module , combined with the real-time six-dimensional force disturbance trend And the end motion state, judge whether the system is currently in the "contact high risk" state, once the preset criteria are met: , the control system will actively adjust the force feedback signal in the following way: Adjust the feedback channel bandwidth and sampling frequency, preferentially transmit key disturbance information; dynamically improve the force feedback gain coefficient , so that the operator can feel the resistance approaching even if he has not actually contacted it; Limit the feedback rate to smooth the transient jump signal and avoid false alarms or oscillation.

[0013] Preferably, the obstacle avoidance and path planning module includes a dynamic obstacle avoidance adjustment mechanism: Real-time monitoring of end force rate And the path tracking deviation , combined with the environment point cloud update frequency, to construct a trigger factor:

[0014] When , it is judged that there is an interference obstacle or environmental mutation outside the path, and local trajectory re-planning is immediately performed to search for a new path segment with the minimum cost at the current state point and splice it to the original trajectory to achieve seamless transition.

[0015] Preferably, the obstacle avoidance and path planning module further includes a soft obstacle volume reasoning mechanism: Maintain a force time series within the control cycle , calculate the average force growth rate in the continuous period , if the following three conditions are met simultaneously: (1) Force slowly rises ; (2) Propulsion speed is zero; (3) The depth data missing rate of the path direction visual image is > 90%; And at this time, the control system detects the continuous but slowly rising end force, and is accompanied by path propulsion difficulty, combined with the lack of visual occlusion information, it is automatically judged that the current path may be occupied by a soft obstacle.

[0016] The application also provides a four-legged robot manipulator end force feedback teleoperation control method, which is used in the four-legged robot manipulator end force feedback teleoperation control system, and the method comprises: Task issuing and inspection point navigation When reaching the specified inspection area, the manipulator starts the binocular vision perception module, collects image data of the current work area, and obtains scene three-dimensional point cloud information through binocular parallax calculation; Once the target pose is determined, the system starts the obstacle avoidance and path planning module; during trajectory execution, when the distance between the end of the mechanical arm and the target surface is <30mm, it automatically switches to the pre-contact mode; During contact, the operator issues teleoperation control instructions through the master control end; the master-slave mapping module maps the master end input to the slave end motion, while fusing sensor force feedback to compare the consistency of the master-slave state; After the current task is completed, the system controller automatically switches to the reset path to control the mechanical arm to retreat to the initial safe position, and updates the task list to enter the task process of the next inspection target point.

[0017] Advantages:

[0018] The present application aims to solve the problems of insufficient remote teleoperation contact precision, large control delay and poor obstacle avoidance robustness in the prior art. In view of the key challenges of control error caused by communication delay and soft obstacle difficult to identify in dynamic environment, a comprehensive control system and method integrating perception, solution, control, feedback and obstacle avoidance functions are proposed. The system introduces a predictive contact response mechanism to model and judge the contact state of the end of the mechanical arm in advance, realizing feedforward control and buffer adjustment for communication delay. At the same time, a dynamic semantic map is constructed combining multi-source sensing information, and the end force sensation change trend is fused to infer soft obstacles and optimize the path, thereby significantly improving the identification and avoidance ability of the system in complex and invisible obstacle environment. The present application has good foresight, adaptability and high redundancy safety characteristics, and is suitable for multi-task inspection operation in industrial sites such as thermal power plants, and is particularly suitable for remote human-machine collaborative operation scenes in small spaces. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A four-legged robot mechanical arm end force feedback teleoperation control system architecture diagram is provided for the embodiments of the present disclosure. DETAILED DESCRIPTION

[0020] To enable those skilled in the art to better understand the technical solutions of the present disclosure, the present disclosure is further described in detail below in conjunction with the drawings and specific embodiments.

[0021] As shown in Figure 1 A four-legged robot mechanical arm end force feedback teleoperation control system includes: Binocular vision perception module, spatial relationship solution module, communication and time synchronization module, teleoperation control module, force feedback judgment module, obstacle avoidance and path planning module, task execution and feedback module: (1) Binocular vision perception module is used to construct a visual point cloud and a depth map, and to identify the spatial pose and surface state of a target object. The binocular vision perception module is a core component of the system for realizing three-dimensional environment perception and target positioning, and is responsible for obtaining a dense depth map, a spatial point cloud, and a spatial pose of an operating target. The core work includes image acquisition, camera calibration, stereo matching, depth post-processing, and target positioning, and through the fusion of vision and multi-source state information, the spatial perception accuracy and stability in a complex dynamic scene are improved. The module provides basic three-dimensional environment information and target geometric description for subsequent spatial relationship solving, teleoperation control, and obstacle avoidance decision-making. Specifically, the following points are included: 1.1 Hardware arrangement The system adopts two industrial-grade low-distortion cameras to form a fixed-baseline binocular system, which is installed in a relatively stable position of the four-legged robot manipulator, ensuring that the camera field of view covers the interaction area. The design of the baseline distance B of the binocular system is optimized according to the target operating distance, with a value range of generally 60-120 mm, balancing the depth precision and field of view range.

[0022] The hardware synchronization method is used to ensure that the two cameras simultaneously capture images, avoiding matching errors caused by time differences. At the same time, the exposure time of the camera is automatically adjusted according to the environmental lighting conditions to ensure that the image brightness and contrast are moderate, improving the image quality.

[0023] 1.2 Camera calibration The calibration board is used to calibrate the intrinsic parameters of each camera, obtaining the intrinsic matrix K of each camera:

[0024] wherein and are the focal lengths of the camera in the x and y directions, and are the coordinates of the image principal point in the x and y directions. Intrinsic calibration determines the geometric and optical characteristics of the camera, providing basic parameters for subsequent image correction and three-dimensional reconstruction.

[0025] By simultaneously shooting the calibration board, the relative position relationship between the left and right cameras, i.e., the extrinsic parameters (rotation matrix R and translation vector t), is obtained. Extrinsic calibration determines the geometric transformation relationship between the left and right images, enabling the mapping of points in the left image to the right image or the unification of points in the left and right images to a common coordinate system.

[0026] The intrinsic and extrinsic parameters obtained by calibration are used to correct the left and right images, eliminating the distortion of the camera, aligning the rows of the left and right images, and simplifying the subsequent stereo matching process. The corrected images satisfy the epipolar constraint, i.e., corresponding points in the left and right images are located on the same horizontal line, greatly reducing the search range of stereo matching.

[0027] 1.3 Stereo matching and depth calculation Traditional industrial binocular vision systems mostly use regular stereo matching methods such as SGBM and BM, which are prone to produce depth holes or parallax errors in texture missing areas. Deep learning methods have stronger feature extraction and context understanding capabilities. The system introduces a DispNet stereo matching model based on an end-to-end convolutional neural network to generate high-precision dense disparity maps, replacing traditional cost aggregation + semi-global matching and other stereo algorithms based on prior assumptions. The model is fine-tuned on image sets of industrial scenes (such as shiny metal equipment and gray monotone walls) to enhance the matching robustness in weak texture and repetitive pattern areas. The system maintains depth estimation stability under different lighting and materials, improving overall mapping accuracy by more than 15%, meeting the spatial error tolerance range (<5mm) required for fine manipulation of robotic arms (such as button pressing and cable plugging).

[0028] The disparity of each pixel is calculated by a stereo matching algorithm where xL and xR are the horizontal coordinates of the corresponding points in the left and right cameras, respectively. Disparity reflects the difference in the positions of objects in left and right images, and is inversely proportional to the depth of the object. Combined with the disparity map and camera parameters, the depth Z of each pixel can be recovered by the following triangulation formula:

[0029] where f is the camera focal length, B is the binocular baseline (the physical distance between the two cameras), and d is the disparity value of the corresponding point. Further, the pixel point (u, v) in the image can be back-projected to the three-dimensional space coordinate system to obtain its spatial coordinates (X, Y, Z):

[0030] The network is fine-tuned on industrial environment image sets to improve its adaptability to special materials such as shiny metal and transparent glass. It is also deployed on edge computing devices (such as Jetson Xavier) through lightweight modification, with an inference speed of up to 20FPS, meeting the real-time requirements of remote teleoperation scenarios.

[0031] 1.4 Depth information post-processing To improve the spatial continuity and geometric consistency of depth estimation, two post-processing methods, voxel filtering and RANSAC plane fitting, are used for global sparsification and local structure optimization, respectively. Voxel filtering divides the three-dimensional space into multiple voxel grids, downsamples the points within each voxel grid, and retains the center point or average value point, thereby reducing the number of point clouds and removing noise points.

[0032] ​

[0033] wherein represents the i-th point within the voxel, and N is the number of points within the voxel.

[0034] For planar regions in the scene, the RANSAC algorithm is used to fit a plane equation, and then the points lying on the plane are projected onto the plane, improving the depth accuracy of the planar region.

[0035] The integrated target detection network YOLOv12 is used to locate interactive objects (such as buttons, surfaces, controllers, etc.) in the image. The target detection network can quickly and accurately identify target objects in the image and give the bounding box of the target object. Combined with depth information, the specific position and attitude of the target object in three-dimensional space can be calculated. For example, for the points within the bounding box, the average depth value can be calculated to obtain the distance between the target object and the camera; by analyzing the direction and size of the bounding box, the attitude of the target object can be estimated.

[0036] 1.5 Data fusion and output In traditional systems, the vision module runs independently and lacks understanding of the robot's own motion state, which can easily cause image jitter and depth jumps when the robot jumps or turns. In this module, IMU and six-axis force sensor data are introduced into the binocular vision system, and an extended Kalman filter (EKF) framework is used for time alignment and state estimation. The point cloud and target pose output by the vision module are jointly estimated with the robot motion state estimated by the IMU, improving the stability of perception under dynamic conditions. Force sensing information is used to determine the occurrence of contact and to compensate for visual occlusion.

[0037] The dense depth map, three-dimensional point cloud, target candidate box, and its spatial position information output by the binocular vision module are fused with the robot state information (such as joint angle, end pose, etc.) and input into the spatial relationship solving module. By fusing vision information and robot state information, the spatial position relationship between the target object and the robot end can be more accurately calculated, providing a more accurate basis for robot motion control.

[0038] (2) The spatial relationship solving module calculates the relative pose relationship between the target attitude and the robot end state information based on the target attitude and the robot end state information, and generates the trajectory parameters required for the operation task.

[0039] The spatial relationship solving module aims to accurately infer the three-dimensional spatial position relationship between the target object and the robot end effector, providing pose reference and path basis for teleoperation control. Based on the multi-source information fusion strategy, the module combines the binocular vision perception results, the current attitude state of the robot, and the coordinate system transformation relationship to complete the whole process of solving from image target positioning to end spatial constraint.

[0040] Firstly, the module receives the three-dimensional target point coordinates provided by the binocular vision module , i.e. the spatial coordinates, which are defined in the camera coordinate system. With the known camera-robot calibration relationship (i.e. the transformation matrix from the camera coordinate system to the robot coordinate system), the coordinate conversion of the target point coordinates can be realized. This process is based on the rigid body transformation formula:

[0041] wherein, is the homogeneous coordinates of the target point; the rotation matrix R and the translation vector t are usually obtained by external parameter calibration. This conversion enables the system to express the target point in the base coordinate system of the robot, thereby directly comparing it with the current pose of the robot end.

[0042] Secondly, in order to estimate the spatial relationship between the end effector and the target point, the system needs to obtain the position and attitude information of the robot end in the base coordinate system in real time, denoted as and . The forward kinematics modeling of the robot uses the Denavit-Hartenberg (D-H) parameter method to define the coordinate system and transformation matrix of each joint, and to construct the homogeneous transformation chain from the base to the end, and the position and attitude information is calculated through the forward kinematics model of the robot:

[0043] wherein n is the number of joints of the robot, is the transformation matrix of the i-th joint, and the parameter is the joint angle. Through forward kinematics, the homogeneous transformation matrix from the end to the base can be obtained , so as to extract the end pose and direction.

[0044] Next, by comparing the spatial relative position of the target point and the end effector , the system can determine whether the current target is within the operating range of the end, and design a pose adjustment strategy based on the vector. In addition, the module can also solve the end pose that the robot should reach to reach the target point position, i.e. the inverse kinematics solution:

[0045] wherein, denotes the inverse kinematics solution function, is the solution set of the robot joint angle. In the solving process, the module needs to consider the feasibility of the solution, obstacle avoidance constraints and control stability, and uses techniques such as numerical optimization to solve it.

[0046] The spatial relationship solving module also undertakes the task of relative positioning and time-varying updating of dynamic obstacles. By combining the IMU data to estimate the motion trend of the robot and the continuous visual input to detect the relative position changes of the target or obstacles, the relative position of the target or obstacles can be continuously updated The vector and the pose error form a real-time feedback mechanism, which improves the stability and accuracy of the control response.

[0047] In summary, the spatial relationship solving module completes the spatial mapping between visual perception and actuator control through multi-source information fusion, coordinate transformation, forward and inverse kinematics modeling, etc., and provides high-precision geometric basis and motion control input for teleoperation control and safe obstacle avoidance.

[0048] (3) The communication and time synchronization module is based on a high-frequency data transmission mechanism and a buffer synchronization mechanism to realize low-latency stable communication between the remote operation end and the robot end.

[0049] The communication and time synchronization module is responsible for coordinating the data interaction and time consistency between the vision system, force feedback unit, robot controller, teleoperation terminal, IMU and other heterogeneous devices, ensuring the real-time, integrity and synchronization of information transmission between modules, and providing support for stable operation and high-precision control of the entire control system.

[0050] The communication architecture adopts a distributed communication system based on ROS2, which realizes efficient data publishing and subscription between multiple modules through DDS (Data Distribution Service) middleware. To reduce development complexity and improve compatibility, the underlying data transmission adopts a TCP / UDP dual protocol support mechanism: large data volume information such as vision images and point clouds uses UDP channel to improve transmission rate, and key control signals and state feedback are transmitted through TCP channel to ensure reliable transmission. All communication nodes use the QoS strategy of ROS2 to manage communication quality, ensuring priority transmission of key data such as teleoperation control and force feedback response.

[0051] To achieve time consistency within the control system, Chrony + NTP time synchronization scheme is selected as the main scheme, and a local NTP server is run on the master control unit. All slave devices (including cameras, IMUs, force sensors, etc.) access this time source through LAN, periodically calibrate the local system clock, and achieve millisecond-level time synchronization accuracy. In the vision module, the image frame is captured with a local timestamp, and the time synchronization mechanism of ROS2 is used to align the data with the robot state. When there are devices that are not synchronized (such as some sensors that do not support system time alignment), the system uses linear time interpolation method for time alignment compensation.

[0052] In terms of data integration, a time synchronization buffer queue scheduler is embedded in the communication module to combine data from different sources according to timestamps and push them to the data fusion module. The scheduler performs sliding matching according to the time window, eliminates the hysteresis frame with large error, and ensures that all sensing information entering the fusion calculation is within the same time period, supporting the synchronization consistency of subsequent path planning and force feedback control.

[0053] (4) The teleoperation control module combines operator input and task requirements to perform position or impedance control, and automatically switches to a pre-contact mode when the robot detects that the end is approaching the target, predicts contact risks and adjusts the control strategy in advance. The teleoperation control module is the core component of the entire system for remote human-computer interaction control, and is mainly responsible for converting the operator's instructions into real-time motion behavior of the robot end. This module not only correctly understands the operator's control intention, but also dynamically adjusts the motion path and end behavior of the robot in combination with multi-source sensing information in complex environments to ensure operation accuracy, safety and response speed. Teleoperation control not only faces the requirement of operation accuracy, but also must overcome key difficulties such as network communication delay, environmental uncertainty and target position disturbance, therefore, based on the traditional teleoperation method, this module proposes a closed-loop control strategy that integrates visual-force feedback, and designs a predictive contact response mechanism to significantly improve the adaptability and stability of the teleoperation system in actual complex task scenarios. Specifically, the following points are included: 4.1 Teleoperation input analysis In terms of teleoperation input analysis, the system is compatible with multiple mainstream human-computer interaction input methods, including VR controllers, teach pendants, and graphical interaction interfaces. All input instructions are first standardized into six-dimensional pose information of the target end, including target position , target attitude and expected speed , through the instruction abstraction conversion mechanism before entering the control system, forming a unified format control command stream:

[0054] The system drives the robot end to move towards the target according to these standardized instructions combined with real-time sensing results. At the same time, in order to ensure the response accuracy of remote teleoperation, all input instructions are attached with timestamps and aligned with visual, force sensing and other sensing data through the time synchronization module to ensure the time consistency of system sensing and control.

[0055] 4.2 Multi-modal fusion end control mechanism Compared with traditional teleoperation systems that rely only on vision or preset paths, this module combines visual target pose recognition + force real-time contact monitoring to adjust control instructions in real time, achieving a soft transition of "approach - touch - hold" and significantly reducing the probability of accidental touch and the risk of target damage. It includes the following three stages: Coarse positioning control stage: the system initially obtains the position of the target object (such as a button or cable interface) in three-dimensional space through the binocular vision module and pose , and then solves the initial path using inverse kinematics:

[0056] In this stage, the robot quickly moves to the target adjacent area, avoiding obstacles but not directly touching the target.

[0057] Fine positioning control stage: when the end of the robot is within 50 mm of the target, the system starts visual closed-loop correction based on the target detection results of consecutive frames and the point cloud projection error:

[0058] This error is used as a controller compensation input to fine-tune the end position and target. It compensates for errors caused by the robot's own movement, vibration, or slight changes in the target's pose.

[0059] Contact control stage: when the system approaches the target and is about to make contact, the system activates the "soft contact mode" to monitor the contact force between the end of the robot and the environment through a six-axis force sensor and compares it with the set contact threshold When the system detects that the end force is gradually approaching the preset threshold, the controller dynamically reduces the end pushing speed and controls the end stiffness through an impedance control strategy to achieve a soft contact and prevent damage to the target. If the force exceeds the limit, the pushing is immediately interrupted and an abnormal state is fed back. The controller is as follows:

[0060] Compared with traditional teleoperation systems that rely only on vision or fixed paths to perform contact operations, this module achieves fine and controllable contact judgment and response through the fusion of visual and force feedback. Its working principle is based on a hybrid control model that uses visual estimates of target position for feedforward path control and force sensor feedback information as a feedback channel to adjust control output. This allows safe and reliable interaction even when position errors accumulate or the target surface structure is unclear.

[0061] 4.3 Predictive force control response mechanism under network latency In remote teleoperation scenarios, due to the inevitable delay in network communication, the system response usually lags behind the actual control intention of the operator, especially when the end of the manipulator approaches the target surface and needs to perform fine actions (such as pressing, plugging, grabbing), it is easy to cause overshoot, collision or target loss and other problems. For this reason, the system proposes a predictive contact response mechanism to reduce the risk of misoperation caused by control delay and improve operation stability and fault tolerance.

[0062] By modeling and judging the state of impending contact in advance, and actively adjusting the control strategy before the contact actually occurs, the potential impact or overshoot is buffered in advance. In specific implementation, when the system detects that the distance between the end of the manipulator and the target is less than 30mm (adjustable threshold), the controller automatically switches to the "pre-contact mode". In this mode, the system no longer relies on the operator's command to trigger deceleration, but introduces multi-source signals (visual distance, end speed, six-axis force sensor disturbance signal) as input to model the contact state in real time.

[0063] The estimation model of contact probability is as follows:

[0064] where v is the speed of the end along the target direction, represents the relative distance change rate between the end and the target, represents the disturbance amplitude in the force sensor, , , is a weighting coefficient, is a Sigmoid activation function.

[0065] When is higher than the set threshold, the controller will actively adjust the control parameters of the manipulator, including reducing the end stiffness, switching to a compliant control state; limiting the maximum propulsion rate of the end to avoid impact; activating the micro-force feedback channel for the operator to perceive that contact is about to occur. The contact risk buffer is completed before the operator explicitly issues a command, significantly reducing the misoperation problem caused by communication delay. This mechanism has a certain foresight and adaptability, so that the system can maintain high operation stability even under low communication quality conditions.

[0066] (5) Force feedback judgment module monitors the robot end-effector force sensor data in real time, forms force sense information, and judges whether the target is contacted and whether the abnormal force changes based on multi-threshold force recognition strategy. The end-effector force feedback judgment module is an important part of the system to ensure the safety and accuracy of interaction in remote teleoperation tasks. The core task is to identify whether the mechanical arm end-effector is in physical contact with the target or obstacles in real time, and trigger the corresponding feedback adjustment and control strategy based on this. Based on the three-dimensional target positioning information provided by the binocular vision perception module and the master control instruction output by the teleoperation control module, this module is responsible for judging the authenticity and rationality of the interaction state, which is a key link to realize the human-machine interaction closed-loop control. Specifically, it includes the following points: 5.1 Dynamic modeling judgment Traditional end-effector contact judgment relies on a single force threshold strategy, which is difficult to adapt to complex disturbance environments in dynamic teleoperation processes, such as light touch misidentification, false collision, and buffer misjudgment, which seriously affects the accuracy and safety of remote control.

[0067] In this module, a dynamic contact determination model is constructed by integrating multi-source perception data. Based on the six-dimensional force / torque sensor, the IMU inertial unit, and the visual spatial data provided by the binocular vision perception module, multi-dimensional information fusion is performed, combined with the motion trend of the mechanical arm end-effector, to complete the comprehensive judgment of the contact state. The system collects the following data streams in each control period: real-time mechanical information output by the end-effector force / torque sensor ; acceleration changes feedback by IMU ; depth distance between the end-effector and the target object provided by the binocular vision perception module ; operator input actions (direction, speed) provided by the teleoperation control module ; The system normalizes these information and inputs into the contact judgment model to determine whether to enter the contact state through the following logic:

[0068] wherein, is the dynamic change threshold, is the visual distance tolerance. The system automatically adjusts the parameters in different task modes to improve adaptability. The contact state label (contact / non-contact) is obtained, and the contact type (effective operation, collision, suspension, etc.) is further identified to provide real contact basis for the obstacle avoidance and path planning module, and to provide force control adjustment signals for the teleoperation control module.

[0069] 5.2 Contact intention recognition and consistency evaluation Existing systems can only determine whether to contact, but cannot distinguish operation intentions. For example, distinguish between "active pressing" and "collision with obstacles", or identify "empty operation" (such as no contact) state. This will cause feedback miscommunication and behavior misjudgment.

[0070] This module introduces the operation behavior atlas mechanism. A large number of samples are used to construct the "position-force" change curve of the typical operation process. For example, the displacement continuously increases and the force accelerates upward and stabilizes when a button is pressed. The displacement is very small and the force instantaneously rises when an obstacle is accidentally touched. The displacement is relatively large and the force is approximately zero when the operation is empty. These curves are converted into standard atlases to form a template library. In real-time operation, the system calculates the distance or similarity between the current data trajectory and the template, uses a classifier to match the intention, and judges the current behavior type. The system can automatically identify various typical operation intentions, effectively distinguish between active behavior and abnormal contact, improve the intelligent understanding ability of the teleoperation system, and improve the operation accuracy and fault tolerance.

[0071] 5.3 Predictive force feedback adjustment During teleoperation, the "master finger command direction" and the "end force direction" are often inconsistent, especially in communication delay, operation drift, and misoperation, which can lead to misfeedback or system oscillation.

[0072] The system monitors the master-slave intention consistency in real time during operation, and calculates the cosine similarity by the following formula: Figure 1

[0073] If is lower than the set threshold, it means that the current force direction and the operation intention direction are obviously inconsistent, and the system determines that it is "unintended contact" or "misoperation", and triggers the feedback suppression mechanism or safety shutdown. This significantly reduces the "misfeedback" in teleoperation, improves the control accuracy and feedback consistency, and improves the system human-machine interaction experience.

[0074] 5.4 Predictive end force feedback adjustment mechanism In the remote control scene, the communication network delay makes the stop command issued by the operator unable to be transmitted in time, and "force overshoot" or misoperation is likely to occur at the actual contact moment. Therefore, a predictive end force feedback adjustment mechanism is designed in this module to provide an early "tactile prompt" to the operator before the actual contact occurs after the controller issues an action instruction through contact probability modeling and feedback channel modulation.

[0075] This mechanism is based on the contact probability estimation results in the teleoperation control module , combined with the real-time six-dimensional force disturbance trend and the end motion state, to determine whether the system is currently in a "high risk of contact" state. Once the preset criteria are met , the system will actively adjust the force feedback signal in the following ways: ​Adjust the feedback channel bandwidth and sampling frequency, prioritize the transmission of critical disturbance information; dynamically increase the force feedback gain coefficient Make the operator feel the "resistance approaching" even if there is no actual contact; limit the feedback rate of change, smooth the transient jump signal, and avoid false alarms or oscillation. This strategy builds a control-sensing two-way predictive closed-loop system: the teleoperation control module is responsible for collision avoidance and shock avoidance in the control path in advance, and the force feedback judgment module is responsible for buffering false perception and delay in the perception path in advance. The two work together to ensure the safety of remote operation while significantly improving the consistency and stability of the system in complex and unstable communication environments.

[0076] (6) The obstacle avoidance and path planning module combines visual point cloud and force information to build a dynamic occupancy map, and uses a two-stage path planning and force dynamic compensation mechanism to achieve sudden obstacle response and path replanning. The obstacle avoidance and path planning module aims to ensure the spatial safety of the robot arm's end-effector during teleoperation, avoiding collisions with obstacles, objects, or human-machine interaction areas in the environment. The main function of this module is to dynamically build a workspace model based on real-time perception data and operation instructions, and generate a safe and feasible end-effector trajectory based on this model. This module is a key part of the "autonomous assistance + human-machine integration" control strategy, especially in narrow spaces, multiple object interference, or frequent changes in operation path scenarios, and is crucial to the flexibility and safety of the system. Specifically, it includes the following points: 6.1 Data fusion and environment mapping To achieve dynamic obstacle avoidance, the system first integrates data streams from multiple sensors, including obstacle three-dimensional point cloud or depth map provided by the binocular vision module, target pose sequence provided by the end-effector trajectory planning module, force abnormal point (used for dynamic adjustment of path tolerance) provided by the force feedback module, and external map information (such as global environment model, historical trajectory, etc.) synchronized by the communication module. Through time synchronization and spatial registration of the above data, the system constructs a real-time updated three-dimensional semantic occupancy grid map, which spatially encodes obstacles, contact points, and operation targets in the region of interest. The occupancy probability is determined by the perception confidence and historical contact information.

[0077] 6.2 Path planning process design Based on the real-time mapping results, a two-stage path planning strategy is used to complete safe obstacle avoidance and path reconstruction, ensuring the reachability, safety, and flexibility of the trajectory: Coarse path search: use RRT (Rapidly-exploring Random Tree) to search for a collision-free path from the current end-effector pose to the target pose in 3D space. The preliminary path only considers obstacle constraints and does not involve dynamics.

[0078] Fine trajectory optimization: Based on the preliminary path, the system introduces the optimization method CHOMP with multiple constraints to further optimize the path smoothness, end pose continuity and motion comfort. The objective function considers the following factors: Obstacle avoidance distance term: maximize the Euclidean distance between the path and the obstacle point cloud; Force safety term: minimize potential force discontinuity on the path; Arm constraint term: consider joint limits, speed, and acceleration limits; Task tracking term: minimize deviation from the original operation trajectory within a controllable range.

[0079] 6.3 Dynamic obstacle avoidance adjustment mechanism based on force feedback guidance The conventional visual obstacle avoidance system can only identify visible obstacles. The reaction to sudden appearance of human bodies, soft objects or occluded areas during operation is lagging, especially in narrow scenes. The system path often relies on global planning and cannot be adjusted in real time, which may cause the robot to forcefully advance, get stuck or accidentally touch. To solve the problem of slow response of traditional path planning system to "sudden interference objects" and inconsistency with the operator's intention, this module designs a path re-planning mechanism based on force feedback guidance. The innovation of this mechanism lies in: by monitoring the force change trend of the robot end and the path tracking error in real time, it is determined whether there is an uncharted obstacle on the path. Once it is determined, local path re-planning is triggered immediately to achieve dynamic obstacle avoidance.

[0080] The system monitors the end force change rate in real time and the path tracking deviation , combined with the environment point cloud update frequency, to construct the trigger factor:

[0081] When , it is determined that there is an interference obstacle outside the path or an environmental mutation, and local trajectory re-planning is immediately performed to search for a new path segment with the minimum cost at the current state point and splice it to the original trajectory to achieve seamless transition.

[0082] It has higher responsiveness to sudden contact and avoids lagging and misjudgment; it ensures consistency with the operator's intention and avoids the system forcibly modifying the trajectory; it supports relying on force sensation to compensate for obstacle avoidance when the point cloud is not updated enough, and enhances the redundancy safety. Figure 1

[0083] 6.4 Active soft obstacle reasoning mechanism This module also introduces a "soft obstacle volume reasoning mechanism" based on force state recognition and historical trajectory inversion, which is used to establish an obstacle "volume shadow zone" for non-rigid, invisible but perceptible objects (such as curtains, soft cushions, hands, etc.).

[0084] ​The system maintains a force time series in the control cycle , calculates the average force growth rate in the continuous period , if the following conditions are met: The force slowly rises ; The propulsion speed tends to zero; The path direction visual image depth data loss rate is > 90%.

[0085] At this time, the system detects the continuous but slowly rising end force, and is accompanied by path propulsion difficulty, combined with visual occlusion information loss, and automatically judges that the current path may be occupied by soft obstacles. The system automatically generates a local "virtual volume" area (such as a sphere, ellipsoid or cube), and sets the occupancy probability of the area to 1. In the next path optimization, the area is forced to avoid, realizing the avoidance of invisible soft objects.

[0086] (7) The task execution and feedback module controls the robot arm to execute specific operation instructions, collects state and abnormal information and feeds back to the operator, forming a task closed loop. The task execution and feedback module is located at the end of the system's execution chain, directly controls the motion output of the robot arm, and at the same time links with sensors, controllers and human-computer interaction interface, builds a complete operation closed loop, and is the core support part to realize the controllability, safety and interaction efficiency of the system.

[0087] This module first receives the task instructions output from the teleoperation control module, and the task types include: end pose point, trajectory action sequence, end operation command (such as clamping, releasing, plugging) and control parameters (such as compliance setting, force threshold limit, etc.). After the task instructions are analyzed and converted into low-level control commands, they are transmitted in real time to the robot arm controller and end effector through the industrial bus (such as CAN or EtherCAT), ensuring the high speed and stable transmission of the instructions. This module uses a non-blocking command management mechanism, which can support multiple tasks in parallel queuing, dynamic insertion, cancellation or adjustment, improving the flexibility of task scheduling and the response speed of the system.

[0088] During task execution, the system synchronously collects the motion state of the robot arm, the end sensor data and the controller feedback information, which are used to evaluate the execution effect of the current operation. Including joint angle and end attitude, six-dimensional force-torque value, task completion state identifier, path deviation information and real-time force control offset, etc. The above data are not only used for the system to judge whether the operation is successful, but also fed back to the operator in real time through the visual interface, providing clear task execution state prompt to improve the operation transparency and trust.

[0089] When the system detects task exceptions during execution, such as end failure to complete operation according to the predetermined trajectory, force control feedback exception (such as touching unexpected objects), target unrecognizable success, mechanical arm trapped in physically unreachable state, etc., the abnormal prompt mechanism will be automatically triggered. The built-in exception recognition rule library in the module can classify and process typical faults, and automatically interrupt the current task or switch to the protection mode, and at the same time, the exception type, occurrence time and related data are pushed to the man-machine interface. The operator can choose "retry", "withdraw" or "switch task mode" and other manual intervention measures based on the system prompt to realize man-machine collaborative decision-making.

[0090] To improve the intelligent linkage capability of the whole system, the module also supports returning the execution result to the upstream module for strategy update of the path planning module or the teleoperation control module, to form a truly closed-loop control logic of task execution-state feedback-strategy optimization. The system can also record successful and failed samples in multiple executions to provide a data basis for future operation learning and model fine-tuning.

[0091] The embodiment of the application also provides a four-legged robot mechanical arm end force feedback teleoperation control method, which is used on the four-legged robot mechanical arm end force feedback teleoperation control system. The specific implementation process is as follows: Task issuing and inspection point navigation: the upper task control center sends inspection instructions to the robot dog through the task scheduling module, and the content includes the spatial position or semantic label of multiple task points (such as switch buttons, thermometers, pressure gauges, etc.). The robot dog autonomously navigates to the vicinity of the task point based on the built-in SLAM system, maintains a stable standing posture, and provides a work basis for the mechanical arm.

[0092] Target recognition and positioning: when reaching the specified inspection area, the mechanical arm starts the binocular vision perception module, collects image data of the current work area, and obtains scene three-dimensional point cloud information through binocular parallax calculation. Then, the semantic recognition and target positioning module identifies target objects (such as red buttons, circular thermometers, etc.) based on a deep learning method, extracts the spatial pose information (position + attitude) of the target, and calculates the surface normal vector and spatial topological relationship thereof in combination with point cloud depth estimation. This step provides key input for path planning and operation strategy formulation.

[0093] Path generation and obstacle avoidance planning: Once the target pose is determined, the system starts the obstacle avoidance and path planning module. In the perception and mapping layer, the system fuses the obstacle point cloud from the vision module and the contact or collision history information provided by the force feedback module to construct a semantic occupancy grid map. The path planning process is divided into two stages: the first stage uses a fast search algorithm based on RRT to generate a rough collision-free path from the current end position to the target point; the second stage uses a CHOMP optimizer to fine-tune the path, considering factors such as obstacle avoidance, pose continuity, task target trajectory accuracy, motion comfort, and joint constraints, to output a smooth, safe, and controllable operation trajectory. When the path planning module identifies potential soft obstacles or insufficient point cloud coverage, it will also actively start the "soft obstacle reasoning mechanism" to construct a virtual obstacle shadow zone using force sensor change trends, achieving path detouring for non-rigid bodies.

[0094] End control and contact prediction adjustment: During trajectory execution, when the robot arm end approaches the target surface (distance < 30mm), the system starts the predictive contact control mechanism. This mechanism fuses end velocity, target surface direction, and force sensor perturbation signal, and performs real-time estimation based on the contact probability model:

[0095] where v is the end velocity along the target direction, represents the relative distance change rate between the end and the target, represents the perturbation amplitude in the force sensor, , , is the weighting coefficient, is the Sigmoid activation function. If the contact probability exceeds the threshold, the controller actively reduces the end stiffness, limits the propulsion rate, and completes the contact buffer in advance, effectively avoiding the impact or overshoot problem caused by time delay.

[0096] Teleoperation instruction driving and human-machine fusion control: During contact, the operator issues teleoperation control instructions through the master control end. The master-slave mapping module maps the master end input to the slave motion, while fusing sensor force feedback for master-slave state consistency. If the system detects that the actual force does not match the operator's intention, it will start force feedback dominant control adjustment, dynamically modify the end path or pose, achieve human-machine collaborative closed-loop control, and improve operation flexibility and safety. If an abnormal force occurs during teleoperation (such as pushing, sticking, sudden change, etc.), the system will immediately trigger local path re-planning or automatic withdrawal instructions to ensure the safety of the equipment and the target.

[0097] Task execution and feedback: After the robot arm completes the specified operation (such as pressing a button or the temperature probe stably contacting the target), the end controller determines whether the task is completed through state machine logic and feeds back the operation status, task success or failure, end force information, etc. to the control end. If the task fails, the system will provide detailed exception codes (such as path failure, unstable contact, force threshold exception, etc.) for the operator to confirm and reissue instructions or switch control modes.

[0098] Task cycle and reset: After the current task is completed, the system controller automatically switches to the reset path to control the robot arm to withdraw to the initial safe position, and updates the task list to enter the task flow of the next inspection target point. This process is executed in a loop throughout the inspection cycle, supporting the continuity of multiple target and batch tasks.

[0099] Compared with the prior art, the present application has the following beneficial effects: 1. Significantly improve the robustness and safety of end interaction judgment: By introducing a multi-source perception fusion contact modeling mechanism, combined with force-position relationship modeling and operation intention recognition method, the system can accurately judge the physical contact state between the robot arm end and the environment, distinguish "effective contact" and "accidental interference", and has higher anti-interference ability and discrimination accuracy compared with traditional fixed threshold judgment method, effectively reducing the false trigger rate.

[0100] 2. Realize predictive response and buffer protection in remote force control scene: To solve the control lag problem caused by network delay, the system designs a predictive contact response mechanism, combines force change trend, visual depth and end dynamic state to build a contact probability model, switches to soft control in advance and adjusts stiffness and speed limit, reduces collision risk, and ensures operation stability in high delay scenarios.

[0101] 3. Enhance obstacle avoidance and path adaptation ability in dynamic environment: Based on the joint judgment mechanism of point cloud mapping, force feedback change and trajectory deviation, the system realizes dynamic identification and path re-planning of "non-visual dominant objects" such as sudden soft obstacles and blocked areas, has the ability to infer the volume of soft obstacles, and effectively avoids the path misjudgment problem caused by perception dead angle or delay in traditional obstacle avoidance systems.

[0102] 4. Support autonomous assistance and human-machine integrated path generation strategy: In terms of path planning, a phased coarse planning + fine trajectory optimization process is adopted, combined with force control safety items and original task tracking constraints, a path generation mechanism that takes into account task achievement and soft safety is realized, meeting the adaptability and flexibility in high complexity scenarios.

[0103] Improve the system's closed-loop decision-making capabilities in variable operational tasks: Through the task execution and feedback module, the system supports the unified collection and feedback of execution status (success / failure), real-time force feedback, and abnormal status. This helps to form an operator-system closed-loop collaboration mechanism, provides support for subsequent human-machine consensus decision-making, and improves overall task execution efficiency and safety controllability.

[0104] It is understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present disclosure, and the present disclosure is not limited thereto. Those skilled in the art may make various modifications and improvements without departing from the spirit and substance of the present disclosure, and such modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A force feedback teleoperation control system for the end of a quadruped robot manipulator arm, characterized in that: include: Binocular vision perception module, spatial relationship solution module, communication and time synchronization module, remote operation control module, force feedback judgment module, obstacle avoidance and path planning module, task execution and feedback module: The binocular vision perception module is used to construct a visual point cloud and a depth map to identify the spatial posture and surface state of the target object; The spatial relationship solving module calculates the relative posture relationship between the target posture and the state information of the end of the manipulator based on the target posture and generates the trajectory parameters required for the operation task; The communication and time synchronization module is based on a high-frequency data transmission mechanism and a buffer synchronization mechanism to achieve low-latency and stable communication between the remote operation terminal and the robot terminal; The teleoperation control module combines operator input with task requirements to perform position or impedance control, and automatically switches to pre-contact mode when the robot detects that the end is approaching the target, predicting contact risks and adjusting the control strategy in advance; The force feedback judgment module monitors the robot's end force sensor data in real time to form force information, and judges whether the target is in contact and whether abnormal force changes occur based on a multi-threshold force recognition strategy; The obstacle avoidance and path planning module combines visual point cloud and force information to construct a dynamic occupancy map, and adopts a two-stage path planning and force dynamic compensation mechanism to achieve sudden obstacle response and path replanning; The task execution and feedback module controls the robotic arm to execute specific operation instructions, collects status and abnormal information and feeds it back to the operator, forming a task closed loop.

2. The quadruped robot manipulator arm end force feedback teleoperation control system according to claim 1, characterized in that: The pre-contact mode includes: When the distance between the end of the robotic arm and the target is less than a preset threshold, the control system automatically enters the pre-contact mode; A contact probability estimation model is constructed using the terminal velocity, target depth variation trend, and force sensor disturbance signal; When the contact probability exceeds the threshold, the control system automatically reduces the control stiffness, limits the propulsion speed and activates the compliant control strategy.

3. The quadruped robot manipulator arm end force feedback teleoperation control system according to claim 1, characterized in that: The force feedback judgment module includes: Active contact event recognition mechanism, which determines whether contact occurs based on the force change rate and direction; Predictive feedback mechanism, which proactively generates tactile feedback before the operator perceives it by fitting force trends; The force channel calibration mechanism corrects the operation path based on tactile information when sensor abnormalities or control drift occur.

4. The quadruped robot manipulator arm end force feedback teleoperation control system according to claim 1, characterized in that: The obstacle avoidance and path planning module includes: The primary path planning uses RRT to quickly search for collision-free paths, and the secondary path optimization stage introduces the CHOMP algorithm to optimize path smoothness, obstacle avoidance distance, and posture continuity; When the force feedback judgment module detects an abnormal sudden change in the force on the robot end and the vision is not updated in time, the local path reconstruction mechanism is triggered; A soft obstacle volume inference mechanism is constructed, and a virtual obstacle avoidance shadow volume is established through the continuous and slowly rising force trend and visual occlusion judgment to bypass invisible obstacles.

5. The quadruped robot manipulator arm end force feedback teleoperation control system according to claim 1, characterized in that: The communication and time synchronization module specifically includes: The underlying data transmission uses a TCP / UDP dual-protocol support mechanism: large amounts of visual image and point cloud information use UDP channels to increase transmission rates; all communication nodes use ROS2's QoS policy to manage communication quality, ensuring priority transmission of critical data such as teleoperation control and force feedback response; A local NTP server runs on the control system's main computing unit. The camera, IMU, and force sensor are all connected to this time source via the LAN. The local system clock is periodically calibrated to achieve millisecond-level time synchronization accuracy. In the vision module, a local timestamp is inserted when the image frame is acquired and aligned with the robotic arm status data through the ROS2 time synchronization mechanism.

6. The quadruped robot manipulator arm end force feedback teleoperation control system according to claim 1, characterized in that: The teleoperation control module includes real-time probability modeling of the contact state. The estimation model of the contact probability is as follows: ； Where v is the velocity of the terminal along the target direction, Indicates the rate of change of the relative distance between the end and the target, represents the amplitude of the micro-disturbance in the force sensor, , , is the weighting coefficient, is the Sigmoid activation function; when When it is higher than the set threshold, the controller will actively adjust the control parameters of the robotic arm, including reducing the end stiffness and switching to a compliant control state; limiting the maximum propulsion rate of the end to avoid impact; and activating the micro-force feedback channel to allow the operator to sense that contact is about to occur.

7. The quadruped robot manipulator arm end force feedback teleoperation control system according to claim 6, characterized in that: The teleoperation control module also includes a predictive end force feedback adjustment mechanism, specifically including: Based on the contact probability estimation results in the teleoperation control module , combined with the real-time six-dimensional force disturbance trend and the terminal motion state to determine whether the system is currently in a "high-risk contact" state. Once the preset criteria are met: , the control system will actively adjust the force feedback signal in the following ways: Adjust the feedback channel bandwidth and sampling frequency to prioritize the transmission of key disturbance information; dynamically increase the force feedback gain coefficient , so that the operator can feel the approach of resistance even without actual contact; Limit the feedback change rate, smooth transient jump signals, and avoid false alarms or oscillations.

8. The quadruped robot manipulator arm end force feedback teleoperation control system according to claim 1, characterized in that: The obstacle avoidance and path planning module includes a dynamic obstacle avoidance adjustment mechanism: Real-time monitoring of the end force change rate Deviation from path tracking , combined with the update frequency of the environment point cloud, construct the trigger factor: ； when , it determines whether there are interfering obstacles outside the path or sudden changes in the environment, and immediately performs local trajectory replanning, searches for a new path segment with the minimum cost at the current state point, and splices it to the original trajectory to achieve seamless transition.

9. The quadruped robot manipulator arm end force feedback teleoperation control system according to claim 1, characterized in that: The obstacle avoidance and path planning module also includes a soft obstacle volume inference mechanism: Maintain a force time series during the control period , calculate the average force growth rate in continuous time , if the following three conditions are met at the same time: (1) Force rises slowly ; (2) The propulsion speed is zero; (3) The missing rate of path direction visual image depth data is greater than 90%; At this time, the control system detects the continuous but slowly increasing end force, which is accompanied by difficulty in path advancement. Combined with the lack of visual occlusion information, it automatically determines that the current path may be occupied by a soft obstacle.

10. A method for remote control of a quadruped robot's manipulator arm with force feedback, characterized in that: The method is used on the force feedback teleoperation control system of the end of the quadruped robot manipulator arm according to any one of claims 1 to 9, and the method comprises: Task distribution and inspection point navigation; When arriving at the designated inspection area, the robotic arm activates the binocular vision perception module to collect image data of the current working area and obtain the three-dimensional point cloud information of the scene through binocular parallax calculation; Once the target pose is determined, the system activates the obstacle avoidance and path planning modules. During trajectory execution, when the distance between the end of the robotic arm and the target surface is less than 30 mm, it automatically switches to pre-contact mode. During the contact process, the operator issues teleoperation control commands through the master terminal; the master-slave mapping module maps the master terminal input to the slave terminal motion, and simultaneously integrates the sensor force feedback to compare the master and slave state consistency; After the current task is completed, the system controller automatically switches to the reset path, controls the robotic arm to withdraw to the initial safe position, and updates the task list at the same time, entering the task process of the next inspection target point.

Images