Force sense feedback control method of intelligent mechanical arm and control system thereof

Abstract

The invention discloses a force sense feedback control method for an intelligent mechanical arm, which comprises the following steps of: 1, acquiring data through a multi-modal sensor and fusing the data to obtain a multi-dimensional perception vector; 3, calculating a force sense tracking error and a change rate and triggering an event-driven control decision mechanism; 4, designing a nonlinear compensation control rule and outputting a control torque instruction, wherein a control system comprises a multi-mode sensing module, a dynamic prediction module, an event-driven control module and a cooperative calculation module; according to the method, multi-mode sensing information is fused with the lifting force sense representation capacity, advanced adjustment is achieved in combination with a dynamic force sense prediction mechanism, event-driven control is used for reducing calculation redundancy, robustness to complex interference is enhanced through a nonlinear compensation strategy, and finally high-precision and low-delay force sense control of the mechanical arm in a dynamic interaction scene is achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot control technology, and in particular to a force feedback control method and control system for an intelligent robotic arm. Background Technology

[0002] With the development of industrial automation and intelligent manufacturing, robotic arms need to perform precision operations in more complex dynamic environments, such as grasping fragile objects, assembling flexible materials, and human-machine collaborative operations. This places higher demands on the real-time performance, robustness, and adaptability of force feedback control.

[0003] However, current force feedback control technology has a single sensing dimension, which makes it difficult to fully characterize the force information of the contact interface, resulting in lag or misjudgment of force perception in complex contact scenarios. At the same time, it has poor dynamic prediction capability and cannot quickly respond to force changes in a short time scale, which can easily lead to control overshoot or contact failure. In addition, it has high computational load and large communication latency, making it difficult to meet the real-time requirements of high-speed operation scenarios. Therefore, this invention proposes a force feedback control method and control system for an intelligent robotic arm to solve the problems existing in the prior art. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to propose a force feedback control method and control system for an intelligent robotic arm. This method and control system enhances force representation capabilities through multimodal perception information fusion, achieves proactive adjustment by combining a dynamic force prediction mechanism, reduces computational redundancy through event-driven control, and enhances robustness against complex disturbances through a nonlinear compensation strategy. Ultimately, it achieves high-precision, low-latency force control of the robotic arm in dynamic interactive scenarios.

[0005] To achieve the objectives of this invention, the invention is implemented through the following technical solution: a force feedback control method for an intelligent robotic arm, comprising the following steps: Step 1: A multimodal sensor array, including a distributed tactile sensor array, a six-dimensional force sensor, an inertial measurement unit, and a vision sensor, is used to collect force, pressure distribution, vibration characteristics, and contact surface feature information from the robotic arm's end effector. Time synchronization and spatial calibration are performed. A cross-modal visual-tactile attention mechanism is used to interact with visual surface features and tactile pressure distribution / vibration features, assigning attention weights to focus on key area features and generating a fusion representation containing the most relevant multimodal information. The fusion yields a multidimensional sensing vector that includes force / torque, pressure distribution matrix, vibration spectrum, and surface texture features. ; Step 2: Based on multidimensional perceptual vectors A prediction model that integrates spatiotemporal graph convolutional networks and gated recurrent units is used to extract spatiotemporal correlation features and predict force perception changes within a short time window in the future. ; Step 3: Based on the current real-time calculated force tracking error and its rate of change This triggers an event-driven control decision-making mechanism; Step 4: Design nonlinear compensation control laws. Based on the differential geometry exact linearization method, linearize the nonlinear dynamic model of the robotic arm, and combine it with adaptive sliding mode control to suppress unmodeled disturbances, outputting control torque commands. To the robotic arm joint actuator.

[0006] A further improvement is made in that: in step one, the distributed tactile sensor array is positioned on the contact surface of the end effector, and the sampling frequency... Spatial resolution Output contact pressure distribution matrix And local deformation information; a six-dimensional force sensor is installed on the wrist of the robotic arm, with a sampling frequency of Output end contact force and torque The inertial measurement unit is integrated into the end effector, with a sampling frequency of... Vibration acceleration during the output contact process and attitude changes; the vision sensor is mounted on the end effector, frame rate Output the surface texture image and deformation features of the contact area. ; In this process, multimodal sensor data is aligned with timestamps and spatially calibrated. Visual and tactile features are input into a cross-modal attention module, which calculates attention weights from vision to touch and from touch to vision. and Dynamically filter the associated regions of visual texture and tactile pressure / vibration to generate fused features. Ultimately, it is fused with other sensor data to form a multidimensional sensing vector. ,in For vibration spectrum, Represents the normalization coefficient. Indicates element-wise multiplication. Indicates splicing.

[0007] A further improvement lies in the following steps: The prediction model in step two is implemented through the following steps: S1, Multidimensional perceptual vector By time step Divided into time windows ; S2. Spatial correlation features of various sensor quantities within a time window are extracted using a spatiotemporal graph convolutional network, where the spatial convolution operation of the spatiotemporal graph convolutional network is further integrated with cross-modal attention generation. To enhance the extraction of local correlation features between pressure distribution and surface texture; S3. Extract dynamic features of time series through gated recurrent units and introduce an attention mechanism to dynamically adjust the weights of historical time steps, focusing on features before the sudden change in force sensation. S4. Output the predicted force vector for the next short time window. Among them, short time window .

[0008] The further improvement lies in the fact that the triggering condition in step three is: or ,in and The preset threshold is pre-set based on the accuracy requirements of the target contact force and the dynamic characteristics of the robotic arm; when or If the control rate is updated, the control output is updated; otherwise, the previous control output is maintained.

[0009] The further improvement lies in the fact that the design of the nonlinear compensation control law in step four specifically includes: S1. Establish the dynamic model of the robotic arm. ,in Joint angle, The inertia matrix, For Coriolis matrix, The gravity vector For external contact force, To control the torque; S2. The nonlinear system is precisely linearized using differential homeomorphism transformation. , where e is the tracking error, and A and B are the linearized system matrices; S3, Design the sliding surface Constructing control laws Among them, equivalent control Switch control , To switch the gain; S4. Introducing the Adaptive Law Adjust K online to compensate for unmodeled nonlinear disturbances. Ensure the sliding surface Converging to zero, where For adaptive gain.

[0010] A force feedback control system for an intelligent robotic arm includes a multimodal perception module, a dynamic prediction module, an event-driven control module, and a collaborative computing module. The multimodal perception module integrates a distributed tactile sensor array, a six-dimensional force sensor, an inertial measurement unit, and a vision sensor to collect force, pressure distribution, vibration, and surface feature information at the end of the robotic arm. It also interacts with visual surface features and tactile pressure distribution / vibration features, assigning attention weights to generate a multimodal fusion representation. The dynamic prediction module is used to predict force changes within a short future time window. The event-driven control module triggers control law updates based on force tracking errors and their rate of change, and performs event trigger judgments. The collaborative computing module calculates and outputs control torque commands.

[0011] Further improvements are made in the following aspects: the dynamic prediction module includes a prediction model that integrates a spatiotemporal graph convolutional network and a gated recurrent unit; the collaborative computing module includes an edge computing unit and a central control unit. The edge computing unit is used for local data preprocessing, short-time window prediction, and preliminary calculation of the control law. The central control unit is used to perform fine calculation of the nonlinear compensation control law and output control torque commands.

[0012] A further improvement is that the edge computing unit and the central control unit communicate via a time-sensitive network, the dynamic prediction module is deployed in the edge computing unit, and the event-driven control module is deployed in both the edge computing unit and the central control unit.

[0013] The beneficial effects of this invention are as follows: By fusing multi-source data, this invention comprehensively captures multimodal information, solving the information loss problem of traditional single-modal perception. By using a prediction model that fuses spatiotemporal graph convolutional networks and gated recurrent units, it predicts force sensation mutations in advance and adjusts the control strategy before contact failure or overshoot occurs, significantly improving robustness in dynamic interactive scenarios. Through an event-driven control decision mechanism, control updates are triggered only when the force sensation error exceeds the limit, effectively reducing the computational load. Furthermore, by designing nonlinear compensation control laws, it effectively suppresses nonlinear interferences such as joint coupling and boom elasticity, effectively reducing force sensation tracking errors, making it suitable for high-speed operation scenarios. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention.

[0015] Figure 2 This is a system architecture diagram of the present invention. Detailed Implementation

[0016] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0017] according to Figure 1 As shown, this embodiment provides a force feedback control method for an intelligent robotic arm, including the following steps: Step 1: A multimodal sensor array, including a distributed tactile sensor array, a six-dimensional force sensor, an inertial measurement unit, and a vision sensor, is used to collect force, pressure distribution, vibration characteristics, and contact surface feature information from the robotic arm's end effector. Time synchronization and spatial calibration are performed. A cross-modal visual-tactile attention mechanism is used to interact with visual surface features and tactile pressure distribution / vibration features, assigning attention weights to focus on key area features and generating a fusion representation containing the most relevant multimodal information. The fusion yields a multidimensional sensing vector that includes force / torque, pressure distribution matrix, vibration spectrum, and surface texture features. ; Distributed tactile sensor arrays are located on the contact surface of the end effector, with a sampling frequency of... Spatial resolution Output contact pressure distribution matrix and local deformation information; The six-dimensional force sensor is installed on the wrist of the robotic arm, with a sampling frequency of... Output end contact force and torque ; The inertial measurement unit is integrated into the end effector, with a sampling frequency of... Vibration acceleration during the output contact process and posture changes, Its frequency response range is 0.1Hz. 1kHz; The vision sensor is mounted on the end effector, frame rate Output the surface texture image and deformation features of the contact area. ; Multimodal sensor data is time-stamped and spatially calibrated. Visual and tactile features are input into a cross-modal attention module, which calculates attention weights from vision to touch and from touch to vision. and Dynamically filter the associated regions of visual texture and tactile pressure / vibration to generate fused features. Ultimately, it is fused with other sensor data to form a multidimensional sensing vector. ,in For vibration spectrum, Represents the normalization coefficient. Indicates element-wise multiplication. Indicates splicing.

[0018] Step 2: Based on multidimensional perceptual vectors A prediction model that integrates spatiotemporal graph convolutional networks and gated recurrent units is used to extract spatiotemporal correlation features and predict force perception changes within a short time window in the future. ; The prediction model is implemented through the following steps: S1, Multidimensional perceptual vector By time step Divided into time windows ; S2. Spatial correlation features of various sensor quantities within a time window are extracted using a spatiotemporal graph convolutional network, where the spatial convolution operation of the spatiotemporal graph convolutional network is further integrated with cross-modal attention generation. To enhance the extraction of local correlation features between pressure distribution and surface texture, such as the correlation between tactile pressure distribution and six-dimensional force; S3. Extract dynamic features of time series through gated loop units and introduce an attention mechanism to dynamically adjust the weight of historical time steps, focusing on features before the change in force sensation, such as a surge in high-frequency components of the vibration spectrum. S4. Output the predicted force vector for the next short time window. Among them, short time window .

[0019] Step 3: Based on the current real-time calculated force tracking error and its rate of change This triggers an event-driven control decision-making mechanism; The trigger condition is: or ,in and The preset threshold is pre-set based on the accuracy requirements of the target contact force and the dynamic characteristics of the robotic arm; when or If the control rate is updated, the control output is updated; otherwise, the previous control output is maintained.

[0020] Step 4: Design nonlinear compensation control laws. Based on the differential geometry exact linearization method, linearize the nonlinear dynamic model of the robotic arm, and combine it with adaptive sliding mode control to suppress unmodeled disturbances, outputting control torque commands. To the robotic arm joint actuator; The design of nonlinear compensation control laws specifically includes: S1. Establish the dynamic model of the robotic arm. ,in Joint angle, The inertia matrix, For Coriolis matrix, The gravity vector For external contact force, To control the torque; S2. The nonlinear system is precisely linearized using differential homeomorphism transformation. , where e is the tracking error, and A and B are the linearized system matrices; S3, Design the sliding surface Constructing control laws Among them, equivalent control Switch control , To switch the gain; S4. Introducing the Adaptive Law Adjust K online to compensate for unmodeled nonlinear disturbances. Ensure the sliding surface Converging to zero, where For adaptive gain.

[0021] according to Figure 2 As shown, a force feedback control system for an intelligent robotic arm includes a multimodal perception module, a dynamic prediction module, an event-driven control module, and a collaborative computing module. The multimodal perception module integrates a distributed tactile sensor array, a six-dimensional force sensor, an inertial measurement unit, and a vision sensor to collect force, pressure distribution, vibration, and surface feature information at the end of the robotic arm. It also interacts with the visual surface features and tactile pressure distribution / vibration features, assigning attention weights to generate a multimodal fusion representation. The dynamic prediction module is used to predict force changes within a short future time window. The event-driven control module triggers control law updates based on force tracking errors and their rate of change, and executes event trigger judgments. The collaborative computing module calculates and outputs control torque commands.

[0022] The distributed tactile sensor array of the multimodal sensing module is a flexible piezoresistive sensor with 16×16 or 32×32 channels; the range of the six-dimensional force sensor is ±5N~±50N, the range of the IMU angular velocity is ±100° / s~±1000° / s, and the resolution of the vision sensor is 1920×1080 pixels.

[0023] The cross-modal attention module includes a visual feature encoder such as a CNN, a tactile feature encoder such as a GCN, and a multi-head attention layer to compute the interaction weights of visual and tactile features.

[0024] The ST-GCN of the dynamic prediction module includes spatial convolution kernels and temporal convolution kernels. The spatial convolution kernel is used to extract the neighborhood correlation of tactile pressure distribution, and the temporal convolution kernel is used to extract the time series features of six-dimensional force and IMU data.

[0025] The spatial convolution operation of ST-GCN further integrates the fusion representation of the cross-modal attention module output to enhance the extraction of local correlation features between visual texture and tactile pressure.

[0026] Trigger threshold of event-driven controller Set to 10%~20% of the target contact force. The value is set to 50% to 100% of the target contact force change rate, and the specific value is dynamically adjusted according to the accuracy requirements of the contact task.

[0027] The dynamic prediction module includes a prediction model that integrates a spatiotemporal graph convolutional network and a gated recurrent unit; the collaborative computing module includes an edge computing unit and a central control unit. The edge computing unit is used for local data preprocessing, short time window prediction, and preliminary calculation of the control law. The central control unit is used to perform fine calculation of the nonlinear compensation control law and output control torque commands.

[0028] The edge computing unit and the central control unit communicate via a time-sensitive network, with communication latency... To ensure the real-time nature of control commands, the dynamic prediction module is deployed in the edge computing unit, and the event-driven control module is deployed in the edge computing unit and the central control unit.

[0029] The computing load of the edge computing unit accounts for 60% to 80% of the total computing load, while the computing load of the central control unit accounts for 20% to 40% of the total computing load.

[0030] The edge computing unit completes high-frequency data preprocessing and short-term prediction, while the central control unit is responsible for low-frequency fine control, balancing real-time performance and computational complexity, and effectively reducing system latency.

[0031] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A force feedback control method for a smart robot arm, characterized in that, The method comprises the following steps: Step one, collect the force sense, pressure distribution, vibration characteristics and contact surface feature information of the manipulator end effector through multi-modal sensors including distributed tactile sensor array, six-axis force sensor, inertial measurement unit and visual sensor, and perform time synchronization and spatial calibration, interact the visual surface features and tactile pressure distribution / vibration characteristics through visual tactile cross-modal attention mechanism, allocate attention weight to focus on key area features, and generate a fusion representation containing the most relevant information of multi-modal , fusion to obtain a multi-dimensional perception vector containing force / torque, pressure distribution matrix, vibration spectrum and surface texture features ; Step two, based on multi-dimensional perception vector , using the prediction model of spatio-temporal graph convolution network and gated recurrent unit, extracting spatio-temporal correlation features and predicting force change in future short time window ; Step three, a force-telepresence tracking error is computed in real-time and its rate of change trigger an event-driven control decision mechanism; Step four, design nonlinear compensation control law, linearize the nonlinear dynamics model of the manipulator based on the differential geometry accurate linearization method, and combine adaptive sliding mode control to suppress unmodeled disturbances, output control torque command to the joint drive of the manipulator. 2.The force sensation feedback control method of the intelligent robot arm according to claim 1, wherein: The distributed tactile sensor array in step one is distributed on the contact surface of the end effector, with a sampling frequency , spatial resolution , outputting a contact pressure distribution matrix and local deformation information; a six-dimensional force sensor is installed on the wrist of the robot arm, with a sampling frequency , outputting the end contact force and torque ; an inertial measurement unit is integrated into the end effector, with a sampling frequency , outputting the vibration acceleration and attitude change during contact; a vision sensor is installed on the end effector, with a frame rate , outputting the surface texture image and deformation features of the contact area ; Wherein, the multimodal sensor data is timestamp aligned and spatially calibrated, the visual features and tactile features are input into the cross-modal attention module, and the attention weights from vision to touch and from touch to vision are calculated and The associated regions of visual texture and tactile pressure / vibration are dynamically screened to generate fusion features Finally, the fusion features are fused with other sensor data into a multi-dimensional perception vector wherein is the vibration spectrum, represents a normalization coefficient, represents element multiplication, represents splicing. 3.The force sensation feedback control method of the intelligent robot arm according to claim 1, wherein: The prediction model in the second step is achieved by the following steps: S1, a multi-dimensional perception vector by time steps partitioned into time windows ; S2, using a spatio-temporal graph convolution network to extract the spatial correlation features of each sensing quantity in the time window, wherein the spatial convolution operation of the spatio-temporal graph convolution network further fuses the cross-modal attention generated to strengthen the extraction of local correlation features of pressure distribution and surface texture; S3, extract time series dynamic features through a gated recurrent unit, and introduce an attention mechanism to dynamically adjust the weight of historical time steps, focusing on features before the sudden change in force sensation; S4, outputting a predicted force vector within a short future time window wherein the short time window . 4.The force sensation feedback control method of the intelligent robot arm according to claim 1, wherein: The trigger condition in the third step is or wherein and is a preset threshold value, which is preset based on the accuracy requirement of the target contact force and the dynamic characteristics of the robot arm; when or the control rate is triggered to update; otherwise, the last control output is maintained. 5.The force sensation feedback control method of the intelligent robot arm according to claim 1, wherein: The nonlinear compensation control law in the fourth step specifically comprises: S1, establish a robot dynamics model wherein is the joint angle, is the inertia matrix, is the Coriolis matrix, is the gravity vector, is the external contact force, is the control torque; S2, the nonlinear system is accurately linearized into where e is the tracking error, A, B are the system matrices after linearization; S3, design a sliding surface , construct a control law where the equivalent control , switch control , is the switching gain; S4, introducing an adaptive law online adjustment of K, compensating for unmodeled nonlinear disturbances , ensuring the sliding surface converges to zero, where is an adaptive gain.

6. A force feedback control system for an intelligent robotic arm, characterized in that: The method comprises a multi-modal perception module, a dynamic prediction module, an event-driven control module, and a collaborative computing module. The multi-modal perception module integrates a distributed tactile sensor array, a six-axis force sensor, an inertial measurement unit, and a vision sensor to collect information about the force sensation, pressure distribution, vibration, and surface characteristics at the end of the robot arm. The module interacts with the visual surface characteristics and tactile pressure distribution / vibration characteristics, assigns attention weights, and generates a multi-modal fusion representation. The dynamic prediction module predicts force changes in a short time window in the future. The event-driven control module triggers control law updates based on force tracking errors and their rates of change, and performs event-triggered judgments. The collaborative computing module calculates and outputs control torque commands. 7.The force sensation feedback control system of an intelligent robot arm according to claim 6, wherein: The dynamic prediction module comprises a prediction model that combines a spatio-temporal graph convolution network and a gated recurrent unit. The collaborative computing module includes an end-edge computing unit and a central control unit. The end-edge computing unit is used for data preprocessing, short-time window prediction, and preliminary control law calculation. The central control unit is used for fine calculation of the nonlinear compensation control law and outputs control torque commands. 8.The force sensation feedback control system of an intelligent robot arm according to claim 7, wherein: The end-edge computing unit and the central control unit communicate through a time-sensitive network. The dynamic prediction module is deployed in the end-edge computing unit, and the event-driven control module is deployed in the end-edge computing unit and the central control unit.

Images