Robotic tactile perception control system and method for a modular electronic skin

By performing time synchronization, spatial coordinate transformation, and sensitivity normalization on the robot's electronic skin module, a unified tactile field data is constructed. Combined with the coupled control of tactile state and motion state, the data inconsistency problem of modular electronic skin is solved, and the stability and adaptability of robot tactile control are realized.

CN123060745APending Publication Date: 2026-11-10SHANGHAI RUITI IND INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611544218.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-08
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

The existing robotic electronic skin modules suffer from asynchronous sampling times, inconsistent spatial coordinates, differences in sensitivity, and discontinuous responses in boundary areas. This makes it difficult to uniformly represent tactile data, and the control methods are unable to adaptively adjust according to tactile states and interaction intentions, thus reducing the stability and accuracy of robotic tactile control.

Method used

By combining time synchronization, spatial coordinate transformation, reference response correction, and sensitivity normalization with module boundary correlation fusion, a unified tactile field data is constructed. Based on spatiotemporal fusion tactile features, tactile state vectors and intent representation data are generated. Combined with the robot's motion state, coupled control is performed to achieve adaptive control of the modular electronic skin.

Benefits of technology

It improves the consistency and continuity of tactile data from multiple modules, enhances the robot's ability to recognize complex contact states and interaction intentions, realizes closed-loop adaptive adjustment of tactile control and module fusion weights, and improves the robot's control adaptability in dynamic contact processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123060745A_ABST
    Figure CN123060745A_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of robot tactile perception and intelligent control, in particular to a robot tactile perception control system and method of a modular electronic skin, the present application first acquires original tactile data, module pose data and sampling time stamp of a plurality of electronic skin modules, forms unified tactile field data through time synchronization, space coordinate conversion, reference correction, sensitivity normalization and boundary correlation fusion, constructs spatial correlation relationship of tactile regions based on the unified tactile field and extracts spatiotemporal fusion tactile features, obtains tactile state and tactile intention, couples the tactile state, the tactile intention and the robot motion state, generates tactile adaptive control quantity, calculates state deviation and module credibility according to the tactile and motion feedback after execution, corrects the robot control quantity and updates the electronic skin module fusion weight, forms a closed-loop control process of tactile perception, state recognition, motion control and module weight adaptive adjustment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of robot tactile perception and intelligent control technology, specifically to a modular electronic skin robot tactile perception control system and method. Background Technology

[0002] As robots evolve from executing pre-defined trajectories in structured environments to autonomous interaction in complex dynamic environments, the demand for real-time perception of contact states, contact positions, forces, and slip states is constantly increasing. Electronic skin can acquire tactile information during robot-to-object contact through flexible pressure, strain, piezoresistive, or other tactile sensing structures, and can cover robot arms, end effectors, and other contact areas. However, existing robot electronic skins typically use fixed areas or single sensing modules for tactile data acquisition. Problems such as asynchronous sampling times, inconsistent spatial coordinates, sensitivity differences, and discontinuous responses in boundary areas can easily arise between different electronic skin modules, making it difficult to uniformly represent the tactile data generated by multiple electronic skin modules.

[0003] On the other hand, existing robot tactile control methods typically adjust control based on a single tactile parameter or preset threshold, making it difficult to simultaneously utilize tactile spatial distribution, dynamic changes in tactile sensation, and robot motion state for collaborative decision-making. When the robot experiences changes in interaction states such as enhanced contact, stable contact, or slippage, fixed control parameters struggle to adjust the control relationship between position, velocity, and force in a timely manner. Furthermore, electronic skin modules may exhibit response deviations under long-term use or different stress conditions, causing data from some abnormal modules to affect the overall tactile perception results, thereby reducing the stability and accuracy of robot tactile control. Therefore, a tactile perception control method and system are needed that can provide a unified tactile representation of multiple electronic skin modules, combine tactile states and interaction intentions to achieve adaptive robot control, and dynamically adjust the module fusion weights based on closed-loop feedback. Summary of the Invention

[0004] To achieve the above-mentioned objectives, the present invention provides the following technical solution: a robot tactile perception control method for modular electronic skin, comprising the following steps: Step S1: Obtain the raw tactile data, module pose data, module identification data and sampling timestamps output by multiple electronic skin modules set on the robot arm, robot end effector and robot contact parts. Perform time synchronization, spatial coordinate transformation, reference response correction, sensitivity normalization and module boundary association fusion on the raw tactile data of different electronic skin modules to obtain unified tactile field data. Step S2: Based on the unified tactile field data, divide the tactile regions according to the spatial position of the robot surface, obtain the pressure characteristics, pressure gradient characteristics, contact area characteristics, tactile response change characteristics and spatial adjacency characteristics of each tactile region, construct the tactile spatial correlation relationship, jointly model the spatial tactile characteristics and temporal change characteristics of each tactile region, obtain spatiotemporal fusion tactile characteristics, and generate tactile state vector and tactile intention representation data based on the spatiotemporal fusion tactile characteristics. Step S3: Obtain the robot's current joint state, end-effector pose state, end-effector velocity, current force, and target motion state. Couple the tactile state vector and tactile intent representation data with the robot's current motion state to obtain a tactile-motion coupling state. Determine the tactile adaptive control weights corresponding to the position, velocity, and force control dimensions based on the tactile-motion coupling state. Coordinate the robot's motion error and tactile state based on the tactile adaptive control weights to obtain the robot's target control quantity. Apply the robot's target control quantity to the robot's actuator. Step S4: Obtain a new round of tactile perception data from the electronic skin module after the robot executes the target control quantity. Obtain new unified tactile field data according to the unified representation method in step S1. Obtain the actual tactile state vector according to the tactile state recognition method in step S2. Compare the actual tactile state vector with the target tactile state to obtain tactile state deviation and motion state deviation. Feedback correction is performed on the target control quantity of the robot based on the tactile state deviation and motion state deviation. At the same time, the module credibility is calculated based on the consistency of the tactile response of each electronic skin module. The module credibility is used to update the weights of each electronic skin module participating in tactile fusion, obtaining the updated robot control quantity and the updated module participation weights. The new unified tactile field data is then re-inputted into step S2 to form a closed-loop control process of modular electronic skin tactile perception, tactile state recognition, robot motion control, and adaptive correction of module weights.

[0005] Preferably, step S1 specifically includes: Get the Each electronic skin module is in constant motion The output raw tactile data, module pose data, module identification data, and sampling timestamps are first time-aligned according to the sampling timestamps of the raw tactile data of different electronic skin modules, and the tactile sampling positions of each electronic skin module are transformed to the robot body coordinate system according to the module pose data to obtain spatially unified tactile sampling data. Further, the non-contact reference response and standard load response of each electronic skin module are obtained. Based on the non-contact reference response and standard load response, the spatially unified tactile response is nonlinearly normalized to obtain the normalized tactile response. For the overlapping areas and module boundary areas between adjacent electronic skin modules, module association weights are established based on the positional distance between tactile sampling points, differences in tactile response, and consistency of response changes between adjacent modules. The normalized tactile responses of adjacent electronic skin modules are then weighted and fused using the module association weights to obtain unified tactile field data.

[0006] Preferably, step S2 specifically includes: The unified tactile field data is divided into multiple tactile regions according to the spatial position on the robot surface. For each tactile region, the average pressure, pressure gradient, contact area, tactile response change rate, and the position of the centroid are extracted. Then, a tactile spatial association map is constructed according to the spatial adjacency relationship and tactile response similarity between each tactile region. For each tactile region in the tactile spatial association graph, the state features of adjacent tactile regions are spatially propagated according to the module association weight, and combined with the state features of the current tactile region itself and the state features of the previous sampling time to obtain spatiotemporal fusion tactile features. Furthermore, based on the spatiotemporal fusion tactile features within the continuous sampling period, dynamic features corresponding to pressure center migration, contact area change, and local pressure gradient change are extracted, and the coupling relationship between contact state and tactile change trend is established to obtain tactile state vector and tactile intention representation data. After obtaining the tactile state vector, the contact intensity, contact area change rate, pressure center movement, and tactile response change rate are extracted from the tactile state vector. Each feature is compared with a preset contact state threshold. Based on the comparison results, the robot's current contact intention category is determined. When the contact intensity continues to increase and the contact area change rate is within a stable range, the contact intention is determined as a contact enhancement intention. When the pressure center continues to move in the same direction and the tactile response change rate exceeds a preset change threshold, the contact intention is determined as a slip correction intention. When both the contact intensity and contact area remain stable, the contact intention is determined as a stability holding intention. The determined contact intention category is combined with the corresponding tactile state intensity to form tactile intention representation data.

[0007] Preferably, step S3 specifically includes: Get the current joint state of the robot End-effector pose Terminal velocity Current force Target motion state and target tactile state vector The tactile state vector and tactile intention representation data are used to... By jointly encoding the tactile feedback with the robot's current motion state, the tactile-motion coupled state is obtained. ; According to the tactile-motor coupling state Calculate the robot position control error separately Speed ​​control error and force control error And using tactile state vectors The contact strength, contact stability, and slip state are dynamically weighted to obtain the tactile adaptive control weights. The tactile adaptive control weights are expressed as follows: ; in, Represents the tactile state mapping matrix. Represents the haptic intent mapping matrix. Represents the tactile rate of change mapping matrix; The position control error Speed ​​control error and force control error Determine them according to the following formulas respectively: ; ; ; Subsequently, based on the tactile adaptive control weights Position control error Speed ​​control error and force control error By performing coordinated adjustments, the target control quantity of the robot can be obtained. The target control quantity of the robot is expressed as: ; in, , and These represent the haptic adaptive control weights corresponding to the position, velocity, and force control dimensions, respectively. , and These represent the position, velocity, and force control gains, respectively. This indicates the tactile feedback gain.

[0008] Preferably, step S4 specifically includes: The robot executes the target control quantity. Then, the tactile perception data output by each electronic skin module is reacquired, and new unified tactile field data is generated according to the data processing method in step S1. Then, the actual tactile state vector is obtained according to the tactile state recognition method in step S2. The actual tactile state vector , and the target tactile state vector generated in step S3 Perform difference calculations to obtain tactile state deviations. and the actual end-effector pose of the robot. With the target end pose By comparison, the deviation of motion state can be obtained. ; According to the tactile state deviation and motion state deviation A control feedback state is established, and the module reliability is calculated based on the correlation between the tactile response of each electronic skin module and the deviation of the tactile state within a continuous control cycle. The reliability of the module is expressed as follows: ; in, Indicates the first The reliability of each electronic skin module. Indicates by the first Each electronic skin module corresponds to the actual tactile state obtained from tactile data. Indicates the target's tactile state. Indicates the first Normalized tactile response of an electronic skin module Indicates the current unified tactile field response. A parameter representing the scale of tactile state deviation. This represents a parameter indicating the scale of tactile response deviation. Indicates the number of electronic skin modules; Further based on the reliability of the module Update the weights of the electronic skin module participating in haptic fusion, and adjust them according to the haptic state deviation. and motion state deviation Target control quantity of robot Perform feedback corrections to obtain updated robot control parameters. The feedback correction process is expressed as follows: ; in, This represents the current target control quantity of the robot. This indicates the tactile state deviation feedback gain. This represents the feedback gain of motion state deviation. The feedback gain represents the rate of change of tactile state deviation, based on the updated robot control quantity. The robot's actuators are controlled, and the updated electronic skin module is used to regenerate unified tactile field data using weights. The new unified tactile field data is then input into step S2 for tactile state recognition, forming a closed-loop iterative process of module weight adaptive adjustment, tactile state recognition, and robot motion control.

[0009] This invention also provides a modular electronic skin robotic tactile perception control system, comprising the following modules: The modular tactile data acquisition and fusion module is used to acquire, process and fuse tactile data generated by multiple electronic skin modules installed on different parts of the robot to form unified tactile perception data on the robot surface, and output the unified tactile perception data to the tactile state recognition and intent analysis module. The tactile state recognition and intent analysis module is used to divide the unified tactile perception data into tactile regions, extract spatiotemporal features and analyze tactile states to obtain the robot's current tactile state and tactile interaction intent, and output the current tactile state and tactile interaction intent to the tactile drive control and closed-loop correction module. The tactile drive control and closed-loop correction module is used to generate robot motion control commands based on the current tactile state, tactile interaction intention and the robot's current motion state, and to correct the robot motion control commands based on the tactile feedback and motion feedback after the robot executes the motion control commands, while adjusting the weights of multiple electronic skin modules participating in tactile data fusion. The modular tactile data acquisition and fusion module, the tactile state recognition and intent analysis module, and the tactile drive control and closed-loop correction module are connected in sequence. The tactile drive control and closed-loop correction module transmits the adjusted electronic skin module fusion weight and feedback data to the modular tactile data acquisition and fusion module to form a closed-loop processing process of tactile data acquisition, tactile data fusion, tactile state recognition, robot motion control and feedback correction.

[0010] The modular tactile data acquisition and fusion module includes: The tactile data acquisition subunit is used to acquire raw tactile data output by multiple electronic skin modules installed on the robot arm, robot end effector, and robot contact parts, as well as the module position and orientation information, module identification information, and sampling time information of each electronic skin module. It performs time alignment on the raw tactile data output by different electronic skin modules according to a unified sampling time reference, and transforms the tactile sampling positions of different electronic skin modules to the robot's unified spatial coordinate system according to the position and orientation information of each electronic skin module. It performs reference correction and sensitivity normalization on the tactile response of each electronic skin module to obtain standardized tactile data with a unified time reference, spatial reference, and response scale. The tactile data fusion subunit is used to acquire tactile data from different electronic skin modules after standardization, identify overlapping areas and module boundary areas between adjacent electronic skin modules, determine the correlation between modules based on the tactile sampling position relationship, tactile response differences, and consistency of tactile response changes of adjacent electronic skin modules, and fuse the tactile data of adjacent electronic skin modules according to the correlation, so that the tactile data between different electronic skin modules transitions continuously at the spatial boundary, forming unified tactile perception data covering multiple contact areas of the robot.

[0011] The tactile state recognition and intent analysis module includes: The spatiotemporal tactile feature construction subunit is used to receive the unified tactile perception data, divide the robot's tactile perception area into multiple tactile regions according to the robot's surface spatial position, extract the pressure features, pressure gradient features, contact area features, tactile response change features, and region position features of each tactile region, and construct the spatial correlation relationship of tactile regions based on the spatial adjacency relationship and tactile response similarity relationship between different tactile regions. At the same time, it combines the tactile feature changes of each tactile region within a continuous sampling period to perform temporal correlation analysis, thereby obtaining spatiotemporal fused tactile features that characterize the robot's tactile spatial distribution and dynamic changes. The tactile state intention recognition subunit is used to receive the spatiotemporal fused tactile features, extract the contact intensity, contact area change, pressure center movement and tactile response change features of the robot's current contact area, determine the robot's current contact state based on the features, and determine the tactile interaction intention between the robot and the external object by combining the change trend of the contact state within a continuous sampling period, thus forming the corresponding tactile state information and tactile intention information.

[0012] The haptic drive control and closed-loop correction module includes: The tactile adaptive control subunit is used to receive the tactile state information and tactile intention information, and to obtain the robot's current joint state, end-effector pose state, end-effector velocity, current force and target motion state. It couples the tactile state information and tactile intention information with the robot's current motion state, determines the control requirements corresponding to the robot's position control, speed control and force control according to the coupling result, and dynamically adjusts each control dimension according to different control requirements, generates robot motion control commands and sends them to the robot actuator. The module collaborative feedback subunit is used to acquire a new round of tactile perception data and actual motion state after the robot executes the motion control command. Based on the new round of tactile perception data, it determines the robot's actual tactile state and compares the actual tactile state with the target tactile state and the actual motion state with the target motion state to obtain tactile state deviation and motion state deviation. Based on the consistency between the tactile response of different electronic skin modules and the tactile state deviation within a continuous control cycle, it determines the module reliability of each electronic skin module and adjusts the weight of each electronic skin module participating in tactile data fusion based on the module reliability. At the same time, it feeds back and corrects the robot motion control command based on the tactile state deviation and motion state deviation, and sends the corrected motion control command to the robot actuator.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: To improve the consistency and continuity of tactile data from multiple modules, time synchronization, spatial coordinate unification, baseline response correction, and nonlinear sensitivity normalization are performed on multiple electronic skin modules. Furthermore, correlation weights are established by combining the spatial location, tactile response, and response changes at the module boundaries. This enables the correlation and fusion of tactile data from different electronic skin modules, reducing response differences between modules and discontinuities in boundary data.

[0014] To improve the robot's ability to recognize complex contact states and interaction intentions, the robot surface is divided into multiple tactile regions. By combining information such as pressure, pressure gradient, contact area, pressure center movement, and tactile response changes, spatiotemporal fusion tactile features are constructed. This enables the robot to distinguish different tactile interaction states such as contact enhancement, stability maintenance, and slip correction, providing tactile state basis for subsequent position, speed, and force coordinated control.

[0015] This method enables closed-loop adaptive adjustment of the fusion weights of robot tactile control and electronic skin modules. It calculates tactile state deviation and motion state deviation based on the actual tactile state and motion state after the robot executes control inputs, and determines the reliability of modules based on the consistency of responses of different electronic skin modules. This allows for the synchronous correction of the fusion weights of robot motion control commands and electronic skin modules, forming a closed-loop iteration of tactile perception, robot motion control, and module state adjustment, thereby improving the robot's control adaptability during dynamic contact processes. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the method steps provided in this application; Figure 2 A schematic diagram of the system modules provided in this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] refer to Figure 1 This invention provides a robotic tactile perception control method for modular electronic skin, comprising the following steps: Step S1: Acquire raw tactile data output from multiple electronic skin modules installed on the robot arm, robot end effector, and robot contact points. Module pose data Module identification data and sampling timestamp The raw tactile data for different electronic skin modules Time synchronization, spatial coordinate transformation, reference response correction, sensitivity normalization, and module boundary correlation fusion are performed to obtain unified tactile field data. ; Step S2: Based on the unified tactile field data The robot surface is divided into tactile regions based on spatial location. Pressure characteristics, pressure gradient characteristics, contact area characteristics, tactile response variation characteristics, and spatial adjacency characteristics of each tactile region are obtained. Tactile spatial relationships are constructed, and the spatial and temporal tactile characteristics of each tactile region are jointly modeled to obtain spatiotemporally fused tactile features. And based on the spatiotemporal fusion tactile features Generate tactile state vector and tactile intention representation data ; Step S3: Obtain the current joint state of the robot. End-effector pose Terminal velocity Current force and the target's motion state The tactile state vector tactile intention representation data Coupled with the robot's current motion state, obtain the tactile-motion coupled state. According to the tactile-motion coupling state Determine the haptic adaptive control weights corresponding to the position, velocity, and force control dimensions. And based on the tactile adaptive control weights By coordinating the adjustment of robot motion error and tactile state, the target control quantity of the robot can be obtained. The target control quantity of the robot Acting on the robot actuator; Step S4: Obtain the target control quantity for the robot to execute. The next round of tactile perception data from the electronic skin module is used to obtain new unified tactile field data according to the unified representation method in step S1. The actual tactile state vector is obtained according to the tactile state recognition method in step S2. The actual tactile state vector By comparing with the target tactile state, the tactile state deviation is obtained. and motion state deviation According to the tactile state deviation and motion state deviation Target control quantity of robot Feedback corrections are made, and the module reliability is calculated based on the consistency of tactile response across all electronic skin modules. Utilizing the module's credibility Update the weights of each electronic skin module participating in haptic fusion to obtain the updated robot control parameters. The updated module participation weights are then incorporated, and the new unified tactile field data is re-input into step S2 to form a closed-loop control process for modular electronic skin tactile perception, tactile state recognition, robot motion control, and adaptive correction of module weights.

[0019] Step S1 specifically includes: Get the Each electronic skin module is in constant motion Raw tactile data output Module pose data Module identification data and sampling timestamp ,in, Indicates the first Raw tactile data from each electronic skin module Indicates the first The position and orientation of each electronic skin module relative to the robot's body coordinate system. Indicates the first The unique identifier for each electronic skin module Indicates the first Sampling timestamps of each electronic skin module; First, according to the sampling timestamp Raw tactile data from different electronic skin modules Perform time alignment and based on module pose data. The tactile sampling positions of each electronic skin module are transformed to the robot's body coordinate system to obtain spatially unified tactile sampling data. The spatial unification process is represented as follows: ; in, Indicates the first The position of the tactile sampling points of each electronic skin module in the robot's body coordinate system. This indicates the position of the tactile sampling point in the local coordinate system of the electronic skin module. This represents the rotation matrix corresponding to the module's attitude. This indicates the position offset of the module relative to the robot's body coordinate system; Further obtain the contactless reference response of each electronic skin module. and standard load response According to the contactless reference response and standard load response The spatially unified tactile response is nonlinearly normalized to obtain a normalized tactile response. The normalized nonlinear sensitivity is expressed as: ; in, Indicates the first The normalized tactile response of each electronic skin module Indicates the first The baseline response of an electronic skin module. Indicates the first The standard load calibration response of an electronic skin module Indicates the first The nonlinear sensitivity index of an electronic skin module, This represents a stable term to prevent the denominator from being zero. For overlapping areas and module boundary areas between adjacent electronic skin modules, module association weights are established based on the positional distance between tactile sampling points, differences in tactile response, and consistency of response changes between adjacent modules. The normalized tactile responses of adjacent electronic skin modules are weighted and fused using the module association weights to obtain unified tactile field data. The module is associated with weights. Represented as: ; in, Indicates the first The tactile sampling point and the first Module association weights between adjacent tactile sampling points and These represent the spatial locations of the two tactile sampling points. and These represent the normalized tactile responses at the two tactile sampling points, and These represent the rates of change of tactile response at the two tactile sampling points, Indicates the first The set of adjacent sampling points of each tactile sampling point This represents a spatial distance scale parameter. Parameters representing the scale of tactile response differences This represents the scale parameter indicating the difference in the rate of change of the response.

[0020] Step S2 specifically includes: The unified tactile field data The robot's surface was divided into multiple tactile zones based on its spatial location, and the average pressure of each zone was extracted. Pressure gradient Contact area tactile response rate of change and the location of the regional centroid ,in The tactile regions are numbered, and a tactile spatial relationship diagram is constructed based on the spatial adjacency relationships and tactile response similarities between the tactile regions. ; For tactile spatial association diagram For each tactile region, the state features of adjacent tactile regions are determined according to the module association weights. Spatial propagation is performed, and the spatiotemporal fused tactile features are obtained by combining the current tactile region's own state characteristics with the state characteristics of the previous sampling time. The spatiotemporal fusion tactile features are represented as follows: ; in, Indicates the first The spatiotemporal fusion tactile characteristics of each tactile region Indicates the first Tactile state characteristics of each tactile region This represents the transformation matrix of its own state characteristics. This represents the state characteristic transformation matrix of the adjacent region. Represents the transformation matrix of time-varying characteristics. Indicates the first The set of adjacent tactile regions of a tactile region Indicates the time interval between adjacent sampling times. Represents a nonlinear feature mapping function; Further based on the spatiotemporal fusion tactile features within continuous sampling periods The dynamic features corresponding to pressure center migration, contact area change, and local pressure gradient change are extracted, and the coupling relationship between contact state and tactile change trend is established to obtain tactile state vector. and tactile intention representation data Among them, the tactile state vector Determined according to the following formula: ; in, This represents the robot's current tactile state vector. This represents the overall feature composed of the spatiotemporal fusion tactile characteristics of each tactile region. Represents the tactile state feature mapping matrix. Represents the tactile dynamic change feature mapping matrix. Represents the contact area feature mapping matrix. Represents the pressure gradient feature mapping matrix. Indicates the overall contact area characteristics. Indicates the overall pressure gradient characteristics; In obtaining the tactile state vector Subsequently, the contact intensity, contact area change rate, pressure center movement, and tactile response change rate are extracted from the tactile state vector. Each feature is then compared with a preset contact state threshold. Based on the comparison results, the robot's current contact intention category is determined. When the contact intensity continuously increases and the contact area change rate remains within a stable range, the contact intention is identified as a contact enhancement intention. When the pressure center continuously moves in the same direction and the tactile response change rate exceeds the preset change threshold, the contact intention is identified as a slip correction intention. When both the contact intensity and contact area remain stable, the contact intention is identified as a stability holding intention. The determined contact intention category is combined with the corresponding tactile state intensity to form tactile intention representation data. .

[0021] Step S3 specifically includes: Get the current joint state of the robot End-effector pose Terminal velocity Current force Target motion state and target tactile state vector The tactile state vector tactile intention representation data By jointly encoding the tactile feedback with the robot's current motion state, the tactile-motion coupled state is obtained. ; According to the tactile-motor coupling state Calculate the robot position control error separately Speed ​​control error and force control error And using tactile state vectors The contact strength, contact stability, and slip state are dynamically weighted to obtain the tactile adaptive control weights. The tactile adaptive control weights are expressed as follows: ; in, Represents the tactile state mapping matrix. Represents the haptic intent mapping matrix. Represents the tactile rate of change mapping matrix; The position control error Speed ​​control error and force control error Determine them according to the following formulas respectively: ; ; ; Subsequently, based on the tactile adaptive control weights Position control error Speed ​​control error and force control error By performing coordinated adjustments, the target control quantity of the robot can be obtained. The target control quantity of the robot is expressed as: ; in, , and These represent the haptic adaptive control weights corresponding to the position, velocity, and force control dimensions, respectively. , and These represent the position, velocity, and force control gains, respectively. This indicates the tactile feedback gain.

[0022] Step S4 specifically includes: The robot executes the target control quantity. Then, the tactile perception data output by each electronic skin module is reacquired, and new unified tactile field data is generated according to the data processing method in step S1. Then, the actual tactile state vector is obtained according to the tactile state recognition method in step S2. The actual tactile state vector , and the target tactile state vector generated in step S3 Perform difference calculations to obtain tactile state deviations. and the actual end-effector pose of the robot. With the target end pose By comparison, the deviation of motion state can be obtained. ; According to the tactile state deviation and motion state deviation A control feedback state is established, and the module reliability is calculated based on the correlation between the tactile response of each electronic skin module and the deviation of the tactile state within a continuous control cycle. The reliability of the module is expressed as follows: ; in, Indicates the first The reliability of each electronic skin module. Indicates by the first Each electronic skin module corresponds to the actual tactile state obtained from tactile data. Indicates the target's tactile state. Indicates the first Normalized tactile response of an electronic skin module Indicates the current unified tactile field response. A parameter representing the scale of tactile state deviation. This represents a parameter indicating the scale of tactile response deviation. Indicates the number of electronic skin modules; Further based on the reliability of the module Update the weights of the electronic skin module participating in haptic fusion, and adjust them according to the haptic state deviation. and motion state deviation Target control quantity of robot Perform feedback corrections to obtain updated robot control parameters. The feedback correction process is expressed as follows: ; in, This represents the current target control quantity of the robot. This indicates the tactile state deviation feedback gain. This represents the feedback gain of motion state deviation. The feedback gain represents the rate of change of tactile state deviation, based on the updated robot control quantity. The robot's actuators are controlled, and the updated electronic skin module is used to regenerate unified tactile field data using weights. The new unified tactile field data is then input into step S2 for tactile state recognition, forming a closed-loop iterative process of module weight adaptive adjustment, tactile state recognition, and robot motion control.

[0023] refer to Figure 2 This invention provides a modular electronic skin robotic tactile perception control system, comprising the following modules: The modular tactile data acquisition and fusion module is used to acquire, process and fuse tactile data generated by multiple electronic skin modules installed on different parts of the robot to form unified tactile perception data on the robot surface, and output the unified tactile perception data to the tactile state recognition and intent analysis module. The tactile state recognition and intent analysis module is used to divide the unified tactile perception data into tactile regions, extract spatiotemporal features and analyze tactile states to obtain the robot's current tactile state and tactile interaction intent, and output the current tactile state and tactile interaction intent to the tactile drive control and closed-loop correction module. The tactile drive control and closed-loop correction module is used to generate robot motion control commands based on the current tactile state, tactile interaction intention and the robot's current motion state, and to correct the robot motion control commands based on the tactile feedback and motion feedback after the robot executes the motion control commands, while adjusting the weights of multiple electronic skin modules participating in tactile data fusion. The modular tactile data acquisition and fusion module, the tactile state recognition and intent analysis module, and the tactile drive control and closed-loop correction module are connected in sequence. The tactile drive control and closed-loop correction module transmits the adjusted electronic skin module fusion weight and feedback data to the modular tactile data acquisition and fusion module to form a closed-loop processing process of tactile data acquisition, tactile data fusion, tactile state recognition, robot motion control and feedback correction.

[0024] The modular tactile data acquisition and fusion module includes: The tactile data acquisition subunit is used to acquire raw tactile data output by multiple electronic skin modules installed on the robot arm, robot end effector, and robot contact parts, as well as the module position and orientation information, module identification information, and sampling time information of each electronic skin module. It performs time alignment on the raw tactile data output by different electronic skin modules according to a unified sampling time reference, and transforms the tactile sampling positions of different electronic skin modules to the robot's unified spatial coordinate system according to the position and orientation information of each electronic skin module. It performs reference correction and sensitivity normalization on the tactile response of each electronic skin module to obtain standardized tactile data with a unified time reference, spatial reference, and response scale. The tactile data fusion subunit is used to acquire tactile data from different electronic skin modules after standardization, identify overlapping areas and module boundary areas between adjacent electronic skin modules, determine the correlation between modules based on the tactile sampling position relationship, tactile response differences, and consistency of tactile response changes of adjacent electronic skin modules, and fuse the tactile data of adjacent electronic skin modules according to the correlation, so that the tactile data between different electronic skin modules transitions continuously at the spatial boundary, forming unified tactile perception data covering multiple contact areas of the robot.

[0025] The tactile state recognition and intent analysis module includes: The spatiotemporal tactile feature construction subunit is used to receive the unified tactile perception data, divide the robot's tactile perception area into multiple tactile regions according to the robot's surface spatial position, extract the pressure features, pressure gradient features, contact area features, tactile response change features, and region position features of each tactile region, and construct the spatial correlation relationship of tactile regions based on the spatial adjacency relationship and tactile response similarity relationship between different tactile regions. At the same time, it combines the tactile feature changes of each tactile region within a continuous sampling period to perform temporal correlation analysis, thereby obtaining spatiotemporal fused tactile features that characterize the robot's tactile spatial distribution and dynamic changes. The tactile state intention recognition subunit is used to receive the spatiotemporal fused tactile features, extract the contact intensity, contact area change, pressure center movement and tactile response change features of the robot's current contact area, determine the robot's current contact state based on the features, and determine the tactile interaction intention between the robot and the external object by combining the change trend of the contact state within a continuous sampling period, thus forming the corresponding tactile state information and tactile intention information.

[0026] The haptic drive control and closed-loop correction module includes: The tactile adaptive control subunit is used to receive the tactile state information and tactile intention information, and to obtain the robot's current joint state, end-effector pose state, end-effector velocity, current force and target motion state. It couples the tactile state information and tactile intention information with the robot's current motion state, determines the control requirements corresponding to the robot's position control, speed control and force control according to the coupling result, and dynamically adjusts each control dimension according to different control requirements, generates robot motion control commands and sends them to the robot actuator. The module collaborative feedback subunit is used to acquire a new round of tactile perception data and actual motion state after the robot executes the motion control command. Based on the new round of tactile perception data, it determines the robot's actual tactile state and compares the actual tactile state with the target tactile state and the actual motion state with the target motion state to obtain tactile state deviation and motion state deviation. Based on the consistency between the tactile response of different electronic skin modules and the tactile state deviation within a continuous control cycle, it determines the module reliability of each electronic skin module and adjusts the weight of each electronic skin module participating in tactile data fusion based on the module reliability. At the same time, it feeds back and corrects the robot motion control command based on the tactile state deviation and motion state deviation, and sends the corrected motion control command to the robot actuator.

[0027] It should be noted that, unless otherwise specified, the embodiments and features and technical solutions in the present invention can be combined with each other.

[0028] Obviously, the embodiments described above are merely some embodiments of the present invention, not all embodiments. The accompanying drawings show preferred embodiments of the present invention, but do not limit the patent scope of the present invention. The present invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the patent protection scope of this invention.

Claims

1. A robotic tactile perception and control method for modular electronic skin, characterized in that, Includes the following steps: Step S1: Obtain the raw tactile data, module pose data, module identification data and sampling timestamps output by multiple electronic skin modules set on the robot arm, robot end effector and robot contact parts. Perform time synchronization, spatial coordinate transformation, reference response correction, sensitivity normalization and module boundary association fusion on the raw tactile data of different electronic skin modules to obtain unified tactile field data. Step S2: Based on the unified tactile field data, divide the tactile regions according to the spatial position of the robot surface, obtain the pressure characteristics, pressure gradient characteristics, contact area characteristics, tactile response change characteristics and spatial adjacency characteristics of each tactile region, construct the tactile spatial correlation relationship, jointly model the spatial tactile characteristics and temporal change characteristics of each tactile region, obtain spatiotemporal fusion tactile characteristics, and generate tactile state vector and tactile intention representation data based on the spatiotemporal fusion tactile characteristics. Step S3: Obtain the robot's current joint state, end-effector pose state, end-effector velocity, current force, and target motion state. Couple the tactile state vector and tactile intent representation data with the robot's current motion state to obtain a tactile-motion coupling state. Determine the tactile adaptive control weights corresponding to the position, velocity, and force control dimensions based on the tactile-motion coupling state. Coordinate the robot's motion error and tactile state based on the tactile adaptive control weights to obtain the robot's target control quantity. Apply the robot's target control quantity to the robot's actuator. Step S4: Obtain a new round of tactile perception data from the electronic skin module after the robot executes the target control quantity. Obtain new unified tactile field data according to the unified representation method in step S1. Obtain the actual tactile state vector according to the tactile state recognition method in step S2. Compare the actual tactile state vector with the target tactile state to obtain tactile state deviation and motion state deviation. Feedback correction is performed on the target control quantity of the robot based on the tactile state deviation and motion state deviation. At the same time, the module credibility is calculated based on the consistency of the tactile response of each electronic skin module. The module credibility is used to update the weights of each electronic skin module participating in tactile fusion, obtaining the updated robot control quantity and the updated module participation weights. The new unified tactile field data is then re-inputted into step S2 to form a closed-loop control process of modular electronic skin tactile perception, tactile state recognition, robot motion control, and adaptive correction of module weights.

2. The robotic tactile perception and control method for modular electronic skin according to claim 1, characterized in that, Step S1 specifically includes: Get the Each electronic skin module is in constant motion The output raw tactile data, module pose data, module identification data, and sampling timestamps are first time-aligned according to the sampling timestamps of the raw tactile data of different electronic skin modules, and the tactile sampling positions of each electronic skin module are transformed to the robot body coordinate system according to the module pose data to obtain spatially unified tactile sampling data. Further, the non-contact reference response and standard load response of each electronic skin module are obtained. Based on the non-contact reference response and standard load response, the spatially unified tactile response is nonlinearly normalized to obtain the normalized tactile response. For the overlapping areas and module boundary areas between adjacent electronic skin modules, module association weights are established based on the positional distance between tactile sampling points, differences in tactile response, and consistency of response changes between adjacent modules. The normalized tactile responses of adjacent electronic skin modules are then weighted and fused using the module association weights to obtain unified tactile field data.

3. The robotic tactile perception control method for modular electronic skin according to claim 1, characterized in that, Step S2 specifically includes: The unified tactile field data is divided into multiple tactile regions according to the spatial position on the robot surface. For each tactile region, the average pressure, pressure gradient, contact area, tactile response change rate, and the position of the centroid are extracted. Then, a tactile spatial association map is constructed according to the spatial adjacency relationship and tactile response similarity between each tactile region. For each tactile region in the tactile spatial association graph, the state features of adjacent tactile regions are spatially propagated according to the module association weight, and combined with the state features of the current tactile region itself and the state features of the previous sampling time to obtain spatiotemporal fusion tactile features. Furthermore, based on the spatiotemporal fusion tactile features within the continuous sampling period, dynamic features corresponding to pressure center migration, contact area change, and local pressure gradient change are extracted, and the coupling relationship between contact state and tactile change trend is established to obtain tactile state vector and tactile intention representation data. After obtaining the tactile state vector, the contact intensity, contact area change rate, pressure center movement, and tactile response change rate are extracted from the tactile state vector. Each feature is compared with a preset contact state threshold. Based on the comparison results, the robot's current contact intention category is determined. When the contact intensity continues to increase and the contact area change rate is within a stable range, the contact intention is determined as a contact enhancement intention. When the pressure center continues to move in the same direction and the tactile response change rate exceeds a preset change threshold, the contact intention is determined as a slip correction intention. When both the contact intensity and contact area remain stable, the contact intention is determined as a stability holding intention. The determined contact intention category is combined with the corresponding tactile state intensity to form tactile intention representation data.

4. The robotic tactile perception and control method for modular electronic skin according to claim 1, characterized in that, Step S3 specifically includes: Get the current joint state of the robot End-effector pose Terminal velocity Current force Target motion state and target tactile state vector The tactile state vector tactile intention representation data By jointly encoding the tactile feedback with the robot's current motion state, the tactile-motion coupled state is obtained. ; According to the tactile-motor coupling state Calculate the robot position control error separately Speed ​​control error and force control error And using tactile state vectors The contact strength, contact stability, and slip state are dynamically weighted to obtain the tactile adaptive control weights. The tactile adaptive control weights are expressed as follows: ; in, Represents the tactile state mapping matrix. Represents the haptic intent mapping matrix. Represents the tactile rate of change mapping matrix; The position control error Speed ​​control error and force control error Determine them according to the following formulas respectively: ; ; ; Subsequently, based on the tactile adaptive control weights Position control error Speed ​​control error and force control error By performing coordinated adjustments, the target control quantity of the robot can be obtained. The target control quantity of the robot is expressed as: ; in, , and These represent the haptic adaptive control weights corresponding to the position, velocity, and force control dimensions, respectively. , and These represent the position, velocity, and force control gains, respectively. This indicates the tactile feedback gain.

5. The robotic tactile perception and control method for modular electronic skin according to claim 1, characterized in that, Step S4 specifically includes: The robot executes the target control quantity. Then, the tactile perception data output by each electronic skin module is reacquired, and new unified tactile field data is generated according to the data processing method in step S1. Then, the actual tactile state vector is obtained according to the tactile state recognition method in step S2. The actual tactile state vector , and the target tactile state vector generated in step S3 Perform difference calculations to obtain tactile state deviations. and the actual end-effector pose of the robot. With the target end pose By comparison, the deviation of motion state can be obtained. ; According to the tactile state deviation and motion state deviation A control feedback state is established, and the module reliability is calculated based on the correlation between the tactile response of each electronic skin module and the deviation of the tactile state within a continuous control cycle. The reliability of the module is expressed as follows: ; in, Indicates the first The reliability of each electronic skin module. Indicates by the first Each electronic skin module corresponds to the actual tactile state obtained from tactile data. Indicates the target's tactile state. Indicates the first Normalized tactile response of an electronic skin module Indicates the current unified tactile field response. A parameter representing the scale of tactile state deviation. This represents a parameter indicating the scale of tactile response deviation. Indicates the number of electronic skin modules; Further based on the reliability of the module Update the weights of the electronic skin module participating in haptic fusion, and adjust them according to the haptic state deviation. and motion state deviation Target control quantity of the robot Perform feedback corrections to obtain updated robot control parameters. The feedback correction process is expressed as follows: ; in, This represents the current target control quantity of the robot. This indicates the tactile state deviation feedback gain. This represents the motion state deviation feedback gain. The feedback gain represents the rate of change of tactile state deviation, based on the updated robot control quantity. The robot's actuators are controlled, and the updated electronic skin module is used to regenerate unified tactile field data using weights. The new unified tactile field data is then input into step S2 for tactile state recognition, forming a closed-loop iterative process of module weight adaptive adjustment, tactile state recognition, and robot motion control.

6. A modular electronic skin robotic tactile perception control system, characterized in that, Includes the following modules: The modular tactile data acquisition and fusion module is used to acquire, process and fuse tactile data generated by multiple electronic skin modules installed on different parts of the robot to form unified tactile perception data on the robot surface, and output the unified tactile perception data to the tactile state recognition and intent analysis module. The tactile state recognition and intent analysis module is used to divide the unified tactile perception data into tactile regions, extract spatiotemporal features and analyze tactile states to obtain the robot's current tactile state and tactile interaction intent, and output the current tactile state and tactile interaction intent to the tactile drive control and closed-loop correction module. The tactile drive control and closed-loop correction module is used to generate robot motion control commands based on the current tactile state, tactile interaction intention and the robot's current motion state, and to correct the robot motion control commands based on the tactile feedback and motion feedback after the robot executes the motion control commands, while adjusting the weights of multiple electronic skin modules participating in tactile data fusion. The modular tactile data acquisition and fusion module, the tactile state recognition and intent analysis module, and the tactile drive control and closed-loop correction module are connected in sequence. The tactile drive control and closed-loop correction module transmits the adjusted electronic skin module fusion weight and feedback data to the modular tactile data acquisition and fusion module to form a closed-loop processing process of tactile data acquisition, tactile data fusion, tactile state recognition, robot motion control and feedback correction.

7. The modular electronic skin robotic tactile perception control system according to claim 6, characterized in that, The modular tactile data acquisition and fusion module includes: The tactile data acquisition subunit is used to acquire raw tactile data output by multiple electronic skin modules installed on the robot arm, robot end effector, and robot contact parts, as well as the module position and orientation information, module identification information, and sampling time information of each electronic skin module. It performs time alignment on the raw tactile data output by different electronic skin modules according to a unified sampling time reference, and transforms the tactile sampling positions of different electronic skin modules to the robot's unified spatial coordinate system according to the position and orientation information of each electronic skin module. It performs reference correction and sensitivity normalization on the tactile response of each electronic skin module to obtain standardized tactile data with a unified time reference, spatial reference, and response scale. The tactile data fusion subunit is used to acquire tactile data from different electronic skin modules after standardization, identify overlapping areas and module boundary areas between adjacent electronic skin modules, determine the correlation between modules based on the tactile sampling position relationship, tactile response differences, and consistency of tactile response changes of adjacent electronic skin modules, and fuse the tactile data of adjacent electronic skin modules according to the correlation, so that the tactile data between different electronic skin modules transitions continuously at the spatial boundary, forming unified tactile perception data covering multiple contact areas of the robot.

8. The modular electronic skin robotic tactile perception control system according to claim 6, characterized in that, The tactile state recognition and intent analysis module includes: The spatiotemporal tactile feature construction subunit is used to receive the unified tactile perception data, divide the robot's tactile perception area into multiple tactile regions according to the robot's surface spatial position, extract the pressure features, pressure gradient features, contact area features, tactile response change features, and region position features of each tactile region, and construct the spatial correlation relationship of tactile regions based on the spatial adjacency relationship and tactile response similarity relationship between different tactile regions. At the same time, it combines the tactile feature changes of each tactile region within a continuous sampling period to perform temporal correlation analysis, thereby obtaining spatiotemporal fused tactile features that characterize the robot's tactile spatial distribution and dynamic changes. The tactile state intention recognition subunit is used to receive the spatiotemporal fused tactile features, extract the contact intensity, contact area change, pressure center movement and tactile response change features of the robot's current contact area, determine the robot's current contact state based on the features, and determine the tactile interaction intention between the robot and the external object by combining the change trend of the contact state within a continuous sampling period, thus forming the corresponding tactile state information and tactile intention information.

9. The modular electronic skin robotic tactile perception control system according to claim 6, characterized in that, The haptic drive control and closed-loop correction module includes: The tactile adaptive control subunit is used to receive the tactile state information and tactile intention information, and to obtain the robot's current joint state, end-effector pose state, end-effector velocity, current force and target motion state. It couples the tactile state information and tactile intention information with the robot's current motion state, determines the control requirements corresponding to the robot's position control, speed control and force control according to the coupling result, and dynamically adjusts each control dimension according to different control requirements, generates robot motion control commands and sends them to the robot actuator. The module collaborative feedback subunit is used to acquire a new round of tactile perception data and actual motion state after the robot executes the motion control command. Based on the new round of tactile perception data, it determines the robot's actual tactile state and compares the actual tactile state with the target tactile state and the actual motion state with the target motion state to obtain tactile state deviation and motion state deviation. Based on the consistency between the tactile response of different electronic skin modules and the tactile state deviation within a continuous control cycle, it determines the module reliability of each electronic skin module and adjusts the weight of each electronic skin module participating in tactile data fusion based on the module reliability. At the same time, it feeds back and corrects the robot motion control command based on the tactile state deviation and motion state deviation, and sends the corrected motion control command to the robot actuator.