An event-driven environment-aware electronic skin system

Abstract

The present invention relates to an event-driven environment-aware electronic skin system, comprising electronic skin hardware composed of expandable modular multimodal perception units, a data processing unit, and a multi-level data fusion environment perception algorithm. The electronic skin system can be wrapped around the entire body of a robot, and can measure multimodal information of the robot's surrounding environment in real time and in all directions. Measurable information includes temperature, proximity, acceleration, and pressure information. The multi-level data fusion environment perception algorithm can realize object motion estimation, positioning, and identity recognition functions. Compared with the prior art, the electronic skin system enables the robot to have a human-like sense of touch and make up for its visual blind spot; on the one hand, the system enables the robot to operate more safely in the environment, and on the other hand, effectively improves the robot's environmental perception ability and improves the utilization rate of information.

Description

Technical Field

[0001] The present invention relates to the field of intelligent perception of collaborative robots, and in particular to an event-driven environment-perceiving electronic skin system. Background Art

[0002] Event-driven refers to a strategy for making decisions in the process of continuous transaction management, that is, following the events that occur at the current time point, mobilizing available resources, and executing related tasks to solve emerging problems and prevent transaction accumulation. It has applications in computer programming, public relations, economic activities and other fields.

[0003] With the development of robotics technology, its application scenarios have undergone major changes. A new generation of robots represented by collaborative robotic arms have begun to complete various tasks in manned and complex unknown environments, which puts higher requirements on the robot's autonomous planning and motion control. In order to be able to complete tasks safely and independently in complex location environments, robots must obtain as much reliable information about the surrounding environment as possible, and make full use of this information to generate reliable, safe and effective motion trajectories and achieve good interaction with the external environment. At present, the external information sources of most robots mainly rely on visual sensors, end force sensors, joint torque and encoders, etc. The above-mentioned robot perception methods cannot obtain all-round information about the surrounding environment, which easily leads to perception blind spots in the robot, making the robot not only insufficiently aware of the surrounding environment, but also reducing the utilization rate of surrounding information. Summary of the invention

[0004] The purpose of the present invention is to provide an event-driven environment-aware electronic skin system in order to overcome the defects of the above-mentioned prior art.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] An event-driven environment-aware electronic skin system is applied to the surface of a robot, comprising a multimodal sensing group, a communication module and a control module, wherein the control module is connected to the multimodal sensing group via the communication module;

[0007] The multimodal sensing group includes a plurality of sensing units, each of which is provided with a main control module, a multimodal sensor, an interface and a software protection structure; the plurality of sensing units are connected via an interface to form a multimodal sensing group;

[0008] The main control module is connected to the multimodal sensor; the software protection structure is used to protect the sensing unit;

[0009] The main control module is deployed with a data fusion environment perception algorithm and a data processing unit;

[0010] The data processing unit is event-driven and is used to process the measurement data of the multimodal sensor;

[0011] The data fusion environment perception algorithm has multiple levels, targeting different perception situations, including the following perception levels:

[0012] Level 1: Using the multimodal sensor to obtain object proximity data and pressure data to achieve object positioning and geometric reconstruction around the robot;

[0013] Level 2: Using the multimodal sensor to obtain acceleration data and pressure data of the object, evaluate the touch condition of the object, and estimate the physical parameter properties of the touched object;

[0014] Level 3: Using the multimodal sensor to obtain object proximity data, pressure data and temperature data, to achieve identity recognition of the touched object.

[0015] Furthermore, the multimodal sensor includes a proximity sensor, an acceleration sensor, a pressure sensor and a temperature sensor.

[0016] Furthermore, the event-driven environment-aware electronic skin system also includes an off-board power supply module, which is respectively connected to the multimodal perception group and the communication module.

[0017] Furthermore, the sensing unit is also provided with an on-board power supply module, and the on-board power supply module is respectively connected to the off-board power supply module, the main control module and the multimodal sensor.

[0018] Furthermore, the main control module is provided with a main control chip and a voltage stabilizing chip, and the main control chip controls the data transmission between each sensing unit through the interface.

[0019] Furthermore, the main control chip is connected to the multimodal sensor via an I2C bus and is used to read data in the acquisition module.

[0020] Further, the event-driven main control module determines the working state of the multimodal sensor, and if the sensor measurement data in the multimodal sensor exceeds a preset threshold, the data processing unit is triggered to process the sensor measurement data;

[0021] The data processing unit includes the following process:

[0022] S101: Data acquisition. The main control module is provided with a main control chip. The main control chip determines the working status of each sensor in the multimodal sensor and sets its working mode during the initialization process. When no abnormality is encountered, the main control chip controls the loop to read the data of each sensor and saves it in an array. When an abnormality is encountered, the data value of the corresponding sensor in the array is filled with 0xff, and then enters the next loop to read the data of each sensor.

[0023] S102: Data preprocessing: the main control chip reads the sensor data stored in the array for local preprocessing. For acceleration data, temperature data, and proximity data, the real-time values ​​obtained are judged and abnormal values ​​are removed.

[0024] For the pressure data, during the initial 20 cycles of reading data, the pressure data is summed, outliers are removed, and the average value is calculated as the initial value F of the pressure sensor in the multimodal sensor when it is not pressed. init ; In the process of cyclically reading the data of each sensor, the real-time pressure data F real With the initial value F init Make a difference and get the pressure change value F i =F real -F init , the pressure change value is used as a reflection of the pressure magnitude received by the pressure sensor;

[0025] S103: Data encoding, encoding the pre-processed data according to a fixed format as a data frame for transmission;

[0026] S104: Data transmission. The data frame is transmitted through the interface between each perception unit. The data of each perception unit is transmitted to the terminal perception unit in sequence. The terminal perception unit uses the communication module to send the data to the control module.

[0027] Furthermore, the data fusion environment perception algorithm of level 1 includes the following steps:

[0028] S201: The control module performs spatiotemporal registration of the proximity data and the pressure data;

[0029] S202: The control module performs data-level fusion on the proximity data and the pressure data to locate the object and reconstruct the three-dimensional geometry;

[0030] The spatiotemporal registration is to convert the coordinate positions of all sensor units into the robot coordinate space, including the following steps:

[0031] Align the sampling time of sensor a to that of reference sensor b. The specific expression is:

[0032]

[0033] Among them, x aibj Indicates that the sampling time of sensor a is aligned to the reference sensor b; x ai represents the sampling value of sensor a at time i; x a(i+1) represents the sampling value of sensor a at time i+1, x a(i-1) represents the sampling value of sensor a at time i-1, T a represents the sampling period of sensor a; t bj represents the sampling time of sensor b at time j; t ai represents the sampling time of sensor a at time i;

[0034] Convert the perception unit from the base coordinate system to the robot's parent joint coordinate system. The specific steps are as follows:

[0035] P i =[x,r sinθ i ,r cosθ i ]

[0036] Among them, P i represents the representation of the base coordinate system of the perception unit in the robot's mother joint coordinate system; r represents the radius of the robot body; x represents the X-axis coordinate of the base coordinate system of the perception unit in the robot's mother joint coordinate system; θ i Represents the angle between the base coordinate system of the perception unit and the Y-axis of the robot's parent joint coordinate system.

[0037] Furthermore, the data fusion environment perception algorithm of level 2 includes the following steps:

[0038] S301: Based on level 1, the control module performs spatiotemporal registration of acceleration data;

[0039] S302: The control module fuses the proximity data and the acceleration data to evaluate the contact situation between the robot and the object;

[0040] S303: The control module fuses the pressure data to estimate the physical parameters of the object touched by the robot.

[0041] Furthermore, the level 3 data fusion environment perception algorithm includes the following steps:

[0042] S401: Based on level 2, the control module performs spatiotemporal registration of the temperature data;

[0043] S402: The control module associates the pressure data, the temperature data and the acceleration data;

[0044] S403: The control module extracts features from the pressure data, the temperature data and the acceleration data to obtain feature values ​​of the pressure, the temperature and the acceleration;

[0045] S404: The control module extracts features of the physical parameters of the object estimated at level 2 to obtain feature values ​​of the physical parameters of the object;

[0046] S405: The control module performs feature layer fusion on the feature values ​​of pressure, temperature, acceleration and physical parameters of the object;

[0047] S406: The control module identifies the object touched by the robot.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. The present invention forms a skin system composed of a large number of expandable modular multimodal perception units, and covers the skin system on the surface of the robot, so that the robot can perceive the surrounding environment information in real time and in all directions, effectively reduce the perception blind spots of the robot, and provide important reference information for the robot in terms of autonomous navigation.

[0050] 2. The present invention can detect the temperature, proximity, pressure and acceleration data of the robot's surrounding environment in real time by arranging multimodal sensors in each multimodal sensing unit. The multi-dimensional data makes the robot's recognition of the surrounding environment more accurate, improves the robot's perception ability, and helps the robot make accurate judgments about the surrounding environment.

[0051] 3. The present invention sets a data processing unit in the main control module of each multimodal sensing unit to collect, preprocess and transmit the data obtained by the multi-dimensional sensor. The data processing unit is event-driven and can greatly reduce the data transmission volume of the system and improve the data processing efficiency.

[0052] 4. The present invention can realize data fusion from the data level to the eigenvalue level by setting a data fusion environment perception algorithm in the main control module of each multimodal perception unit, thereby improving the utilization rate of information and further improving the robot's environmental perception ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a hardware system block diagram of the present invention;

[0054] Figure 2 It is a front view of the sensing unit of the present invention;

[0055] Figure 3 It is a back view of the sensing unit of the present invention;

[0056] Figure 4 It is a flow chart of data collection;

[0057] Figure 5 It is a data transmission flow chart;

[0058] Figure 6 This is the flow chart of the data fusion algorithm. DETAILED DESCRIPTION

[0059] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0060] In the drawings, components with the same structure are indicated by the same numerical labels, and components with similar structures or functions are indicated by similar numerical labels. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, some parts in the drawings are appropriately exaggerated.

[0061] The present invention proposes an event-driven environment-aware electronic skin system, which can cover the entire robot body like animal skin and collect all-round information about the robot's surrounding environment in real time. On the one hand, it can supplement the robot's visual blind spot, and on the other hand, it can realize the perception of the surrounding physical parameters, human emotions, etc., which is an effective means to improve the robot's perception ability.

[0062] The multi-modal perception intelligent electronic skin system is composed of numerous regular hexagonal electronic skin units, each of which can collect real-time information such as proximity, pressure, temperature and acceleration. At the same time, the event-driven large-area skin data processing algorithm greatly improves the efficiency of robot information processing, and the multi-level data fusion environment perception algorithm is now data fusion from the data level to the eigenvalue level, which improves the utilization rate of information. This will greatly improve the robot's environmental perception ability.

[0063] like Figure 1 As shown, an event-driven environment-aware electronic skin system is applied to the surface of a robot, including a multimodal sensing group, a communication module 2, an off-board power supply module 3 and a PC control module module 4. The multimodal sensing group includes a plurality of multimodal sensing units 1. The PC control module module 4 is connected to the multimodal sensing unit 1 through the communication module 2. The off-board power supply module 3 is connected to the multimodal sensing unit 1 and the communication module 2 respectively.

[0064] The multimodal sensing unit 1 is an expandable modular multimodal sensing unit, each of which is provided with a main control module 105, a multimodal sensor, an interface, an on-board power supply module 106 and a software protection structure; multiple sensing units are connected through interfaces to form a multimodal sensing group; the off-board power supply module 3 is connected to the on-board power supply module 106 to supply a rated voltage, and the on-board power supply module 106 is connected to the pressure sensor 101, the temperature sensor 102, the acceleration sensor 103, the proximity sensor 104, the UART interface 107 and the main control module 105 to supply a transformed rated voltage.

[0065] The main control module 105 is connected to the multimodal sensor.

[0066] In this embodiment, the multimodal sensor includes a pressure sensor 101, a temperature sensor 102, an acceleration sensor 103 and a proximity sensor 104. Preferably, the acceleration sensor 103 used in this embodiment is a six-axis acceleration sensor.

[0067] In this embodiment, the interface used is a UART interface, and includes a total of 4 interfaces, namely UART1, UART2, UART3 and UART4.

[0068] In this embodiment, the soft protection structure is manufactured by 3D printing, and the material includes one or more of resin, polylactic acid, polyvinyl alcohol, etc., which is used to flexibly wrap the sensing unit and play a passive and compliant role for the robot and the sensing unit.

[0069] like Figure 2 , which is a schematic diagram of the front of the sensing unit, on which a pressure sensor 101, a temperature sensor 102, an acceleration sensor 103, a proximity sensor 104, an on-board power module 106 and an interface 107 are provided;

[0070] like Figure 3 , which is a schematic diagram of the back side of the sensing unit, is provided with a main control module 105.

[0071] The main control module 105 is provided with a main control chip and a voltage stabilizing chip, and the main control chip controls the data transmission between each sensing unit through the UART interface. The main control chip is connected to the pressure sensor 101, the temperature sensor 102, the acceleration sensor 103 and the proximity sensor 104 through the I2C bus, and reads the corresponding data of each sensor through the standard I2C communication protocol; secondly, the main control module 105 processes the read sensor data locally, and sends the encoded data to the adjacent multimodal sensing unit through the serial port. Each sensing unit stores the data of the previous unit, adds the data of this unit and sends it to the next adjacent unit.

[0072] The sensing units are connected to each other through the UART interface to form a multimodal sensing group. The UART3 of the current sensing unit is connected to the UART2 of the adjacent sensing unit, and the UART4 of the current sensing unit is connected to the UART2 of the adjacent sensing unit. The UART interface completes the connection and communication of the sensing units.

[0073] The data of each multimodal perception group is sent to the communication module 2 through the UART interface at the terminal perception unit, and then sent to the PC control module. The advantage of the multimodal perception group communicating with the PC control module through the communication module 2 is that as long as there are enough communication module interfaces, the number of perception groups can be easily and quickly expanded.

[0074] In this embodiment, a data fusion environment perception algorithm and a data processing unit are deployed in the main control module; the data processing unit is event-driven and is used to process the measurement data of the multimodal sensor, including data acquisition, data preprocessing and data transmission;

[0075] Since the electronic skin system proposed in the present invention covers the entire body of the robot and is composed of many sensing units, it has high requirements for real-time performance. The large-area skin data collection and processing algorithm based on event-driven can greatly reduce the amount of data collected and improve the efficiency of information processing.

[0076] The working principle of the event-driven sensor is as follows: the signal change detector directly detects the change of the analog signal and converts the signal x(t) in the continuous time range into the event e i The analog change detector detects the signal x(t) and tracks the change of the signal until the cumulative change exceeds a predefined threshold δ, then triggers the event e i The sensor starts collecting data after the creation of the sensor. The principle can be explained by the following formula:

[0077]

[0078] In this embodiment, the event-driven main control module determines the working status of the multimodal sensor. If the sensor measurement data in the multimodal sensor exceeds a preset threshold, the data processing unit is triggered to process the sensor measurement data. The preset threshold here is formulated by professional and technical personnel in the relevant field according to specific needs.

[0079] The data processing unit includes the following processes:

[0080] like Figure 4 As shown, the data collection process includes the following steps:

[0081] Initialization step: During the initialization process, the main control module determines the working status of each sensor in the multimodal sensor, reads each sensor ID, and sets its working mode;

[0082] Data reading steps: When the main control module encounters no abnormality, it loops to read the data of each sensor and saves it in the memory array; when an abnormality is encountered, the data value of the corresponding sensor in the array is filled with 0xff, and then enters the next loop to read the data of each sensor;

[0083] Data preprocessing steps:

[0084] Processing steps of acceleration data, temperature data and proximity data: the main control module judges the real-time values ​​obtained and removes abnormal values;

[0085] Pressure data processing steps: During the initial 20 cycles of data reading, the pressure data are summed, outliers are removed, and the average value is calculated as the initial value F of the pressure sensor in the multimodal sensor when it is not pressed. init ; In the process of cyclic reading of data, the real-time pressure data F real With the initial value F init Make a difference and get the pressure change value F i =F real -F init , the pressure change value is used as a reflection of the pressure magnitude received by the pressure sensor;

[0086] In this embodiment, the acceleration data is acquired by the acceleration sensor, the temperature data is acquired by the temperature sensor, the proximity data is acquired by the proximity sensor, and the pressure data is acquired by the pressure sensor.

[0087] Data encoding step: Encode the preprocessed data in a fixed format as a data frame for transmission; the data frame format is shown in Table 1.

[0088] Table 1 Data transmission format

[0089] Data content Sensing unit ID Pressure value Approximate value Acceleration value Temperature value Data length (bytes) 1 2 2 12 2

[0090] Data transmission steps: The head node perception unit of the multimodal perception group has a head node identifier and does not need to wait for the input data interrupt. It only needs to send the data of this unit to the next unit in a loop at the set rate. Each perception unit turns on the serial port interrupts of UART1 and UART3. When there is data input, the serial port interrupt is triggered, the input data is received, and the sensor data of this unit is spliced ​​to the end of the input data. According to the input serial port of UART1 or UART3, the output serial port is selected as UART2 or UART4, and the data is sent to the next unit. The data of each perception unit is transmitted to the terminal perception unit in turn, and the terminal perception unit uses the communication module to send the data to the control module.

[0091] like Figure 5As shown in the figure, it is a data transmission flow chart. After the program starts to execute, it will determine whether the sensing unit is the head node. If it is the head node, the data of this unit will be sent to the next unit through the serial port UART2 or UART4; if it is not the head node, the receiving interrupt of the serial port UART1 or UART3 will be enabled. If the receiving interrupt is successfully triggered, the data sent by the previous unit will be received and the data of this unit will be added to the end, and then the unit data will be sent to the next unit through the serial port UART2 or UART4; if the receiving interrupt fails to be triggered, it will return to the step of enabling the receiving interrupt of the serial port UART1 or UART3, and execute the subsequent steps in sequence.

[0092] In this embodiment, the data fusion environment perception algorithm is a multi-level data fusion environment perception algorithm, including the following perception levels for different perception situations:

[0093] Level 1: Use multimodal sensors to obtain object proximity data and pressure data to achieve object positioning and geometric reconstruction around the robot;

[0094] Level 2: Use multimodal sensors to obtain acceleration data and pressure data of objects, evaluate the touch conditions of objects, and estimate the physical parameter properties of the touched objects;

[0095] Level 3: Use multimodal sensors to obtain object proximity data, pressure data, and temperature data to identify the touched object.

[0096] Specifically, the level 1 data fusion environment perception algorithm includes the following steps:

[0097] S201: The control module performs spatiotemporal registration of the proximity data and the pressure data;

[0098] S202: The control module performs data-level fusion on the proximity data and the pressure data to locate the object and reconstruct the three-dimensional geometry.

[0099] The level 2 data fusion environment perception algorithm includes the following steps:

[0100] S301: Based on level 1, the control module performs spatiotemporal registration of acceleration data;

[0101] S302: The control module fuses the proximity data and the acceleration data to evaluate the contact situation between the robot and the object;

[0102] S303: The control module fuses the pressure data to estimate the physical parameters of the object touched by the robot.

[0103] The level 3 data fusion environment perception algorithm includes the following steps:

[0104] S401: Based on level 2, the control module performs spatiotemporal registration of the temperature data;

[0105] S402: The control module associates the pressure data, the temperature data and the acceleration data;

[0106] S403: The control module extracts features from the pressure data, the temperature data and the acceleration data to obtain feature values ​​of the pressure, the temperature and the acceleration;

[0107] S404: The control module extracts features of the physical parameters of the object estimated at level 2 to obtain feature values ​​of the physical parameters of the object;

[0108] S405: The control module performs feature layer fusion on the feature values ​​of pressure, temperature, acceleration and physical parameters of the object;

[0109] S406: The control module identifies the object touched by the robot.

[0110] In this embodiment, a PC is used as the control module.

[0111] like Figure 6 As shown in the figure, it is a flow chart of data fusion algorithm, including:

[0112] (1) Spatiotemporal registration step: The coordinate positions of all sensor units are converted to the robot polar coordinate space, including the following steps:

[0113] Align the sampling time of sensor a to that of reference sensor b. The specific expression is:

[0114]

[0115] Among them, x aibj Indicates that the sampling time of sensor a is aligned to the reference sensor b; x ai represents the sampling value of sensor a at time i; x a(i+1) represents the sampling value of sensor a at time i+1, x a(i-1) represents the sampling value of sensor a at time i-1, T a represents the sampling period of sensor a; t bj represents the sampling time of sensor b at time j; t ai represents the sampling time of sensor a at time i;

[0116] Convert the perception unit from the base coordinate system to the robot's parent joint coordinate system. The specific steps are as follows:

[0117] P i =[x,r sinθ i ,r cosθ i ]

[0118] Among them, P i represents the representation of the base coordinate system of the perception unit in the robot's mother joint coordinate system; r represents the radius of the robot body; x represents the X-axis coordinate of the base coordinate system of the perception unit in the robot's mother joint coordinate system; θ i Represents the angle between the base coordinate system of the perception unit and the Y-axis of the robot's parent joint coordinate system.

[0119] (2) Data association step: The PC parses the unit data packet and associates the data according to the sensor data category of the sensing unit ID.

[0120] (3) Data-level fusion steps:

[0121] The proximity data and pressure data are fused at data level to locate the object and reconstruct the 3D geometry.

[0122] In this embodiment, a cylinder is used as an example, and positioning and three-dimensional geometric reconstruction of an object based on a parameter method include the following steps:

[0123] Determine the baseline:

[0124] r=r0+λu,u=n j-1 ×n j

[0125] Where: n j represents the tangent space at point j on the cylindrical surface; n j-1 represents the tangent space at point j-1 on the cylindrical surface, r0 represents the starting point of the reference line; λ is a coefficient, and the value of λ is defined by professional and technical personnel in the relevant field according to actual conditions;

[0126] Determine the radius and curvature of the cylinder:

[0127]

[0128] Where: R is the radius of the cylinder, k represents the curvature of the cylindrical surface; n j,0 represents the coordinates of point j on the cylindrical surface; n j-1,0 Represents the coordinates of point j-1 on the cylindrical surface.

[0129] The proximity data and acceleration data are further fused to evaluate the contact between the robot and the object.

[0130] The pressure data is further fused to estimate the physical parameters of the object the robot touches.

[0131] (4) Feature extraction step: The PC performs feature extraction on the pressure sensor data, the temperature sensor data, and the acceleration sensor data; and performs feature extraction on the physical parameters of the touched object.

[0132] (5) Feature-level data fusion step: The PC performs feature-level fusion on the characteristic values ​​of pressure, temperature, acceleration, and physical parameters of the object.

[0133] (6) Identity recognition: Identify the touched objects based on the Bayse method and DS evidence theory.

[0134] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. An event-driven environment-aware electronic skin system, applied to the surface of a robot, characterized in that: It includes a multimodal sensing group, a communication module and a control module, wherein the control module is connected to the multimodal sensing group via the communication module; The multimodal sensing group includes a plurality of sensing units, each of which is provided with a main control module, a multimodal sensor, an interface and a software protection structure; the plurality of sensing units are connected via an interface to form a multimodal sensing group; The main control module is connected to the multimodal sensor; the software protection structure is used to protect the sensing unit; The main control module is deployed with a data fusion environment perception algorithm and a data processing unit; The data processing unit is event-driven and is used to process the measurement data of the multimodal sensor; The data fusion environment perception algorithm includes the following perception levels: Level 1: Using the multimodal sensor to obtain object proximity data and pressure data to achieve object positioning and geometric reconstruction around the robot; Level 2: Using the multimodal sensor to obtain acceleration data and pressure data of the object, evaluate the touch condition of the object, and estimate the physical parameter properties of the touched object; Level 3: Using the multimodal sensor to obtain object proximity data, pressure data and temperature data to achieve identity recognition of the touched object; The data fusion environment perception algorithm of level 1 includes the following steps: S201: The control module performs spatiotemporal registration of the proximity data and the pressure data; S202: The control module performs data-level fusion on the proximity data and the pressure data to locate the object and reconstruct the three-dimensional geometry; The spatiotemporal registration is to convert the coordinate positions of all sensor units into the robot coordinate space, including the following steps: Align the sampling time of sensor a to that of reference sensor b. The specific expression is: in, Align the sampling time of sensor a to that of reference sensor b; a The sampling value of the sensor at time i; represents the sampling value of sensor a at time i+1, represents the sampling value of sensor a at time i-1, The sampling period of sensor a; The sampling time of sensor b at time j; The sampling time of sensor a at time i; Convert the perception unit from the base coordinate system to the robot's parent joint coordinate system. The specific steps are as follows: in, The representation of the base coordinate system of the perception unit in the coordinate system of the robot's parent joint; Robot body radius; The X-axis coordinate of the base coordinate system of the sensing unit in the coordinate system of the robot's parent joint; The angle between the base coordinate system of the sensing unit and the Y-axis of the robot's parent joint coordinate system.

2. The event-driven environment-aware electronic skin system according to claim 1, characterized in that: The multimodal sensor includes a proximity sensor, an acceleration sensor, a pressure sensor and a temperature sensor.

3. The event-driven environment-aware electronic skin system according to claim 1, characterized in that: It also includes an off-board power supply module, which is connected to the multimodal sensing group and the communication module respectively.

4. The event-driven environment-aware electronic skin system according to claim 3, characterized in that: The sensing unit is also provided with an on-board power supply module, and the on-board power supply module is respectively connected to the off-board power supply module, the main control module and the multimodal sensor.

5. The event-driven environment-aware electronic skin system according to claim 1, characterized in that: The main control module is provided with a main control chip and a voltage stabilizing chip, and the main control chip controls the data transmission between various sensing units through the interface.

6. The event-driven environment-aware electronic skin system according to claim 5, characterized in that: The main control chip is connected to the multimodal sensor via the I2C bus and is used to read the data in the acquisition module.

7. The event-driven environment-aware electronic skin system according to claim 1, characterized in that: The event-driven main control module determines the working state of the multimodal sensor, and if the sensor measurement data in the multimodal sensor exceeds a preset threshold, the data processing unit is triggered to process the sensor measurement data; The data processing unit includes the following process: S101: Data acquisition. The main control module is provided with a main control chip. The main control chip determines the working status of each sensor in the multimodal sensor and sets its working mode during the initialization process. When no abnormality is encountered, the main control chip controls the loop to read the data of each sensor and saves it in an array. When an abnormality is encountered, the data value of the corresponding sensor in the array is filled with 0xff, and then enters the next loop to read the data of each sensor. S102: Data preprocessing: the main control chip reads the sensor data stored in the array for local preprocessing. For acceleration data, temperature data, and proximity data, the real-time values ​​obtained are judged and abnormal values ​​are removed. For the pressure data, during the initial 20 cycles of reading data, the pressure data is summed, outliers are removed, and the average value is calculated as the initial value of the pressure sensor in the multimodal sensor when it is not pressed. ; In the process of cyclically reading the data of each sensor, the real-time pressure data read is With initial value The pressure change is obtained by subtracting , the pressure change value is used as a reflection of the pressure magnitude received by the pressure sensor; S103: Data encoding, encoding the pre-processed data according to a fixed format as a data frame for transmission; S104: Data transmission. The data frame is transmitted through the interface between each perception unit. The data of each perception unit is transmitted to the terminal perception unit in sequence. The terminal perception unit uses the communication module to send the data to the control module.

8. The event-driven environment-aware electronic skin system according to claim 1, characterized in that: The data fusion environment perception algorithm of level 2 includes the following steps: S301: Based on level 1, the control module performs spatiotemporal registration of acceleration data; S302: The control module fuses the proximity data and the acceleration data to evaluate the contact situation between the robot and the object; S303: The control module fuses the pressure data to estimate the physical parameters of the object touched by the robot.

9. The event-driven environment-aware electronic skin system according to claim 8, characterized in that: The Level 3 data fusion environment perception algorithm includes the following steps: S401: Based on level 2, the control module performs spatiotemporal registration of the temperature data; S402: The control module associates the pressure data, the temperature data and the acceleration data; S403: The control module extracts features from the pressure data, the temperature data and the acceleration data to obtain feature values ​​of the pressure, the temperature and the acceleration; S404: The control module extracts features of the physical parameters of the object estimated at level 2 to obtain feature values ​​of the physical parameters of the object; S405: The control module performs feature layer fusion on the feature values ​​of pressure, temperature, acceleration and physical parameters of the object; S406: The control module identifies the object touched by the robot.

Images