A multimodal, reconfigurable, and scalable interactive flexible electronic skin

Abstract

This invention discloses a multimodal, reconfigurable, and scalable interactive flexible electronic skin. The flexible electronic skin proposed in this invention includes a flexible sensing array, a light-emitting interaction unit, a data reading and processing unit, and an array reconstruction unit. The flexible sensing array is connected to the data reading and processing unit and a microcontroller unit. The microcontroller unit is connected to the light-emitting interaction unit, the flexible electronic skin expansion interface, a bus communication unit, and the flexible electronic skin data output interface. Adjacent skin structure units are connected through the flexible electronic skin expansion interface. The microcontroller unit controls the data reading and processing unit to achieve switching and reconstruction of the sensing modes of the flexible sensing array. The flexible electronic skin of this invention features multimodal proximity sensing, intuitive visualization of detection results, and reconfigurable and scalable functions, which is beneficial for achieving large-area coverage on the robot body, ensuring the safety of the human-computer interaction process, and improving the smoothness of the human-computer interaction process.

Description

Technical Field

[0001] This invention relates to a flexible electronic skin in the field of flexible sensors, specifically a multimodal, reconfigurable, and scalable interactive flexible electronic skin. Background Technology

[0002] Since the beginning of the 21st century, robots have played an indispensable role in various fields such as national defense, industry, agriculture, education, healthcare, and entertainment. As a highly versatile, portable, and easily deployed automated machine system, collaborative robots have gradually expanded their applications beyond traditional production lines and promoted a shift in decision-making models from machine-driven to human-centered approaches, thereby strengthening the direct interaction between humans and robots. With the continuous transformation of industries and the gradual expansion of robot families such as industrial robots and collaborative robots, the working model between robots and humans has undergone several transformations. Compared with the fixed industrial environment, the workspace of collaborative robots is mostly highly dynamic and unstructured. One of the main challenges in dealing with this generally unknown, human-dominated work environment is developing strategies for safe and reliable interaction with humans.

[0003] Sensors deployed on collaborative robots are crucial components for ensuring safe interaction between them. Flexible electronic skin, based on the principles of human skin and made of flexible materials, is attached to the surface of collaborative robots, acting as a new physical barrier between the robot and its environment. With the rapid development of intelligent technologies, the demand for flexible electronic skin with intelligent sensing and interaction capabilities is constantly increasing. While traditional flexible electronic skin has achieved some degree of environmental perception, it still suffers from drawbacks such as narrow sensing range, limited functionality, and lack of autonomous learning and adaptability.

[0004] Current research and applications of robotic skin primarily focus on optimizing the structure and performance of flexible sensors. Safety strategies rely on emergency stopping and avoidance after a collision, failing to prevent injury from the contact itself. Furthermore, it places high demands on the robot's speed and reaction time, making it difficult to meet the safety requirements of human-computer interaction. In addition, most current research on flexible electronic skin focuses on the development and performance improvement of the flexible sensor itself, with less consideration for the cascading and expansion of individual skin units. This makes it difficult to cover large areas of the robot body, hindering practical applications. Moreover, typical electronic skin cannot provide direct interactive feedback; instead, it transmits the generated interaction data to a computer, controlling the robot's movement to provide feedback to the interacting object. This interaction strategy suffers from low efficiency, lack of intuitiveness, and high latency, which to some extent restricts the further development of human-computer interaction technology. Summary of the Invention

[0005] To address the problems of current interactive flexible electronic skin, such as limited sensing modes, inflexible reconfiguration, inability to cover large areas, and unintuitive interaction results, this invention proposes a multimodal, reconfigurable, and scalable interactive flexible electronic skin.

[0006] The technical solution adopted in this invention is as follows:

[0007] The interactive flexible electronic skin includes a flexible sensing array, a flexible printed circuit board substrate, a light-emitting interactive unit, a data reading and processing unit, a flexible electronic skin expansion connection interface, a microcontroller unit, and a bus communication unit. The flexible sensing array is fixedly mounted on the flexible printed circuit board substrate, and the light-emitting interactive unit is fixedly mounted on the flexible sensing array. The flexible sensing array is connected to the data reading and processing unit and the microcontroller unit respectively. The light-emitting interactive unit, the data reading and processing unit, the flexible electronic skin expansion connection interface, and the bus communication unit are all connected to the microcontroller unit. The data reading and processing unit, the flexible electronic skin expansion connection interface, the microcontroller unit, the flexible electronic skin data output interface, and the bus communication unit are all mounted on one side of the flexible printed circuit board substrate. The flexible sensing array achieves multimodal sensing function by switching between long-distance sensing mode, short-distance sensing mode, and contact sensing mode. The microcontroller unit controls the data reading and processing unit to realize the switching and reconstruction of the sensing mode of the flexible sensing array.

[0008] The flexible sensing array includes multiple flexible sensing light-emitting units mounted on a flexible printed circuit board substrate in an array arrangement. Each flexible sensing light-emitting unit includes two flexible proximity sensing unit row electrodes and two flexible proximity sensing unit column electrodes. The two flexible proximity sensing unit row electrodes are arranged at intervals, and a corresponding flexible proximity sensing unit column electrode is arranged between each side of the two flexible proximity sensing unit row electrodes, thereby forming a square-shaped flexible sensing light-emitting unit. Some of the flexible sensing light-emitting units in the flexible sensing array also include time-of-flight sensing units, which are arranged between the two flexible proximity sensing unit row electrodes and the two flexible proximity sensing unit column electrodes. Multiple time-of-flight sensing units are arranged in an array arrangement. The flexible proximity sensing unit row electrodes, flexible proximity sensing unit column electrodes, and time-of-flight sensing units are all connected to the data reading and processing unit.

[0009] The data reading and processing unit includes a multimodal sensing module, a time-of-flight sensing module, and an array reconstruction unit. The multimodal sensing module is connected to the array reconstruction unit, the multimodal sensing module is connected to the flexible sensing array, the time-of-flight sensing module is connected to the flexible sensing array, both the multimodal sensing module and the array reconstruction unit are connected to the microcontroller unit, and the flexible sensing array is connected to the array reconstruction unit.

[0010] The multimodal sensing module includes a multimodal sensing detection chip and a multimodal sensing voltage regulator chip connected together. The multimodal sensing detection chip is connected to the array reconstruction unit, the multimodal sensing detection chip is connected to the column electrode of the flexible proximity sensing unit of the flexible sensing array, the multimodal sensing voltage regulator chip is connected to the row electrode of the flexible proximity sensing unit of the flexible sensing array, and the multimodal sensing voltage regulator chip is also connected to the microcontroller unit.

[0011] The time-of-flight sensing module includes a connected time-of-flight sensing read selection unit and a time-of-flight sensing read extension unit; the time-of-flight sensing read extension unit is connected to the time-of-flight sensing unit of the flexible sensing array.

[0012] The array reconstruction unit includes an array reconstruction control unit, an array reconstruction selection unit, and an array reconstruction logic processing unit connected in sequence; the array reconstruction selection unit is connected to the flexible proximity sensing unit column electrode of the flexible sensing array, the array reconstruction selection unit is also connected to the multimodal sensing module, and the array reconstruction control unit is connected to the microcontroller unit.

[0013] The array reconfiguration selection unit includes K input terminals, K output terminals, and K single-pole double-throw switches, where K is the number of column electrodes in a single row of flexible proximity sensing units; the reconfiguration control unit includes L reconfiguration control terminals, where L is log2K rounded up; the array reconfiguration logic processing unit includes several OR gates; all L reconfiguration control terminals are connected to the microcontroller unit, and each of the L reconfiguration control terminals is connected to the control pins of the K single-pole double-throw switches through several OR gates, providing control signals to the control pins of the K single-pole double-throw switches;

[0014] Each single-pole double-throw switch has two normally open pins S1A and S1B as output terminals, and input pin D as an input terminal. The column electrodes of the K flexible proximity sensing units in the row are respectively connected to the K input terminals of the single-pole double-throw switch. Among the K input terminals, the first input terminal is directly connected to the first output terminal, and the remaining K-1 input terminals are respectively connected to the input pins of the single-pole double-throw switch.

[0015] Both the multimodal sensing and detection chip and the time-of-flight sensing readout extension unit support the IIC communication protocol, and the multimodal sensing and detection chip has a programmable IIC address. The scalable interactive flexible electronic skin system enables communication between the microcontrollers of different flexible electronic skins via the IIC communication protocol, thus achieving scalability. Up to 128 identical flexible electronic skins can be extended onto the flexible electronic skin of this invention.

[0016] The flexible electronic skin is equipped with multi-level distance detection thresholds, and the data reading and processing unit and the microcontroller unit realize dynamic reconstruction and dynamic dereconstruction in the proximity sensing mode of the flexible sensing and light-emitting array.

[0017] Multiple multimodal reconfigurable and scalable interactive flexible electronic skins with the same structure are connected in series via a flexible electronic skin expansion connection interface. One of the multi-level sensing flexible robot skins is used as the master node, and the other multi-level sensing flexible robot skins are used as slave nodes. The microcontroller unit of the master node selectively reads the sensing data of the flexible sensing light-emitting units of the slave nodes by polling the addresses configured for the microcontroller units of the other slave nodes. The read sensing data is finally aggregated in the microcontroller unit of the master node, thereby forming a scalable interactive flexible robot skin.

[0018] The light-emitting interaction unit is an addressable programmable light-emitting device, which generates light-emitting interaction modes according to the different working modes of the flexible electronic skin.

[0019] The beneficial effects of this invention are as follows:

[0020] This invention utilizes flexible sensing arrays, array reconstruction, and data reading circuit technology to achieve dynamic adjustment of maximum detection distance and spatial resolution. Through the design based on multimodal sensing principles, the flexible robot skin can achieve a large-range proximity sensing and high spatial resolution touch sensing according to the needs of specific working environments. The flexible sensing array allows the robot to flexibly perceive its surroundings at different distances, ensuring its safety and adaptability in various working scenarios. Simultaneously, through array reconstruction and data reading and processing units, the robot can dynamically adjust the layout of the sensing array and the data acquisition method, achieving flexible control of spatial resolution. This dynamic adjustment characteristic not only facilitates safe, non-contact human-machine interaction between the user and the robot but also enables the flexible electronic skin to become a window and interface for user input commands. By recognizing the user's touch trajectory, the flexible robot skin of this invention can complete some low-repetition or unstructured tasks, improving the robot's intelligence and flexibility and providing new possibilities for human-machine interaction.

[0021] This invention has strong scalability. The flexible electronic skin system described herein can realize communication between the microcontrollers of different flexible electronic skins through the IIC communication protocol, so that the sensing data of multiple flexible electronic skins as slave nodes can be collected into a microcontroller of a flexible robot skin as the master node. This invention can realize the expansion of up to 128 flexible electronic skins with the same structure, which can easily achieve large-area coverage on the robot body and greatly help to improve the robot's environmental perception capabilities.

[0022] The flexible electronic skin described in this invention has multimodal sensing capabilities, including long-distance sensing, short-distance sensing, and contact sensing modes. The sensing modes of the flexible electronic skin can be switched by adjusting the operating states of the row and column electrodes of the time-of-flight sensing unit and the flexible proximity sensing unit. In actual detection, the operating mode of the flexible electronic skin is determined based on the minimum distance between the external environmental object or user and the electronic skin. Only one sensing mode is activated at a time, which not only significantly reduces the power consumption of the flexible electronic skin but also improves its sensitivity and extends its service life.

[0023] The flexible electronic skin described in this invention is equipped with multi-level detection thresholds. Dynamic reconstruction and de-reconstruction of the flexible sensing and luminescent array in proximity sensing mode are achieved through the array reconstruction unit and microcontroller unit within the data reading and processing unit. When the flexible electronic skin detects an object or user moving away, the array reconstruction unit controls short circuits between adjacent row electrodes and adjacent column electrodes. These short-circuited proximity sensing electrodes can be considered as a reconstructed proximity sensing unit with an increased area. A larger electrode area results in a greater detection distance, while a smaller electrode area provides higher detection resolution, thus achieving the effect of reducing resolution and increasing detection distance. When the flexible electronic skin detects an object or user approaching, the array reconstruction unit controls the flexible proximity sensing electrodes to reverse reconstruct, forming multiple small-area sensing units. Therefore, the flexible proximity sensing unit achieves the goal of reducing detection distance in exchange for higher spatial resolution by implementing the array reverse dynamic reconstruction process.

[0024] This invention integrates addressable programmable light-emitting devices into electronic skin, enabling the robot skin to emit light and interact. This allows information detected by sensors to be quickly and intuitively fed back to the operator, thus optimizing the human-computer interaction process in various scenarios. The visualized interaction not only improves the efficiency and safety of human-computer interaction but also enhances the smoothness of the interaction process and user engagement, potentially bringing positive impacts to human-computer interaction in various fields. Attached Figure Description

[0025] Figure 1 This is an overall structural diagram of the flexible electronic skin of the present invention;

[0026] Figure 2 This is a schematic diagram of the appearance of the flexible sensing electrode of the flexible electronic skin of the present invention.

[0027] Figure 3 This is an extended connection diagram of the flexible electronic skin of the present invention;

[0028] Figure 4 This is a schematic diagram of the reconstruction of the flexible proximity sensing unit of the flexible electronic skin of the present invention;

[0029] Figure 5 This is a schematic diagram of the electrical connections of the various components of the flexible electronic skin of the present invention;

[0030] Figure 6 This is a circuit diagram of the array reconstruction part of the flexible electronic skin of the present invention;

[0031] In the figure: 1. Flexible sensing array; 101. Row electrode of flexible proximity sensing unit; 102. Column electrode of flexible proximity sensing unit; 103. Time-of-flight sensing unit; 2. Flexible printed circuit board substrate; 3. Light-emitting interaction unit; 4. Data reading and processing unit; 4. Multimodal sensing module; 410. Multimodal sensing detection chip; 411. Multimodal sensing voltage regulator chip; 412. Time-of-flight sensing module; 420. Time-of-flight sensing reading selection unit; 421. Time-of-flight sensing reading expansion unit; 422. Array reconstruction unit; 430. Array reconstruction control unit; 431. Array reconstruction selection unit; 432. Array reconstruction logic processing unit; 433. Flexible electronic skin expansion connection interface; 5. Microcontroller unit; 6. Bus communication unit; 7. Flexible electronic skin data output interface; 8. Data output circuit; 9. Detailed Implementation

[0032] The invention will be further described below with reference to the accompanying drawings.

[0033] like Figure 1 As shown, the present invention includes a flexible sensing array 1, a flexible printed circuit board substrate 2, a light-emitting interactive unit 3, a data reading and processing unit 4, a flexible electronic skin extension connection interface 5, a microcontroller unit 6, a bus communication unit 7, a flexible electronic skin data output interface 8, and a data output circuit 9. The flexible sensing array 1 is fixedly mounted on the flexible printed circuit board substrate 2, and the light-emitting interactive unit 3 is fixedly mounted on the flexible sensing array 1. The flexible sensing array 1 is connected to the data reading and processing unit 4 and the microcontroller unit 6, respectively. The light-emitting interactive unit 3, the data reading and processing unit 4, the flexible electronic skin extension connection interface 5, the bus communication unit 6, the flexible electronic skin data output interface 7, the flexible electronic skin data output interface 8, and the bus communication unit 9 are all connected to the flexible electronic skin substrate 2. The communication unit 7 and the flexible electronic skin data output interface 8 are both connected to the microcontroller 6. The flexible electronic skin data output interface 8 is connected to the data output circuit 9. The data reading and processing unit 4, the flexible electronic skin expansion connection interface 5, the microcontroller 6, the flexible electronic skin data output interface 8, and the bus communication unit 7 are all installed on one side of the flexible printed circuit board substrate 2. The flexible sensing array realizes multimodal sensing function by switching between long-distance sensing mode, short-distance sensing mode, and contact sensing mode. The microcontroller 6 realizes the sensing mode switching and reconstruction of the flexible sensing array by controlling the data reading and processing unit 4.

[0034] The flexible sensing array 1 includes multiple flexible sensing light-emitting units mounted on a flexible printed circuit board substrate 2 in an array arrangement. Each flexible sensing light-emitting unit includes two flexible proximity sensing unit row electrodes 101 and two flexible proximity sensing unit column electrodes 102. The two flexible proximity sensing unit row electrodes 101 are arranged at intervals, and a corresponding flexible proximity sensing unit column electrode 102 is arranged between each side of the two flexible proximity sensing unit row electrodes 101. The two flexible proximity sensing unit row electrodes 101 and the two flexible proximity sensing unit column electrodes 102 are also arranged at intervals, thereby forming a square-shaped flexible sensing light-emitting unit, i.e., the second basic structure. Some of the flexible sensing light-emitting units in the flexible sensing array 1 also include time-of-flight sensing units 103. The time-of-flight sensing units 103 are arranged in the middle of the two flexible proximity sensing unit row electrodes 101 and the two flexible proximity sensing unit column electrodes 102, and the two flexible proximity sensing unit row electrodes 101, the two flexible proximity sensing unit column electrodes 102 and the time-of-flight sensing units 103 are all arranged at intervals, i.e., the first basic structure. The multiple time-of-flight sensing units 103 are arranged in an array arrangement. The first and second basic structures are arranged according to a certain pattern, as shown in the figure. There is a second basic structure between two adjacent first basic structures, and a first basic structure between two adjacent second basic structures. The row electrode 101, column electrode 102, and time-of-flight sensing unit 103 of the flexible proximity sensing unit are all connected to the data reading and processing unit 4. The row electrode 101 of the flexible proximity sensing unit is connected to the multimodal sensing voltage regulator chip 412, the column electrode 102 of the flexible proximity sensing unit is connected to the multimodal sensing detection chip 411 and the array reconstruction selection unit 432, and the time-of-flight sensing unit 103 is connected to the time-of-flight sensing reading extension unit 422.

[0035] The light-emitting interaction unit 3 is an addressable programmable light-emitting device that generates light-emitting interaction modes according to the different working modes of the flexible electronic skin.

[0036] The data reading and processing unit 4 includes a multimodal sensing module 410, a time-of-flight sensing module 420, and an array reconstruction unit 430. The multimodal sensing detection chip 411 of the multimodal sensing module 410 is connected to the array reconstruction selection unit 432 of the array reconstruction unit 430. The multimodal sensing voltage regulator chip 412 of the multimodal sensing module 410 is connected to the row electrode 101 of the flexible proximity sensing unit of the flexible sensing array 1. The time-of-flight sensing reading extension unit 422 of the time-of-flight sensing module 420 is connected to the time-of-flight sensing unit 103 of the flexible sensing array 1. Both the multimodal sensing module 410 and the array reconstruction unit 430 are connected to the microcontroller unit 6. The column electrode 102 of the flexible proximity sensing unit of the flexible sensing array 1 is connected to the multimodal sensing detection chip 411 and the array reconstruction selection unit 432 of the array reconstruction unit 430.

[0037] The multimodal sensing module 410 includes a multimodal sensing detection chip 411 and a multimodal sensing voltage regulator chip 412 connected together. The multimodal sensing detection chip 411 is connected to the array reconstruction selection unit 432 of the array reconstruction unit 430, and is also connected to the column electrode 102 of the flexible proximity sensing unit of the flexible sensing array 1. The multimodal sensing voltage regulator chip 412 is connected to the row electrode 101 of the flexible proximity sensing unit of the flexible sensing array 1, and is also connected to the microcontroller unit 6. When an external object or a human body approaches or comes into contact with the flexible sensing array 1, the multimodal sensing detection chip 411 outputs a corresponding value in real time.

[0038] The time-of-flight sensing module 420 includes a connected time-of-flight sensing readout selection unit 421 and a time-of-flight sensing readout extension unit 422; the time-of-flight sensing readout extension unit 422 is connected to the time-of-flight sensing unit 103 of the flexible sensing array 1. When an external environmental object or a human body approaches the time-of-flight sensing unit 103, the real-time output value of the time-of-flight sensing unit 103 changes accordingly. When the real-time output value change of the row electrode 101 and column electrode 102 of the flexible proximity sensing unit is simultaneously detected by the multimodal sensing detection chip 411, the corresponding intersection area can be accurately located. Therefore, the intersection area of ​​the row electrode 101 and column electrode 102 of the flexible proximity sensing unit naturally forms the basic sensing unit of the multimodal reconfigurable and expandable interactive flexible electronic skin.

[0039] In a multimodal, reconfigurable, and scalable interactive flexible electronic skin, at the initial power-on moment, the proximity sensing chip 411 measures the values ​​of the row electrode 101 of the proximity sensing unit and the column electrode 102 of the flexible proximity sensing unit several times and takes the average as the initial reference value of the row electrode 101 of the proximity sensing unit and the column electrode 102 of the flexible proximity sensing unit, respectively.

[0040] The array reconstruction unit 430 includes an array reconstruction control unit 431, an array reconstruction selection unit 432, and an array reconstruction logic processing unit 433 connected in sequence. The array reconstruction selection unit 432 is connected to the flexible proximity sensing unit column electrode 102 of the flexible sensing array 1. The array reconstruction selection unit 432 is also connected to the multimodal sensing detection chip 411 of the multimodal sensing module 410. The array reconstruction control unit 431 is connected to the microcontroller unit 6.

[0041] When only the time-of-flight sensing unit 103 is working, the multimodal reconfigurable and scalable interactive flexible electronic skin operates in a long-distance sensing mode. The flexible robot skin is equipped with proximity sensing detection thresholds and contact sensing detection thresholds. When only the row electrodes 101 and column electrodes 102 of the flexible proximity sensing unit are working, and the difference between the real-time output value and the initial reference value of the row electrodes 101 and column electrodes 102 of the flexible proximity sensing unit and the initial reference value is greater than or equal to the proximity sensing detection threshold and less than or equal to the contact sensing detection threshold, the flexible electronic skin operates in a short-distance sensing mode. When only the row electrodes 101 and column electrodes 102 of the flexible proximity sensing unit are working, and the difference between the real-time output value and the initial reference value of the proximity sensing detection chip 411 in the data reading and processing unit 4 is greater than the contact threshold or the contact sensing threshold, the flexible electronic skin operates in a contact sensing mode.

[0042] The flexible electronic skin is equipped with multi-level distance detection thresholds. Through the array reconstruction unit 430 and microcontroller unit 6 in the data reading and processing unit 4, dynamic reconstruction and dynamic dereconstruction of the flexible sensing light-emitting array in the proximity sensing mode are realized.

[0043] This invention employs proximity sensing based on the time-of-flight principle and approach and contact sensing based on the capacitance principle, arranging flexible sensing units in an array to form the flexible sensing array of this invention. This enables the flexible electronic skin to have proximity sensing with a high detection range and approach and contact sensing with high spatial resolution.

[0044] The flexible sensing array 1 endows the flexible electronic skin of this invention with proximity sensing capabilities, effectively compensating for the shortcomings of traditional external cameras in robot vision. Compared with depth cameras and image processing methods, this invention provides robots with accurate distance sensing capabilities unaffected by environmental complexity and visibility, enabling robots to detect the approach of external objects before collision, achieving non-contact and safe human-machine interaction. Furthermore, based on the reconfigurable characteristics of the flexible electronic skin of this invention, the maximum detection distance and spatial resolution can be dynamically adjusted according to the needs of different working environments, maximizing its functionality. It not only achieves non-contact and safe human-machine interaction but can also serve as an input terminal for operator commands, controlling the robot to complete corresponding tasks. Since the flexible robot skin of this invention can be widely deployed on the robot surface, the proximity sensing function can also significantly reduce the robot's visual blind spots, greatly enhancing the robot's perception of the external environment.

[0045] The array reconstruction unit 430 and microcontroller unit 6 in the data reading and processing unit 4 realize dynamic reconstruction and dynamic dereconstruction in the proximity sensing mode of the flexible sensing light-emitting array. When the flexible electronic skin senses that an object or user is moving away, the array reconstruction unit 430 controls the adjacent row electrodes and adjacent column electrodes of the flexible sensing array 1 to be short-circuited. The multiple proximity sensing electrodes after short-circuiting can be regarded as a reconstructed proximity sensing unit with an increased area. The larger the electrode area, the greater the detection distance; the smaller the electrode area, the higher the detection resolution, thereby achieving the effect of reducing resolution and increasing detection distance. When the flexible electronic skin senses that an object or user is approaching, the array reconstruction unit 430 controls the flexible proximity sensing electrodes to reverse reconstruct to form multiple small-area sensing units. Therefore, the flexible proximity sensing unit achieves the purpose of reducing detection distance and obtaining higher spatial resolution by implementing the array reverse dynamic reconstruction process.

[0046] With this structure, the flexible sensing array 1 of the present invention consists of multiple orthogonally arranged flexible proximity sensing electrodes, according to... Figure 2 The basic structure shown is an array of size including but not limited to 6×2. The array structure of more flexible sensing units is the same, which can realize a larger detection range for proximity and contact sensing.

[0047] The specifically implemented flexible sensing array 1 is a 6×2 array composed of row electrodes 101 and column electrodes 102 of flexible proximity sensing units, with a time-of-flight sensing unit 103 installed at the intersection of the row electrodes 101 and column electrodes 102. The row electrodes 101 and column electrodes 102 are made by plating copper foil with gold. The gold-plated sensing electrodes have good conductivity and can utilize the principle of self-capacitance to reflect changes in the distance of an object or user approaching as changes in capacitance, thereby detecting the approach of an object. The presence of the gold plating layer not only improves the conductivity of the sensing electrodes, which is beneficial for expanding the sensing range, but also enhances the wear resistance of the sensing electrodes, extending the lifespan of the flexible electronic skin. The electrode shape in the specific implementation is as follows: Figure 2 As shown, each row electrode 101 and column electrode 102 of the flexible proximity sensing unit has the same shape. The time-of-flight sensing unit 103 is located at the intersection of the row electrode 101 and column electrode 102 of the flexible proximity sensing unit, which not only makes up for the short detection distance of the flexible sensing electrode, but also realizes the detection of the approach of external objects at the intersection.

[0048] The function of the data reading and processing unit 4 is to filter the capacitance value detected by the flexible sensing array 1 and convert it from analog signal to digital signal to obtain a stable digital signal, and then send it to the external microcontroller unit 6. The external microcontroller unit 6 then applies the data to control the interaction mode of the light-emitting interactive unit 3, and sends the data to the robot control system for further motion control.

[0049] The multimodal sensing and detection chip 411 is the MPR121, which can connect to 12 external electrodes, fully meeting the 6×2 array requirement of the flexible sensing array 1. The MPR121 chip also provides an internal short-circuit mode, allowing the 12 external electrodes to be used as a single large-area electrode. Since the detection distance of the capacitive proximity sensor is proportional to the electrode area, this mode can significantly improve the measurement range of the flexible robotic skin of this invention.

[0050] The flexible robotic skin of this invention can operate in an internal short-circuit mode by default after power-on, thereby providing the maximum proximity detection range. When the distance between an external object and the flexible robotic skin is less than a limit distance, i.e., the capacitance detection result is greater than the limit value, the external microcontroller switches to normal mode. In this mode, the 12 electrodes work separately, thereby improving spatial resolution for sensing the specific position of external objects and receiving user gesture commands. Furthermore, the MPR121 chip supports the IIC communication protocol, which can send the sensing data generated by the flexible sensing array 1 to the external microcontroller 6 in real time. The MPR121 chip has four built-in IIC communication addresses. Connecting the ADDR pin to VDD (i.e., 3.3V high level), VSS (i.e., GND low level), SDA, and SCL pins respectively enables the selection of four IIC addresses. Based on the feature of the data processing chip that allows hardware configuration of four different communication addresses, one external microcontroller 6 can simultaneously control four flexible robotic skins of this invention and read data. This is achieved by connecting the four flexible robotic skins of this invention as follows: Figure 3 The components are connected in series as shown, which realizes the scalability of the flexible robot skin of this invention and greatly saves the hardware resources of the external microcontroller. With the help of mature communication protocols such as SPI, IIC and CAN bus between external microcontrollers, the flexible robot skin of this invention can be easily implemented to cover a large area of ​​the robot body, thereby reducing or even eliminating detection blind spots.

[0051] The working principle of the time-of-flight sensing unit 103 is based on the time-of-flight sensing principle. The emitter of the time-of-flight sensor emits infrared light pulses through a built-in light source. After propagation through the medium, the pulses are emitted onto the surface of the target object. Part of the light is reflected by the surface of the target object and then received by the receiver of the time-of-flight sensor. Based on the time interval between the emitter and receiver and the speed of infrared light propagation in the medium, the distance between the object or human body and the sensor can be calculated. The time-of-flight sensor exhibits many advantages in terms of long-distance measurement accuracy, response speed, and applicability to various materials, making it particularly suitable for flexible electronic skin to detect the approach of external obstacles or human bodies at long distances. It can accurately measure the position and movement of objects at considerable distances.

[0052] like Figure 3As shown, multiple multimodal reconfigurable and scalable interactive flexible electronic skins with the same structure are connected in series via flexible electronic skin expansion connection interface 5. One of the multi-level sensing flexible robot skins is used as the master node, and the other multi-level sensing flexible robot skins are used as slave nodes. The microcontroller unit 6 of the master node selectively reads the sensing data of the flexible sensing light-emitting units 2 of the slave nodes from the addresses configured on the microcontroller units 6 of the other slave nodes in a polling manner. The read sensing data is finally summarized in the microcontroller unit 6 of the master node, thereby forming a scalable interactive flexible robot skin.

[0053] The reconfigurable characteristic of the flexible sensing array 1 is achieved under the control of the array reconstruction unit 430. Short circuits are implemented between adjacent row electrodes and adjacent column electrodes, causing the flexible proximity sensing array to be gradually reconfigured from the initial 6×2 to 3×2, 2×2, 3×1, 2×1 and 1×1 arrays. The two or more flexible proximity sensing electrodes after short circuit can be regarded as a single electrode proximity sensor with an increased area. Since the detection distance of the single electrode proximity sensor is positively correlated with the electrode area, the short circuit reconstruction process actually improves the maximum detection distance of the flexible robot skin of the present invention. However, the short circuit reconstruction also renders the original basic sensing unit ineffective, and replaces it with an enlarged reconfigured sensing unit with an area that is 2, 3, 4, 6 and 12 times larger than the basic sensing unit. Since the area of ​​the sensing unit determines the spatial resolution of the flexible robot skin of the present invention, the short circuit reconstruction process actually reduces the spatial resolution. That is, the short circuit reconstruction process is a process of reducing spatial resolution in exchange for a larger detection distance, while the reverse reconstruction process of short circuit reconstruction is a process of reducing detection distance in exchange for higher spatial resolution.

[0054] The specific implementation of the flexible proximity sensor array reconfiguration is as follows: Figure 4 As shown, Figure 4 (a) is a schematic diagram of the basic sensory unit structure before reconstruction. Figure 4 (b) Figure 4 (c) Figure 4 (d) Figure 4 of (e) Figure 4(f) represents the reconstructed 3×2, 2×2, 3×1, 2×1, and 1×1 arrays, respectively. The dashed boxes represent the reconstructed sensing units. It can be seen that the area of ​​the reconstructed sensing unit of the 3×2 array is twice that of the basic sensing unit, the area of ​​the reconstructed sensing unit of the 2×2 array is three times that of the basic sensing unit, the area of ​​the reconstructed sensing unit of the 3×1 array is four times that of the basic sensing unit, the area of ​​the reconstructed sensing unit of the 2×1 array is six times that of the basic sensing unit, and the area of ​​the reconstructed sensing unit of the 1×1 array is twelve times that of the basic sensing unit. Obviously, the short-circuit reconstruction process is a process of reducing spatial resolution in exchange for the maximum detection distance.

[0055] The data reading and processing unit 4 controls the reconstruction of the sensing array, reads the sensing data from the flexible sensing array 1, and transmits the read capacitive data to the microcontroller unit 6. The microcontroller unit 6 then issues commands to control the light-emitting interactive unit 3. The electrical connections of each part are as follows: Figure 5 As shown. The specific implementation of the array reconstruction unit 430 includes an array reconstruction control unit 431, an array reconstruction selection unit 432, and an array reconstruction logic processing unit 433. The multimodal sensing detection chip 411 is connected to the microcontroller unit 6 and the row electrode 101 of the flexible proximity sensing unit through the multimodal sensing voltage regulator chip 412. The multimodal sensing detection chip 411 is directly or through the array reconstruction unit 430 connected to the column electrode 102 of the flexible proximity sensing unit. When an external environmental object or a human body approaches or comes into contact with the flexible sensing array 1, the real-time output value of the multimodal sensing detection chip 411 changes accordingly.

[0056] The circuit diagram of the array reconfiguration unit 430 implemented in this way is as follows: Figure 6 As shown, Figure 6 The circuit shown is the reconstruction circuit for the column electrodes in the flexible sensing array 1. The reconstruction circuit for the row electrodes follows the same principle and is therefore not shown. The array reconstruction selection unit 432 includes K input terminals, K output terminals, and K single-pole double-throw switches, where K is the number of column electrodes 102 in a single row of flexible proximity sensing units. The reconstruction control unit 430 includes L reconstruction control terminals, where L is log2K rounded up. The array reconstruction logic processing unit 433 includes several OR gates.

[0057] All L reconfiguration control terminals are connected to the microcontroller 6. Each of the L reconfiguration control terminals is connected to the control pins of the K single-pole double-throw switches through several OR gates. The L reconfiguration control terminals provide control signals to the control pins of the K single-pole double-throw switches. In specific implementation, depending on the needs of the reconfiguration array, different single-pole double-throw switches are controlled by different numbers of control signals. A single-pole double-throw switch controlled by one control signal directly connects its corresponding reconfiguration control terminal to the control pin, while a single-pole double-throw switch controlled by multiple control signals aggregates the multiple reconfiguration control terminals through an OR gate and then connects them to the control pin.

[0058] Each single-pole double-throw switch has two normally open pins S1A and S1B as output terminals, and an input pin D as an input terminal. The K flexible proximity sensing unit column electrodes 102 on the row are respectively connected to the K input terminals of the single-pole double-throw switch. Among the K input terminals, the first input terminal is directly connected to the first output terminal, and the remaining K-1 input terminals are respectively connected to the input pins of the single-pole double-throw switch.

[0059] DIGITPIN2, DIGITPIN3, and DIGITPIN6 are the input pins for the reconstruction control signal and are connected to the array reconstruction unit 430. Figure 6 The P1-P6 pins on the left correspond to the six sensing input pins of the multimodal sensing and detection chip 411. Figure 6 The E1-E6 pins on the right correspond to the six column electrodes of the flexible sensing array 1. The working principle of the array reconstruction selection unit 432 of the array reconstruction unit 430 is described below:

[0060] When the reconfiguration control terminal DIGITPIN2 is high and the reconfiguration control terminals DIGITPIN3 and DIGITPIN6 are low, the OR gates OR1 and OR4 output high levels, and the OR gates OR2 and OR3 output low levels. At this time, the control pin IN of the single-pole double-throw switches SPTD1, SPTD3, and SPTD5 receives a high level, and the switches are switched to the normally open pin S1A, which shorts E2 with E1, E4 with E3, and E6 with E5, thus reconfiguring the original 6 column electrodes of the flexible proximity sensing unit column electrode 102 into 3. The received capacitance values ​​are sent to the P1, P3, and P5 pins respectively.

[0061] When the reconfiguration control terminal DIGITPIN3 is high and the reconfiguration control terminals DIGITPIN2 and DIGITPIN6 are low, the control gates OR1, OR2, OR3, and OR4 output high levels. At this time, the control pin IN of the single-pole double-throw switches SPTD1, SPTD2, SPTD4, and SPTD5 receives a high level, and the switches are switched to the normally open pin S1A, which shorts E1, E2, and E3, and shorts E4, E5, and E6. This reconfigures the original six column electrodes of the flexible proximity sensing unit column electrode 102 into two, and the received capacitance values ​​are sent to the P1 and P4 pins respectively.

[0062] When the reconfiguration control terminal DIGITPIN6 is high and the reconfiguration control terminals DIGITPIN2 and DIGITPIN3 are low, the control gates OR1, OR2, OR3, and OR4 output high levels. At this time, the control pin IN of the single-pole double-throw switches SPTD1, SPTD2, SPTD4, SPTD5, and SPTD6 receives a high level, and the switches are switched to the S1A terminal, causing E1, E2, E3, E4, E5, and E6 to be shorted. This reconfigures the original six row electrodes of the flexible proximity sensing unit column electrode 102 into one, and the received capacitance value is sent to the P1 pin.

[0063] The specific reconfiguration devices used in the array reconfiguration selection unit 432 circuit are: SN74HCS4075 OR gate and ADG884BRMZ single-pole double-throw switch.

[0064] In the practical application of the flexible electronic skin of the present invention, it is generally necessary to detect the entire range from the maximum detection distance to the highest spatial resolution. Therefore, the flexible sensing array 1 after power-on can be reconstructed into a 1×1 array first. When an external object is detected approaching, the received capacitance value gradually increases as the object gets closer. The reconstruction is performed sequentially as follows: 2×1 array, 3×1 array, 2×2 array, and 3×2 array, and finally reconstructed into the most basic 6×2 array, thereby gradually obtaining the highest spatial resolution. When the object gradually moves away from the flexible robot skin of the present invention, the flexible sensing array 1 is reconstructed into 3×2, 2×2, 3×1, 2×1, and 1×1 arrays in sequence, and the maximum detection distance is obtained again step by step.

[0065] The implemented flexible electronic skin has multiple sensing modes. When only the time-of-flight sensing unit 103 is working, the multimodal reconfigurable and scalable interactive flexible electronic skin operates in a long-range sensing mode. The flexible robotic skin is equipped with proximity sensing detection thresholds and contact sensing detection thresholds. When only the row electrodes 101 and column electrodes 102 of the flexible proximity sensing unit are working, and the proximity sensing detection chip 411 detects that the difference between the real-time output value of the row electrodes 101 and column electrodes 102 and the initial reference value is greater than or equal to the proximity sensing detection threshold and less than or equal to the contact sensing detection threshold, the flexible electronic skin operates in a short-range sensing mode. When only the row electrodes 101 and column electrodes 102 of the flexible proximity sensing unit are working, and the difference between the real-time output value of the proximity sensing detection chip 411 and the initial reference value is greater than or equal to the contact sensing threshold, the flexible electronic skin operates in a contact sensing mode.

[0066] like Figure 3 The flexible electronic skin expansion interface 5 shown connects multiple flexible electronic skin units of the present invention in series. Communication between the microcontroller units 6 of different skin structure units is achieved through the IIC communication protocol, allowing sensing data from multiple skin structure units to be aggregated into a single microcontroller unit 6. This skin structure unit is referred to as the master node, and the microcontroller unit 6 is the master of IIC communication. The remaining skin structure units are slave nodes, and the microcontroller units 6 of the slave nodes are slaves in IIC communication. Each slave microcontroller unit 6 is programmed with a unique IIC address. The microcontroller unit 6 of the master node communicates with and receives data from the specific microcontroller unit 6 of the skin structure unit through this address. According to the address specifications of the IIC communication protocol, the flexible robot skin of the present invention can expand to a maximum of 128 units. The robot skin expansion interface 4 in the specific implementation consists of four pads: 5V and GND for power supply, and SCL and SDA for IIC communication between different skin structure units. In practical use, the connection between the various parts is completed by soldering ribbon cables to the different parts of the robot flexible electronic skin expansion interface 5.

[0067] like Figure 3As shown, multiple skin structure units connected in series collect data into a single microcontroller unit 6, which then sends the data to the robot control end via the flexible electronic skin data output interface 8 and data output circuit 9, participating in the control and decision-making of robot movements. The flexible electronic skin data output interface 8 establishes communication between the flexible electronic skin of this invention and the robot control end (generally a computer) via serial communication. Since most computers provide a Universal Serial Bus (USB) interface, the data output circuit 9 is used to convert between the serial communication interface and the USB interface. In the specific implementation, after the slave unit of the flexible electronic skin collects all the data to the host unit, the host unit communicates with the data output circuit 9 through the flexible electronic skin data output interface 8. The flexible electronic skin data output interface 8 consists of 5 pads: a 5V pad for power supply, a GND pad for grounding, RX and TX pads for serial communication between the flexible electronic skin of this invention and the data output circuit 9, and a RESET pad for synchronously resetting all expandable interactive flexible electronic skins.

[0068] The data output circuit 9, specifically implemented, includes five pads, a serial-to-USB chip FT232RL, and a 5-pin microUSB connector interface. One end of the data output circuit 9 has five pads, identical to the five pads of the flexible electronic skin data output interface 8, for connection to the flexible electronic skin data output interface 8; the other end is soldered with a 5-pin micro USB connector. Connecting one end of a micro USB data cable to the micro USB connector and the other end to the universal serial bus (USB) interface of the host computer completes the connection between the flexible electronic skin (as the master node) and the host computer. The serial-to-USB chip FT232RL converts the data transmitted through the RX and TX pads into data that can be transmitted via micro USB, enabling communication between the flexible electronic skin and the host computer.

[0069] The material of the flexible printed circuit board substrate 2 in the specific implementation is polyethylene terephthalate (PET).

[0070] The light-emitting interaction unit 3 endows the flexible electronic skin of this invention with light-emitting interaction functionality. By changing the color and brightness and setting different light-emitting modes, it realizes visual feedback on the detection results of the flexible sensing array 1, and provides intuitive light-emitting interaction between the detected distance parameters and the user. The user can directly read the sensor detection results from the electronic skin itself without the need for a display or other devices, which not only helps to improve the efficiency and smoothness of the human-computer interaction process, but also can issue an alarm by flashing or other light-emitting modes when the robot is too close to the user or other external objects, so that the robot and the user can simultaneously perceive dangerous situations and take corresponding actions, effectively improving the safety of the human-computer interaction process.

[0071] Specifically, the light-emitting interaction unit 3 uses WS2812B LED beads as the light-emitting device. Five light-emitting interaction units 3 are integrated on the flexible robot skin of this invention, which can effectively provide visual feedback. Since the WS2812B LED beads have addressable and programmable characteristics, the detection results of the flexible sensing array 1 can be visualized by turning the beads on and off and changing their colors under the control of the microcontroller unit 6 of the electronic skin.

[0072] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal, reconfigurable, and scalable interactive flexible electronic skin, characterized in that, The system includes a flexible sensing array (1), a flexible printed circuit board substrate (2), a light-emitting interactive unit (3), a data reading and processing unit (4), a flexible electronic skin extension connection interface (5), a microcontroller unit (6), and a bus communication unit (7). The flexible sensing array (1) is fixedly mounted on the flexible printed circuit board substrate (2), and the light-emitting interactive unit (3) is fixedly mounted on the flexible sensing array (1). The flexible sensing array (1) is connected to the data reading and processing unit (4) and the microcontroller unit (6) respectively. The light-emitting interactive unit (3), the data reading and processing unit (4), and the flexible electronic skin are connected to each other. The extended connection interface (5) and the bus communication unit (7) are both connected to the microcontroller (6). The data reading and processing unit (4), the flexible electronic skin extended connection interface (5), the microcontroller (6), the flexible electronic skin data output interface (8), and the bus communication unit (7) are all installed on one side of the flexible printed circuit board substrate (2). The flexible sensing array realizes multimodal sensing function by switching between long-distance sensing mode, short-distance sensing mode and contact sensing mode. The microcontroller (6) realizes the switching and reconstruction of the sensing mode of the flexible sensing array by controlling the data reading and processing unit (4). The flexible sensing array (1) includes multiple flexible sensing light-emitting units mounted on a flexible printed circuit board substrate (2) in an array arrangement. Each flexible sensing light-emitting unit includes two flexible proximity sensing unit row electrodes (101) and two flexible proximity sensing unit column electrodes (102). The two flexible proximity sensing unit row electrodes (101) are arranged at intervals, and a corresponding flexible proximity sensing unit column electrode (102) is arranged between each side of the two flexible proximity sensing unit row electrodes (101), thereby forming a square flexible sensing light-emitting unit. Some of the flexible sensing light-emitting units in the flexible sensing array (1) also include time-of-flight sensing units (103). The time-of-flight sensing units (103) are arranged in the middle of the two flexible proximity sensing unit row electrodes (101) and the two flexible proximity sensing unit column electrodes (102), and the multiple time-of-flight sensing units (103) are arranged in an array. The flexible proximity sensing unit row electrodes (101), flexible proximity sensing unit column electrodes (102) and time-of-flight sensing units (103) are all connected to the data reading and processing unit (4).

2. The multimodal, reconfigurable, and scalable interactive flexible electronic skin according to claim 1, characterized in that, The data reading and processing unit (4) includes a multimodal sensing module (410), a time-of-flight sensing module (420), and an array reconstruction unit (430). The multimodal sensing module (410) is connected to the array reconstruction unit (430), the multimodal sensing module (410) is connected to the flexible sensing array (1), the time-of-flight sensing module (420) is connected to the flexible sensing array (1), the multimodal sensing module (410) and the array reconstruction unit (430) are both connected to the microcontroller unit (6), and the flexible sensing array (1) is connected to the array reconstruction unit (430).

3. The multimodal, reconfigurable, and scalable interactive flexible electronic skin according to claim 2, characterized in that, The multimodal sensing module (410) includes a multimodal sensing detection chip (411) and a multimodal sensing voltage regulator chip (412) connected together. The multimodal sensing detection chip (411) is connected to the array reconstruction unit (430), the multimodal sensing detection chip (411) is connected to the column electrode (102) of the flexible proximity sensing unit of the flexible sensing array (1), the multimodal sensing voltage regulator chip (412) is connected to the row electrode (101) of the flexible proximity sensing unit of the flexible sensing array (1), and the multimodal sensing voltage regulator chip (412) is also connected to the microcontroller unit (6).

4. The multimodal, reconfigurable, and scalable interactive flexible electronic skin according to claim 2, characterized in that, The time-of-flight sensing module (420) includes a connected time-of-flight sensing read selection unit (421) and a time-of-flight sensing read extension unit (422); the time-of-flight sensing read extension unit (422) is connected to the time-of-flight sensing unit (103) of the flexible sensing array (1).

5. The multimodal, reconfigurable, and scalable interactive flexible electronic skin according to claim 2, characterized in that, The array reconstruction unit (430) includes an array reconstruction control unit (431), an array reconstruction selection unit (432), and an array reconstruction logic processing unit (433) connected in sequence. The array reconstruction selection unit (432) is connected to the flexible proximity sensing unit column electrode (102) of the flexible sensing array (1). The array reconstruction selection unit (432) is also connected to the multimodal sensing module (410). The array reconstruction control unit (431) is connected to the microcontroller unit (6).

6. The multimodal, reconfigurable, and scalable interactive flexible electronic skin according to claim 5, characterized in that, The array reconstruction selection unit (432) includes K input terminals, K output terminals, and K single-pole double-throw switches, where K is the number of column electrodes (102) of a single row of flexible proximity sensing units; the reconstruction control unit (430) includes L reconstruction control terminals, where L is log2K rounded up; the array reconstruction logic processing unit (433) includes several OR gates; all L reconstruction control terminals are connected to the microcontroller unit (6), and the L reconstruction control terminals are respectively connected to the control pins of the K single-pole double-throw switches through several OR gates, and the L reconstruction control terminals provide control signals to the control pins of the K single-pole double-throw switches; Each single-pole double-throw switch has two normally open pins S1A and S1B as output terminals and an input pin D as an input terminal. The K flexible proximity sensing unit column electrodes (102) on the row are respectively connected to the K input terminals of the single-pole double-throw switch. Among the K input terminals, the first input terminal is directly connected to the first output terminal, and the remaining K-1 input terminals are respectively connected to the input pins of the single-pole double-throw switch.

7. The multimodal, reconfigurable, and scalable interactive flexible electronic skin according to claim 3, characterized in that, Both the multimodal sensing and detection chip (411) and the time-of-flight sensing and reading extension unit (422) support the IIC communication protocol, and the multimodal sensing and detection chip (411) has a programmable IIC address; the scalable interactive flexible electronic skin system can realize communication between the microcontroller units (6) of different flexible electronic skins through the IIC communication protocol, thereby realizing the scalable function; up to 128 flexible electronic skins with the same structure can be extended on the flexible electronic skin of the present invention.

8. The multimodal, reconfigurable, and scalable interactive flexible electronic skin according to claim 1, characterized in that, The flexible electronic skin is equipped with multi-level distance detection thresholds, and the data reading and processing unit (4) and the micro control unit (6) realize dynamic reconstruction and dynamic dereconstruction in the proximity sensing mode of the flexible sensing light-emitting array.

9. The multimodal, reconfigurable, and scalable interactive flexible electronic skin according to claim 1, characterized in that, Multiple multimodal reconfigurable and scalable interactive flexible electronic skins with the same structure are connected in series through a flexible electronic skin expansion connection interface (5). One of the multi-level sensing flexible robot skins is used as the master node, and the other multi-level sensing flexible robot skins are used as slave nodes. The microcontroller unit (6) of the master node selectively reads the sensing data of the flexible sensing light-emitting units (2) of the slave nodes by polling the addresses configured in the microcontroller units (6) of the other slave nodes. The reading sensing data is finally summarized in the microcontroller unit (6) of the master node, thereby forming an scalable interactive flexible robot skin.

Images