A grasping control method based on tactile sensing, a dexterous hand, a system and a storage medium

Abstract

The application discloses a grasping control method based on tactile sensing, a dexterous hand, a system and a storage medium. The method comprises the following steps: emitting a plane ultrasonic wave in a first period, and receiving a first echo signal reflected by a target object surface; emitting a focused ultrasonic wave in a second period, and receiving a second echo signal reflected by the target object inside; extracting a surface space feature embedding vector according to the first echo signal; extracting a longitudinal time sequence and frequency domain feature embedding vector according to the second echo signal; fusing the surface space feature embedding vector and the longitudinal time sequence and frequency domain feature embedding vector based on a cross-attention mechanism, and outputting a global comprehensive feature vector representing a target object category and a physical state; and adjusting a grasping parameter in real time according to the global comprehensive feature vector, and controlling the dexterous hand to execute or adjust a grasping operation, which overcomes the limitation that traditional visual and tactile sensing can only obtain a surface image.

Description

Technical Field

[0001] This invention belongs to the field of tactile perception technology for robot dexterous hands, and particularly relates to a grasping control method, dexterous hand, system and storage medium based on tactile sensing. Background Technology

[0002] With the deepening application of robotics technology in industrial manufacturing, medical rehabilitation, and service robots, the ability of robotic dexterous hands to precisely manipulate target objects has become a key indicator for measuring the level of intelligence. Tactile perception, as a crucial channel for robots to acquire contact information, can provide information such as surface texture, contact force, sliding, and material properties that are not available through vision. This is of great significance for achieving stable grasping, object recognition, and compliant control. An ideal tactile sensor should possess multimodal sensing capabilities, including multidimensional force, vibration, sliding, texture, material, and temperature. Among these, texture and material information are particularly crucial for establishing tactile models of objects and achieving intelligent manipulation.

[0003] Currently, common tactile sensing technologies mainly include piezoresistive, piezoelectric, capacitive, and optical imaging-based visual tactile technologies. Piezoresistive, piezoelectric, and capacitive tactile sensors have advantages such as simple structure and fast response, but their spatial resolution is usually on the order of millimeters, which is difficult to meet the needs of detecting fine textures with feature sizes of tens of micrometers. Visual tactile technologies developed in recent years (such as GelSight and DIGIT) capture images of elastic body contact deformation using high-definition cameras, enabling the acquisition of high-resolution surface textures and contact force distributions, and have been widely used in robot grasping and object recognition.

[0004] However, visual-tactile technology still has significant limitations in practical applications: First, its core components, including cameras, light sources, and elastomers, result in large sensor sizes, high costs, and difficulties in achieving flexibility and large-area integration. Second, visual-tactile technology can only perceive surface information of objects and cannot detect deeper properties such as internal structure, density, and material properties; for example, it cannot distinguish between raw and cooked eggs or determine the ripeness of fruit. Traditional ultrasonic nondestructive testing technology, while possessing penetrating capabilities, is mainly used for industrial flaw detection or medical imaging. Its probes are rigid, large, and have long working distances, making them unsuitable for tactile scenarios requiring conformal attachment by dexterous hands. Summary of the Invention

[0005] In view of this, the present invention provides a gripping control method, dexterous hand, system and storage medium based on tactile sensing, which solves the problems of existing visual-tactile technologies being large in size, difficult to integrate flexibly and unable to detect the deep internal properties of objects.

[0006] To achieve the above objectives, in a first aspect, the technical solution of the present invention to solve the technical problem is to provide a gripping control method based on tactile sensing, applied to a dexterous hand. The dexterous hand includes a conformally attached flexible ultrasonic tactile sensor, the flexible ultrasonic tactile sensor including a flexible substrate, and an array management circuit and a touch element array fabricated on the flexible substrate. The touch element array includes multiple touch elements arranged in an array. The method includes: controlling the touch element array through the array management circuit to emit planar ultrasonic waves in a first time period and receiving a first echo signal reflected from the surface of a target object by the planar ultrasonic waves; and controlling the touch element array through the array management circuit to emit focused ultrasonic waves in a second time period. The system uses focused ultrasound and receives the second echo signal reflected from the inside of the target object. Based on the first echo signal, it acquires a two-dimensional acoustic phase diagram of the target object and extracts a surface spatial feature embedding vector. Based on the second echo signal, it acquires a time-of-flight sequence, spectral distribution, and attenuation characteristics, and extracts depth temporal and frequency domain feature embedding vectors. Based on a cross-attention mechanism, it fuses the surface spatial feature embedding vector and the depth temporal and frequency domain feature embedding vectors to output a global comprehensive feature vector characterizing the target object's category and physical state. Based on the global comprehensive feature vector, it adjusts the gripping parameters in real time and controls the dexterous hand to perform or adjust gripping operations.

[0007] In one optional embodiment, the array management circuit includes a phase array charging source and a scanning bus, and each of the touch elements integrates a charge memory and a voltage comparator; the charge memory is connected to the phase array charging source, and the input terminal of the voltage comparator is connected to both the charge memory and the scanning bus; controlling the touch element array to emit focused ultrasonic waves in a second time period through the array management circuit includes: charging the charge memory in each of the touch elements with a target voltage corresponding to its respective preset emission phase through the phase array charging source according to a preset focused phase distribution map; inputting a linear scanning voltage to the scanning bus; and triggering the touch element to emit ultrasonic waves when its corresponding target voltage is equal to the linear scanning voltage, thereby forming focused ultrasonic waves with a phase difference.

[0008] In one optional embodiment, the touch element integrates a pulse window control circuit and a transmitting transistor. The output terminal of the charge memory is connected to the input terminal of the pulse window control circuit, the output terminal of the voltage comparator is connected to the gate of the transmitting transistor, and the output terminal of the pulse window control circuit is connected to the drain of the transmitting transistor. The array management circuit controls the touch element array to emit focused ultrasonic waves in a second time period, including: inputting an excitation signal to the pulse window control circuit; when the high level output of the voltage comparator triggers the transmitting transistor to turn on, the transmitting transistor conducts the excitation signal to the touch element, triggering the touch element to emit ultrasonic waves.

[0009] In one optional embodiment, the array management circuit includes a timing control circuit, and each of the touch elements integrates an ultrasonic transducer, as well as a column control switch and a phase control switch connected in series on the pulse drive path of the ultrasonic transducer; controlling the touch element array to emit focused ultrasonic waves in a second time period through the array management circuit includes: generating a delay time corresponding to the touch element array through the timing control circuit; controlling the column control switch to be turned on; and controlling the phase control switch in each of the touch elements to be turned on at the corresponding time node according to the delay time, so that the ultrasonic pulse voltage is applied to the corresponding ultrasonic transducer through the turned-on column control switch and the phase control switch, thereby forming focused ultrasonic waves with a phase difference.

[0010] In one optional embodiment, the ultrasonic transducer includes a first electrode and a second electrode, the first electrode being connected to a transmitting end; the column control switch and the phase control switch are both field-effect transistors, and the step of triggering the array of elements to emit focused ultrasonic waves specifically includes: applying an ultrasonic pulse voltage to the first electrode through the transmitting end, so that the ultrasonic pulse voltage forms a ground-based driving circuit through the ultrasonic transducer, the conducting phase control switch and the column control switch, thereby triggering the ultrasonic transducer to emit ultrasonic waves.

[0011] In one optional implementation, the focused ultrasonic wave emitted in the second time period is a spherical focused ultrasonic wave, and the time delay required for each of the touch elements to emit the spherical focused ultrasonic wave is expressed as: ; in, The time delay required for each of the aforementioned touch elements to emit the spherically focused ultrasonic wave; R is the speed of sound of ultrasound. F d is the radius of the spherical focused ultrasonic wave, d is the distance between the touch elements in the touch element array, N is the total number of touch elements in a single column or row in the touch element array, and n is the position number of the corresponding touch element from the center of the column or row.

[0012] In one optional implementation, the touch element integrates an active pixel sensor circuit. Receiving the first echo signal and the second echo signal includes: during the first time period, after emitting the planar ultrasonic wave, the array management circuit synchronously switches all the touch elements to a receiving state to acquire the first echo signal through the active pixel sensor circuit; during the second time period, after emitting the focused ultrasonic wave, the array management circuit switches each touch element to a receiving state with a corresponding delay based on the time delay when each touch element emits the focused ultrasonic wave to acquire the second echo signal through the active pixel sensor circuit.

[0013] In one optional implementation, the step of fusing the surface spatial feature embedding vector and the deep temporal and frequency domain feature embedding vectors based on the cross-attention mechanism to output a global comprehensive feature vector characterizing the target object category and physical state includes: mapping the surface spatial feature embedding vector to a first query vector, mapping the deep temporal and frequency domain feature embedding vectors to a first key vector and a first value vector, and obtaining a first fused feature through attention calculation; mapping the deep temporal and frequency domain feature embedding vectors to a second query vector, mapping the surface spatial feature embedding vector to a second key vector and a second value vector, and obtaining a second fused feature through attention calculation; and concatenating the first fused feature and the second fused feature and performing nonlinear mapping processing to generate the global comprehensive feature vector.

[0014] In one optional implementation, the extraction of surface spatial feature embedding vectors includes: extracting features from the two-dimensional acoustic phase diagram using a convolutional neural network; the extraction of deep temporal and frequency domain feature embedding vectors includes: extracting features from the temporal sequence using a one-dimensional convolutional neural network and a long short-term memory network or a gated recurrent unit.

[0015] In one optional embodiment, the flexible substrate is a polyvinylidene fluoride substrate, and the flexible ultrasonic tactile sensor is formed by sequentially fabricating the flexible substrate, the array management circuit, and the touch element array on a base substrate, and then peeling the flexible substrate off the base substrate using a laser lift-off process.

[0016] In one optional embodiment, the touch element array is arranged in a double layer, including a transmitting touch element array and a receiving touch element array stacked together; the transmitting touch element array performs the transmitting of planar ultrasonic waves in the first time period and the transmitting of focused ultrasonic waves in the second time period, and the receiving of the first echo signal and the second echo signal is performed by the receiving touch element array.

[0017] In one optional implementation, the physical state includes the internal acoustic features and surface features of the target object; based on the global integrated feature vector, the gripping parameters are adjusted in real time, and the dexterous hand is controlled to perform or adjust the gripping operation, including: if the internal acoustic features indicate that the target object has a flexible deformation tendency or is a vulnerable part, then the normal gripping force threshold of the dexterous hand is reduced; if the surface features indicate that the target object has a sliding tendency in the current contact state, then the tangential gripping force of the dexterous hand is increased or the contact posture is adjusted; based on the adjusted normal gripping force threshold and the tangential gripping force, the dexterous hand is driven to close and maintain a stable grip.

[0018] Secondly, the present invention provides a dexterous hand, comprising: a robotic hand body; and a controller connected to the robotic hand body for executing a tactile sensing-based grasping control method.

[0019] Thirdly, the present invention provides a gripping control system based on tactile sensing, including a dexterous hand and a storage unit. The storage unit stores a gripping strategy library, which includes multiple target object categories and their corresponding gripping parameters. The controller of the dexterous hand is further configured to retrieve the corresponding gripping parameters from the gripping strategy library according to the target object category, and control the dexterous hand to perform or adjust gripping operations according to the gripping parameters.

[0020] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements a gripping control method based on tactile sensing.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: By using a conformally attached flexible ultrasonic tactile sensor, planar ultrasonic waves and focused ultrasonic waves are emitted in sequence, enabling dual-modal synchronous detection of the surface texture (resolution up to 50 micrometers) and internal depth structure (such as layering and stiffness distribution) of the target object. This overcomes the limitation of traditional visual-tactile sensors, which can only acquire surface images.

[0022] Furthermore, the cross-attention mechanism is used to perform bidirectional weighted fusion of surface spatial features and deep temporal features, enabling the model to automatically learn cross-modal associations, which significantly improves the recognition accuracy and grasp adaptability of complex heterogeneous objects (such as raw and cooked eggs, fruits of different ripeness).

[0023] Meanwhile, it provides a variety of touch element circuit implementation schemes based on TFT or single-crystal silicon technology (such as voltage comparison triggering and dual-switch drive circuit), achieving a good balance between miniaturization, flexibility and low-cost mass production, and is suitable for large-area integration on the surface of dexterous hands. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the touch element array structure provided in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the flexible tactile sensor conformally attached to a dexterous hand in the first embodiment of this application; Figure 3 A schematic diagram illustrating the wavefront synthesis principle of planar ultrasonic waves and spherically focused ultrasonic waves emitted by a touch element array. Figure 4 A flowchart illustrating the steps of a tactile sensing-based grip control method provided by the present invention; Figure 5 This is a schematic diagram of a touch element emission circuit module based on charge comparison triggering according to an embodiment of the present invention; Figure 6 A topology diagram of a touch transceiver circuit based on a field-effect transistor according to another embodiment of the present invention; Figure 7 This is a schematic diagram of the stacked structure and fabrication process of the flexible ultrasonic tactile sensor provided in an embodiment of the present invention; Figure 8 This is a schematic diagram illustrating the working principle of the touch array under the transceiver separation stacked architecture provided in an embodiment of the present invention. Figure 9 This is a diagram of a multimodal fusion tactile model and robot system control architecture provided for embodiments of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] like Figures 1 to 2 As shown, the gripping control method based on tactile sensing provided in the first embodiment of the present invention is applied to a dexterous hand. The dexterous hand includes a conformally attached flexible ultrasonic tactile sensor. The flexible ultrasonic tactile sensor includes a flexible substrate, an array management circuit and a touch element array fabricated on the flexible substrate, and the touch element array includes multiple touch elements arranged in an array.

[0027] Figure 3The diagram schematically illustrates the wavefront synthesis principle of planar ultrasonic waves emitted by a touch element array and spherically focused ultrasonic waves. Taking five touch elements (elements 1 to 5) as an example, when each element emits ultrasonic pulses simultaneously, the synthesized wavefront is a planar wave parallel to the array element arrangement direction, used to detect the surface morphology of the target object. When each touch element emits ultrasonic pulses sequentially according to a preset time delay, the synthesized wavefront becomes a concave surface, forming an adjustable-focus spherically focused ultrasonic wave. Its energy is concentrated at a preset focal point, used to detect the internal depth structure of the target object. Furthermore, by changing the emission delay sequence of each touch element, the synthesized wavefront can be deflected, enabling scanning and detection of cross-sections in different directions or at different depths.

[0028] Figure 4 A flowchart illustrating the steps of a tactile sensing-based grasp control method is shown, which includes: S100 controls the touch element array through the array management circuit to emit planar ultrasonic waves in the first time period and receive the first echo signal reflected by the planar ultrasonic waves from the surface of the target object. Specifically, a single touch element has a length and width of approximately 10 to 500 micrometers and a thickness of approximately 20 to 200 micrometers. Multiple touch elements can be constructed into a large-area touch element array (e.g., a two-dimensional array of 100 × 200 = 20,000 touch elements). During the first phase of emitting planar ultrasonic waves, all touch elements in a column or row of the array are simultaneously excited to emit ultrasonic waves (frequency > 20 kHz), thus synthesizing a planar wavefront parallel to the touch element array surface. After the plane wave reaches the surface of the target object, it is reflected, and the first echo signal is received by the original touch element array. Plane waves are very suitable for detecting the surface texture of various work objects (such as textiles, leather, or paper products, specifically cleanroom wipes, mouse pads, labels, or linen). In practical applications, the resolution of this ultrasonic imaging can reach 50 micrometers, which can fully meet the detection requirements of tactile perception for minimum feature sizes of 100 to 200 micrometers.

[0029] S200 controls the touch element array to emit focused ultrasonic waves in the second time period through the array management circuit, and receives the second echo signal reflected by the focused ultrasonic waves through the inside of the target object; Specifically, during the second phase of transmitting focused ultrasonic waves, each sensor element on the array initiates transmission with a specific regular time difference (i.e., phase difference), forming a composite wavefront that is curved relative to the array plane (such as a parabolic cylindrical wave or a spherical wave). This beam can focus ultrasonic energy, enabling it to penetrate the surface of the object under test, penetrate into the object's interior, and detect deep structures and materials in the depth direction (e.g., detecting different depth interfaces of eggshell, albumen, and yolk). The received second echo signal carries the temporal magnitude characteristics of the internal structure. The entire array can be scanned column by column. The GOA (Gate on Array) sequentially scans each column to complete the transmission and reception operations. For example, for a 100×100 array with 10MHz timing scanning, completing the full array dual-modal scan takes only about 1.6 milliseconds, fully meeting the dynamic response requirements of robot tactile perception.

[0030] S300: Based on the first echo signal, acquire the two-dimensional acoustic phase diagram of the target object and extract the surface spatial feature embedding vector. Specifically, after preprocessing the acquired first echo signal, including denoising, acoustic imaging, and normalization, a two-dimensional acoustic phase diagram reflecting the surface morphology, roughness, and acoustic impedance distribution of the object is generated. Denoising employs bandpass filtering (passband frequency ±30% of the ultrasonic transducer center frequency) combined with median filtering (window size 3×3 to 7×7). Acoustic imaging uses a delay-and-sum (DAS) algorithm, which calculates the time delay of the received echoes for each array element at each imaging point and superimposes the amplitudes. Normalization uses maximum and minimum values ​​to normalize to the [0,1] interval. Subsequently, feature extraction is performed on this two-dimensional acoustic image, mapping it to a fixed-dimensional surface spatial feature embedding vector. This vector encodes spatial information such as the object's micro-texture and surface hardness distribution.

[0031] S400: Based on the second echo signal, acquire the time series of flight time, spectral distribution and attenuation characteristics, and extract the in-depth time series and frequency domain feature embedding vectors. Specifically, the acquired second echo signal from the depth is preprocessed, including frequency domain transformation and time alignment, to obtain a time series containing Time of Flight (ToF), spectral distribution, and attenuation characteristics. The time of flight is determined using a peak detection method (finding the moment corresponding to the maximum echo envelope value) combined with a cross-correlation method (cross-correlation with the transmitted pulse to obtain the correlation peak position). The spectral distribution is obtained by extracting the center frequency, bandwidth, and high-frequency attenuation slope after performing a Fast Fourier Transform (FFT) on the echo signal. The attenuation characteristics are estimated by fitting the echo amplitude versus depth curve and using logarithmic spectral subtraction to estimate the ultrasonic attenuation coefficient (unit: dB / cm / MHz). For example, linear least squares is used to fit the logarithm of the echo peak amplitude with depth, and the slope is the attenuation coefficient. Based on this, temporal or frequency domain features are extracted and mapped into depth feature embedding vectors to encode depth physical information such as the object's internal density, elastic modulus, and layered interfaces.

[0032] In addition, the frequency domain features in the in-depth time-series and frequency domain feature embedding vectors can be obtained by performing a Fast Fourier Transform (FFT) on the time-series sequence to obtain a spectrum, and then using a one-dimensional convolutional neural network to extract the frequency domain feature embedding vectors to characterize the frequency response characteristics of the object.

[0033] S500, based on the cross-attention mechanism, fuses the surface spatial feature embedding vector and the deep temporal and frequency domain feature embedding vectors to output a global comprehensive feature vector representing the target object category and physical state. Specifically, within the AI ​​(Artificial Intelligence) ecosystem, separate surface texture features and internal depth material features are fused across modalities. The cross-attention mechanism enables the model to automatically learn the correlation patterns between surface texture and internal properties, generating a globally integrated feature vector that simultaneously contains information about both surface and depth structures.

[0034] The S600 adjusts the gripping parameters in real time based on the global comprehensive feature vector and controls the dexterous hand to perform or adjust the gripping operation. Specifically, the output global comprehensive features are fed into a multimodal recognition model to achieve high-precision identification and state judgment of object types (such as judging the ripeness of fruit and distinguishing between raw and cooked eggs). The control program compares the data with the backend strategy library in a timely manner and adjusts the gripping action of the robot's dexterous hand actuator with an extremely short response time, replacing pure trial and error feedback.

[0035] In this embodiment, the emitted planar ultrasonic waves and focused ultrasonic waves are controlled in time sequence by the touch element array, and the surface two-dimensional texture and the depth temporal features are fused based on the cross-attention mechanism. This overcomes the shortcomings of traditional optical tactile technology, which is large in size and cannot detect the internal deep material. It realizes accurate dual-modal tactile perception of high-resolution surface texture and internal physical structure characteristics, which greatly improves the efficiency of dexterous hand in operating complex and unknown objects and the gripping robustness.

[0036] In an optional embodiment of this application, the array management circuit includes a phase array charging source and a scanning bus, and each touch element integrates a charge memory and a voltage comparator; the charge memory is connected to the phase array charging source, and the input terminal of the voltage comparator is connected to the charge memory and the scanning bus respectively; controlling the touch element array to emit focused ultrasonic waves in the second time period through the array management circuit includes: According to the preset focusing phase distribution diagram, the phase array charging power supply charges the charge storage in each contact element with the target voltage corresponding to its preset emission phase.

[0037] Specifically, to address the technical challenge of independently controlling the emission time (i.e., phase) of each contact element on the large array surface, a programmable emission phase array library is incorporated into the system. Before emission, the phase array charging source pre-charges the charge storage within each contact element to a set target voltage. This voltage value determines the emission phase of that contact element: the lower the voltage, the earlier the emission time; the higher the voltage, the later the emission time. The phase distribution diagram of the entire contact element array strictly corresponds to the pre-charged voltage distribution.

[0038] Input a linear scan voltage to the scan bus; Specifically, when a transmission operation needs to be triggered, the phase scanning voltage module outside the contact element inputs a scanning voltage that increases linearly with time to all contacts in the array through the scanning bus.

[0039] The voltage comparator in each contact element triggers the contact element to emit ultrasonic waves when its corresponding target voltage is equal to the linear scanning voltage, thus forming focused ultrasonic waves with a phase difference. Specifically, the voltage comparator inside each contact element compares the linear scanning voltage with the pre-stored target voltage in real time. When the two are equal, the comparator flips and outputs a trigger signal, causing different contact elements to emit sequentially according to a preset time difference, thus synthesizing the required curved wavefront.

[0040] In this embodiment, by integrating a charge storage device with a preset phase voltage in each touch element and using a global linear scanning voltage and a local comparator for triggering, the complex extremely short time delay control is converted into a reliable local voltage comparison action, thereby achieving high-precision spatial phase control and beam focusing for a large-area ultrasonic array.

[0041] In an optional embodiment of this application, a pulse window control circuit and a transmitting transistor are integrated within the touch element. The output terminal of the charge memory is connected to the input terminal of the pulse window control circuit, the output terminal of the voltage comparator is connected to the gate of the transmitting transistor, and the output terminal of the pulse window control circuit is connected to the drain of the transmitting transistor. The touch element array is controlled by an array management circuit to emit focused ultrasonic waves in a second time period, including: Input the excitation signal to the pulse window control circuit.

[0042] Specifically, during the period when a linear scanning voltage is input via the scanning bus, the pulse generation and amplification module outside the contact element inputs an ultrasonic excitation pulse signal containing a high-frequency AC component to all contact elements via the global excitation bus.

[0043] When the high level output of the voltage comparator triggers the transmitting transistor to turn on, the transmitting transistor will conduct the excitation signal to the contact element, triggering the contact element to emit ultrasonic waves.

[0044] Specifically, the emitter transistor acts as a direct switch controlling the transducer, only turning on its gate after the voltage comparator flips. Simultaneously, an integrated pulse window control circuit limits the effective time window for the excitation signal, preventing high-frequency signal overlap or crosstalk between adjacent emitter cycles.

[0045] As a feasible example, the pulse window control circuit can be composed of a monostable multivibrator whose trigger input is connected to the output of a voltage comparator, and whose output controls the enable terminal of an AND gate. The excitation signal can only reach the emitter transistor through the AND gate within a set time window after the voltage comparator flips.

[0046] In this embodiment, by cooperating with the pulse window control circuit and the transmitting transistor, the high-frequency driving energy is released to the corresponding transducer only at the precise phase comparison reversal point, which effectively avoids crosstalk of large-scale high-frequency signals in the array and improves the signal purity of the ultrasonic transmitter.

[0047] Combined with appendix Figure 5As shown, this embodiment provides a trigger-based emitter circuit based on charge comparison triggering. Each trigger element integrates a charge memory, a voltage comparator, an emitter transistor, and a pulse window control circuit. Regarding the specific circuit connections and signal flow, the output of the phase array charging source located outside the trigger element is connected via an addressing circuit to the input of the charge memory (e.g., a sample-and-hold capacitor with a capacitance between 0.1pF and 1pF, used to store the voltage written by the phase array charging source and kept in an open state by a transistor switch to maintain a constant voltage) inside the trigger element, for pre-charging the target voltage that determines the emitter phase; the positive and negative comparison inputs of the voltage comparator are respectively connected to the output of the charge memory and the global scan bus (this bus is used to input a scan voltage that increases linearly with time); the signal output of the voltage comparator is directly connected to the gate of the emitter transistor; simultaneously, the pulse window control circuit, used to receive external high-frequency AC excitation signals, has its output connected to the drain of the emitter transistor; the source of the emitter transistor is electrically connected to the corresponding ultrasonic transducer. In the above structure, when the scanning voltage on the scanning bus rises over time to equal the pre-stored target voltage, the output level of the voltage comparator instantly flips, outputting a high level to the gate of the emitter transistor, opening the gate channel of the emitter transistor, allowing the high-frequency excitation signal, limited by the pulse window control circuit, to flow instantaneously through the emitter transistor into the ultrasonic transducer. This topology cleverly converts the time-dimensional delay into a voltage-dimensional comparison, significantly reducing the risks of parasitic capacitance and timing offset caused by directly wiring the global high-speed clock signal to each tiny contact element.

[0048] In an optional embodiment of this application, the array management circuit includes a timing control circuit, and each touch element integrates an ultrasonic transducer, as well as a column control switch and a phase control switch connected in series on the pulse drive path of the ultrasonic transducer; controlling the touch element array to emit focused ultrasonic waves in the second time period through the array management circuit includes: The delay time of the corresponding touch array is generated by the timing control circuit.

[0049] Specifically, based on an alternative hardware implementation architecture, the phase delay is not generated by voltage comparison, but is directly issued by phase-controlled timing circuitry (such as a digital delay circuit containing a reference clock and a counter). For tactile sensing applications, the minimum required time delay is typically in the range of a few nanoseconds, and a system clock in the frequency range of 100MHz to 200MHz can meet the spatial phase adjustment accuracy requirements of microsecond-level ultrasound.

[0050] The control column control switch is turned on.

[0051] Specifically, in the column-by-column scanning mode, the column drive circuit first uniformly controls the column control switch of all contacts in the column to open, establishing the basic communication and drive preparation state for the column.

[0052] Based on the delay time, the phase control switch in each contact element is turned on at the corresponding time node, so that the ultrasonic pulse voltage is applied to the corresponding ultrasonic transducer through the turned-on column control switch and phase control switch, forming focused ultrasonic waves with phase difference.

[0053] Specifically, with the column control switch turned on, the phase control switches connected in series with each contact element will be turned on sequentially according to their respective nanosecond-level time delays issued by the timing control circuit. Only when both switches are turned on can the ultrasonic driving voltage be applied to the ultrasonic transducer to excite the piezoelectric effect, thereby producing a specific beam deflection or depth focusing effect at the physical level.

[0054] In this embodiment, by using a two-stage control structure that connects a series control switch and a phase control switch in the drive circuit of the ultrasonic transducer, the device quantity limitation of TFT (Thin Film Transistor) polysilicon process is compatible, and flexible, digital phase-controlled delayed emission of ultrasonic beams is achieved while simplifying the circuit complexity within the pixel.

[0055] In an optional embodiment of this application, the ultrasonic transducer includes a first electrode and a second electrode, the first electrode being connected to a transmitting end; both the column control switch and the phase control switch are field-effect transistors, and the step of triggering the array of elements to emit focused ultrasonic waves specifically includes: An ultrasonic pulse voltage is applied to the first electrode by the transmitting end, so that the ultrasonic pulse voltage forms a ground drive circuit through the ultrasonic transducer, the conducting phase control switch and the column control switch, triggering the ultrasonic transducer to emit ultrasonic waves.

[0056] Specifically, the ultrasonic transducer has a structure with upper and lower electrodes clamping piezoelectric material. During the transmission phase, an ultrasonic AC pulse voltage is directly applied to the first electrode of the transducer (such as the upper electrode), while the second electrode of the transducer (such as the lower electrode) is connected in series to a phase control switch and a column control switch, both of which are field-effect transistors. When both transistors are controlled to conduct, a high-frequency drive current flows from the transmitting end through the transducer, through these two transistors, and finally discharged to the system ground, forming a complete dynamic drive circuit, which excites the inverse piezoelectric effect and thus radiates ultrasonic mechanical waves outward.

[0057] In this embodiment, by using a field-effect transistor as a ground discharge switch to construct the driving circuit, the risk of high voltage from the driving electrode impacting the fragile internal readout circuit is greatly reduced, enhancing the electrical safety and system stability of the miniature touch element during high-frequency ultrasonic transmission.

[0058] Combined with appendix Figure 6The specific circuit topology shown in this embodiment provides a touch element transceiver circuit based on field-effect transistors. Each touch element integrates an ultrasonic transducer and four field-effect transistors: a first transistor M1, a second transistor M2, a column control switch M3 (whose gate is connected to a reset signal), and a phase control switch M4. The first transistor M1 and the second transistor M2 constitute an active pixel sensor (APS) readout circuit, while the column control switch M3 and the phase control switch M4 constitute a pulse drive path. In terms of specific circuit connections, the first electrode of the ultrasonic transducer is directly connected to the transmitting end of the ultrasonic pulse generator; the second electrode of the ultrasonic transducer is connected to the gate of the first transistor M1 and the drain (or source) of the phase control switch M4; the source (or drain) of the phase control switch M4 is connected in series with the drain of the column control switch M3; the gate of the column control switch M3 receives the reset signal, and its source is connected to system ground or a reference low level. Meanwhile, the drain of the first transistor M1 is connected to the power supply voltage, its source is connected to the drain of the second transistor M2, the gate of the second transistor M2 is connected to the row read control line, and its source is connected to the column data read line.

[0059] With the aforementioned 4T (four transistors) architecture, during the transmission of the focused ultrasonic wave in the second phase, the high-frequency excitation signal from the transmitting end can only be successfully discharged to ground via the transducer, M4, and M3 when M3 and M4 are simultaneously controlled to conduct, forming a complete transmission drive circuit. This specific topology ensures miniaturization of the single pixel area while physically achieving nanosecond-level phase delay control and high-voltage crosstalk protection.

[0060] In an optional embodiment of this application, the focused ultrasonic wave emitted in the second time period is a spherical focused ultrasonic wave, and the time delay required for each touch element to emit the spherical focused ultrasonic wave is expressed as follows: ; in, The time delay required for each contact element to emit spherically focused ultrasonic waves; R is the speed of sound of ultrasound. F d is the radius of the spherical focused ultrasound wave, d is the distance between the touch elements in the touch element array, N is the total number of touch elements in a single column or row in the touch element array, and n is the position number of the corresponding touch element from the center of the column or row.

[0061] Specifically, when performing the detection task in the second phase, in order to detect internal structures at specific depths (such as penetrating the skin layer to detect blood vessels, or penetrating the peel to detect the pulp structure), the array needs to emit converging spherical waves. The array management circuit calculates the time difference between each column (or each row) of contact elements and the contact element at the geometric center of the array. By adjusting the set spherical radius parameter, the final focusing depth of the beam can be dynamically changed. For example, by assigning different time delay sequences corresponding to different curved wavefronts, high-resolution tomographic detection of different depth sections within the target object can be achieved.

[0062] In this embodiment, by constructing a precise mathematical mapping model between geometric distance and acoustic time of flight and issuing delay commands, the traditional static mechanical focusing is transformed into dynamic electronic phased-array focusing, thereby realizing high-precision three-dimensional acoustic scanning and detection of variable depth inside an object.

[0063] In an optional embodiment of this application, an active pixel sensor circuit is integrated within the touch element, and receiving the first echo signal and the second echo signal includes: In the first time period, after emitting planar ultrasonic waves, all touch elements are synchronously switched to receiving state through the array management circuit so as to obtain the first echo signal through the active pixel sensor circuit.

[0064] Specifically, the touch element not only contains a transmission circuit but also integrates an APS (Active Pixel Sensor) circuit for receiving. After transmitting a plane wave without phase difference, the echo is also a plane wave. Therefore, the array management circuit will uniformly cut off the transmission control switches of all touch elements before the echo arrives at the sensor array, so that all array units switch to the receiving state at the same absolute point in time. The reflected ultrasonic waves generate a positive piezoelectric effect in the sensing layer, exciting a weak induced charge. This charge is directly connected to the gate of the first transistor M1 via the second electrode of the ultrasonic transducer, changing the gate voltage of the first transistor M1. The first transistor M1, acting as a source follower, amplifies the gate voltage change and outputs it row by row to the column data readout line through the row selection switch of the second transistor M2, thereby realizing the acquisition of the echo signal, changing the gate voltage of the signal amplification transistor in the APS, and then reading it row by row by the external acquisition device through the subsequent readout switch.

[0065] In the second time period, after the focused ultrasonic wave is emitted, the array management circuit switches each touch element to the receiving state according to the time delay when each touch element emits the focused ultrasonic wave, so as to obtain the second echo signal through the active pixel sensor circuit.

[0066] Specifically, in the focused wave (curved wave) operating mode, due to the phase difference in transmission, the arrival times of the echoes at different touch elements will inevitably differ. Therefore, during the reception preparation phase, the array management circuit cannot implement synchronous shutdown. Instead, the control center unit first shuts down the transmission path and switches to reception, and then the touch elements at the edge positions proportionally delay their transmission switches according to their respective corresponding transmission delay times.

[0067] In this embodiment, by matching the spatiotemporal physical characteristics of simultaneous plane wave transmission and reception and delayed spherical wave transmission and reception, the synchronous or delayed switching of the receiving state is precisely executed, preventing abnormal leakage of induced charges excited by weak echoes into the transmission path, and greatly improving the receiving sensitivity and signal-to-noise ratio of signals in dual modes.

[0068] In an optional embodiment of this application, based on a cross-attention mechanism, the surface spatial feature embedding vector and the deep temporal and frequency domain feature embedding vectors are fused to output a global comprehensive feature vector representing the target object category and physical state, including: The surface spatial feature embedding vector is mapped to the first query vector, and the deep temporal and frequency domain feature embedding vectors are mapped to the first key vector and the first value vector. The first fusion feature is obtained by attention calculation.

[0069] Specifically, in the training and inference process of the multimodal AI model, the model first uses surface features as Q (Query) and depth features as K (Key) and V (Value) for calculation. This step enables the network to adaptively learn the dependencies between different modalities. For example, when there is no significant difference in surface texture, the model will automatically calculate and increase the attention weights assigned to depth features. The attention calculation uses a standard scaled dot product attention mechanism, which involves scaling the query vector and key vector after dot product, normalizing the result using Softmax to obtain the weights, and then adding the weighted vector.

[0070] It should be noted that the mapping can be achieved through three independent linear transformation matrices (fully connected layers), which multiply the input feature vector by a learnable weight matrix to obtain the query vector, key vector, and value vector. The dimension of each vector can be set to 128 dimensions.

[0071] The deep temporal and frequency domain feature embedding vectors are mapped to the second query vector, and the surface spatial feature embedding vectors are mapped to the second key vector and the second value vector. The second fusion feature is obtained by attention calculation.

[0072] Specifically, a reverse attention process is performed: the internal depth features are used as the second Q vector, and the surface features are used as the second K and V vectors for matrix inner product and Softmax probability normalization. This process is used to correct for interference and environmental noise that may exist in the regular surface texture features. The second fused feature is obtained through attention calculation, and the same standard scaled dot product attention mechanism is used.

[0073] The first and second fusion features are concatenated and nonlinearly mapped to generate a global comprehensive feature vector.

[0074] Specifically, the high-dimensional feature tensors output by the bidirectional attention network are concatenated and combined, and nonlinear mapping is completed through a multilayer perceptron or activation function, ultimately outputting a global high-dimensional representation that includes both surface details and deep material properties.

[0075] In this embodiment, a bidirectional cross-attention mechanism is used to replace the traditional feature splicing method, constructing a deep semantic association between soft and hard representation information. This enables the model to adaptively allocate decision weights between surface and deep information according to the difficulty of the object being identified, significantly improving the recognition accuracy of the multimodal perception system when facing complex and heterogeneous objects.

[0076] In an optional embodiment of this application, the specific steps for extracting surface spatial features and depth features include: Extracting surface spatial feature embedding vectors includes: using convolutional neural networks to extract features from two-dimensional acoustic phase diagrams.

[0077] Specifically, for two-dimensional images representing the microscopic texture and contour of objects generated by transverse ultrasound imaging, local spatial features are extracted hierarchically using CNN (Convolutional Neural Network), which is adept at processing spatial grid data.

[0078] As an example of what can be achieved, the convolutional neural network consists of three convolutional layers (3×3 kernels, stride 1, number of channels 32-64-128), each followed by batch normalization and ReLU activation, and then 2×2 max pooling; finally, it outputs a 128-dimensional feature embedding vector through two fully connected layers (256-128 dimensions).

[0079] Extracting in-depth temporal and frequency domain feature embedding vectors includes: using one-dimensional convolutional neural networks and long short-term memory networks or gated recurrent units to extract features from time series sequences.

[0080] Specifically, depth echo is a time-series signal containing Time-of-Flight (ToF) and its amplitude decay sequence. The system extracts local waveform abrupt changes through a front-end 1D-CNN (1-Dimensional Convolutional Neural Network), and then concatenates it with LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit) to capture its long-range temporal dependencies, thereby accurately encoding it into a temporal embedding vector that can characterize elasticity and density.

[0081] As a feasible example, a one-dimensional convolutional neural network consists of two convolutional layers (with kernels of 5 and 3, and 64 channels each), followed by ReLU and max pooling with a stride of 2 after each layer; the output sequence is fed into a two-layer LSTM with 64 hidden units; and finally, a 128-dimensional feature embedding vector is output through a fully connected layer.

[0082] In this embodiment, by introducing CNN and temporal recurrent neural networks for targeted front-end feature isolation and extraction, which are different from spatial imaging and time series data structures, the high-frequency texture components and low-frequency material components in the original physical signal are preserved to the greatest extent, providing a high-quality data foundation for the back-end fusion model.

[0083] like Figure 7 As shown, in an optional embodiment of this application, the flexible substrate (direct substrate) is a polyvinylidene fluoride substrate, and the flexible ultrasonic tactile sensor is formed by sequentially fabricating the flexible substrate, array management circuit and touch element array on the base substrate, and then peeling the flexible substrate off the base substrate by a laser lift-off process.

[0084] Specifically, the fabrication of the ultrasonic sensor employs a unique process combining in-situ growth and exfoliation. First, a layer of polyvinylidene fluoride (PVDF) or polyimide (PI) film is coated onto a rigid substrate (such as high-transmittance glass) with sufficient structural strength as a flexible substrate (direct substrate). Then, on the working surface based on this flexible material, a transistor transceiver array and a piezoelectric transducer layer are fabricated using mature, high-precision semiconductor microelectronics processes. After fabrication, LLO (Laser Lift-off) technology is used, employing a high-energy laser of a specific wavelength to penetrate the base glass substrate, ablate the interface between it and the PVDF flexible layer, and finally completely exfoliate it, obtaining a large-area flexible thin-film device with a thickness of less than 1 mm and semiconductor precision.

[0085] In this embodiment, by employing a basic hard substrate transfer and laser lift-off process, the dependence of high-resolution array microelectronic semiconductor processing on rigid substrates and the strong demand for flexible characteristics in conformal attachment by robot dexterity hands are perfectly reconciled, greatly reducing the manufacturing difficulty of mass production of ultra-miniature ultrasonic array sensors.

[0086] like Figure 8 As shown, in an optional embodiment of this application, the touch element array is distributed in two layers, including a transmitting touch element array and a receiving touch element array stacked together; the transmitting touch element array performs the function of transmitting planar ultrasonic waves in the first time period and transmitting focused ultrasonic waves in the second time period, and the receiving touch element array performs the function of receiving the first echo signal and the second echo signal.

[0087] Specifically, in addition to integrating the transmitting and receiving functions onto a piezoelectric transducer on the same wafer plane using time-division multiplexing, this application also provides a transceiver-separated stacked physical architecture. The sensor is configured three-dimensionally as two independent arrays: the bottom array is dedicated to performing phased-array ultrasonic wave transmission, and the top array is dedicated to monitoring and receiving the echo. The two arrays are isolated from each other by an acoustic matching layer or a substrate. In this architecture, the transmission timing and reception timing are completely decoupled, eliminating the need to design a high-frequency switching leakage protection switch within the contact element.

[0088] In this embodiment, by introducing a stacked array architecture with physical isolation between the transmitter and receiver layers, the complexity of the high-speed transmit and receive timing control logic within the system is greatly simplified. It also allows for the use of dedicated piezoelectric materials with different optimal impedance characteristics for the transmitter and receiver, thereby improving the electroacoustic conversion efficiency from the hardware perspective.

[0089] In an optional embodiment of this application, the physical state includes the internal acoustic characteristics (such as stiffness and absorption properties) and surface characteristics (such as texture, reflection, and sliding properties) of the target object; based on the globally integrated feature vector, the grasping parameters are adjusted in real time, and the dexterous hand is controlled to perform or adjust the grasping operation, including: If the internal acoustic characteristics indicate that the target object has a tendency to deform flexibly or is a vulnerable part, then the normal gripping force threshold of the dexterous hand should be reduced.

[0090] Specifically, the gripping control system can further integrate traditional multi-dimensional force sensors with auxiliary sensing modes to form multi-mode fusion. When the material of the object is determined to be soft and easily damaged, such as tofu or ripe fruit, based on the depth echo analysis, the system directly applies force limitation to the servo motor on the output command side, actively lowering the threshold warning line of the normal gripping force.

[0091] If surface features indicate that the target object has a tendency to slide in the current contact state, increase the tangential grip force of the dexterous hand or adjust the contact posture.

[0092] Specifically, if, during operation, the system detects insufficient friction coefficient at the contact interface or a slippage trend due to relative displacement by combining acoustic texture and mechanical sensing, it will intervene in real time, instructing the mechanical finger to increase the tangential friction grip force generated by the normal positive pressure, or to fine-tune the envelope contact posture of each phalanx to change the constraint boundary.

[0093] Based on the adjusted normal gripping force threshold and tangential gripping force, the dexterous hand is driven to close and maintain a stable grip.

[0094] Specifically, the control system drives the finger joint motor to close by issuing corrected dynamic parameters, thus completing a full dynamic closed-loop operation from macroscopic spatial approximation to microscopic force stability.

[0095] In this embodiment, by directly mapping the high-level semantic features (softness, hardness, and sliding) decoded by AI down to the underlying normal and tangential physical torque limit thresholds, a rapid feedback channel from tactile perception and state determination to the intervention of the actuator is constructed, which effectively avoids the engineering problems of crushing objects or grasping slippage that are easy to occur when traditional robotic arms are in a blind grasping state.

[0096] In a second embodiment of the present invention, a dexterous hand is provided, comprising: The robotic arm itself; The controller, connected to the robotic arm body, is used to execute the tactile sensing-based grasping control method of any of the above embodiments.

[0097] Specifically, the dexterous hand has the aforementioned flexible ultrasonic tactile sensor array laid on the force-bearing surfaces of the workpiece, such as the bionic knuckles and / or the palm. The controller incorporates a neural network computing accelerator chip (such as a tensor operation unit) to process the massive multimodal feature matrix.

[0098] like Figure 9 As shown, in this embodiment, by equipping a dexterous hand body structure with a flexible ultrasonic array and an AI controller, microscopic beam control and macroscopic mechanical control are organically integrated, giving the robot end effector a three-dimensional tactile perception ability similar to human skin, which can identify materials and textures.

[0099] In a third embodiment of the present invention, a gripping control system based on tactile sensing is proposed, including a dexterous hand and a storage unit as described in the above embodiments. The storage unit stores a gripping strategy library, which includes multiple target object categories and their corresponding gripping parameters. The controller of the dexterous hand is further configured to retrieve the corresponding gripping parameters from the gripping strategy library according to the identified target object category, and control the dexterous hand to perform or adjust the gripping operation according to the gripping parameters.

[0100] Specifically, the storage unit is pre-trained offline and loaded with an operational logic experience library covering hundreds of common industrial and everyday objects. During online grasping, the system does not need to test the mechanical limits online every time, but directly retrieves the optimal feedforward control parameters based on the acoustic feature collision strategy library.

[0101] In this embodiment, by introducing a strategy library architecture that combines offline training and online invocation, the computational latency and trial-and-error time costs of dexterous hands when facing complex and unknown environments are greatly reduced, enabling the system to have both powerful generalization judgment capabilities and extremely high industrial execution efficiency.

[0102] It should be noted that if an unknown object that does not exist in the grasp strategy library is encountered during the grasping process, the grasping operation will be abandoned and an alarm will be issued.

[0103] The fourth embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0104] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these. The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; however, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0105] The specific embodiments of the present invention described above do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made in accordance with the technical concept of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A gripping control method based on tactile sensing, characterized in that, The method is applied to a dexterous hand, the dexterous hand including a conformally attached flexible ultrasonic tactile sensor, the flexible ultrasonic tactile sensor including a flexible substrate, and an array management circuit and a touch element array fabricated on the flexible substrate, the touch element array including a plurality of touch elements arranged in an array; the method includes: The array management circuit controls the touch element array to emit planar ultrasonic waves in the first time period and receive the first echo signal reflected by the planar ultrasonic waves from the surface of the target object. The array management circuit controls the touch element array to emit focused ultrasonic waves in the second time period and receives the second echo signal reflected by the focused ultrasonic waves through the inside of the target object. Based on the first echo signal, a two-dimensional acoustic phase diagram of the target object is obtained, and the surface spatial feature embedding vector is extracted. Based on the second echo signal, the time series of flight time, spectral distribution and attenuation characteristics are obtained, and the in-depth time series and frequency domain feature embedding vectors are extracted. Based on the cross-attention mechanism, the surface spatial feature embedding vector and the deep temporal and frequency domain feature embedding vector are fused to output a global comprehensive feature vector that characterizes the target object category and physical state. Specifically, the surface spatial feature embedding vector is mapped to a first query vector, and the deep temporal and frequency domain feature embedding vectors are mapped to a first key vector and a first value vector, and the first fusion feature is obtained through attention calculation; The in-depth temporal and frequency domain feature embedding vectors are mapped to a second query vector, and the surface spatial feature embedding vectors are mapped to a second key vector and a second value vector. The second fusion feature is obtained through attention calculation. The first fused feature and the second fused feature are concatenated and subjected to nonlinear mapping to generate the global comprehensive feature vector. Based on the global integrated feature vector, the grasping parameters are adjusted in real time, and the dexterous hand is controlled to perform or adjust the grasping operation. The physical state includes the internal acoustic features and surface features of the target object; If the internal acoustic features indicate that the target object has a tendency to deform flexibly or is a vulnerable part, then the normal gripping force threshold of the dexterous hand is reduced. If the surface features indicate that the target object has a tendency to slide in the current contact state, then increase the tangential grip force of the dexterous hand or adjust the contact posture; Based on the adjusted normal gripping force threshold and the tangential gripping force, the dexterous hand is driven to close and maintain a stable grip.

2. The grip control method based on tactile sensing as described in claim 1, characterized in that, The array management circuit includes a phase array charging source and a scanning bus, and each of the contact elements integrates a charge memory and a voltage comparator; the charge memory is connected to the phase array charging source, and the input terminal of the voltage comparator is connected to the charge memory and the scanning bus respectively; The array management circuit controls the touch element array to emit focused ultrasonic waves in the second time period, including: According to the preset focusing phase distribution diagram, the phase array charging source charges the charge storage in each of the contact elements with a target voltage corresponding to their respective preset emission phase. A linear scan voltage is input to the scan bus; The voltage comparator within each of the touch elements triggers the touch element to emit ultrasonic waves when its corresponding target voltage is equal to the linear scanning voltage, thereby forming focused ultrasonic waves with a phase difference.

3. The grip control method based on tactile sensing as described in claim 2, characterized in that, The touch element integrates a pulse window control circuit and an emitting transistor. The output terminal of the charge memory is connected to the input terminal of the pulse window control circuit. The output terminal of the voltage comparator is connected to the gate of the emitting transistor. The output terminal of the pulse window control circuit is connected to the drain of the emitting transistor. The array management circuit controls the touch element array to emit focused ultrasonic waves in the second time period, including: An excitation signal is input to the pulse window control circuit; When the high level output of the voltage comparator triggers the transmitting transistor to turn on, the transmitting transistor conducts the excitation signal to the contact element, triggering the contact element to emit ultrasonic waves.

4. The grip control method based on tactile sensing as described in claim 1, characterized in that, The array management circuit includes a timing control circuit, and each of the touch elements integrates an ultrasonic transducer, as well as a column control switch and a phase control switch connected in series on the pulse drive path of the ultrasonic transducer. The array management circuit controls the touch element array to emit focused ultrasonic waves in the second time period, including: The timing control circuit generates a delay time corresponding to the touch element array; Control the column control switch to be turned on; According to the delay time, the phase control switch in each of the touch elements is controlled to be turned on at the corresponding time node, so that the ultrasonic pulse voltage is applied to the corresponding ultrasonic transducer through the turned-on column control switch and the phase control switch to form a focused ultrasonic wave with a phase difference.

5. The grip control method based on tactile sensing as described in claim 4, characterized in that, The ultrasonic transducer includes a first electrode and a second electrode, the first electrode being connected to a transmitting end; both the column control switch and the phase control switch are field-effect transistors, and the step of triggering the touch element array to emit focused ultrasonic waves specifically includes: An ultrasonic pulse voltage is applied to the first electrode through the transmitting end, so that the ultrasonic pulse voltage forms a ground drive circuit through the ultrasonic transducer, the activated phase control switch and the column control switch, triggering the ultrasonic transducer to emit ultrasonic waves.

6. A gripping control method based on tactile sensing as described in any one of claims 1 to 5, characterized in that: The focused ultrasonic wave emitted in the second time period is a spherical focused ultrasonic wave, and the time delay required for each of the touch elements to emit the spherical focused ultrasonic wave is expressed as follows: ； in, The time delay required for each of the aforementioned touch elements to emit the spherically focused ultrasonic wave; R is the speed of sound of ultrasound. F d is the radius of the spherical focused ultrasonic wave, d is the distance between the touch elements in the touch element array, N is the total number of touch elements in a single column or row in the touch element array, and n is the position number of the corresponding touch element from the center of the column or row.

7. A gripping control method based on tactile sensing as described in any one of claims 1 to 5, characterized in that, The touch element integrates an active pixel sensor circuit, and the receiving of the first echo signal and the second echo signal includes: During the first time period, and after the planar ultrasonic wave is emitted, the array management circuit synchronously switches all the touch elements to the receiving state so as to obtain the first echo signal through the active pixel sensor circuit. During the second time period, and after the focused ultrasonic wave is emitted, the array management circuit switches each of the touch elements to a receiving state according to the time delay when each touch element emits the focused ultrasonic wave, so as to obtain the second echo signal through the active pixel sensor circuit.

8. The grip control method based on tactile sensing as described in claim 1, characterized in that: The step of extracting surface spatial feature embedding vectors includes: extracting features from the two-dimensional acoustic phase diagram using a convolutional neural network; The extraction of in-depth temporal and frequency domain feature embedding vectors includes: using a one-dimensional convolutional neural network and a long short-term memory network or a gated recurrent unit to extract features from the temporal sequence.

9. The grip control method based on tactile sensing as described in claim 1, characterized in that: The flexible substrate is a polyvinylidene fluoride substrate, and the flexible ultrasonic tactile sensor is formed by sequentially fabricating the flexible substrate, the array management circuit, and the touch element array on a base substrate, and then peeling the flexible substrate off the base substrate using a laser lift-off process.

10. The grip control method based on tactile sensing as described in claim 1, characterized in that: The touch element array is distributed in two layers, including a transmitting touch element array and a receiving touch element array stacked together; the transmitting touch element array performs the transmitting planar ultrasonic wave in the first time period and the transmitting focused ultrasonic wave in the second time period, and the receiving touch element array performs the receiving of the first echo signal and the second echo signal.

11. A dexterous hand, characterized in that, include: The robotic arm itself; A flexible ultrasonic tactile sensor conformally attached to the body of the robotic arm, the flexible ultrasonic tactile sensor including a flexible substrate, and an array management circuit and a touch element array fabricated on the flexible substrate, the touch element array including multiple touch elements arranged in an array; The controller, connected to the robotic arm body and the array management circuit, is used to execute the tactile sensing-based grasping control method as described in any one of claims 1 to 10.

12. A gripping control system based on tactile sensing, characterized in that, Includes a dexterous hand and storage unit as described in claim 11, wherein the storage unit stores a grasping strategy library, the grasping strategy library including multiple target object categories and their corresponding grasping parameters; The controller of the dexterous hand is also used to retrieve the corresponding grasping parameters from the grasping strategy library according to the target object category, and control the dexterous hand to perform or adjust the grasping operation according to the grasping parameters.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the grip control method based on tactile sensing as described in any one of claims 1 to 10.

Images