Flexible optoelectronic multimodal sensing device and measurement method

Abstract

The application provides a flexible photoelectric multi-modal sensing device and a measuring method, the device has an arrayed flexible optical structure, including a flexible proximity-temperature-haptic multi-modal sensor, a signal processing circuit and a precise grabbing embedded control system, and can be used for realizing automatic precise grabbing control; the measuring method completes precise grabbing based on the flexible proximity-temperature-haptic multi-modal sensor, and automatically distinguishes three working tasks of proximity sensing, haptic normal force measurement and sliding measurement through time sequence voltage information, thereby providing technical support for realizing an automatic, high-precision and highly expandable precise grabbing control system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot intelligent grasping and multi-modal sensing technology, and in particular to a flexible photoelectric multi-modal sensing device and a measuring method. BACKGROUND

[0002] In robot work tasks, grasping operation as a key part, the continuous advancement of intelligent grasping field is the prerequisite for robots to achieve tasks in industrial and civil fields. However, the measurement requirement of grasping is not limited to tactile measurement, because tactile cannot fully perceive the pre-grasping link in the grasping process, and precise grasping usually shows instability. To solve this problem, the proximity sensor with its near scene high sensitivity and high precision characteristics becomes an ideal choice to realize dual modal sensing combined with tactile, thereby improving the effect of precise grasping. Therefore, the application of proximity-temperature-tactile multi-modal sensor to robot precise grasping measurement and control has become the current mainstream research direction. This research aims to realize the measurement and control of the pre-grasping link through the proximity sensor, and complete the measurement and control of the grasping link through the tactile sensor, so as to comprehensively measure and control the whole grasping process, so as to achieve the best grasping effect.

[0003] However, there are still some challenges. First, the effective range of the current proximity sensor is low, making it difficult to measure the whole grasping process. Second, the outputs of the proximity-tactile-slip multi-modal sensor are coupled on the analog output, lacking intelligent and stable discrimination basis to decouple the output curves of the two modalities, and manual intervention is needed for mode recognition. Third, there is currently a lack of an online full-automatic self-calibration method to improve the accuracy of the proximity sensor in measuring unknown targets. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a flexible photoelectric multi-modal sensing device.

[0005] Another technical problem to be solved by the present application is to provide a measuring method for the above-mentioned flexible photoelectric multi-modal sensing device.

[0006] To solve the above technical problems, the technical solution of the present application is:

[0007] A flexible photoelectric multi-modal sensing device, comprising a flexible proximity-temperature-tactile multi-modal sensor, a signal processing circuit and a precise grasping embedded control system, wherein,

[0008] The flexible proximity-temperature-tactile multi-modal sensor comprises a flexible light-sensitive contact layer (1), a flexible light barrier layer (2) and a flexible photoelectric conversion layer (3), wherein the flexible light-sensitive contact layer (1) is provided with a flexible contact array (4), a positioning pin (6) for assembly positioning, and a through hole (11) for smooth light emission; the flexible light barrier layer (2) is provided with a light barrier hole (7), a positioning hole (8) for cooperation with the positioning pin (6) for fixing, and a through hole (11) for smooth light emission; the flexible photoelectric conversion layer (3) is respectively provided with an interface (10), a near-infrared light source array (5), a near-infrared photocell array (9), a far-infrared photocell array (12), and a positioning hole (8) for cooperation with the positioning pin (6) for fixing; the flexible contact array (4) is composed of a plurality of hemispherical flexible contacts, the near-infrared photocell array (9) is composed of a plurality of near-infrared photocells, the far-infrared photocell array (12) is composed of a plurality of far-infrared photocells, and the flexible contact, the light barrier hole (7) and the near-infrared photocell or the far-infrared photocell are coaxial;

[0009] The signal processing circuit comprises an IV amplifier (17) and an ADC digital-analog converter (18), the IV amplifier (17) and the ADC digital-analog converter (18) are electrically connected, and the IV amplifier (17) is electrically connected with the interface (10);

[0010] The embedded control system is provided with a host computer main control board (16) and an ADC digital-analog converter (18).

[0011] The flexible proximity-temperature-tactile multi-modal sensor is connected to the IV amplifier (17) of the signal processing circuit through the interface (10), at this time, the output is an analog voltage value entering the ADC analog-digital converter (18) of the embedded control system, the output is a digital signal, the digital signal processing and analysis are realized through the host computer main control board (16), the control instructions are divided into two paths, one path controls the opening and closing speed and position of the manipulator (13) through a TTL-to-485 chip (19), and the other path controls the lower computer main control board (22) in the mechanical arm control cabinet (21) through wireless communication, so as to realize the motion control of the mechanical arm tool end (20).

[0012] The above-mentioned embedded control system, manipulator (13), mechanical arm control cabinet (21) and mechanical arm tool end (20) are all general-purpose devices, the mechanical arm control cabinet (21) and the mechanical arm tool end (20) are two components of one product, which are forcedly connected together, the manipulator (13) is fixed on the tool end and is fixedly connected with the flexible proximity-temperature-tactile multi-modal sensor in a pasting manner.

[0013] Preferably, the flexible photoelectric multi-modal sensing device, when the flexible proximity-temperature-haptic multi-modal sensor is used as a coarse sensor unit (15), is located in front of the target object (23) to guide the target object (23) into the clamping range; and / or when the flexible proximity-temperature-haptic multi-modal sensor is used as a fine sensor unit (14), is located on the side of the target object (23) to complete the pre-grabbing function through tilt elimination, eccentricity elimination, self-calibration, to realize the grabbing function through normal force measurement and sliding detection, and to complete all links required for precise grabbing.

[0014] Preferably, the flexible photoelectric multi-modal sensing device, the installation positions of the near-infrared light source array (5) and the near-infrared photocell array (9) should be spaced apart by a distance, and the top of the near-infrared light source array (5) should be lower than the lower surface of the flexible photosensitive contact layer (1), so as to meet the condition of establishing a light guide.

[0015] Preferably, the flexible photoelectric multi-modal sensing device, the curvature radius of the flexible contact is 1.5-2.5mm, the contact diameter is not less than 3mm, and the thickness is 1mm-2mm, so as to ensure that the flexible contact can fully complete the function of modulating the light path, and ensure that the thickness of the sensor is small.

[0016] Preferably, the flexible photoelectric multi-modal sensing device, the flexible photosensitive contact layer (1), the flexible diaphragm layer (2) and the flexible photoelectric conversion layer (3) are positioned and assembled through the positioning cooperation of the positioning pins (6) and the positioning holes (8), and are integrally prepared by combining the plasma bonding process and the injection molding process.

[0017] Preferably, the flexible photoelectric multi-modal sensing device, each flexible contact of the flexible contact array (4) presents a chessboard-like distribution, the spacing between adjacent flexible contacts is the same, and a through hole (11) is arranged at the middle position of the four adjacent flexible contacts, so as to ensure that the light emitted by the near-infrared light source array (5) can smoothly exit.

[0018] Preferably, the flexible photoelectric multi-modal sensing device, the positioning pins (6) are arranged at the four corners of the flexible photosensitive contact layer (1) for assembly positioning.

[0019] Preferably, the flexible photoelectric multi-modal sensing device, the flexible photosensitive contact layer (1) is prepared by the following method: the preparation material is PDMS and filter dye, a high-precision mold is combined, and the uniform glue, vacuum defoaming, injection molding, drying and film peeling processes are sequentially performed to prepare the flexible photosensitive contact layer (1).

[0020] Preferably, the flexible photoelectric multi-modal sensing device, the mass ratio of each component in the preparation material is PDMS A glue: PDMS B glue: 850NM filter dye = 10:1:0.1.

[0021] Preferably, the flexible optoelectronic multi-modal sensing device, the material of the flexible diaphragm layer (2) is black silicone, the diaphragm hole (7) and the positioning hole (8) are made by precise punching equipment, the relative position of the positioning hole (8) is consistent with the positioning pin (6).

[0022] Preferably, the flexible optoelectronic multi-modal sensing device, the flexible photoelectric conversion layer (3) is a FPC flexible circuit board, the relative position of the near-infrared light source array (5) on the flexible photoelectric conversion layer (3) is consistent with the through hole (11), so that the light emitted by the near-infrared light source array (5) can smoothly exit through the through hole (11).

[0023] Preferably, the flexible optoelectronic multi-modal sensing device, the number of the far-infrared photovoltaic cell array (12) is 2, which is diagonally arranged at both ends of the flexible photoelectric conversion layer (3); the near-infrared light source array (5) and the near-infrared photovoltaic cell array (9) are uniformly dispersed on the flexible photoelectric conversion layer (3).

[0024] Preferably, the flexible optoelectronic multi-modal sensing device, the flexible proximity-temperature-haptic multi-modal sensor transmission characteristic should have a proximity sign, the center wavelength of the near-infrared light source array (5) is 850nm, the sensitivity peak wavelength of the near-infrared photovoltaic cell array (9) is 850nm, and the sensitivity peak wavelength of the far-infrared photovoltaic cell array (12) is 2-7μm, preferably 4-5μm.

[0025] Preferably, the flexible optoelectronic multi-modal sensing device, the data transmission frequency of the flexible proximity-temperature-haptic multi-modal sensor is 500HZ, which can ensure the timely acquisition of proximity distance or contact force sliding state measurement tasks with sufficient real-time performance, and ensure timely control of the robot or other equipment to complete the protection or other tasks of the proximity object.

[0026] The measurement method of the flexible optoelectronic multi-modal sensing device, based on the flexible proximity-temperature-haptic multi-modal sensor, can automatically distinguish the proximity, tactile normal force measurement and sliding measurement three working tasks through time sequence voltage information.

[0027] Preferably, the measurement method of the flexible optoelectronic multi-modal sensing device can be directly installed with the flexible proximity-temperature-haptic multi-modal sensor, its supporting IV amplifier circuit and embedded control system for precise on-site grabbing control in an environment without high time effectiveness requirements; in an environment with high time effectiveness requirements, the user needs to fix the flexible proximity-temperature-haptic multi-modal sensor on a part where proximity and touch and slip are to be measured, connect the sensor-IV amplifier-embedded system to a computer, and use a trained network to predict the results, which can obtain sensor state prediction and measurement value calculation according to the time sequence change of the voltage.

[0028] Preferably, when the proximity measurement task is performed, the near-infrared light source array (5) emits light energy, which is diffusely reflected on the surface of the detected object. The reflected part of the light energy can enter the near-infrared photocell array (9) through the diaphragm hole (7), form a photocurrent, and be converted into a voltage value by the IV amplifier (17) for collection; as the proximity of the object to be detected, more reflected light energy can enter the near-infrared photocell array (9) through the restriction of the diaphragm hole (7), making the photocurrent larger. This change has the following corresponding relationship:

[0029]

[0030] Due to the existence of the bias light source (the light source emitted by the near-infrared light source array (5)), an output maximum value phenomenon occurs at a distance of 5mm from the target object (23), which becomes the proximity landmark point of the sensor to the target object. This point does not change with the properties of the target object and can be used as the eccentricity elimination target point and the self-calibration starting point. After self-calibration, a more precise transfer function can be obtained. By analyzing the collected voltage data, the distance data of the detected object can be obtained, and the proximity measurement can be completed.

[0031] Preferably, when the touch measurement task is performed, the bias light source generates an optical waveguide in the flexible photosensitive touch head layer (1). After contact, the leaked light makes the photocurrent have an increasing trend; but the loading of the detected object can make the flexible touch head sink downward, causing the focal point of the flexible touch head to move downward, and the diaphragm hole (7) to restrict more light energy from entering, making the photocurrent have a downward trend, resulting in a difference between the touch part transfer function and the proximity transfer function. The mutation position of the transfer function is easy to detect and can be used as a contact landmark. The photocurrent is converted into a voltage value by the IV amplifier (17) for collection. This change has the following corresponding relationship:

[0032]

[0033] By analyzing the collected voltage data, pressure data can be obtained to complete the tactile measurement; when critical contact occurs, the infrared reflectivity of different materials is different, and the collected voltage values are also different. The material of the detected object can be distinguished by this feature.

[0034] Preferably, the measurement method of the flexible optoelectronic multi-modal sensing device, when relative sliding occurs, the flexible contact will tilt and vibrate, the focal point will also shift and vibrate, the aperture hole (7) limits more light energy to be incident, the photoelectric current presents the phenomenon of overall decline and accompanying fluctuation, the computer can detect this feature and judge whether sliding occurs, and complete sliding measurement.

[0035] Preferably, the measurement method of the flexible optoelectronic multi-modal sensing device, when the target object is a heat-emitting object, the target object becomes a far-infrared energy source, and the far-infrared cell array (12) works as a far-infrared detection unit (temperature measurement unit) (starts to work, according to the thermal radiation power-distance solving formula: The closer the target object is to the flexible proximity-temperature-tactile multi-modal sensor, the stronger the far-infrared energy incident on the flexible proximity-temperature-tactile multi-modal sensor. The far-infrared cell array (12) can be used as an auxiliary ranging module of the flexible proximity-temperature-tactile multi-modal sensor. The intensity of far-infrared radiation is only related to the temperature of the target object, and is not related to the color and roughness of the object. This feature compensates for the error of the light reflection proximity sensor and improves the measurement accuracy. After contacting the target object, the far-infrared light no longer passes through the air, but directly enters the flexible photosensitive contact layer, the heat conduction efficiency is improved, the far-infrared energy entering the flexible proximity-temperature-tactile multi-modal sensor is greatly improved, and the far-infrared intensity changes abruptly, which marks the contact between the object and the flexible contact array, and assists the flexible proximity-temperature-tactile multi-modal sensor to complete the proximity-tactile modal recognition.

[0036] Preferably, the measurement method of the flexible optoelectronic multi-modal sensing device, the working process of the robot is divided into two processes of pre-grasping and grasping, the pre-grasping process is aimed at eliminating eccentricity and inclination, and the grasping process realizes the monitoring of the normal force and the pre-warning and control of sliding; wherein the coarse measurement sensor unit (15) undertakes the task of guiding the target object (23) to the measurement range of the fine measurement sensor unit (14), after detecting that the output presents a low-high-low feature, an instruction of forward movement of the robot tool end (20) is issued to the robot control cabinet (21), at this time the fine measurement sensor unit (14) starts to work, and the robot tool end (20) stops moving after detecting the low-high-low feature, and then starts to eliminate the inclination, adopts the binocular lateral tilt method, uses the forward and backward movement of the robot tool end (20), calculates the voltage difference Diff of the fine measurement sensor unit (14), when the Diff value is within the set threshold range, the inclination elimination is completed; and then starts to eliminate the eccentricity, the robot tool end (20) first moves to the right until the proximity sign of the left unit in the fine measurement sensor unit (14) is reached, then the robot tool end (20) moves to the left and enables pulse counting, and after reaching the proximity sign of the right unit in the fine measurement sensor unit (14), the robot tool end (20) moves to the right, and the movement distance is determined by the pulse counting result; at this time, the pre-grasping work is completed, the host computer mainboard (16) issues a clamping instruction to the robot (13), and stops moving after the fine measurement sensor unit (14) detects the proximity sign, and completes the self-calibration data fitting; then the robot (13) continues to clamp, and the stepping speed decreases until the contact sign and the object tilt sign are detected, the fine measurement sensor unit (14) starts to monitor the normal force value, until the normal force reaches the preset value, the robot tool end (20) is controlled to move upward to complete clamping; when the fine measurement sensor unit (14) detects the frequency spectrum signal caused by sliding, the robot (13) increases the clamping force until the preset value is reached.

[0037] Preferably, the measurement method of the flexible optoelectronic multi-modal sensing device, when the target object (23) is a heat object, the inclination elimination work and the eccentricity elimination work in the pre-grasping process can be completed by the far-infrared photocell array (12): since the emission of far-infrared energy is only determined by the temperature of the target object and does not change with the surface characteristics of the target object, in the eccentricity elimination, only the outputs of the left and right far-infrared photocell arrays (12) of the fine measurement sensor unit (14) need to be consistent; the output of the target object after contacting the fine measurement sensor unit (14) can be used as a contact sign, each far-infrared photocell unit of the far-infrared photocell array (12) is distributed on the diagonal of the flexible optoelectronic conversion layer (3), when two far-infrared photocell units simultaneously detect the contact sign, it is considered that the inclination is completely eliminated.

[0038] Beneficial effects:

[0039] The flexible photoelectric multi-modal sensing device has an arrayed flexible optical structure, which includes processing and acquisition of flexible proximity-temperature-haptic multi-modal sensor raw signals, data analysis and control instruction sending of a master processor, and serves as a flexible photoelectric proximity-touch-slip multi-modal arrayed sensing system, which can be used to realize automatic precise grabbing control and provide technical support for realizing an automatic, high-precision and scalable precise grabbing control system. Specifically,

[0040] 1. The sensor has the advantages of strong anti-interference, small size and light weight, is more suitable for the use scene of robot precise grabbing, has stronger anti-interference ability to environmental factors such as electromagnetism, and can still guarantee the measurement accuracy of the sensor in different use scenes.

[0041] 2. The sensor as a whole is a flexible system, can be adapted to the surface of a jig of any shape, has low manufacturing process difficulty and low cost, and the consistency can be effectively controlled by using injection molding and film stripping process, thereby guaranteeing the economy and feasibility of large-scale production.

[0042] 3. The introduction of the photosensitive contact optical layer modulates the receiving light path of the sensor, expands the effective range of the sensor, and deduces a physical model, which gives the qualitative influence of the setting of the structure parameters on the transmission curve. At the same time, the influence of stray light on the system is reduced, and the signal-to-noise ratio of the system is improved. Further, the high sensitivity interval of the system has high resolution, which can fully guarantee that the system can realize high-precision measurement and control in the measurement tasks of eccentricity and normal force required in precise grabbing.

[0043] 4. The introduction of the flexible photosensitive contact layer and the biased light source enables the sensor to have fixed proximity and contact marks, which provides a solution to the difficulties of eccentricity elimination, self-calibration and proximity-haptic modal recognition in precise grabbing. At the same time, it is found in experimental verification that these mark points do not change with the properties of the target object, so that the accuracy of the sensor is well guaranteed under any use condition.

[0044] 5. The sensor has temperature sensing capability, can detect the distance and contact signal of a heating object by using heat, improves the proximity sensing accuracy and contact mark discrimination stability, and provides a more efficient tilt elimination method.

[0045] 6. The flexible proximity-temperature-haptic multi-modal sensor uses its proximity mark, contact mark and far infrared temperature measurement to design an automatic precise grabbing control scheme, which includes automatic grabbing guidance, automatic tilt and eccentricity elimination, automatic self-calibration, automatic normal force value prediction and automatic slip sensing, and covers the whole process of pre-grabbing and grabbing in precise grabbing work. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a structural schematic diagram of the flexible proximity-temperature-haptic multi-modal sensor described in the present invention.

[0047] Figure 2 is a structural schematic diagram of the flexible optoelectronic multi-modal sensing device described in the present invention.

[0048] Figure 3 is a schematic diagram of the optical path simulation in ZEMAX simulation software described in the present invention.

[0049] Figure 4 is a schematic diagram of the light waveguide construction and light leakage principle described in the present invention.

[0050] Figure 5 is a schematic diagram for high-sensitivity haptic sensing described in the present invention.

[0051] Figure 6 is an optical path simulation diagram for high-sensitivity haptic sensing described in the present invention.

[0052] Figure 7 is a schematic diagram for large-range haptic sensing described in the present invention.

[0053] Figure 8 is an optical path simulation diagram for large-range haptic sensing described in the present invention.

[0054] Figure 9 is an optical path schematic diagram for three-dimensional force haptic sensing described in the present invention.

[0055] Figure 10 is a schematic diagram of the thermal sensing unit generating a contact mark described in the present invention.

[0056] Figure 11 is a sensor layout schematic diagram for precise gripping of a manipulator described in the present invention.

[0057] Figure 12 is a control logic schematic diagram for precise gripping experiments of a manipulator described in the present invention.

[0058] Figure 13 is an electrical connection schematic diagram for precise gripping experiments of a manipulator described in the present invention.

[0059] Figure 14 is a mark stability verification diagram for precise gripping experiments of a manipulator described in the present invention.

[0060] In the diagram: 1- Flexible photosensitive contact layer (PDMS model RTV615, manufactured by Momentive), 2- Flexible aperture layer, 3- Flexible photoelectric conversion layer, 4- Flexible contact array, 5- Near-infrared light source array, 6- Positioning pin, 7- Aperture hole, 8- Positioning hole, 9- Near-infrared photocell array (near-infrared photocell model Siemens BPW34 S), 10- Interface, 11- Through hole, 12- Far-infrared photocell array (far-infrared photocell model VIGO PVI-4TE-5 photodiode), 13- Robotic arm (EasyClaw Robot ZFG-50), 14- Precision sensor unit, 15- Coarse sensor unit, 16- Host computer main control board (STMicroelectronics STM32 F411CEU6), 17- IV amplifier, 18- ADC analog-to-digital converter, 19- TTL to 485 chip (model MAX485) 20- Robotic arm tool end (robotic arm UR5), 21- Robotic arm control cabinet (robotic arm UR5), 22- Lower computer main control board (ST Semiconductor STM32 F411 CEU6), 23- Target object. Detailed Implementation

[0061] The flexible optoelectronic multimodal sensing device, the flexible optoelectronic multimodal sensing device, and the measurement method of the present invention will be described below with reference to embodiments and accompanying drawings.

[0062] Example 1

[0063] like Figures 1-13 As shown, a flexible optoelectronic multimodal sensing device includes a flexible proximity-temperature-touch multimodal sensor, a signal processing circuit, and a precision grasping embedded control system.

[0064] The flexible proximity-temperature-tactile multi-modal sensor comprises a flexible light-sensing contact layer 1, a flexible light diaphragm layer 2 and a flexible photoelectric conversion layer 3, wherein the flexible light-sensing contact layer 1 is provided with a flexible contact array 4, positioning pins 6 for assembly positioning, and through holes 11 for smooth light emission; the flexible light diaphragm layer 2 is provided with light diaphragm holes 7, positioning holes 8 for fixing in cooperation with the positioning pins 6, and through holes 11 for smooth light emission; the flexible photoelectric conversion layer 3 is respectively provided with interfaces 10, a near-infrared light source array 5, a near-infrared photocell array 9, a far-infrared photocell array 12, and positioning holes 8 for fixing in cooperation with the positioning pins 6, wherein the near-infrared light source array 5 and the near-infrared photocell array 9 and the far-infrared photocell array 12 are directly welded on the flexible photoelectric conversion layer 3, the installation positions of the near-infrared light source array 5 and the near-infrared photocell array 9 should be spaced apart, and the top of the near-infrared light source array 5 is lower than the lower surface of the flexible light-sensing contact layer 1, so as to meet the conditions for establishing a light waveguide: 1, the light source and the signal receiver have a certain distance so that the light path is not vertically emitted and emitted, 2, the light source emission position should be lower than the lower surface of the layer level for establishing the light waveguide. The flexible contact array 4 is composed of a plurality of hemispherical flexible contacts, the near-infrared photocell array 9 is composed of a plurality of near-infrared photocells, the far-infrared photocell array 12 is composed of a plurality of far-infrared photocells, and the flexible contacts, the light diaphragm holes 7 and the near-infrared photocells or the far-infrared photocells are coaxial; wherein the preparation process is as follows:

[0065] (1) The flexible light-sensing contact layer 1 is composed of a flexible contact array 4 and a PDMS substrate, and is prepared by injection molding. A mold is manufactured by using a high-precision lathe with a tolerance less than 0.05 mm. In order to reduce the interference factors caused by visible light, the PDMS needs to be mixed with a 780 nm high-pass filter dye. The three are poured into a glue mixer in the proportion of A glue: B glue (curing agent): filter dye = 10:1:0.01, and taken out after stirring for not less than 30 min. The mixed glue is placed in a vacuum box for defoaming, and the defoaming time is not less than 40 min. The first defoamed glue is taken out, injected into the mold, and placed in the vacuum box again for secondary defoaming, with the same time of not less than 40 min. The mold with the injected glue after the second defoaming is placed in a constant temperature heating box at 70 degrees Celsius for 3 hours. After taking out, the mold is removed and can be used. The prepared device substrate has a thickness of 1.0 mm, a length of 28.0 mm, a width of 28.0 mm, a contact curvature radius of 2.0 mm, and four positioning pins with a designed diameter of 0.9 mm for positioning of the sensor assembly.

[0066] (2) Flexible diaphragm layer 2 is made of black silicone with a thickness of 0.5 mm, and the punching process is completed by a high-precision lathe driven punch head, with a tolerance of less than 0.05 mm. Three positioning holes are consistent with the positioning pins, with a diameter of 1.0 mm. At the same time, nine diaphragm holes are designed to correspond to the positions of the photodiodes, with a diameter of 0.6 mm. According to the matching relationship of the positioning holes and the positioning pins, the flexible diaphragm layer 2 is assembled below the flexible photosensitive contact layer 1.

[0067] (3) The substrate of the flexible photoelectric conversion layer 3 is FPC (Flexible Printed Circuit), and the devices welded on it (including interface 10, near-infrared photodiode array 9, far-infrared photodiode array 12, and near-infrared light source array 5) are rigid devices, but due to their small size, they do not affect the compliance of the circuit board, and can still be firmly attached even on complex surfaces. The FPC is designed with positioning holes corresponding to the positions of the positioning pins, with a diameter of 1.0 mm, and is assembled below the diaphragm layer according to the matching relationship.

[0068] (4) After assembling the above three components, place them in the encapsulation mold and use silicone encapsulation. After the silicone is cured, the finished sensor is obtained by demolding. The total thickness of the device is 3.5 mm, the length is 28.0 mm, and the width is 28.0 mm.

[0069] The signal processing circuit includes an IV amplifier 17 and an ADC digital-to-analog converter 18, and the IV amplifier 17 and the ADC digital-to-analog converter 18 are electrically connected, and the IV amplifier 17 is electrically connected with the interface 10;

[0070] The embedded control system includes an upper computer main control board 16, a lower computer main control board 22, a TTL to 485 chip 19, and a Bluetooth communication module (CC2540 TI Bluetooth chip, Texas Instruments), wherein the upper computer main control board 16, the TTL to 485 chip 19, the IV amplifier 17, and the ADC digital-to-analog converter 18 are integrated on one circuit board, and can be installed on the manipulator 13. The ADC digital-to-analog converter 18 is used as an interface with the previous stage IV amplifier to obtain raw data. The lower computer main control board 22 and another Bluetooth chip are integrated on another circuit board and installed in the control cabinet for receiving and processing instructions.

[0071] As Figure 13As shown, in the application process, the flexible proximity-temperature-tactile multi-modal sensor transmission characteristics should have proximity signs, the center wavelength of the near-infrared light source array 5 is 850 nm, the sensitivity peak wavelength of the near-infrared photocell array 9 is 850 nm, and the sensitivity peak wavelength of the far-infrared photocell array 12 is 2-7 μm (preferably 4 μm). When the data transmission frequency of the flexible proximity-temperature-tactile multi-modal sensor is 500 HZ, it can be ensured that the proximity distance or contact force sliding state measurement task is obtained in time, and the robot or other equipment is controlled in time to complete the protection or other tasks of the proximity object. The flexible proximity-temperature-tactile multi-modal sensor is connected to the IV amplifier 17 of the signal processing circuit through the interface 10, the sensor obtains the original current value, enters the IV amplifier 17 for amplification, and then the amplified voltage value is converted into digital by the ADC sampling module 18. The data is transmitted to the upper control board 16 for operation through the I2C protocol. The control result obtained by operation will be divided into two ways, one way is to control the opening and closing speed and position of the robot 13 through the TTL to 485 chip 19, and the other way is to control the lower machine main control board 22 in the mechanical arm control cabinet 21 through the wireless communication of the Bluetooth module. The lower control board 22 translates the instruction into control information and sends it to the UR5 robot arm to realize the motion control of the robot tool end 20 and complete the closed-loop control. The embedded control system, the robot 13, the robot arm control cabinet 21 and the robot tool end 20 used above are general-purpose devices. The robot arm control cabinet 21 and the robot tool end 20 are two components of one product, which are forcedly connected together. The robot 13 is fixed on the tool end and is fixedly connected with the flexible proximity-temperature-tactile multi-modal sensor in a conforming manner.

[0072] The working principle of the above flexible proximity-temperature-tactile multi-modal sensor is as follows:

[0073] The working principle of the proximity sensor is as Figure 2 shown. Since the surface of the object can be considered as a rough surface, when the infrared light of the light source irradiates the surface of the target object 23, the light will be diffusely reflected on the rough surface. For this sensor, only the diffuse reflection light intensity needs to be obtained to calculate the voltage value.

[0074] V=A v σI x (1)

[0075] In the formula, I x is the light intensity, the coefficient σ is the sensitivity of the detector A v is the gain multiple

[0076] Suppose the emitted light intensity is 0 and the measured object reflection light intensity is 1. Suppose the thickness of the rubber is d and the distance from the object is x, as shown in Figure 1According to the different refractive index of rubber material, a part of R light will be reflected from the interface of rubber and air, a part of k light will be scattered, and a part of a light will be reflected on the target surface.

[0077] The classical model of diffuse reflection intensity and reflection distance is:

[0078]

[0079] According to Fresnel equation Born and Wolf (1999), the reflection at the interface of PDMS / air

[0080]

[0081] However, this classical model cannot be fitted to the real curve, which shows that the structure of the sensor photosensitive contact and diaphragm will bring the influence of transfer function model. In order to find the factor of this influence, two correction operators are proposed to modify the classical model.

[0082] 1. The influence of oblique incidence, the oblique incidence of light source will introduce a cosine correction term to the formula

[0083]

[0084] In the formula:

[0085] θ = 0.5arctan(4 / (x+2)) (5)

[0086] The general formula is:

[0087] θ = 0.5arctan(l / (x+d+h)) (6)

[0088] In the formula, I is the distance between the silicon photocell and the emission end, d is the thickness of the photosensitive contact base, and h is the thickness of the silica gel. A new formula is obtained because the up and down surfaces of PDMS are reflected once, so

[0089]

[0090] 2. β correction operator

[0091] Due to the existence of flexible contact and diaphragm hole, the system has a field of view angle

[0092]

[0093] Put d = 0.3, f = 4mm into it to get: θ = 1.1°, that is, the farther the distance, the larger the area of the reflecting surface that can contribute to the light flux of the sensor And is proportional to the illumination lx, and

[0094]

[0095] This leads to a quadratic term of x, and also explains why in the above θ = 0.5arctan(l / (x+d+h)), I is put in as the full distance from the detector to the emitter, not half of it, because in the design without the photosensitive contact and the aperture hole, light can enter the detector from any direction, so the average point of incidence is considered to be the intersection of the perpendicular line from the detector to the emitter and the target object. But for this model, since only the circular region with the center at the point directly opposite the center of the photosensitive contact and with a radius of l / 2 contributes, the average point of incidence can be considered to be the center of this circle, i.e. the point directly opposite the center of the photosensitive contact. Figure 2 The radius of this circular region is r.

[0096] Therefore, an operator β(x) = l x *S x can be written, where has been explained, and l x is defined as the luminous flux per unit area that contributes to the photosensitive contact.

[0097] But now the operator is a quadratic term, so it should be an operator that is inversely proportional to x.

[0098]

[0099] The conclusion of the Lambert model is that L0 in one direction is an operator that is directly proportional to the cosine of the angle of emission cosω0. The conclusion of my previous simulation that uses a point source to emit with a probability of cosω0 is exactly the contribution of the light emitted with a probability of cosω0 per unit area to the incident light intensity. The simulation transmission curve is well fitted by an inverse proportional function. The simulation model is shown in Figure 3 .

[0100] Therefore, β(x) = l x *S x = β a (2.5+x), β a is proportional to This operator is modified back to

[0101]

[0102]

[0103] Optical layer brings close mark and contact mark

[0104] ​​The current proximity sensor is limited by the lack of reliable proximity and contact marks, so the measurement accuracy is affected by the surface state of the target object and decreases, but the sensor introduces a photosensitive contact layer and an oblique incident light source to produce a fixed mark point on the proximity transmission curve. The V-X transmission curve changes from monotonic increasing to monotonic decreasing, and this point is fixed at 5mm, and its accuracy is not lower than 30μm and is not affected by factors such as the color and roughness of the target object. At the same time, as shown in the principle Figure 5 , the oblique incident light source produces a light waveguide in the PDMS photosensitive contact layer, and the contact produces a leakage light, which changes the transmission curves of proximity and touch. The critical point of this change can be used as a contact discrimination mark. Its accuracy is not lower than 10μN.

[0105] The working principle of touch is divided into two stages, namely the 0-3N high-sensitivity photosensitive contact deformation stage and the 3-10N large-range substrate deformation stage. The elastic modulus of PDMS is E=2.25MPa.

[0106] The principle of the high-sensitivity photosensitive contact deformation stage is shown in Figure 6 : After the target object 23 completes the approach process, it starts to contact the flexible photosensitive contact 4 and starts the touch process. Under the action of the normal force, the target object 23 presses the photosensitive contact 4, the top of the photosensitive contact is flattened, and in the ZEMAX simulation (as shown in Figure 7 ), the flattened part (a circular surface) is obtained, and the angle of the light incident into the detector becomes smaller. In this model, the reason for the change in light flux is that part of the photosensitive contact changes from a spherical surface to a flat surface. According to the Hertz Formula in contact mechanics, the radius of the part of the hemisphere that changes to a flat surface can be described as:

[0107]

[0108] where a is the radius of the contact surface, F is the normal force, ρ1 is the curvature radius of the loaded sphere, ρ2 is the curvature radius of the loaded plane, μ1 is the Poisson's ratio of the loaded sphere, μ2 is the Poisson's ratio of the loaded plane, E1 is the Young's modulus of the loaded sphere, and E2 is the Young's modulus of the loaded plane. Generally, formula (12) can be simplified as:

[0109]

[0110] According to the simulation results, the light intensity loss rate of the photosensitive contact that changes to a flat surface is constant, which can be represented by σ l . Therefore, the change in light intensity is:

[0111]

[0112] The change in voltage is:

[0113]

[0114] The total voltage is: V f =V max -△V

[0115] In the formula V max The voltage output by the sensor upon contact is the maximum value.

[0116] When the force exceeds 3N, the clipping deformation ratio of the photosensitive contact decreases, and the main factor affecting luminous flux becomes the degree of substrate concavity. Meanwhile, the sensor output voltage at 3N is denoted as V. fh .like Figure 8 As shown, under the action of the normal force, the target object 23 presses down on the photosensitive contact 4, causing the photosensitive contact to indent downwards. In ZEMAX simulation (e.g. Figure 9 This results in a smaller angle at which light can enter the detector due to the downward indentation of the photosensitive contact.

[0117] In the tactile measurement operation of this device, the reason why pressure affects the amount of incident light energy is due to the longitudinal displacement of the photosensitive contact, which causes the focal point to shift downwards. Because of this downward shift, the aperture 7 restricts more light energy from entering the photovoltaic cell 9, resulting in a decrease in photocurrent. This displacement can be expressed by the formula: The formula is derived as follows: F represents the normal pressure; l0 represents the substrate thickness: l0 = 1 mm; and A represents the bottom area of ​​the photosensitive contact. The maximum voltage V can be collected when the contact is imminent. max Under maximum voltage conditions, the angle formed by the light passing through the aperture is... In the formula, r s Radius of the aperture This is the thickness of the aperture. The displacement Δl causes this included angle to become...

[0118] In the derivation of proximity perception, it is proposed that the intensity of light rays entering the aperture is considered to be uniform in all directions. Therefore:

[0119]

[0120] Will By substituting, we can deduce:

[0121]

[0122] Formula (11) shows that as F increases, the measured voltage V... f The trend shows a downward trend. Experiments revealed a correlation between the magnitude of the normal force and the decrease in photocurrent, and this correlation exhibits good repeatability and accuracy. Therefore, this sensor can efficiently perform tactile measurements.

[0123] The working principle of the sliding sense is as shown in the figure Figure 10 As shown in the figure, after the target object 23 is subjected to the joint action of the normal force and the tangential force, the flexible contact 4 is deflected, the focal point is offset from the center, the aperture hole 7 limits more light energy, the light intensity received by the photocell 9 is reduced, and the photoelectric current is reduced. The decrease in photoelectric current caused by the joint action of the normal force and the tangential force has a corresponding relationship and a high degree of repetition. At the same time, due to the different directions of the tangential force, the change in the photoelectric current of the four photocells corresponding to the four light-sensitive contacts will be different, and by sampling different direction tangential force data combination, the corresponding relationship between the tangential force and the voltage can be fitted. Since the output changes of the corresponding photocells of the four light-sensitive contacts are different, the tangential force and the normal force can be distinguished, and the direction of the tangential force can also be determined. In this way, a three-dimensional force measurement scheme is completed, and the critical point of the sliding of the target object 23 can be determined through the formula F f = μF N , (μ is the roughness of the sensor surface, F N is the normal force, and F f is the maximum frictional force) to determine whether the object has a risk of falling.

[0124] When the temperature sense measurement task is performed, the infrared light emitted by the object is received by the far-infrared sensor through the filtering dye. Generally, the intensity of the far-infrared light received by the sensor receiving diode can be represented by the following formula:

[0125]

[0126] Where P is the power of the light source, and r represents the distance of light propagation. The power of the far-infrared radiation of the object can be calculated as follows:

[0127] P = εAσT 4 (19)

[0128] Where ε represents the emissivity, A represents the area of the radiating object, which is the object area here, σ represents the Stefan-Boltzmann constant, and T is the temperature of the object. The semiconductor in the sensor converts the infrared signal emitted by the surrounding object into an electrical signal after receiving it, outputs the measurement result signal after processing by the far-infrared photocell processing special circuit, and finally outputs to the computer after digital-to-analog conversion. The output voltage and the object temperature have a precise transfer function relationship. So that the sensor can accurately complete the measurement.

[0129] According to the radiation heat transfer rate formula:

[0130]

[0131] In the formula: q: heat flux density per unit area; ε: emissivity (0-1); δ: Stefan-Boltzmann constant; T1: temperature of radiation surface 1; T2: temperature of radiation surface 2; S: heat storage coefficient of the material

[0132] When the target comes into contact with the sensor, the medium changes from target-air-sensor to target-sensor, and the reduction of the intermediate medium improves the heat conduction efficiency, which makes the output of the far-infrared temperature measurement unit suddenly increase, thereby becoming a stable contact mark. Compared with the optical waveguide contact mark, this mark has higher sensitivity and is more stable.

[0133] Example 2

[0134] The specific process of precise grasping by the mechanical hand is as follows:

[0135] In a conventional demand scenario, the sensor does not need to be pre-trained or calibrated, and the sensor can be arranged on the clamp by using a scheme such as Figure 11 According to the connection order of the sensor-signal processing circuit-embedded control system, the connection and driving of the sensor are completed. The embedded control system deploys the program of the precise grasping work according to the main control logic in Figure 12 , and performs electrical deployment according to the logic block diagram in Figure 13 .

[0136] In actual operation, the first step of the precise grasping work is the coarse positioning of the target, and the mechanical arm moves laterally from left to right. The coarse sensor works, and when the target object passes in front of the sensor, the sensor output presents the rules of rising, maintaining, and falling. The rising edge and the falling edge also represent the coarse position of the edge of the target object. After detecting the falling edge, the mechanical arm steps forward.

[0137] At this time, the fine sensor starts to work, and the midpoint position of the maintaining is taken as the center of the object to start the tilt measurement. In the light reflection measurement model (see formula 11), the tilt measurement can use the scheme of the binocular side tilt method (formula 21):

[0138]

[0139] d1 and d2 are two-point measurement values, and L is the distance between the two eyes. Although the sensor does not obtain a precise measurement transfer function at this time, the approach of the target object can also increase the output of the photocurrent. Therefore, when the mechanical arm moves forward and backward, data of one point is collected from the front end and the rear end of the target object, and the tilt of the target object will cause the output results of the two points to be inconsistent. We can use the size and sign of the difference Diff to eliminate the tilt until the difference Diff of the two points is within the threshold range. Combined with the torsion accuracy of the mechanical arm, the maximum angle measurement accuracy that can be obtained is 0.1 degrees.

[0140] After the tilt measurement is completed, the centering measurement is started, which is mainly realized by the proximity sensing of the proximity marker (5mm for this sensor design). Since this inflection point is determined by the sensor structure and does not change with the properties of the target object, different target objects do not affect the accuracy of the centering measurement. First, the robot arm finds the 5mm position on the right side of the target object, and uses the swing method to find the extreme value of the proximity output. The swing method is a commonly used algorithm for extreme value finding of discrete signals, and finding the 5mm position only requires finding the maximum value. After detecting the 5mm distance marker on the right side, the robot arm moves to the right to find the 5mm position on the left side, and uses the pulse counting method to calculate the movement distance of the robot arm. The midpoint from the 5mm on the right to the 5mm on the left represents the centering position. The robot arm is guided to the midpoint to complete the centering. From this point, the pre-grasping is completed. Then, self-calibration is performed, the robot arm steps by 1mm, data is collected, and when the 5mm is detected, data fitting is performed, the 0-5mm measurement curve is fitted, and then measurement and control are continued. When the tactile inflection point is detected, the 0-5N tactile measurement curve is fitted again. The robot arm stops when the preset force value is reached, and then the target object is lifted to complete the grasping.

[0141] For a heated target object, the elimination of eccentricity and tilt is more simple. Since the output of the far infrared energy is only determined by the temperature, if the temperature distribution of the target object is uniform, the output of the far infrared sensing unit (12) is only determined by the distance between the target object and the sensor and the temperature of the target object, and the output difference between the two sides of the precision measurement unit (14) is only determined by the distance difference between the target object and the two sensors. When the outputs on both sides are consistent, it means that the eccentricity of the target object is eliminated. The heat exchange formula proves that the far infrared temperature sensing unit is more accurate in identifying contact, and in this study, the far infrared sensing unit (12) is arranged on the diagonal of the sensing array. Only when both sensors on the diagonal detect contact at the same time, it is considered that the target object is parallel to the sensor, and the tilt problem is eliminated.

[0142] Example 3

[0143] Reliability test of flexible optoelectronic multi-modal sensing device

[0144] Verification includes four colors (white, orange, yellow, green) and three roughness levels (600, 800, 100 grit sandpaper) to test the reliability of the sensor control logic under different target colors, roughness, and materials. The target is a 50x50mm square fixed on a black square platform.

[0145] The basic performance calibration includes Z-axis stage, X-axis stage, force sensor, calibration stage, target stage and data acquisition card. The Z-axis and X-axis stage (recommended model Zolix PA200) has a 200mm travel range and 1μm accuracy. In the experiment, the displacement stage will step from the starting position of 200mm by 1mm, and sample 50 times after each step and calculate the root mean square result. After contact, the displacement stage steps by 0.1mm to provide the required pressure, and the sampling logic is the same as above, and the reference value of the pressure is obtained by the force sensor (recommended as ATI-MINI40).

[0146] In order to verify the maximum accuracy provided by the proximity mark and the contact mark, a calibration experiment with a 0.1mm step size is designed near these marks, and detailed proximity mark (red box) and contact mark detail information (blue box) charts are obtained. The details of the proximity mark show that the positioning accuracy exceeds 0.1mm, and it is not affected by the target object. These stable marks become the basis for realizing automated precision grasping.

[0147] As shown in Figure 14 The results of the sensor transmission curve obtained using four colors (white, orange, yellow, green) and three roughness levels (1000, 800, 600 grit sandpaper) are shown in the figure. As can be seen from the results, the sensor can achieve a 0-200mm effective range of distance measurement and a 0-40N effective range of force measurement. More importantly, the sensor shows a fixed polarity change at 5mm, a gradient change when in contact, and provides reliable proximity marks and contact marks, which are not affected by the characteristics of the target.

[0148] In summary, under the test conditions of targets of various colors and materials, the marks of the sensor remain consistent, so the measurement stability and mark stability of the sensor are very reliable.

[0149] The above only describes the preferred embodiments of the present application, and it should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A flexible optoelectronic multi-modal sensing device, characterized by: The application relates to a flexible proximity-temperature-haptic multi-modal sensor, a signal processing circuit and a precise grasping embedded control system, wherein, The flexible proximity-temperature-haptic multi-modal sensor comprises a flexible light-sensing contact layer (1), a flexible light-shield layer (2) and a flexible photoelectric conversion layer (3), the flexible light-sensing contact layer (1) is provided with a flexible contact array (4), positioning pins (6) for assembly positioning and through holes (11) for enabling smooth light emission, the flexible light-shield layer (2) is provided with light-shield holes (7), positioning holes (8) for cooperation with the positioning pins (6) for fixing and through holes (11) for enabling smooth light emission, the flexible photoelectric conversion layer (3) is respectively provided with interfaces (10), a near-infrared light source array (5), a near-infrared photocell array (9), a far-infrared photocell array (12) and positioning holes (8) for cooperation with the positioning pins (6) for fixing, the flexible contact array (4) is composed of a plurality of hemispherical flexible contacts, the near-infrared photocell array (9) is composed of a plurality of near-infrared photocells, the far-infrared photocell array (12) is composed of a plurality of far-infrared photocells, the flexible contacts, the light-shield holes (7) and the near-infrared photocells or the far-infrared photocells are coaxial; The signal processing circuit comprises an IV amplifier (17) and an ADC digital-analog converter (18), the IV amplifier (17) and the ADC digital-analog converter (18) are electrically connected, and the IV amplifier (17) is electrically connected with the interface (10); The embedded control system is electrically connected with the ADC digital-analog converter (18) through an upper computer main control board.

2. The flexible optoelectronic multi-modal sensing device of claim 1, wherein: The installation positions of the near-infrared light source array (5) and the near-infrared photocell array (9) should be spaced apart, and the top of the near-infrared light source array (5) should be lower than the lower surface of the flexible light-sensing contact layer (1).

3. The flexible optoelectronic multi-modal sensing device of claim 1, wherein: The flexible light-sensing contact layer (1), the flexible light-shield layer (2) and the flexible photoelectric conversion layer (3) are positioned and assembled through the positioning cooperation relationship between the positioning pins (6) and the positioning holes (8), and are integrally prepared through a plasma bonding process and an injection molding process.

4. The flexible optoelectronic multi-modal sensing device of claim 1, wherein: The flexible contacts of the flexible contact array (4) are distributed in a chessboard shape, the spacing between adjacent flexible contacts is the same, and through holes (11) are arranged at the middle positions of the four adjacent flexible contacts.

5. The flexible optoelectronic multi-modal sensing device of claim 1, wherein: The far-infrared photocell array (12) has two elements and is arranged at the two ends of the flexible photoelectric conversion layer (3) in a diagonal mode, and the near-infrared light source array (5) and the near-infrared photocell array (9) are uniformly and dispersedly arranged on the flexible photoelectric conversion layer (3).

6. A measurement method based on the flexible optoelectronic multi-modal sensing device according to one of claims 1 to 5, characterized in that: The precise grasping is completed based on the flexible proximity-temperature-haptic multi-modal sensor, and proximity, haptic normal force measurement and sliding measurement can be automatically distinguished through time sequence voltage information.

7. The method of measuring according to claim 6, wherein: The flexible proximity-temperature-haptic multi-modal sensor and its matching IV amplifier circuit and embedded control system can be directly installed for on-site precise grasping control. If the control efficiency needs to be improved, the user needs to fix the flexible proximity-temperature-haptic multi-modal sensor at a position where one or more of proximity and touch and slip need to be measured, connect the sensor-IV amplifier-embedded system to a computer, and use a trained network to predict the results, which can obtain sensor state prediction and measurement value calculation according to the time sequence change of the voltage.

8. The method of measuring according to claim 6, wherein: When measuring proximity, the near-infrared light source array (5) emits light energy, which is diffusely reflected on the surface of the detected object. Part of the reflected light energy can enter the near-infrared photocell array (9) through the diaphragm hole (7), forming a photocurrent, which is converted into a voltage value by the IV amplifier (17) and collected; As the detected object approaches, more reflected light energy can enter the near-infrared photocell array (9) through the restriction of the diaphragm hole (7), making the photocurrent larger. This change has the following corresponding relationship: ， In the formula, I x is the actual collected light intensity of the near-infrared photocell array, x is the real-time distance from the surface of the object being detected, β is the correction operator, d is the thickness of the photosensitive contact base, h is the thickness of the silica gel; Due to the existence of the bias light source, i.e., the near-infrared light source array (5), an output maximum value will occur at a distance of 5mm from the target object (23), which becomes the proximity landmark point of the sensor to the target. This point does not change with the properties of the target object and can be used as the eccentricity elimination target point and the self-calibration starting point. After self-calibration, a more precise transfer function can be obtained. By analyzing the collected voltage data, the distance data of the detected object can be obtained, and the proximity measurement can be completed. When measuring touch, the bias light source produces optical waveguide in the flexible photosensitive contact layer (1). After contact, the leakage light is generated, making the photocurrent have a tendency to increase. However, the loading of the detected object can cause the flexible contact to sag downward, resulting in the downward movement of the focal point of the flexible contact. The diaphragm hole (7) restricts more light energy from entering, making the photocurrent have a downward trend, resulting in a difference between the touch part transfer function and the proximity transfer function. The mutation position of the transfer function is easy to detect and can be used as a contact mark. The photocurrent is converted into a voltage value by the IV amplifier (17) and collected. This change has the following corresponding relationship: ， wherein F is the normal force, A is the photosensitive contact base area, E is the elastic modulus of the flexible material, l 0 is the substrate thickness, is the aperture hole radius, is the aperture hole thickness, V f is the real-time output voltage under the normal force F action; By analyzing the collected voltage data, the pressure data can be obtained, and the touch measurement can be completed. In the critical contact, due to the different infrared reflectivities of different materials, the collected voltage values are also different. This feature can be used to distinguish the materials of the detected object.

9. The method of claim 6, wherein: When relative sliding occurs, the flexible contact will tilt and vibrate, and the focal point will also shift and vibrate. The diaphragm hole (7) restricts more light energy from entering, and the photocurrent presents a general downward trend with fluctuations. The computer can detect this feature and determine whether sliding occurs, completing the sliding measurement.

10. The measuring method according to one of claims 6-9, characterized in that: The working process of the robot connected with the flexible proximity-temperature-haptic multi-modal sensor is divided into two processes of pre-grasping and grasping. The pre-grasping process is aimed at eliminating eccentricity and tilt, and the grasping process realizes the monitoring of normal force and the early warning and control of sliding. When the flexible proximity-temperature-haptic multi-modal sensor is used as a coarse sensor unit (15), it is located in front of the target object (23), and when the flexible proximity-temperature-haptic multi-modal sensor is used as a fine sensor unit (14), it is located on the side of the target object (23). The coarse sensor unit (15) undertakes the task of guiding the target object (23) to the measurement range of the fine sensor unit (14). After detecting that the output presents a low-high-low feature, the robot control cabinet (21) is instructed to move the robot tool end (20) forward. At this time, the fine sensor unit (14) starts to work, and the robot tool end (20) stops moving after detecting the low-high-low feature, and then starts to eliminate tilt. The double-eye lateral tilt method is used, the robot tool end (20) moves forward and backward, and the voltage difference Diff of the fine sensor unit (14) is calculated. When the Diff value is within the set threshold range, the tilt elimination is completed. Then the eccentricity elimination work starts. The robot tool end (20) first moves to the right until it reaches the proximity mark of the left unit in the fine sensor unit (14), then the robot tool end (20) moves to the left and enables pulse counting, and after reaching the proximity mark of the right unit in the fine sensor unit (14), the robot tool end (20) moves to the right, and the movement distance is determined by the pulse counting result. At this time, the pre-grasping work is completed, the host control board (16) of the upper computer issues a clamping instruction to the robot (13), and the movement is stopped after the fine sensor unit (14) detects the proximity mark, and the self-calibration data fitting is completed. Then the robot (13) continues to clamp, and the step speed decreases until the contact mark and the object tilt mark are detected, and the fine sensor unit (14) starts to monitor the normal force value until the normal force reaches the preset value, and the robot tool end (20) is controlled to move upward to complete clamping. When the fine sensor unit (14) detects the frequency spectrum signal caused by sliding, the robot (13) increases the clamping force until the preset value is reached.

Images