Flexible photoelectric multi-mode sensing device and measuring method

Abstract

The invention provides a flexible photoelectric multi-mode sensing device and a measuring method, the device is provided with an array flexible optical structure, comprises a flexible proximity-temperature-touch multi-mode sensor, a signal processing circuit and a precise grabbing embedded control system, and can be used for realizing automatic precise grabbing control. According to the measurement method, precise grabbing is completed based on the flexible proximity-temperature-touch multi-mode sensor, three work tasks of proximity and touch normal force measurement and sliding measurement are automatically distinguished through time sequence voltage information, and technical support is provided for achieving an automatic, high-precision and high-expandability precise grabbing control system.

Description

Technical Field

[0001] The present invention relates to the field of robot intelligent grasping and multimodal sensing technology, and in particular to a flexible photoelectric multimodal sensing device and a measuring method. Background Art

[0002] Grasping is a key component of robotic tasks, and the continued advancement of intelligent grasping is a prerequisite for robots to achieve these tasks in both industrial and civilian applications. However, grasping measurement needs extend beyond tactile measurement, as tactile perception cannot fully perceive the pre-grasping phase of the grasping process, and precision grasping often exhibits instability. To address this issue, proximity sensors, with their high sensitivity and high precision in close range, are ideal for combining tactile sensing with dual-modal sensing, thereby enhancing precision grasping. Therefore, the application of proximity-temperature-tactile multimodal sensors for precision grasping measurement and control in robots has become a current mainstream research direction. This research aims to achieve measurement and control of the pre-grasping phase using proximity sensors and of the grasping phase using tactile sensors, thereby comprehensively measuring and controlling the entire grasping process to achieve optimal grasping results.

[0003] However, several challenges remain. First, the current effective range of proximity sensors is relatively low, making it difficult to measure the entire grasping process. Second, the outputs of proximity-touch-slip multimodal sensors are coupled to analog outputs, lacking an intelligent and stable basis for decoupling the output curves of the two modes, requiring manual intervention for pattern recognition. Third, there is currently a lack of an online, fully automatic self-calibration method to improve the accuracy of proximity sensors in measuring unknown targets. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a flexible photoelectric multimodal sensing device.

[0005] Another technical problem to be solved by the present invention is to provide a measurement method for the above-mentioned flexible photoelectric multimodal sensing device.

[0006] In order to solve the above technical problems, the technical solution of the present invention is:

[0007] A flexible photoelectric multimodal sensing device includes a flexible proximity-temperature-tactile multimodal sensor, a signal processing circuit and a precision grasping embedded control system, wherein:

[0008] The flexible proximity-temperature-tactile multimodal sensor comprises a flexible photosensitive contact layer (1), a flexible aperture layer (2) and a flexible photoelectric conversion layer (3). The flexible photosensitive contact layer (1) is provided with a flexible contact array (4), a positioning pin (6) for assembly positioning, and a through hole (11) for allowing light to be emitted smoothly; the flexible aperture layer (2) is provided with an aperture hole (7), a positioning hole (8) for cooperating with the positioning pin (6) for fixing, and a through hole (11) for also allowing light to be emitted smoothly; the flexible photoelectric conversion layer (3) is provided with a flexible photosensitive contact layer (1) and a flexible contact array (4), a positioning pin (6) for assembly positioning, and a through hole (11) for allowing light to be emitted smoothly. The layer (3) is respectively provided with an interface (10), a near-infrared light source array (5), a near-infrared photocell array (9), a far-infrared photocell array (12), and a positioning hole (8) which is also fixed in conjunction with the positioning pin (6); the flexible contact array (4) is composed of a plurality of hemispherical flexible contacts, the near-infrared photocell array (9) is composed of a plurality of near-infrared photocells, the far-infrared photocell array (12) is composed of a plurality of far-infrared photocells, and the flexible contacts and the aperture (7) are coaxial with the near-infrared photocell or the far-infrared photocell;

[0009] The signal processing circuit includes an IV amplifier (17) and an ADC digital-to-analog converter (18), wherein the IV amplifier (17) is electrically connected to the ADC digital-to-analog converter (18), and the IV amplifier (17) is electrically connected to the interface (10);

[0010] The embedded control system has a host computer main control board (16) electrically connected to an ADC digital-to-analog converter (18).

[0011] The flexible proximity-temperature-tactile multimodal sensor is connected to an IV amplifier (17) of a signal processing circuit using an interface (10). At this time, the output is an analog voltage value that enters the ADC analog-to-digital converter (18) of the embedded control system. The output is a digital signal that is processed and analyzed by a host computer main control board (16). The control instructions are divided into two paths. One path controls the opening and closing speed and position of the manipulator (13) through a TTL to 485 chip (19), and the other path controls the lower computer main control board (22) in the manipulator control cabinet (21) through wireless communication to achieve motion control of the manipulator tool end (20).

[0012] The above-mentioned embedded control system, the manipulator (13), the manipulator control cabinet (21) and the manipulator tool end (20) are all universal devices. The manipulator control cabinet (21) and the manipulator tool end (20) are two components of a product and are forcibly connected together. The manipulator (13) is fixed on the tool end and fixedly connected to the flexible proximity-temperature-tactile multimodal sensor in a fitting manner.

[0013] Preferably, in the flexible photoelectric multimodal sensing device, the flexible proximity-temperature-tactile multimodal sensor is located in front of the target object (23) when serving as a rough measurement sensor unit (15), guiding the target object (23) into the clamping range; and / or the flexible proximity-temperature-tactile multimodal sensor is located on the side of the target object (23) when serving as a fine measurement sensor unit (14), completing the pre-grasping function through tilt elimination, eccentricity elimination, and self-calibration, and achieving the grasping function through normal force measurement and sliding detection, thereby completing all the links required for precise grasping.

[0014] Preferably, in the flexible photoelectric multimodal sensing device, the installation positions of the near-infrared light source array (5) and the near-infrared photocell array (9) should be separated by a distance, and the top of the near-infrared light source array (5) should be lower than the lower surface of the flexible photosensitive contact layer (1), so as to meet the conditions for establishing an optical waveguide.

[0015] Preferably, in the above-mentioned flexible photoelectric multimodal sensing device, the curvature radius of the flexible contact is 1.5-2.5 mm, the contact diameter is not less than 3 mm, and the thickness is 1 mm-2 mm, thereby ensuring that the flexible contact can fully complete the function of modulating the optical path and ensuring that the thickness of the sensor is small.

[0016] Preferably, in the above-mentioned flexible photoelectric multimodal sensing device, the flexible photosensitive contact layer (1), the flexible aperture layer (2) and the flexible photoelectric conversion layer (3) are positioned and assembled through the positioning and matching relationship between the positioning pins (6) and the positioning holes (8), and are integrated by combining the plasma bonding process and the injection molding process.

[0017] Preferably, in the flexible photoelectric multimodal sensing device, the flexible contacts of the flexible contact array (4) are arranged in a checkerboard pattern, the spacing between adjacent flexible contacts is the same, and through holes (11) are provided in the middle positions of four adjacent flexible contacts to ensure that the light emitted by the near-infrared light source array (5) can be emitted smoothly.

[0018] Preferably, in the flexible photoelectric multimodal sensing device, the positioning pins (6) are provided at the four corners of the flexible photosensitive contact layer (1) for assembly positioning.

[0019] Preferably, in the above-mentioned flexible photoelectric multimodal sensing device, the flexible photosensitive contact layer (1) is prepared by the following method: the preparation materials are PDMS and filter dye, combined with a high-precision mold, and prepared in sequence through the processes of glue uniformity, vacuum defoaming, injection molding, drying, and film peeling.

[0020] Preferably, in the above-mentioned flexible photoelectric multimodal sensing device, the mass ratio of the components in the preparation material is PDMSA glue:PDMS B glue:850NM filter dye=10:1:0.1.

[0021] Preferably, in the above-mentioned flexible photoelectric multimodal sensing device, the material of the flexible aperture layer (2) is black silicone, and the aperture hole (7) and the positioning hole (8) are made by precision punching equipment, and the relative position of the positioning hole (8) is consistent with the positioning pin (6).

[0022] Preferably, in the flexible photoelectric multimodal sensing device, the flexible photoelectric conversion layer (3) serves as an FPC flexible circuit board, and the relative position of the near-infrared light source array (5) thereon is consistent with the through hole (11), so that the light emitted by the near-infrared light source array (5) can be smoothly emitted through the through hole (11).

[0023] Preferably, in the flexible photoelectric multimodal sensing device, the number of the far-infrared photocell arrays (12) is two, which are diagonally arranged at both ends of the flexible photoelectric conversion layer (3); and the near-infrared light source array (5) and the near-infrared photocell array (9) are evenly dispersed and arranged on the flexible photoelectric conversion layer (3).

[0024] Preferably, in the above-mentioned flexible photoelectric multimodal sensing device, the flexible proximity-temperature-tactile multimodal sensor transmission characteristics should have a proximity mark, the center wavelength of the near-infrared light source array (5) is 850nm, the sensitivity peak wavelength of the near-infrared photocell array (9) is 850nm, and the sensitivity peak wavelength of the far-infrared photocell array (12) is 2μm-7μm, preferably 4-5μm.

[0025] Preferably, the above-mentioned flexible photoelectric multimodal sensing device, the flexible proximity-temperature-tactile multimodal sensor has a data transmission frequency of 500HZ, which can ensure timely acquisition of measurement tasks such as proximity distance or contact force sliding state with sufficiently strong real-time performance, and ensure timely control of the manipulator or other equipment to complete the protection of approaching objects or other tasks.

[0026] The measurement method of the above-mentioned flexible photoelectric multimodal sensing device completes precise grasping based on a flexible proximity-temperature-tactile multimodal sensor, and automatically distinguishes the three work tasks of proximity sensing, tactile normal force measurement and sliding measurement through timing voltage information.

[0027] Preferably, the measurement method of the above-mentioned flexible photoelectric multimodal sensing device can directly install the flexible proximity-temperature-tactile multimodal sensor and its supporting IV amplifier circuit and embedded control system for on-site precision grasping control in an environment where there are no high timeliness requirements. In an environment with high timeliness requirements, the user needs to fix the flexible proximity-temperature-tactile multimodal sensor to the location where one or more quantities of proximity and touch and slip need to be measured, connect the sensor-IV amplifier-embedded system to the computer, and use the trained network to predict the results. The network can obtain sensor state prediction and measurement value solution based on the time series changes of voltage.

[0028] Preferably, in the measurement method of the flexible photoelectric multimodal sensing device, when performing a proximity measurement task, the near-infrared light source array (5) emits light energy, which is diffusely reflected on the surface of the detected object, and part of the reflected light energy can enter the near-infrared photocell array (9) through the aperture (7), forming a photocurrent, which is converted into a voltage value by the IV amplifier (17) and collected; as the object to be detected approaches, more reflected light energy can enter the near-infrared photocell array (9) through the restriction of the aperture (7), making the photocurrent larger, and this change has the following corresponding relationship:

[0029]

[0030] Due to the presence of the bias light source (the light source emitted by the near-infrared light source array (5)), a maximum output value phenomenon will be generated at a distance of 5 mm from the target object (23), which becomes the proximity mark point of the sensor approaching the target object. This point does not change with the properties of the target object and can be used as the eccentricity elimination target point and the starting point of self-calibration. After self-calibration, a more precise transfer function can be obtained. By analyzing the collected voltage data, the distance data of the detected object can be obtained to complete the proximity measurement.

[0031] Preferably, in the measurement method of the flexible photoelectric multimodal sensing device, when performing a tactile measurement task, the bias light source generates an optical waveguide in the flexible photosensitive contact layer (1), and after contact, leakage light is generated, so that the photocurrent has an increasing trend; however, the loading of the detected object can cause the flexible contact to sag downward, resulting in the focus of the flexible contact moving downward, and the aperture (7) limits more light energy from entering, so that the photocurrent has a decreasing trend, resulting in a difference between the transfer function of the tactile part and the transfer function of the proximity sense, and the mutation position of the transfer function is easy to detect and can be used as a contact mark; the photocurrent is converted into a voltage value by the IV amplifier (17) and collected, and this change has the following corresponding relationship:

[0032]

[0033] By analyzing the collected voltage data, pressure data can be obtained to complete tactile measurement; at critical contact, due to the different infrared reflectivity of different materials, the collected voltage values ​​are also different, and this feature can be used to distinguish the materials of the detected objects.

[0034] Preferably, in the measurement method of the flexible photoelectric multimodal sensing device, when relative sliding occurs, the flexible contact will tilt and vibrate, the focus will also shift and vibrate, the aperture (7) limits more light energy from entering, and the photocurrent shows an overall decrease accompanied by fluctuations. The computer can detect this feature and determine whether sliding occurs, thereby completing the sliding measurement.

[0035] Preferably, in the measurement method of the flexible photoelectric multimodal sensing device, when the target object is a heat-generating object, the target object becomes a far-infrared energy source, and the far-infrared photocell array (12) serves as a far-infrared detection unit (temperature measurement unit) (starts working, according to the thermal radiation power-distance solution formula: The closer the distance between the target object and the flexible proximity-temperature-tactile multimodal sensor, the stronger the far-infrared energy incident on the flexible proximity-temperature-tactile multimodal sensor. The far-infrared photocell array (12) can be used as an auxiliary ranging module of the flexible proximity-temperature-tactile multimodal sensor. At the same time, the far-infrared radiation intensity is only related to the temperature of the target object and has nothing to do with the color, roughness, etc. of the object. This feature enables the error of light reflection type proximity sensing to be compensated, thereby improving the measurement accuracy. After contacting the target object, the far-infrared light no longer passes through the air, but directly enters the flexible photosensitive contact layer, thereby improving the heat conduction efficiency, greatly increasing the far-infrared energy entering the flexible proximity-temperature-tactile multimodal sensor, and causing a sudden change in the far-infrared intensity. This sign determines the contact between the object and the flexible contact array, thereby assisting the flexible proximity-temperature-tactile multimodal sensor to complete proximity-tactile modal recognition.

[0036] Preferably, in the measurement method of the flexible photoelectric multimodal sensing device, the working process of the manipulator is decomposed into two processes: pre-grasping and grasping. The pre-grasping process eliminates eccentricity and tilt, while the grasping process realizes normal force monitoring and early warning and control of sliding; wherein, the coarse measurement sensor unit (15) is responsible for guiding the target object (23) to the measurement range of the fine measurement sensor unit (14), and after detecting that its output presents a low-high-low feature, it issues an instruction to the manipulator tool end (20) to move forward to the manipulator control cabinet (21), at which time the fine measurement sensor unit (14) starts to work, and after detecting the low-high-low feature, the manipulator tool end (20) stops moving, and then starts the tilt elimination work, adopts the binocular tilt method, uses the manipulator tool end (20) to move back and forth, calculates the voltage difference Diff between two points of the fine measurement sensor unit (14), and when the Diff value is within the set threshold range, the tilt elimination is completed; then the eccentricity elimination work is started, and the manipulator works The tool end (20) first moves to the right until it reaches the proximity mark of the left unit in the precision sensor unit (14), and then the tool end (20) of the manipulator moves to the left and starts pulse counting. After reaching the proximity mark of the right unit in the precision sensor unit (14), the tool end (20) of the manipulator moves to the right, and the movement distance is determined by the pulse counting result; at this time, the pre-grabbing work is completed, and the host computer main control board (16) issues a clamping instruction to the manipulator (13), and stops moving after the precision sensor unit (14) detects the proximity mark, and completes the self-calibration data fitting; then the manipulator (13) continues to clamp, and the stepping speed decreases until the contact mark and the object no tilt mark are detected, and the precision sensor unit (14) starts to monitor the normal force value until the normal force reaches a preset value, and controls the tool end (20) of the manipulator to move upward to complete the clamping; when the precision sensor unit (14) detects the spectrum signal caused by the sliding, the manipulator (13) increases the clamping force until the preset value.

[0037] Preferably, in the measurement method of the flexible photoelectric multimodal sensing device, when the target object (23) is a heat-generating object, the tilt elimination work and the eccentricity elimination work in the pre-grabbing process can be completed with the assistance of the far-infrared photocell array (12): since the emission of far-infrared energy is only determined by the temperature of the target object and does not change with its surface characteristics, in the eccentricity elimination, it is only necessary for the far-infrared photocell arrays (12) on the left and right sides of the precision measurement sensor unit (14) to have the same output; the output surge after the target object contacts the precision measurement sensor unit (14) can be used as a contact mark, and each far-infrared photocell unit of the far-infrared photocell array (12) is distributed on the diagonal line of the flexible photoelectric conversion layer (3). When two far-infrared photocell units detect the contact mark at the same time, it can be considered that the tilt is completely eliminated.

[0038] Beneficial effects:

[0039] The flexible photoelectric multimodal sensing device has an arrayed flexible optical structure, which includes the processing and acquisition of raw signals from the flexible proximity-temperature-tactile multimodal sensor, data analysis and control instruction transmission by the main control processor. As a flexible photoelectric proximity-tactile-slip multimodal array sensing system, it can be used to achieve automated precision grasping control, providing technical support for the realization of an automated, high-precision, and highly scalable precision grasping control system. Specifically:

[0040] 1. It has the advantages of strong anti-interference, small size and light weight, and is more suitable for the use scenarios of robot precision grasping. It has stronger ability to resist interference from environmental factors such as electromagnetic interference, and can still ensure the measurement accuracy of the sensor in different use scenarios.

[0041] 2. The sensor is a flexible system that can be adapted to the surface of any shape of fixture. At the same time, its manufacturing process is easy and low-cost. Its consistency can also be effectively controlled by the injection molding process, ensuring the economy and feasibility of large-scale production.

[0042] 3. The introduction of a photosensitive contact optical layer modulates the sensor's receiving optical path, expanding the sensor's effective range. A physical model was derived, demonstrating the qualitative impact of structural parameter settings on the transmission curve. This also reduces the impact of stray light on the system and improves the system's signal-to-noise ratio. This further ensures high resolution within the system's high-sensitivity range, fully guaranteeing high-precision measurement and control for tasks such as eccentricity and normal force measurement required for precision grasping.

[0043] 4. The introduction of a flexible photosensitive contact layer and a biased light source provides the sensor with fixed proximity and contact markers, providing solutions to the challenges of offset elimination, self-calibration, and proximity-tactile modal recognition in precision grasping. Furthermore, experimental verification has shown that these markers do not change with the properties of the target, ensuring the sensor's accuracy under all operating conditions.

[0044] 5. It has temperature sensing capability and can use heat to detect the distance and contact signals of heating objects, thereby improving the proximity measurement accuracy and contact mark discrimination stability, and providing a more efficient tilt elimination method.

[0045] 6. The flexible proximity-temperature-tactile multimodal sensor uses its proximity mark, contact mark and far-infrared temperature measurement to design an automated precision grasping control solution, which includes automated grasping guidance, automated tilt and eccentricity elimination, automated self-calibration, automated normal force prediction and automated slip measurement, covering the entire process of pre-grasping and grasping of precision grasping work. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a structural schematic diagram of the flexible proximity-temperature-tactile multimodal sensor of the present invention.

[0047] Figure 2 It is a structural schematic diagram of the flexible photoelectric multimodal sensing device described in the present invention.

[0048] Figure 3 It is a schematic diagram of light path simulation performed in ZEMAX simulation software according to the present invention.

[0049] Figure 4 It is a schematic diagram of the optical waveguide construction and the principle of light leakage of the present invention.

[0050] Figure 5 Schematic diagram of the present invention for high-sensitivity tactile sensing.

[0051] Figure 6 This is a simulation diagram of the optical path used for high-sensitivity tactile sensing in the present invention.

[0052] Figure 7 It is a schematic diagram of the present invention used for large-scale tactile sensing.

[0053] Figure 8 This is a simulation diagram of the optical path used for large-scale tactile sensing according to the present invention.

[0054] Figure 9 It is a schematic diagram of the optical path used for three-dimensional force tactile sensing of the present invention.

[0055] Figure 10 It is a schematic diagram of the contact mark generated by the thermal sensing unit of the present invention.

[0056] Figure 11 It is a schematic diagram of the sensor layout used for precise clamping of a manipulator according to the present invention.

[0057] Figure 12 It is a control logic diagram of the present invention used in a manipulator precision clamping experiment.

[0058] Figure 13 It is a schematic diagram of the electrical connection of the present invention for a manipulator precision clamping experiment.

[0059] Figure 14 This is a diagram for verifying the stability of the landmark used in the robot's precision clamping experiment.

[0060] In the figure: 1-flexible photosensitive contact layer (PDMS model is RTV615, produced by Momentive), 2-flexible aperture layer, 3-flexible photoelectric conversion layer, 4-flexible contact array, 5-near-infrared light source array, 6-positioning pin, 7-aperture hole, 8-positioning hole, 9-near-infrared photocell array (near-infrared photocell model is Siemens BPW34 S), 10-interface, 11-through hole, 12-far-infrared photocell array (far-infrared photocell model is VIGO PVI-4TE-5 photodiode), 13-manipulator (Easy Claw Robot ZFG-50), 14-precision measurement sensor unit, 15-coarse measurement sensor unit, 16-host computer main control board (STMicroelectronics (ST) STM32 F411CEU6), 17-IV amplifier, 18-ADC analog-to-digital converter, 19-TTL to 485 chip (model is MAX485 ADI), 20-robotic arm tool end (robotic arm UR5), 21-robotic arm control cabinet (robotic arm UR5), 22-lower computer main control board (STMicroelectronics (ST) STM32 F411 CEU6), 23-target object. DETAILED DESCRIPTION

[0061] The flexible photoelectric multimodal sensing device, the flexible photoelectric multimodal sensing device, and the measurement method of the present invention are described below with reference to the embodiments and the accompanying drawings.

[0062] Example 1

[0063] like Figure 1-13 As shown, a flexible photoelectric multimodal sensing device includes a flexible proximity-temperature-tactile multimodal sensor, a signal processing circuit and a precision grasping embedded control system, wherein:

[0064] The flexible proximity-temperature-tactile multimodal sensor comprises a flexible photosensitive contact layer 1, a flexible aperture layer 2 and a flexible photoelectric conversion layer 3. The flexible photosensitive contact layer 1 is provided with a flexible contact array 4, a positioning pin 6 for assembly positioning and a through hole 11 for allowing light to be emitted smoothly; the flexible aperture layer 2 is provided with an aperture hole 7, a positioning hole 8 for cooperating with the positioning pin 6 for fixation and a through hole 11 for also allowing light to be emitted smoothly; the flexible photoelectric conversion layer 3 is provided with an interface 10, a near-infrared light source array 5, a near-infrared photocell array 9, a far-infrared photocell array 10 and a far-infrared photocell array 11. The columns 12 and the positioning holes 8 that are also fixed in conjunction with the positioning pins 6, the near-infrared light source array 5 and the near-infrared photocell array 9 and the far-infrared photocell array 12 can be directly welded on the flexible photoelectric conversion layer 3. The installation positions of the near-infrared light source array 5 and the near-infrared photocell array 9 should be separated by a certain distance, and the top of the near-infrared light source array 5 is lower than the lower surface of the flexible photosensitive contact layer 1, so as to meet the conditions for establishing an optical waveguide: 1. There is a certain distance between the light source and the signal receiver so that the light path is not vertically emitted and injected, and 2. The light source emission position should be lower than the lower surface of the layer for establishing the optical waveguide. The flexible contact array 4 is composed of a number of hemispherical flexible contacts, the near-infrared photocell array 9 is composed of a number of near-infrared photocells, the far-infrared photocell array 12 is composed of a number of far-infrared photocells, and the flexible contacts and aperture holes 7 are coaxial with the near-infrared photocell or the far-infrared photocell; wherein, the preparation process is as follows:

[0065] (1) The flexible photosensitive contact layer 1 is composed of a flexible contact array 4 and a PDMS substrate, and is made by injection molding. The mold is made using a high-precision lathe with a tolerance of less than 0.05mm. In order to reduce the interference factors caused by visible light, PDMS needs to be mixed with a 780nm high-pass filter dye. These three are poured into the glue machine in the ratio of mass ratio A glue: B glue (curing agent): filter dye = 10:1:0.01. The glue is stirred for no less than 30 minutes before being taken out. The mixed glue is placed in a vacuum box for defoaming, and the defoaming time is no less than 40 minutes. The colloid that has been defoamed for the first time is taken out, injected into the mold, and placed in the vacuum box again for a second defoaming, and the time is also no less than 40 minutes. The mold injected with the colloid after the second defoaming is placed in a constant temperature heating box and maintained at 70 degrees Celsius for 3 hours. After taking it out, the mold is removed and used. The device substrate is 1.0 mm thick, 28.0 mm long, and 28.0 mm wide. The contact curvature radius is 2.0 mm, and there are four positioning pins with a designed diameter of 0.9 mm for positioning the sensor assembly.

[0066] (2) The flexible aperture layer 2 is made of black silicone with a thickness of 0.5 mm. The drilling process is completed by a high-precision lathe driven punching head, and the required tolerance is less than 0.05 mm. The position of the three positioning holes is consistent with the positioning pins, and the diameter is 1.0 mm. At the same time, 9 aperture holes are designed to correspond to the positions of the photodiodes, and their diameter is 0.6 mm. According to the matching relationship between the positioning holes and the positioning pins, the flexible aperture layer 2 is assembled below the flexible photosensitive contact layer 1.

[0067] (3) The base of the flexible photoelectric conversion layer 3 is an FPC (flexible printed circuit). Although the devices welded thereto (including the interface 10, the near-infrared photocell array 9, the far-infrared photocell array 12 and the near-infrared light source array 5) are rigid devices, their small size does not affect the compliance of the circuit board and they can still be firmly attached even on complex surfaces. The FPC is designed with positioning holes corresponding to the positioning pins, with a diameter of 1.0 mm. The FPC is assembled below the aperture layer according to the matching relationship.

[0068] (4) After assembling the above three components, place them in a sealing mold and seal them with silicone. When the silicone is cured, demould to obtain a finished sensor. The total thickness of the device is 3.5 mm, the length is 28.0 mm, and the width is 28.0 mm.

[0069] The signal processing circuit includes an IV amplifier 17 and an ADC digital-to-analog converter 18, wherein the IV amplifier 17 is electrically connected to the ADC digital-to-analog converter 18, and the IV amplifier 17 is electrically connected to the interface 10;

[0070] The embedded control system includes a host computer main control board 16, a slave computer main control board 22, a TTL-to-485 chip 19, and a Bluetooth communication module (CC2540 TI Bluetooth chip, Texas Instruments). The host computer main control board 16, the TTL-to-485 chip 19, the IV amplifier 17, the ADC digital-to-analog converter 18, and the Bluetooth chip are integrated on a circuit board and can be installed on the manipulator 13. The ADC digital-to-analog converter 18 serves as an interface with the upper-level IV amplifier for obtaining raw data. The slave computer main control board 22 and another Bluetooth chip are integrated on another circuit board and installed in the control cabinet for receiving and processing instructions.

[0071] like Figure 13As shown, during the application process, the transmission characteristics of the flexible proximity-temperature-tactile multimodal sensor should have a proximity mark, the center wavelength of the near-infrared light source array 5 is 850nm, the sensitivity peak wavelength of the near-infrared photocell array 9 is 850nm, and the sensitivity peak wavelength of the far-infrared photocell array 12 is 2-7μm (preferably 4μm). When the data transmission frequency of the flexible proximity-temperature-tactile multimodal sensor is 500HZ, it can ensure timely acquisition of measurement tasks such as proximity distance or contact force sliding state with sufficient real-time performance, and ensure timely control of the manipulator or other equipment to complete the protection of approaching objects or other tasks. The flexible proximity-temperature-tactile multimodal sensor is connected to the IV amplifier 17 of the signal processing circuit using the interface 10. The sensor obtains the original current value, enters the IV amplifier 17 for amplification, and then the amplified voltage value is converted into digital by the ADC sampling module 18. The data enters the upper control board 16 for calculation through the I2C protocol. The control results obtained by the calculation are divided into two paths. One path controls the opening and closing speed and position of the manipulator 13 through the TTL-to-485 chip 19. The other path uses a Bluetooth module to control the lower computer main control board 22 in the manipulator control cabinet 21 through wireless communication. The lower control board 22 translates the instructions into control information and sends it to the UR5 manipulator, realizing motion control of the manipulator tool end 20 and completing closed-loop control. The embedded control system used above, as well as the manipulator 13, manipulator control cabinet 21, and manipulator tool end 20 are all universal devices. The manipulator control cabinet 21 and manipulator tool end 20 are two components of a single product, forcibly connected together. The manipulator 13 is fixed to the tool end and fixedly connected to the flexible proximity-temperature-tactile multimodal sensor in a fitted manner.

[0072] The working principle of the flexible proximity-temperature-tactile multimodal sensor is as follows:

[0073] How proximity perception works Figure 2 As shown, since the surface of the object can be considered as a rough surface, when the infrared light of the light source irradiates the surface of the target object 23, the light will be diffusely reflected on the rough surface. For this sensor, it is only necessary to obtain the diffusely reflected light intensity to calculate the voltage value.

[0074] V=A v σI x (1)

[0075] Where I x is the light intensity, and the coefficient σ is the detector sensitivity A v is the gain multiple

[0076] Assume that the emitted light intensity is 0 and the measured reflected light intensity is 1. Assume that the thickness of the rubber is d and the distance from the object is x. Figure 1Depending on the refractive index of the rubber material, part of the R light will be reflected from the interface between the rubber and the air, part of the k light will be scattered, and part of the a light will be reflected on the target surface.

[0077] The classic model of diffuse reflection intensity and reflection distance is:

[0078]

[0079] According to the Fresnel equation Born and Wolf (1999), the reflection at the PDMS / air interface

[0080]

[0081] However, this classic model cannot fit the real curve, indicating that the structure of the sensor's photosensitive contact and aperture affects the transfer function model. In order to find the influencing factor, two correction operators are proposed to modify the classic model.

[0082] 1. The oblique incidence of the light source will introduce a cosine correction term into the formula.

[0083]

[0084] Where:

[0085] θ=0.5arctan(4 / (x+2))(5)

[0086] The general formula is:

[0087] θ=0.5arctan(l / (x+d+h))(6)

[0088] Where I is the distance between the silicon photocell and the emitter, d is the thickness of the photosensitive contact substrate, and h is the thickness of the silicone. A new formula is obtained because the reflection occurs once on the upper and lower surfaces when passing through PDMS. Therefore

[0089]

[0090] 2.β correction operator

[0091] Due to the presence of the flexible contacts and the aperture, the system has a field of view angle

[0092]

[0093] Substituting d = 0.3 and f = 4 mm, we get: θ = 1.1°. In other words, the farther the distance, the greater the reflective surface area that can contribute to the sensor's luminous flux. The bigger it is. Proportional to the illuminance lx, and

[0094]

[0095] This leads to a quadratic term related to x, and also explains that in the above θ = 0.5arctan(l / (x+d+h)), I refers to the full distance from the detector to the emitter instead of half of this distance. Because in a design without a photosensitive contact and an aperture, light from any direction can enter the detector, so the average incident point is considered to be the intersection of the perpendicular bisector between the detector and the emitter and the target. However, for this model, since the only contribution is the point with the center of the circle facing the center of the photosensitive contact, The average incident point is a circular domain with a radius of , and the center of the circle can be considered as the center of the circle, that is, the position facing the center point of the photosensitive contact. Figure 2 The radius of this circle is r.

[0096] Therefore, we can write an operator: β(x) = l x *S x In the formula It has been explained, and l x It is defined as the luminous flux per unit area contributed to the photosensitive contact.

[0097] But now the operator is a quadratic term, proving that / x should be an operator inversely proportional to x.

[0098]

[0099] The conclusion of the Lambert model is that L0 in a certain direction is a formula that is directly proportional to the cosine value of the emission angle, cosω0. My previous simulation using a point light source to make a cosω0 probability emission conclusion just reflects the contribution of light emitted with a cosω0 probability per unit area to the incident light intensity. The simulated transmission curve is fitted with a linear inverse proportional function, which has a high degree of fit. The simulation model is shown below. Figure 3 .

[0100] Thus, β(x)=l x *S x =β a (2.5+x), β a Proportional to Correct this operator back to

[0101] get:

[0102]

[0103] Proximity and contact marks brought by the optical layer

[0104] Currently, proximity sensors are limited by the lack of reliable proximity and contact markers, so the measurement accuracy will be affected by the surface condition of the target object and decrease. However, the photosensitive contact layer and oblique incident light source introduced by this sensor create a fixed marker point in the proximity transmission curve, and its VX transmission curve changes from monotonically increasing to monotonically decreasing. This point is fixed at 5mm, and its accuracy is not less than 30μm and is not affected by factors such as the color and roughness of the target object. At the same time, as the principle Figure 5 As shown, an obliquely incident light source creates an optical waveguide in the PDMS photosensitive contact layer. Contact generates leaked light, causing a change in the transmission curves of proximity and touch. This critical point of change can be used as a contact detection marker, with an accuracy of no less than 10μN.

[0105] The haptic principle works in two stages: a high-sensitivity photosensitive contact deformation stage of 0-3N and a large-scale substrate deformation stage of 3-10N. The elastic modulus of PDMS is E = 2.25MPa.

[0106] The principle of deformation stage of high-sensitivity photosensitive contact is as follows Figure 6 As shown in the figure: after the target object 23 completes the approach process, it begins to contact the flexible photosensitive contact 4 and starts the tactile process. Under the action of the normal force, the target object 23 presses the photosensitive contact 4, and the top of the photosensitive contact becomes flat. In the ZEMAX simulation (as shown in Figure 7 ) results in a flattened top (a circular surface), and the angle at which light can enter the detector decreases. In this model, the change in luminous flux occurs because a portion of the photosensitive contact changes from a spherical surface to a flat surface. According to the Hertz formula in contact mechanics, the radius of the flat portion of the hemisphere can be described as:

[0107]

[0108] Where: a is the contact surface radius, F is the normal pressure, ρ1 is the curvature radius of the load sphere, ρ2 is the curvature radius of the load surface, μ1 is the Poisson's ratio of the load sphere, μ2 is the Poisson's ratio of the load surface, E1 is the Young's modulus of the load sphere, and E2 is the Young's modulus of the load plane. In general, Equation (12) can be simplified to:

[0109]

[0110] According to the simulation results, the loss rate of light intensity per unit area is constant for the photosensitive contact that becomes a plane, which can be expressed as σ l In this way, the change in illumination is:

[0111]

[0112] The change in voltage is:

[0113]

[0114] The total voltage is: V f =V max -△V

[0115] Where V max The sensor output voltage is the maximum value when in contact.

[0116] When the force exceeds 3N, the deformation ratio of the top of the photosensitive contact decreases, and the main factor affecting the light flux becomes the degree of depression of the base. At the same time, the output voltage of the sensor at 3N is V fh .like Figure 8 As shown, under the action of the normal force, the target object 23 presses the photosensitive contact 4, and the photosensitive contact 4 is depressed downward. In the ZEMAX simulation (as shown in FIG. Figure 9 ) It is found that because the photosensitive contact is concave downward, the angle at which light can enter the detector becomes smaller.

[0117] When this device is measuring tactile sensation, the reason pressure affects incident light energy is because the longitudinal displacement of the photosensitive contact causes the focal point to shift downward. Due to the downward shift of the focal point, aperture 7 restricts more light energy from entering photocell 9, causing the photocurrent to decrease. This displacement can be expressed by the formula: Where F is the normal pressure. l0 is the substrate thickness: l0 = 1mm. A is the bottom area of ​​the photosensitive contact: The maximum voltage V can be collected when close to contact max , under the condition of maximum voltage, the angle of light that can pass through the aperture is Where r s is the aperture radius is the aperture thickness. The displacement △l makes this angle become

[0118] In the derivation of proximity perception, it is proposed that the intensity of light entering the aperture is assumed to be consistent in all directions. Therefore:

[0119]

[0120] Will Substituting in, we can deduce:

[0121]

[0122] From formula (11), we can see that as F increases, the voltage measurement value V f The normal force shows a downward trend. Experiments have shown a corresponding relationship between the magnitude of the normal force and the decrease in the photocurrent, and this relationship has good repeatability and accuracy. Therefore, this sensor can effectively perform tactile measurements.

[0123] How does the sense of slip work? Figure 10 As shown, after the target object 23 is subjected to the combined action of the normal force and the tangential force, the flexible contact 4 is deflected, causing the focus to shift from the center, the aperture 7 to limit more light energy, the illumination received by the photocell 9 to decrease, and the photocurrent to decrease. This reduction in photocurrent caused by the combined action of the normal force and the tangential force has a corresponding relationship and a high degree of repeatability. At the same time, due to the different directions of the tangential force, the photocurrent changes of the four photocells corresponding to the four photosensitive contacts will be different. By sampling the data combination of tangential forces in different directions, the corresponding relationship between the tangential force and the voltage can be fitted. Since the output changes of the photocells corresponding to the four photosensitive contacts are different, the tangential force and the normal force can be distinguished, and the direction of the tangential force can also be determined, thus completing a three-dimensional force measurement scheme. By using the formula F f =μF N The critical point of the target object 23 sliding can be determined (μ is the sensor surface roughness, F N is positive pressure, F f is the maximum friction force) to determine whether the object has the risk of slipping.

[0124] When the present invention performs temperature measurement tasks, the infrared light emitted by the object is received by the far-infrared sensor through the filter dye. Generally, the intensity of the far-infrared light received by the sensor receiving diode can be expressed by the following formula:

[0125]

[0126] Where P is the power of the light source and r represents the distance the light travels. The power of the object radiating in the far infrared can be calculated as follows:

[0127] P=εAσT 4 (19)

[0128] Here, ε represents the emissivity, A represents the area of ​​the radiating object (in this case, the object's area), σ represents the Stefan-Boltzmann constant, and T represents the object's temperature. The semiconductor in the sensor receives infrared signals emitted by surrounding objects and converts them into electrical signals. After processing through a dedicated circuit in a far-infrared photoelectric tube, the output signal is converted to a measurement result. Finally, after digital-to-analog conversion, the output signal is sent to a computer. This output voltage has a precise transfer function relationship with the object's temperature, enabling the sensor to accurately perform measurements.

[0129] According to the radiation heat transfer rate formula:

[0130]

[0131] Where: q: heat flux density per unit area; ε: emissivity (0-1); δ: Stefan-Porhertz constant; T1: temperature of radiating surface 1; T2: temperature of radiating surface 2; S: thermal storage coefficient of the material

[0132] When the target contacts the sensor, the medium changes from target-air-sensor to target-sensor. The reduction of the intermediate medium improves heat transfer efficiency, causing the output of the far-infrared temperature measurement unit to surge, creating a stable contact mark. This mark is more sensitive and stable than an optical waveguide contact mark.

[0133] Example 2

[0134] The specific process of the robot implementing precision grasping is as follows:

[0135] In conventional demand scenarios, sensors do not need to be pre-trained or calibrated. Figure 11 The sensor is placed on the fixture according to the solution. The sensor is connected and driven according to the connection sequence of sensor-signal processing circuit-embedded control system. Figure 12 The master control logic in the program deployment of precise crawling work, and according to Figure 13 The electrical deployment is based on the logic block diagram.

[0136] In actual operation, coarse sensing is the first step in precision grasping: roughly locating the target object. The robotic arm moves horizontally from left to right, and the coarse sensing sensor operates. When the target object passes in front of the sensor, the sensor output rises, remains, and then falls. The rising and falling edges also represent the coarse position of the target object's edge. After detecting the falling edge, the robotic arm steps forward.

[0137] At this point, the precision sensor begins to work, detecting the same pattern of rise, hold, and fall. The midpoint of the hold is taken as the center of the object and tilt measurement begins. In the light reflection measurement model (see Equation 11), tilt measurement can use the binocular tilt method (Equation 21):

[0138]

[0139] d1 and d2 are the measured values ​​at two points, and L is the binocular distance. Although the sensor doesn't yet have a precise measurement transfer function, the proximity of the target can still increase the photocurrent output. Therefore, when the robotic arm moves back and forth, collecting data from one point at the front and one at the back of the target, the tilt of the target can cause inconsistencies in the output results. We can use the magnitude and sign of this difference (Diff) to eliminate the tilt until the difference between the two points is within the threshold. Combined with the torsional accuracy of the robotic arm, the maximum angular measurement accuracy is 0.1 degrees.

[0140] After the tilt measurement is complete, the centering measurement begins. This measurement primarily utilizes the proximity sensor's inflection point (designed at 5mm for this sensor). Since this inflection point is determined by the sensor's structure and does not vary with the target's properties, varying the target's position does not affect the accuracy of the centering measurement. The robotic arm first searches 5mm to the right of the target and uses the swinging method to find the extreme value of the proximity sensor output. The swinging method is a common algorithm for finding extreme values ​​in discrete signals; finding the 5mm position simply requires finding the maximum value. After detecting the 5mm distance mark to the right, the robotic arm moves right, searching for a position 5mm to the left. Pulse counting is used to calculate the distance the robotic arm has moved. The midpoint of the movement from 5mm to the left represents the centering position. Guiding the robotic arm to this midpoint completes the centering. Pre-grasping is now complete. Self-calibration then occurs, with the robot moving in 1mm steps, collecting data. When it detects 5mm, it pauses for data fitting, creating a 0-5mm measurement curve. Measurement and control then continue. When the tactile inflection point is detected, it refits the 0-5N tactile measurement curve. The robot stops when it reaches the preset force value, then lifts the object to complete the grasp.

[0141] For a target object that is heated, the elimination of eccentricity and tilt is simpler. Since the output of far-infrared energy is determined only by temperature, if the target temperature is evenly distributed, the output of the far-infrared sensing unit (12) is determined only by the distance between the target and the sensor and the target temperature, while the output difference between the two sides of the precision measurement unit (14) is determined only by the difference in the distance between the target and the two sensors. When the outputs on both sides are consistent, it means that the eccentricity of the target has been eliminated. The heat exchange formula proves that the far-infrared temperature sensing unit is more accurate in identifying contact. In this study, the far-infrared sensing unit (12) is set at the diagonal corner of the sensor array. If and only if the two diagonal sensors detect the contact mark at the same time, it can be considered that the target and the sensor are completely parallel, and the tilt problem is eliminated.

[0142] Example 3

[0143] Reliability testing of flexible optoelectronic multimodal sensing devices

[0144] The validation included four colors (white, orange, yellow, green) and three roughness levels (600, 800, 100 grit sandpaper) to test the reliability of the sensor's control logic under different target colors, roughness, and materials. The target was a 50x50 mm square fixed on a black square platform.

[0145] Basic performance calibration involves a Z-axis stage, X-axis stage, force sensor, calibration platform, target platform, and data acquisition card. The Z-axis and X-axis stages (Zolix PA200 is recommended) have a 200mm travel range and 1μm accuracy. During the experiment, the stage will be stepped in 1mm increments from a 200mm starting position. After each step, 50 samples are taken and the RMS result is calculated. After contact, the stage will step in 0.1mm increments to provide the required pressure. The sampling logic is the same as above, and the pressure reference value is obtained by the force sensor (ATI-MINI40 is recommended).

[0146] To verify the maximum accuracy provided by the proximity and contact markers, a calibration experiment with a 0.1 mm step size was designed near these markers, resulting in detailed graphs of the proximity markers (red box) and contact marker details (blue box). The proximity marker details demonstrate positioning accuracy exceeding 0.1 mm and are unaffected by the target object. These stable markers form the foundation for automated precision grasping.

[0147] like Figure 14 The figure shows the sensor's transfer curves obtained using four colors (white, orange, yellow, and green) and three roughness levels (1000, 800, and 600 grit sandpaper). The results demonstrate that the sensor is capable of measuring an effective range of 0-200 mm for distance and 0-40 N for force. More importantly, the sensor exhibits a fixed polarity change at 5 mm and a gradient change upon contact, providing reliable proximity and contact markings regardless of target characteristics.

[0148] In summary, the sensor's markings remain consistent under test conditions for targets of various colors and materials, so both the sensor's measurement stability and marking stability are very reliable.

[0149] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A flexible photoelectric multimodal sensing device, characterized in that: It includes flexible proximity-temperature-tactile multimodal sensor, signal processing circuit and precision grasping embedded control system, among which, The flexible proximity-temperature-tactile multimodal sensor comprises a flexible photosensitive contact layer (1), a flexible aperture layer (2) and a flexible photoelectric conversion layer (3); the flexible photosensitive contact layer (1) is provided with a flexible contact array (4), a positioning pin (6) for assembly positioning and a through hole (11) for enabling light to be emitted smoothly; the flexible aperture layer (2) is provided with an aperture hole (7), a positioning hole (8) for fixing with the positioning pin (6) and a through hole (11) for enabling light to be emitted smoothly; the flexible photoelectric conversion layer (3) is provided with a flexible photosensitive contact layer (1) and a flexible contact array (4), a positioning pin (6) for assembly positioning and a through hole (11) for enabling light to be emitted smoothly; The layer (3) is respectively provided with an interface (10), a near-infrared light source array (5), a near-infrared photocell array (9), a far-infrared photocell array (12), and a positioning hole (8) which is also used for fixing with the positioning pin (6); the flexible contact array (4) is composed of a plurality of hemispherical flexible contacts, the near-infrared photocell array (9) is composed of a plurality of near-infrared photocells, the far-infrared photocell array (12) is composed of a plurality of far-infrared photocells, and the flexible contact and the aperture hole (7) are coaxial with the near-infrared photocell or the far-infrared photocell; The signal processing circuit comprises an IV amplifier (17) and an ADC digital-to-analog converter (18), wherein the IV amplifier (17) is electrically connected to the ADC digital-to-analog converter (18), and the IV amplifier (17) is electrically connected to the interface (10); The embedded control system has a host computer main control board electrically connected to an ADC digital-to-analog converter (18).

2. The flexible photoelectric multimodal sensing device according to claim 1, characterized in that: The installation positions of the near-infrared light source array (5) and the near-infrared photocell array (9) should be spaced apart by a certain distance, and the top of the near-infrared light source array (5) should be lower than the lower surface of the flexible photosensitive contact layer (1).

3. The flexible photoelectric multimodal sensing device according to claim 1, characterized in that: The flexible photosensitive contact layer (1), the flexible aperture layer (2) and the flexible photoelectric conversion layer (3) are assembled by positioning through the positioning matching relationship between the positioning pins (6) and the positioning holes (8), and are integrated by combining the plasma bonding process with the injection molding process.

4. The flexible photoelectric multimodal sensing device according to claim 1, characterized in that: The flexible contacts of the flexible contact array (4) are distributed in a chessboard shape, the spacing between adjacent flexible contacts is the same, and through holes (11) are provided in the middle of four adjacent flexible contacts.

5. The flexible photoelectric multimodal sensing device according to claim 1, characterized in that: The number of the far-infrared photocell arrays (12) is two and they are arranged diagonally at two ends of the flexible photoelectric conversion layer (3); the near-infrared light source array (5) and the near-infrared photocell array (9) are evenly dispersed and arranged on the flexible photoelectric conversion layer (3).

6. The method for measuring the flexible photoelectric multimodal sensing device according to any one of claims 1 to 5, characterized in that: Precision grasping is accomplished based on a flexible proximity-temperature-tactile multimodal sensor, which automatically distinguishes the three tasks of proximity sensing, tactile normal force measurement, and sliding measurement through timing voltage information.

7. The method for measuring a flexible photoelectric multimodal sensing device according to claim 6, characterized in that: The flexible proximity-temperature-tactile multimodal sensor and its supporting IV amplifier circuit and embedded control system can be directly installed to perform on-site precise grasping control; if the control efficiency needs to be improved, the user needs to fix the flexible proximity-temperature-tactile multimodal sensor to the location where one or more quantities of proximity and tactile sensation need to be measured, connect the sensor-IV amplifier-embedded system to the computer, and use the trained network to predict the results. The network can obtain sensor state prediction and measurement value solution based on the time series changes of voltage.

8. The method for measuring a flexible photoelectric multimodal sensing device according to claim 6, characterized in that: When performing a proximity measurement task, the near-infrared light source array (5) emits light energy, which is diffusely reflected on the surface of the detected object. Part of the reflected light energy can enter the near-infrared photocell array (9) through the aperture (7), forming a photocurrent, which is converted into a voltage value by a 1V amplifier (17) and collected; As the object to be detected approaches, more reflected light energy can pass through the aperture (7) and enter the near-infrared photocell array (9), causing the photocurrent to increase. This change has the following corresponding relationship: Due to the presence of the bias light source (the light source emitted by the near-infrared light source array (5)), a maximum output value phenomenon will be generated at a distance of 5 mm from the target object (23), which becomes the proximity mark point of the sensor approaching the target object. This point does not change with the properties of the target object and can be used as the eccentricity elimination target point and the self-calibration starting point. After self-calibration, a more precise transfer function can be obtained. By analyzing the collected voltage data, the distance data of the detected object can be obtained to complete the proximity measurement. When performing a tactile measurement task, the biased light source generates an optical waveguide in the flexible photosensitive contact layer (1). After contact, leakage light is generated, causing the photocurrent to increase. However, the loading of the detected object can cause the flexible contact to sag downward, causing the focus of the flexible contact to move downward. The aperture hole (7) limits more light energy from entering, causing the photocurrent to decrease, resulting in a difference between the transfer function of the tactile part and the transfer function of the proximity sense. The mutation position of the transfer function is easy to detect and can be used as a contact mark. The photocurrent is converted into a voltage value by a 1V amplifier (17) and collected. This change has the following corresponding relationship: By analyzing the collected voltage data, pressure data can be obtained to complete tactile measurement; at critical contact, due to the different infrared reflectivity of different materials, the collected voltage values ​​are also different, and this feature can be used to distinguish the materials of the detected object.

9. The method for measuring a flexible photoelectric multimodal sensing device according to claim 6, characterized in that: When relative sliding occurs, the flexible contact will tilt and vibrate, the focus will also shift and vibrate, the aperture (7) will limit more light energy from entering, and the photocurrent will show an overall decrease accompanied by fluctuations. The computer can detect this feature and determine whether sliding occurs, thereby completing the sliding measurement.

10. The method for measuring a flexible photoelectric multimodal sensing device according to any one of claims 6 to 9, characterized in that: The working process of the manipulator connected to the flexible proximity-temperature-tactile multimodal sensor is decomposed into two processes: pre-grasping and grasping. The pre-grasping process eliminates eccentricity and tilt, while the grasping process realizes normal force monitoring and early warning and control of sliding; wherein the coarse measurement sensor unit (15) is responsible for guiding the target object (23) to the measurement range of the fine measurement sensor unit (14), and after detecting that its output presents a low-high-low feature, it issues an instruction to the manipulator tool end (20) to move forward to the manipulator control cabinet (21). At this time, the fine measurement sensor unit (14) starts to work. After detecting the low-high-low feature, the movement of the manipulator tool end (20) stops, and then the tilt elimination work starts. The binocular tilt method is adopted, and the manipulator tool end (20) moves forward and backward to calculate the voltage difference Diff between two points of the fine measurement sensor unit (14). When the Diff value is within the set threshold range, the tilt elimination is completed; then the eccentricity elimination work starts, and the manipulator tool end (20) first moves to the right until it reaches the proximity mark of the left unit in the precision sensor unit (14), then the robot arm tool end (20) moves to the left and starts pulse counting, and after reaching the proximity mark of the right unit in the precision sensor unit (14), the robot arm tool end (20) moves to the right, and the movement distance is determined by the pulse counting result; at this time, the pre-grasping work is completed, the host computer main control board (16) issues a clamping instruction to the robot (13), and stops moving after the precision sensor unit (14) detects the proximity mark, and completes the self-calibration data fitting; then the robot (13) continues to clamp, and the stepping speed decreases until the contact mark and the object no tilt mark are detected, the precision sensor unit (14) starts to monitor the normal force value until the normal force reaches the preset value, and controls the robot arm tool end (20) to move upward to complete the clamping; when the precision sensor unit (14) detects the spectrum signal caused by sliding, the robot (13) increases the clamping force until the preset value.

Images