Sensors and their manufacturing methods, sensing methods

Abstract

This invention relates to the field of robot perception technology, and provides a sensor and its manufacturing and perception methods. The sensor's perception method includes the following steps: S1: Pre-calibrating the parameter curve between the tentacle and the substrate to construct a calibration model; S2: A camera unit acquires images of the root of each tentacle on the substrate surface to form an image sequence; S3: A processing unit receives the image sequence and combines it with the calibration model and a built-in image processing algorithm to calculate and form multimodal perception information. This invention employs an optical imaging structure with the tentacle embedded in a silicone layer and a camera mounted on its back. It can calculate multimodal information such as three-dimensional contact force, fluid velocity and direction, and object surface texture / contour, achieving integrated detection of contact force perception and non-contact flow field perception, and has the advantages of being lightweight, compact, and highly robust.

Description

Technical Field

[0001] This invention relates to the field of robot perception technology, specifically to a sensor and its manufacturing and perception methods, and more particularly to a composite sensor integrating optical tactile sensing and bionic tentacles and its multimodal information perception method, applicable to scenarios such as robot navigation, fluid dynamics monitoring, fine manipulation and environmental interaction. Background Technology

[0002] In nature, the tentacles of many organisms (such as rodents, seals, and insects) are crucial multimodal sensory organs, enabling efficient navigation, fluid detection, and mechanical signal decoding in visually limited environments. The sensory mechanism of biological tentacles is highly inspiring: the tentacle shaft itself has no sensing function, but rather is a sophisticated force transmission structure; the actual sensing occurs in the hair follicles at the root of the tentacle, where dense nerve endings convert minute mechanical deformations into neural signals. This "end-sensing, shaft-transmission" mechanism, along with the behavior of animals actively exploring their environment by controlling their tentacles to perform high-frequency sweeping movements, provides an ideal paradigm for the design of biomimetic tactile sensors.

[0003] Inspired by this, researchers have developed biomimetic tentacle sensors based on various principles, aiming to extend robots' sensing capabilities in dark, confined, or dynamic fluid environments. Based on sensing principles, existing technologies mainly include sensors based on magnetoinduction, piezoresistive, piezoelectric, and capacitive principles. While these approaches have made progress in specific applications, they generally face a series of common challenges: Limited sensing dimensions: Most sensors struggle to decouple three-dimensional contact forces synchronously and with high precision, or are unable to effectively sense non-contact flow field information.

[0004] Arraying and integration difficulties: Due to limitations in sensing principles (such as electromagnetic interference and wiring complexity) or structural design, it is difficult to construct large-scale, high-density tentacles to achieve distributed and comprehensive sensing of spatial force fields / flow fields.

[0005] The system is bulky and lacks dynamic performance: some sensors have complex structures and large size, which limits their application in small or dexterous robots; at the same time, their signals are prone to saturation or distortion under large deformation or high-frequency excitation, and their dynamic range and reliability need to be improved.

[0006] In recent years, the technological approach of combining visual haptic sensing with biomimetic tentacles has shown significant potential. Visual haptic sensing captures high-resolution deformation of elastomer surfaces through cameras, offering advantages such as resistance to electromagnetic interference, decoupling of multidimensional information, and ease of implementation of flexible and thin structures, making it theoretically very suitable for constructing highly sensitive tentacle-based sensing units. However, existing visual haptic tentacles still have significant limitations: First, it is difficult to construct tentacle arrays with large coverage and high spatial resolution within limited physical space and camera field of view; second, their force sensing resolution (typically at the sub-millinewton level) still has room for improvement compared to some micro-nano sensing principles; and third, complex image processing algorithms may lead to insufficient real-time performance, making it difficult to meet the rapid response requirements in dynamic scenes.

[0007] Therefore, existing biomimetic tentacle sensors, whether based on traditional physical principles or emerging visual-tactile solutions, all have shortcomings in balancing high-resolution contact force sensing, large-scale flow field sensing, high-density array integration, and system lightweighting and real-time performance. This invention aims to solve these problems by providing a biomimetic tentacle-type visual-tactile sensor solution that achieves multimodal operation, high sensitivity, high spatial resolution, high real-time performance, and is easy to integrate. Summary of the Invention

[0008] In view of the deficiencies in the prior art, the purpose of this invention is to provide a sensor and its manufacturing method and sensing method.

[0009] A sensor according to the present invention includes: A sensing unit includes one or more tentacles, one end of which is a free end and the other end is a fixed end; The base, with the fixing end fixed on the front; A camera unit is disposed on the back side of the substrate; The processing unit is connected to the camera unit via signals. When an external force is applied to the free end, the image of the tendril root on the substrate surface acquired by the camera unit forms an image sequence, which is then processed by the processing unit in combination with the calibration model and the built-in image processing algorithm to form multimodal sensing information.

[0010] A sensing method for a sensor according to the present invention includes the following steps: S1: Pre-calibrate the parametric curves between the tentacles and the substrate to construct a calibration model; S2: The camera unit captures images of the root of each tentacle on the substrate surface to form an image sequence; S3: The processing unit receives the image sequence and combines it with the calibration model and the built-in image processing algorithm to form multimodal perception information.

[0011] Preferably, different calibration models are used to determine the magnitude and direction of the contact external force and to determine the magnitude and direction of the fluid velocity; Contact-type external force is a sudden change in displacement value. By taking the "displacement" value of the root of the tentacle at each moment, the magnitude of the force on the tentacle at that moment can be obtained. The force generated by the fluid causes the displacement at the root of the tendril to oscillate at a certain frequency, which is obtained by taking the average value of the oscillating displacement amplitude.

[0012] Preferably, the top surface of the substrate is uniformly coated with a reflective coating, which is either ink spraying or spray paint.

[0013] Preferably, the tendrils are cylindrical rods made of highly elastic high-carbon steel wire.

[0014] Preferably, an extended optical component is also provided, which is disposed between the substrate and the camera unit; The extended optical component includes a small parabolic mirror and a large parabolic mirror, the focal points of which coincide. The large parabolic mirror is aligned with the substrate, and the small parabolic mirror is aligned with the camera unit.

[0015] Preferably, it further includes at least one of the following structures; The housing, the base, the camera unit, and the processing unit are all disposed on the housing. Bottom cover; Acrylic material is disposed between the substrate and the housing; A light source is used to provide uniform illumination to the substrate.

[0016] Preferably, the substrate is a transparent material that is liquid before curing and elastic after curing: Silicone; Polyurethane elastomer (CPU); UV-cured acrylate elastomers; Hydrogel.

[0017] A method for manufacturing a sensor according to the present invention includes the following steps: M1: Prepare the base and fabricate the tentacle retaining plate and depth control plate; M2: Pass one end of the tentacle through the tentacle retaining plate and make its end abut against the depth control plate; M3: Secure the tentacles to the tentacles holding plate and remove the depth control plate; M4: Place the tentacle retaining plate on the surface of the uncured base material liquid, ensuring that the tentacle tip is inserted into the base to a predetermined depth; M5: After the liquid base material has completely solidified, remove the tentacle retaining plate to obtain a base with embedded tentacles.

[0018] Preferably, it further includes: M6: A reflective layer is made on the back of the substrate, a camera unit and a light source are installed, and the camera unit is connected to the processing unit.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention deeply mimics the natural sensing mechanism of biological tentacles. It employs an optical imaging structure with a biomimetic tentacle array embedded in a silicone layer and a camera mounted on the back. Through end-interaction, force transmission at the shaft, and base sensing of the tentacles, it deeply integrates the long-range spatial sensing capability of the biomimetic tentacle array with the high resolution and multi-parameter decoupling advantages of optical visual-tactile sensing. A single camera module captures micron-level displacement and substrate deformation images of the tentacle root within a transparent silicone substrate. Combined with a self-developed image processing algorithm and a pre-calibrated model, it can simultaneously and accurately calculate multimodal physical information such as three-dimensional contact force, contact torque, fluid velocity and direction, and object surface texture / contour. This achieves integrated detection of contact force sensing and non-contact flow field sensing. It is an integrated, lightweight, compact, and highly robust sensing solution that combines direct contact force detection with large-scale flow field sensing. It overcomes the core bottlenecks of traditional sensors, such as limited sensing range, difficulty in array integration, and single sensing dimension. It also solves the problems of small sensing radius and inability to effectively sense non-contact flow fields in traditional visual-tactile sensors.

[0020] 2. This invention embeds the tendrils directly into a soft substrate, naturally integrating them with the optical sensing system. It eliminates the need for complex wiring, resulting in a lightweight, thin, and compact structure that is easy to integrate into large-scale arrays.

[0021] 3. This invention solves the problem of the constraint between the camera's field of view (FOV) and the sensor's thickness in large-area tentacle array sensing by using a double parabolic mirror array composed of small and large parabolic mirrors, combined with a disk-shaped silicone layer and the camera's optical structure. Under the premise of limited sensor thickness, it can achieve full coverage of the field of view of a single camera for a large-scale tentacle array, and can achieve complete imaging of a large-area array of 55 tentacles, adapting to the large-scale flow field reconstruction needs of carriers such as aircraft and submarines.

[0022] 4. The overall optical sensing principle of this invention is not affected by external electromagnetic fields. The silicone cover and sealed structure can effectively isolate the disturbance of environmental airflow to high-sensitivity measurement. Through dynamic analysis, it can distinguish its own motion inertial signal from external stimulus signal, and has strong anti-interference ability.

[0023] 5. This invention utilizes a combination of highly elastic steel wire and a soft silicone substrate to achieve micro-Newton level force resolution and high sensitivity.

[0024] 6. This invention solves the problems of traditional bionic tentacle sensors being difficult to integrate in an array and having poor array sensing consistency by using a manufacturing structure that combines a depth control plate and a tentacle holding plate with a silicone layer to pre-embed and solidify the tentacle. It can accurately control the insertion depth and vertical posture of the tentacle, achieving a high-density, high-consistency tentacle array arrangement, and adapting to the distributed sensing needs of multiple scenarios such as robot navigation and flow field reconstruction.

[0025] 7. By adopting a pre-calibrated force-displacement and torque-displacement normalization model, combined with the structural design for detecting the displacement at the root of the tentacles, this invention solves the problems of low force measurement accuracy and complex data calculation under different external force application points of traditional tentacles sensors, and greatly improves sensing sensitivity and measurement accuracy. Attached Figure Description

[0026] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A schematic diagram of the basic sensor mechanism design; Figure 2 Exploded view of the basic sensor; Figure 3 Schematic diagram of a spherical tentacle array sensor mechanism; Figure 4 Exploded view of a spherical tentacle array sensor; Figure 5 Schematic diagram of a large planar tentacle array sensor mechanism; Figure 6 Exploded view of a large planar tentacle array sensor machine; Figure 7 Diagram of an airflow sensing experimental setup (wind tunnel); Figure 8 This is a schematic diagram of the optical principle of a double parabolic mirror. Figure 9 This is a schematic diagram illustrating the relationship between tentacle deformation and external force (used for mathematical modeling). Figure 10 For small tentacle retaining plates; Figure 11 For holding plates for large array of tentacles; Figure 12 Insert the tentacles into the depth control plate.

[0027] The diagram shows: Sensing Unit 1; Reflective coating 2; Base 3; Acrylic 4; Light source 5; Camera unit 6; Casing 7; Bottom cover 8; Small parabolic mirror 9; Large parabolic mirror 10; Tentacle 11; Plug 12; Sensor locator 13; Anemometer measuring head 14; PWM speed control fan 15; Small tentacle retaining plate 16; Support column 161; Positioning hole 162; Large array of tentacles holding plate 17; Depth control board 18; A protrusion of 181. Detailed Implementation

[0028] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0029] Example 1: To address the limitations of existing visual-tactile sensors in terms of sensing range and inability to effectively sense non-contact flow fields, as well as the problems of existing whisker sensors such as single sensing dimension, difficulty in array integration, bulky systems, and insufficient three-dimensional force decoupling capability, this invention provides a sensor that is lightweight, highly integrated, highly sensitive, and easily arrayed, capable of simultaneously achieving high-precision contact force sensing and wide-range fluid sensing. Its core idea is to mimic the sensing mechanism of biological whiskers, combining the "long-range sensing" capability of whisker arrays with the "high resolution and multi-parameter decoupling" advantages of optical visual-tactile sensors.

[0030] Specifically, the sensor includes a sensing unit 1, a substrate 3, a camera unit 6, a processing unit, and a housing 7. The sensing unit 1 includes one or more tentacles 11 arranged in an array, such as... Figure 1 , Figure 2 , Figure 5 , Figure 6 As shown, the tentacles 11 are made of highly elastic materials, such as high-carbon steel wire. One end of the tentacles 11 is a free end for interacting with the environment, and the other end of the tentacles 11 is a fixed end and is disposed on the front side of the substrate 3. The fixed ends of the multiple tentacles 11 are preferably vertically fixed inside the substrate 3 at a specific depth and spacing. The substrate 3 is preferably in the form of a thin sheet with a thickness ranging from 3 to 5 mm, preferably 3 mm.

[0031] The camera unit 6 is located on the back of the substrate 3 and is used to continuously capture images of the root area of ​​the tentacle 11 and the silicone surface. The substrate 3, the camera unit 6, and the processing unit are all located on the housing 7. The camera unit 6 can be a camera.

[0032] Furthermore, the processing unit is signal-connected to the camera unit 6 and can receive the image sequence fed back by the camera unit 6. The processing unit can analyze the pixel displacement changes of the root of the tentacle 11 on the silicone surface through computer vision algorithms, and then calculate multi-dimensional physical information.

[0033] The substrate 3 is preferably made of transparent silicone. The silicone used in the substrate 3 has a high light transmittance and high elasticity structure. PDMS silicone is preferred as the silicone layer. It can be processed into a planar sheet or a disc shape according to application requirements. It can serve as a fixing carrier for the tentacles 11, accurately converting the external force on the free end of the tentacles 11 into micro-displacement at the root. It can also serve as an optical propagation medium, providing an unobstructed imaging optical path for the camera unit 6. For example, in flow field reconstruction applications, the silicone layer can be processed into a 15cm diameter disc shape, with 55 tentacles 11 arranged along an Archimedean spiral to form a large-scale flow field sensing array. Besides PDMS silicone, the silicone layer can also be made of Ecoflex silicone, optical-grade flexible silicone, etc. This invention utilizes the planar substrate 3 and the rear camera layout to easily construct a large-area sensing "skin" covering the robot body, extending the "surface perception" of visual and tactile perception to the "spatial perception" (centimeter-level) of the tentacles' length, achieving effective detection of non-contact flow fields and mid-range obstacles, thus expanding the sensing range.

[0034] The top surface of the substrate 3 is uniformly coated with a reflective coating 2, which is preferably an ink spray coating. Specifically, the ink spray coating is a reflective layer formed by spraying a mixture of PANTONE Cool Gray 4C ink, curing agent, and diluent in a certain proportion. This can significantly improve light reflection efficiency and enhance the image contrast of the imaging unit 6. Moreover, because the coating thickness is uniform and there are no specular reflective spots, it can ensure that the micro-displacement of the root of the tentacle 11 and the deformation characteristics of the silicone layer are clearly and completely captured. For example, when working inside the robot cabin in low light, it can avoid the problems of image blurring and feature loss caused by uneven light transmission, ensuring the accuracy of subsequent image processing. The material of the reflective coating 2 can be light-colored ink, light-colored spray paint, etc. This invention utilizes the high spatial resolution of optical imaging to detect the micron-level displacement of the root of the tentacle 11, achieving micro-Newton-level force resolution, which is superior to many existing visual-tactile tentacle sensors and has the advantages of high sensitivity and resolution.

[0035] In practical applications, in addition to transparent silicone, substrate 3 can also be made of transparent materials such as polyurethane elastomer (CPU), UV-curable acrylate elastomer, and hydrogel, which are liquid before curing and elastic after curing.

[0036] Specifically, the sensor is also equipped with a light source 5, which is disposed on the housing 7 to provide uniform illumination for the substrate 3, facilitating image acquisition by the camera unit 6. The light source 5 can be an LED. When the LED light source 5 is lit, it can provide uniform, shadow-free illumination for the camera. When the camera is in working condition, it can continuously acquire image sequences of the surface of the substrate 3 and the root area of ​​the tentacle 11, and transmit them to the processing unit connected to the communication. When the free end of the tentacle 11 is deflected by external stimulation, the micro-displacement generated by the fixed end of the tentacle 11 within the substrate 3 can be completely captured by the camera.

[0037] Furthermore, the light source 5 is a surface-mount LED strip, evenly arranged around the perimeter of the camera. This provides shadowless, uniform, and stable surface illumination to the back of the silicone layer, ensuring that the camera can clearly capture the positional changes of the tentacle 11 roots under different working environments. Moreover, due to the low power consumption and small size of the LED light source 5, it can adapt to the design requirements of sensor miniaturization and lightweighting. For example, it can be compactly arranged without occupying extra space in the narrow head mounting space of a micro-mechanical mouse. The LED light source 5 can be a surface-mount white LED, or a red, green, and blue tri-color LED divided into three equal parts along the circumference.

[0038] Specifically, the tentacle 11 is a cylindrical rod made of high-elasticity high-carbon steel wire. The geometric parameters of a single tentacle 11 preferably conform to a length-to-diameter ratio of 400:3. For example, the tentacle (11) is 4cm long, 0.3mm in diameter, and has an insertion depth of 2mm in the substrate 3. It has high sensitivity and good wind speed-displacement quadratic fitting characteristics. It can generate a high-sensitivity response to micro-Newton level external forces and low-velocity flow fields while ensuring the strength of the mechanical structure. Moreover, since multiple tentacles 11 can be arranged into an array according to preset rules, it can realize spatially distributed multi-dimensional environmental perception. The arrangement of the tentacles 11 can be a uniform dot matrix, an Archimedean spiral, etc. This invention extends the sensing range from the silicone layer The surface extends to the space of the tentacle length 11. By using a camera to capture the micro-displacement of the tentacle root, the highly elastic bionic tentacle array is combined with visual-tactile sensing technology to realize the mechanism of "end contact, base sensing". Through a single visual channel, it can simultaneously realize the decoupled perception of three-dimensional contact force (normal / tangential), contact torque, fluid velocity (airflow / waterflow), flow field direction and surface texture. Combined with the sensing structure that calculates multi-dimensional information by the processing unit, it can simultaneously calculate multi-modal physical information such as three-dimensional contact force, flow field velocity and direction, and object surface texture. It solves the problem that traditional bionic tentacle sensors such as magnetic induction, piezoresistive and capacitive sensors have a single sensing dimension and are difficult to simultaneously decouple three-dimensional contact force.

[0039] The sensor is also equipped with acrylic 4, which is a highly flat transparent acrylic sheet that is bonded and fixed between the substrate 3 and the housing 7. The housing 7 is preferably made by 3D printing, which can provide stable rigid support for the substrate 3, the light source 5, and the camera, ensuring that there is no relative displacement between the optical components. Moreover, due to the high light transmittance of acrylic 4, it will not obstruct or interfere with the imaging light path of the camera, thus avoiding imaging deviation caused by structural shaking and ensuring the stability and consistency of the images acquired by the camera.

[0040] The camera is a high-frame-rate miniature industrial camera with its lens facing the silicone layer and reflective coating 2. It can continuously and at high resolution capture image sequences including the root region of the fixed end of the tentacle 11 and the surface deformation of the silicone layer. Furthermore, due to the camera's high frame-rate acquisition characteristics, it can capture the high-frequency oscillation signals of the tentacle 11 in the flow field, providing complete data for dynamic characteristic analysis. For example, when acquiring images at a 30Hz frame rate, it can completely record the oscillation timing signals of the tentacle 11 under different wind speeds, and analyze the velocity and characteristics of the flow field through Fast Fourier Transform. The camera can be a miniature CMOS camera, etc.

[0041] The bottom of the outer shell 7 is also equipped with a bottom cover 8, which is a 3D-printed encapsulation structure made of PLA material. When the outer shell 7 and the bottom cover 8 are fastened together, they form a sealed internal cavity that can accommodate and fix all core components, isolating them from external dust and ambient airflow interference with high-sensitivity measurements. Furthermore, the shape of the outer shell can be designed according to the structure of the mounting carrier, thus adapting to the installation requirements of different carriers such as robots and aircraft. For example, when designed as a small cylindrical shell, it can be installed on the head as a bionic nose tip for a mechanical mouse, achieving compact integration. The materials of the outer shell 7 and the bottom cover 8 can be PLA, as well as resin, nylon, ABS engineering plastics, etc. This invention deeply imitates biological tentacle systems from the sensing mechanism of the base sensor and structural array to navigation and flow field sensing functions, providing robots with biological-like sensing capabilities and exhibiting excellent biomimetic characteristics.

[0042] The sensor is also equipped with extended optical components, which are disposed between the substrate 3 and the camera unit 6. These extended optical components include a small parabolic mirror 9 and a large parabolic mirror 10, such as... Figure 6 , Figure 8 As shown, both the small parabolic mirror 9 and the large parabolic mirror 10 are high-precision optical mirrors with their focal points overlapping. The large parabolic mirror 10 is aligned with the silicone layer, and the small parabolic mirror 9 is aligned with the camera. In this way, the large parabolic mirror 10 can converge the image reflected from the roots of all the tentacles 11 to the small parabolic mirror 9 at its focal point, and then reflect it to the camera. Moreover, without increasing the overall thickness of the sensor, the effective imaging field of view of the camera is significantly expanded. Therefore, it can solve the imaging coverage problem of large-scale tentacles arrays under small thickness constraints. For example, 55 tentacles 11 arranged on a 15cm diameter disc-shaped silicone layer can achieve full coverage imaging by a single camera through this dual parabolic mirror array. The small parabolic mirror 9 and the large parabolic mirror 10 can be optical glass aluminized mirrors, polymer coated mirrors, highly polished aluminum alloy machined parts, etc. By employing a double parabolic mirror array consisting of a small parabolic mirror 9 and a large parabolic mirror 10, combined with a disk-shaped silicone layer and the optical structure of the camera, the problem that a single camera cannot fully cover the field of view of a large-scale tentacle 11 array under the premise of limited sensor thickness is solved. It can achieve complete imaging of a large-area array of 55 tentacles 11, and is suitable for the large-scale flow field reconstruction needs of carriers such as aircraft and submarines.

[0043] The mounting and fixing of the fixed end of the tentacle 11 is carried out using special auxiliary tools. These tools, used in the sensor manufacturing stage, include a small tentacle holding plate 16, a large array tentacle holding plate 17, and a depth control plate 18. The depth control plate 18 has four protrusions 181 with the same insertion depth of the fixed end of the tentacle 11 into the silicone. Both the small tentacle holding plate 16 and the large array tentacle holding plate 17 have upper and lower array-type positioning holes 162. First, the four support pillars 161 of the tentacle holding plate are fixed to the four protrusions 181 of the depth control plate 18 with hot melt adhesive. The tentacle 11 is then inserted into the positioning holes 162, and the upper end of the tentacle 11 is fixed to the upper layer of the tentacle holding plate with hot melt adhesive. Next, the depth control plate 18 is removed, and the retaining plate holding the tentacles 11 is placed on the acrylic plate used for casting silicone. The four support pillars 161 are fixed with hot melt adhesive. This allows for precise control of the insertion depth and vertical orientation of the tentacles 11 before the silicone layer cures, and also ensures that the insertion depth and vertical orientation of all tentacles 11 in the array are completely consistent, ensuring the sensing consistency and accuracy of the sensor array. For example, when mass-producing sensors, this auxiliary tool can ensure that the tentacle array parameters of each batch are completely uniform. The materials of the depth control plate 18 and the tentacle retaining plate can be PLA, etc.

[0044] The present invention also provides a method for manufacturing a sensor, comprising the following steps: M1: Prepare base 3 and make tentacle retaining plate and depth control plate 18; M2: Pass one end of the tentacle 11 through the tentacle retaining plate and make its end abut against the depth control plate 18; M3: Temporarily fix the tentacle 11 to the tentacle retaining plate and remove the depth control plate 18; M4: Place the tentacle retaining plate on the surface of the uncured substrate 3 material liquid, ensuring that the ends of the tentacle 11 are inserted into the substrate 3 to the predetermined depth; M5: After the liquid substrate 3 material has completely solidified, remove the tendril retaining plate to obtain the substrate 3 with the embedded vertical tendril array; M6: A reflective layer is made on the back of the substrate 3, and a camera unit 6 and a light source 5 are installed. The camera unit 6 is then connected to the processing unit.

[0045] In this invention, the fixed end of the tentacle 11 is pre-embedded into the substrate 3 at a set size before the substrate 3 is cured, and then the substrate 3 is cured. The pre-embedding and curing process is achieved by a specially designed depth control plate 18 and tentacle retaining plate to ensure that the tentacle 11 remains vertical and is precisely inserted to a predetermined depth before the silicone is cured, and a firm connection is formed after curing.

[0046] The processing unit communicates with the camera and has built-in image processing algorithms and a pre-calibrated calibration model library. It can receive image sequences captured by the camera, extract the pixel displacement values ​​of each of the 11 roots of each tentacle and the texture deformation features of the silicone layer surface. Moreover, it can quickly solve the corresponding multimodal physical information through the pre-calibrated calibration model. Therefore, it can realize the synchronous perception of information such as three-dimensional contact force, flow field velocity and direction, and object surface texture. For example, when the robot comes into contact with an object, the image processing unit can calculate the magnitude, direction and point of application of the contact force in real time, providing feedback for fine operation.

[0047] Furthermore, the present invention also provides a sensing method for a sensor, comprising the following steps: S1: Pre-calibrate the parameter curves between the tentacles 11 and the substrate 3 to construct the calibration model; the parameter curves include force-displacement curves, torque-displacement curves, flow velocity-displacement curves, etc., where the flow velocity can be gas flow velocity or liquid flow velocity; S2: Camera unit 6 captures images of the root of each tentacle 11 on the surface of substrate 3 to form an image sequence; S3: The processing unit receives the image sequence and combines it with the calibration model and built-in image processing algorithms to calculate multimodal sensing information. Specifically, the algorithm extracts the contours to obtain the distance and direction of the line connecting the two farthest points at the base of each tentacle 11, thus obtaining pixel displacement. When the tentacle 11 is in a vertical position without external force, the image captured by the camera at the base of the tentacle 11 is a black circle, such as... Figure 9 As shown, when an external force is applied to the tentacle 11, the tentacle 11 tilts, and the image captured by the camera changes from a black circle to a thick black line segment. The greater the tilt, the longer this thick line segment becomes. We use the Canny edge detection algorithm to extract the contour of the root image of the tentacle 11 and calculate the distance between the two farthest pixels in the contour. Theoretically, this distance is proportional to the external force applied to the tentacle 11. The "distance between the two farthest pixels in the contour and the direction of the connection" is simply referred to as "displacement". This invention uses the pixel distance between the two farthest points in the root image of the tentacle 11 combined with a calibration model to determine the flow velocity magnitude, and the pixel connection between the two farthest points to determine the flow velocity direction. The determination of the flow velocity magnitude and direction falls under the tangential category, and the measurement of the tangential value does not require the use of a neural network.

[0048] It should be noted that different calibration models are used to determine the magnitude and direction of contact forces and fluid velocity. Contact forces are concentrated point forces, while fluid forces are distributed forces. Regarding how to distinguish whether an external stimulus is a specific contact force or a fluid-generated force, if it is a fluid, the magnitude of the "displacement" at the root of tentacle 11 will oscillate at a certain frequency, which can be obtained by averaging the amplitude of the oscillations. If the external stimulus is a contact force, it is generally not an oscillation at a certain frequency, but rather a sudden change in displacement value; therefore, simply taking the "displacement" value at the root of tentacle 11 at each moment will yield the magnitude of the force acting on tentacle 11 at that moment. This invention can simultaneously acquire high-resolution three-dimensional contact force (micro-Newton level), flow field velocity and direction, surface texture, and other multimodal information through a single sensor platform, greatly expanding the robot's environmental perception capabilities.

[0049] The sensing method in this invention is theoretically based on analyzing the static and dynamic equilibrium of the tentacle-base system under external loads (force F or fluid drag force). The external load generates an upper torque on the tentacle 11, which is then used... This indicates that the constraint of base 3 on the root of the tentacle generates a root restoring torque, which is expressed as... This indicates that the two are equal when in equilibrium. The contribution from both shear and normal deformation of base 3 is expressed as follows: Where π is the mathematical constant Pi, and E is the Young's modulus of the base material 3. θ is Poisson's ratio, θ is the tilt angle of tentacle 11, d is the insertion depth, and r is the radius of tentacle 11.

[0050] External loads generate an upper torque on the tentacle 11. For fluid sensing, we have: ∝ in, Let be the density of the fluid, V be the fluid velocity, r be the radius of tentacle 11, and θ be the tilt angle of tentacle 11. Let θ be the length of the tentacle. At equilibrium, the displacement of the root of tentacle 11 due to the external load is Δ. When θ is a small angle, it is approximately Δ∝θ. At equilibrium, we can see that the resulting Δ due to θ is proportional to the square of V, i.e., Δ ∝ It is consistent with the quadratic curve calibrated in the experiment.

[0051] For the perception of contact force, we have: Where F is the force exerted on the tentacle 11 by the external load, and s is the action distance generated on the tentacle 11 due to the force F. At equilibrium, θ and Δ are proportional to the product of F and the action distance s, i.e., Δ ∝ Fs.

[0052] Specific sensing modes include contact force sensing, flow field sensing, and other sensing methods, specifically: Contact force sensing: When the tentacle 11 comes into contact with an object, the force-displacement curve is calculated based on a pre-calibrated curve. Within the linear region (e.g., external force 0-1350μN), the pixel displacement and external force are linearly related, with a sensitivity of up to 0.056 pixels / μN and a force resolution of approximately 17.85μN. When the distance *s* from the point of application of the external force to the root varies, it can be normalized using the torque M = Fs. Experiments show that within the range of *s* = 1.2-3.0 cm, the torque-displacement curves almost coincide, and the displacement and torque correspond approximately one-to-one, simplifying the force calculation. The force-displacement curve or torque-displacement curve mentioned above is the calibration model for contact force.

[0053] Flow field sensing includes steady-state velocity estimation, flow direction estimation, and dynamic analysis. Velocity (range 0-7.5 m / s) is estimated by combining steady-state pixel displacement with a calibrated quadratic function-type "wind speed-displacement" curve (e.g., y=0.65x²+0.35x+0.12). The wind speed-displacement curve serves as the calibration model for the flow field force. Flow direction is calculated by analyzing the response differences of the tentacles 11 at different positions in the array. It is important to note that different types of fluids (such as airflow and water flow) have their own specific pre-calibration curves. By performing a Fast Fourier Transform (FFT) analysis on the time-series displacement signal, the dynamic characteristics were found to be stable at approximately 5Hz (determined by the natural frequency of the tentacles), while the oscillation amplitude and average displacement increase with increasing wind speed, which can be used to comprehensively judge the flow field intensity.

[0054] Other sensing capabilities: By analyzing the contact patterns of the multiple tentacles 11, timing signals, and surface deformation images of the substrate 3 itself, it is also possible to perceive the surface texture and shape of objects, as well as the relative distance between the sensor and obstacles.

[0055] Example 2: This embodiment is a preferred example of Embodiment 1. This embodiment provides a method for manufacturing a sensor, including the following steps: M1: Prepare a transparent, highly elastic silicone as substrate 3, for example, substrate 3 is made of PDMS; process a depth control plate 18 and a tentacle holding plate, wherein the depth control plate 18 has through holes for controlling the insertion depth of the tentacle 11, the insertion depth is preferably 2mm, and the tentacle holding plate has array holes for fixing the tentacle 11 to keep it vertical. M2: Multiple carbon spring steel wires with a diameter of 0.3mm and a length of 4cm are used as tendons 11. One end is blackened with quick-drying ink, and then they are passed through the tendon retaining plate and the end of the tendon is pressed against the depth control plate 18. M3: Use hot melt glue to temporarily fix the tentacle 11 to the tentacle retaining plate, and then remove the depth control plate 18; M4: Place the entire tentacle retainer plate on the surface of uncured silicone liquid, ensuring that the end of tentacle 11 is inserted into the silicone to a predetermined depth, preferably 2 mm; M5: After the silicone has fully cured, remove the tentacle retainer plate to obtain the substrate 3 with an embedded vertical tentacle array; M6: Gray ink is sprayed onto the back of substrate 3 as a reflective layer, and then a miniature camera and LED light strip are installed as the light source 5 to form a complete sensor. The camera is connected to a host computer via a USB interface, and the host computer is the processing unit.

[0056] In this embodiment, the force sensing calibration and implementation are specifically carried out as follows: The single tentacle 11 was calibrated using a gravity compensation method. Tentacle 11 is 4 cm long, 0.3 mm in diameter, and inserted 2 mm into the substrate 3. The sensor was horizontally fixed on an analytical balance (Mettler-Toledo MA204), and the free end of tentacle 11 was connected to the slider groove of a high-precision Z-axis displacement stage (LWZ25-L100). The balance reading when tentacle 11 was horizontal was recorded as a reference. The stage was controlled to tilt tentacle 11, and the change in the balance reading was the external force (F) acting on tentacle 11. Camera images were acquired simultaneously, and the pixel displacement Δ of the root of tentacle 11 relative to the reference image was calculated using an image processing algorithm. By changing the magnitude of the external force and the distance s of the point of application, a series of force-displacement curves were obtained.

[0057] Experiments revealed that when s = 2.7 cm, Δ and F exhibit a linear relationship (linear region) within the range of 0-1350 μN, with a sensitivity of 0.056 pixels / μN and a force resolution of approximately 17.85 μN; above 1350 μN, the force enters the saturation region. After normalizing the curves under different s values ​​using the torque M = Fs, the curves essentially overlapped, verifying the one-to-one correspondence between displacement and torque.

[0058] In practical applications, when the tentacle 11 contacts an object, the system calculates Δ in real time. By querying the calibration model, the magnitude of the contact force and the torque can be quickly calculated.

[0059] In this embodiment, the flow field sensing is implemented in the following specific way: The sensor was placed in a 3D-printed wind tunnel, which used a PWM speed-controlled fan 15 (Delta 8025). A thermal anemometer (Xinster HT9829) was used to calibrate the relationship between the PWM value and the wind speed (v). The wind speed adjustment range was 0-7.5 m / s. The wind tunnel was equipped with a fixture 141, a plug 12, and a sensor locator 13. The anemometer measuring head 14 was placed in the plug 12 and fixed by the fixture 141. The sensor locator 13 was used to fix the sensor, such as... Figure 7 As shown.

[0060] Steady-state wind speed measurement: Under different wind speeds, record the average pixel displacement (Δavg) at the base of tentacle 11. Through data fitting, establish a quadratic model of Δavg = k1 * v² + k2 * v + k3 (for example, when the tentacle diameter is 0.3mm, the length is 4cm, and the insertion depth is 2mm, y = 0.65x² + 0.35x + 0.12). During sensing, the wind speed can be inferred by measuring Δavg and substituting it into this model.

[0061] Dynamic Characteristic Analysis: High-speed acquisition of the oscillation timing signal of the antenna 11 under different wind speeds (e.g., 100Hz). FFT analysis of the signal reveals that the frequency corresponding to the peak value of the spectrum is the main oscillation frequency (approximately 5Hz). Analysis shows that the main oscillation frequency is independent of wind speed and remains essentially constant. Higher wind speeds result in increased time-domain amplitude and Δavg of the oscillation signal; therefore, the wind speed can be comprehensively judged by combining amplitude and average displacement.

[0062] Flow direction perception: When using an array composed of multiple tentacles (11), the steady-state response (Δavg) of tentacles 11 at different positions in the array to the same flow field is compared. The direction of the tentacles 11 with the strongest response indicates the main flow direction of the flow field. The two-dimensional flow direction can be calculated by interpolation or synthesis of multi-tentacle response vectors.

[0063] Water flow sensing: Similarly, sensors can be placed in a water tank to calibrate and sense the speed and direction of water flow, based on the same principle.

[0064] Example 3: This embodiment is another preferred example of Embodiment 1, and it is a design for a large planar tentacled array sensor for flow field reconstruction. To achieve the perception and reconstruction of large-scale flow fields (such as airflow around an aircraft or water flow around a submarine), this embodiment provides a large planar tentacled array sensor.

[0065] Specifically, a disc-shaped planar base 3 is used, with a diameter of, for example, 15 cm. On this base, such as Figure 5 , Figure 6As shown, 55 tentacles 11 are arranged in an Archimedean spiral pattern, forming a large-scale array. However, due to limitations in sensor thickness, a single camera's field of view may not be able to cover the roots of all tentacles 11. To address this issue, a dual parabolic mirror array optical system is added in front of the camera unit 6. This system consists of two parabolic mirrors, one large and one small, with their focal points overlapping. The smaller parabolic mirror is placed at the focal point of the larger parabolic mirror. The large parabolic mirror converges the images reflected from the roots of all tentacles 11 to its focal point (i.e., the position of the smaller parabolic mirror), which then reflects this image back to the camera. In this way, the camera's field of view only needs to cover the smaller parabolic mirror to indirectly obtain an image of the entire large tentacle array, thus achieving effective imaging of a large-scale array while maintaining a thin and lightweight structure. This sensor can be installed on the surface of aircraft or submarines for measuring fluid velocity and direction and reconstructing flow fields.

[0066] The large planar tentacled array sensor in this embodiment forms a sensing "skin" covering the surface of underwater vehicles, ships, or aircraft, enabling real-time, large-area sensing of the speed, direction, and turbulence of surrounding water or air currents for attitude control and energy-efficient navigation. Simultaneously, it enables contactless communication by using coded airflow sequences (such as airflows of varying durations and intensities) as a carrier. This visual-tactile sensor can decode airflow signals, achieving a novel contactless information transmission method.

[0067] Example 4: The difference between this embodiment and Embodiment 1 is that the substrate 3 is hemispherical, serving as a fixed carrier for the tentacles 11. To enable navigation and obstacle avoidance for robots (such as mechanical mice) in visually limited environments (such as darkness or narrow pipes), a spherical tentacles array sensor can be designed. This can be applied to mechanical mice navigating in dark, narrow pipe environments. Multiple tentacles 11 arranged on the hemispherical silicone layer 3 detect the location, distance, and outline of surrounding obstacles, thereby triggering turning or obstacle avoidance actions. This design represents a spherical tentacles array sensor for navigation applications.

[0068] Specifically, the base 3 is made into a hemispherical shape, such as... Figure 3 , Figure 4 As shown, five tentacles 11 are vertically inserted into each of the two sides of the hemispherical base 3, and the orientation of each tentacle 11 is distributed along the normal of the hemisphere and is different from that of the others. Compared with a planar array of tentacles 11 with the same orientation, this spherical design allows the free ends of the tentacles 11 to point in different directions in space, thereby greatly expanding the spatial range in which the sensor can perceive obstacles at a single point. This sensor can act as the "nose" of a robot, sensing the orientation, distance, and outline of obstacles through contact between the tentacles 11 and the obstacles, thus achieving biomimetic tentacle navigation.

[0069] The spherical tentacle array sensor in this embodiment is installed on the head of a mobile robot or drone (such as a mechanical mouse). In dark, smoke, or narrow environments, the multi-directional tentacles 11 detect the distance and outline of surrounding obstacles, enabling efficient navigation.

[0070] The working principle of this invention is as follows: When external contact force, airflow, or water flow acts on the free end of the tentacle 11, the tentacle 11 will deflect, and its fixed end will produce a small displacement within the silicone layer, while simultaneously causing elastic deformation of the surrounding silicone layer. Under uniform illumination from the LED light source 5, the camera continuously captures image sequences of these small displacements and deformations and transmits them to the processing unit. The processing unit analyzes the pixel displacement at the root of the tentacle 11 and the deformation characteristics of the silicone layer in the image, and combines pre-calibrated calibration models such as force-displacement curves, torque-displacement curves, and flow velocity-displacement quadratic curves to calculate specific information such as the magnitude, direction, and point of application of the external stimulus, ultimately achieving synchronous perception of multimodal information such as three-dimensional contact force, flow field velocity and direction, and object surface texture.

[0071] In the description of this application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0072] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A sensor, characterized in that, include: The sensing unit (1) includes one or more tendons (11), one end of which is a free end and the other end is a fixed end; The base (3) is fixed to the fixed end on the front side; A camera unit (6) is disposed on the back side of the substrate (3); The processing unit is connected to the camera unit (6) via a signal. When an external force is applied to the free end, the image of the root of the tendril (11) on the surface of the substrate (3) acquired by the camera unit (6) forms an image sequence and is solved by the processing unit in combination with the calibration model and the built-in image processing algorithm to form multimodal perception information.

2. A sensing method for a sensor, characterized in that, Includes the following steps: S1: Construct a calibration model by pre-calibrating the parametric curves between the tentacles (11) and the substrate (3); S2: The camera unit (6) acquires images of the root of each tentacle (11) on the surface of the base (3) to form an image sequence; S3: The processing unit receives the image sequence and combines it with the calibration model and the built-in image processing algorithm to form multimodal perception information.

3. The sensing method of the sensor according to claim 1 or the sensor according to claim 2, characterized in that, The magnitude of the force is determined by the pixel distance between the two farthest points in the root image of the tentacle (11) combined with the calibration model, and the direction of the force is determined by the pixel connection between the two farthest points, thus forming multimodal perception information.

4. The sensing method of the sensor according to claim 1 or the sensor according to claim 2, characterized in that, It is also equipped with an extended optical component, which is disposed between the substrate (3) and the camera unit (6); The extended optical component includes a small parabolic mirror (9) and a large parabolic mirror (10), the focal points of which coincide with each other, the large parabolic mirror (10) is aligned with the substrate (3), and the small parabolic mirror (9) is aligned with the camera unit (6).

5. The sensing method of the sensor according to claim 1 or the sensor according to claim 2, characterized in that, The top surface of the substrate (3) is uniformly coated with a reflective coating (2), which is either ink spraying or spray paint. The tendrils (11) are cylindrical rods made of high-elasticity high-carbon steel wire.

6. The sensing method of the sensor according to claim 1 or the sensor according to claim 2, characterized in that, Different calibration models are used to determine the magnitude and direction of contact external forces and to determine the magnitude and direction of fluid flow velocity; Contact-type external force is a sudden change in displacement value. By taking the "displacement" value at the root of the tentacle (11) at each moment, the magnitude of the force on the tentacle (11) at that moment can be obtained. The force generated by the fluid causes the displacement of the root of the tendril (11) to oscillate at a certain frequency, which is obtained by taking the average value of the oscillation displacement amplitude.

7. The sensing method of the sensor according to claim 1 or the sensor according to claim 2, characterized in that, It also includes at least one of the following structures; Shell (7); Bottom cover (8); Acrylic (4) is disposed between the base (3) and the shell (7); A light source (5) is used to provide uniform illumination to the substrate (3).

8. The sensing method of the sensor according to claim 1 or the sensor according to claim 2, characterized in that, The substrate (3) is made of any of the following transparent materials that are liquid before curing and elastic after curing: silicone; Polyurethane elastomer (CPU); UV-cured acrylate elastomers; Hydrogel.

9. A method for manufacturing a sensor, characterized in that, Includes the following steps: M1: Prepare the base (3) and make the tentacle retaining plate and the depth control plate (18). M2: Pass one end of the tentacle (11) through the tentacle retaining plate and make its end abut against the depth control plate (18). M3: Fix the tentacle (11) to the tentacle holding plate and remove the depth control plate (18). M4: Place the tentacle retaining plate on the surface of the uncured substrate (3) material liquid, ensuring that the ends of the tentacles (11) are inserted into the substrate (3) to a predetermined depth; M5: After the liquid substrate (3) material has completely solidified, remove the tendril retaining plate to obtain the substrate (3) with the tendrils (11) embedded in it.

10. The method for manufacturing a sensor according to claim 9, characterized in that, Also includes: M6: A reflective layer is made on the back of the substrate (3), a camera unit (6) and a light source (5) are installed, and the camera unit (6) is connected to the processing unit.

Images