A reflective tactile sensor and a force field reconstruction method thereof

Abstract

The application discloses a reflective tactile sensor and a force field reconstruction method thereof, the sensor comprising a substrate layer, a transparent elastic layer and a continuous optical modulation layer; the substrate layer is provided with an array of light emitting units and an array of light receiving units; the transparent elastic layer is arranged on the substrate layer and is used for transmitting optical signals and generating elastic deformation when subjected to force; the continuous optical modulation layer is arranged on or in the transparent elastic layer and is a continuous layer covering the entire sensing area and having a preset fine texture or pattern, and is used for modulating incident light into a light field having spatial coding characteristics, and does not have a physical alignment relationship with the array of light emitting units and the array of light receiving units. The application eliminates the problems of complex process and low yield caused by one-by-one alignment in the related art, reduces manufacturing cost, and is sensitive to small tangential deformation, thereby providing a physical basis for super-resolution force field reconstruction.

Description

Technical Field

[0001] This invention relates to the field of sensor technology, and in particular to a reflective tactile sensor and its force field reconstruction method. Background Technology

[0002] Tactile sensors play a crucial role in fields such as dexterous robot manipulation, intelligent prosthetics, human-computer interaction, and wearable health monitoring. Among various sensing principles, optical tactile sensors have attracted widespread attention due to their strong resistance to electromagnetic interference, fast response speed, and ease of arraying. Among these, reflective architectures have become one of the important technological paths in the field of flexible electronic skin due to their simple structure, low power consumption, and ease of integration.

[0003] In existing implementation schemes, a typical architecture involves arranging a sensing element array consisting of light-emitting units and multiple light-receiving units on a substrate layer. A transparent elastic layer is placed above the substrate layer, and discrete microstructures, such as dome-shaped, frustum-shaped, or columnar structures, are arranged above or inside the elastic layer and aligned one-to-one with the sensing elements below. Under external force, the microstructures undergo compression, tilting, or shear deformation, causing changes in the direction and intensity of the reflected light. Light receivers at different positions receive differentiated light intensity signals accordingly. By calculating the differences in signals from each channel, three-dimensional force decoupling of normal and tangential forces is achieved.

[0004] However, in the above scheme, the microstructure needs to be precisely aligned with the center of each sensing unit below. This alignment step places extremely high precision requirements on the processing technology and stack-up assembly. Especially when manufacturing on large-area or flexible substrates, alignment deviations will significantly affect device consistency and yield, leading to a substantial increase in manufacturing costs and becoming a key bottleneck restricting the large-scale application of this type of sensor. Summary of the Invention

[0005] This invention provides a reflective tactile sensor and its force field reconstruction method, which solves the technical problems in related technologies, such as high manufacturing precision requirements, complex processes, low yield, and limited sensing resolution due to the need for physical alignment between elastic microstructures and photoelectric sensing units, as well as insensitivity to minute tangential deformations. The invention enables the encoding of continuous surface deformation fields into differentiated light intensity signals for each light receiving unit without the need for physical alignment between the modulation layer and the photoelectric unit. This reduces manufacturing difficulty and cost, improves the detection sensitivity to tangential forces and minute slippages, and provides a physical basis for super-resolution force field reconstruction.

[0006] This invention provides a reflective tactile sensor and a force field reconstruction method thereof, wherein the reflective tactile sensor includes: A substrate layer, on which an array of light emitting units and an array of light receiving units are disposed; A transparent elastic layer is disposed on the substrate layer to transmit optical signals and generate elastic deformation when subjected to force; An optical modulation layer is disposed on or inside the transparent elastic layer. The optical modulation layer is a continuous layer with a preset fine texture or pattern that covers the entire sensing area. It is used to modulate the incident light into a light field with spatial coding characteristics, and the optical modulation layer covers the entire sensing area of ​​the sensor. There is no physical alignment between the optical modulation layer and the optical emitting unit array and the optical receiving unit array.

[0007] Optionally, the optical modulation layer is any one of the following: A patterned reflective layer comprising a high-reflectivity region and a low-reflectivity / light-absorbing region, wherein the high-reflectivity region and the low-reflectivity / light-absorbing region are alternately arranged along at least one direction; Pseudo-random speckle pattern layer; A textured layer with a three-dimensional microstructure, wherein the three-dimensional microstructure is arranged in an array on the surface of the optical modulation layer; A multi-scale superimposed pattern containing both coarse and fine periodic components.

[0008] Optionally, a light-shielding protective layer is also included, disposed above the optical modulation layer.

[0009] Optionally, the optical emitting unit array and the optical receiving unit array are arranged in an alternating pattern on the substrate layer.

[0010] Optionally, the spacing between adjacent feature units of the texture or pattern of the optical modulation layer is smaller than the unit spacing of the light emitting unit array or the light receiving unit array.

[0011] Furthermore, to achieve the above objectives, the present invention also provides a force field reconstruction method for a reflective tactile sensor, the method comprising: Acquire the baseline light intensity signal of each optical receiving unit under no-force conditions; Acquire the current light intensity signal of each optical receiving unit under stress, and calculate the differential light intensity signal of each optical receiving unit based on the baseline light intensity signal and the current light intensity signal; The differential light intensity signals of each optical receiving unit are arranged into a two-dimensional differential light intensity image according to their spatial positions on the substrate layer; The two-dimensional differential light intensity image is input into a trained convolutional neural network, which outputs the three-dimensional deformation field of the optical modulation layer surface. Based on the physical relationship between surface force and surface deformation in elasticity, the three-dimensional force field distribution on the sensor surface is obtained by inversion from the three-dimensional deformation field.

[0012] Optionally, the step of obtaining the three-dimensional force field distribution of the sensor surface from the three-dimensional deformation field based on the physical relationship between surface force and surface deformation in elasticity includes: A linear relationship matrix is ​​constructed between the three-dimensional deformation field and the three-dimensional force field. The linear relationship matrix characterizes the deformation response generated at each position on the sensor surface under the action of a unit force. Based on the linear relationship matrix, a smoothness constraint is introduced to minimize the deviation between the reconstructed deformation field and the measured deformation field, while suppressing the solution instability caused by measurement noise, thus obtaining a stable three-dimensional force field distribution.

[0013] Optionally, the training data for the convolutional neural network is obtained in the following way: A mechanical model of the sensor structure is established to simulate the three-dimensional deformation field of the transparent elastic layer and the optical modulation layer under the action of external forces of different positions, directions and magnitudes; A light field propagation model is established to simulate the light intensity distribution received by each light receiving unit corresponding to the three-dimensional deformation field. The simulated light intensity distribution is processed into a differential light intensity image, which is then paired with the corresponding three-dimensional deformation field ground value to form a training dataset.

[0014] Optionally, after the step of arranging the differential light intensity signals of each optical receiving unit according to their spatial positions on the substrate to form a two-dimensional differential light intensity image, the method further includes: The two-dimensional differential light intensity image is directly input into a trained physical information neural network, and a one-step mapping is directly performed through the physical information neural network to output the three-dimensional force field distribution on the sensor surface. The physical information neural network is pre-trained using multiple sets of training data, each set of training data containing the differential light intensity image and the corresponding three-dimensional force field true value.

[0015] Optionally, the training steps of the physical information neural network include: Multiple sets of training data are acquired, each set of training data including the differential light intensity image and the corresponding three-dimensional force field true value; During training, a loss function containing data fitting terms and residual terms of the elasticity control equations is used for iterative optimization; wherein, the data fitting terms are used to measure the deviation between the force field output by the network and the true value of the force field in the training data, and the residual terms of the elasticity control equations are used to constrain the force field output by the network to satisfy the basic equations of elasticity. The physical information neural network is iteratively trained using the training data until the loss function converges.

[0016] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages: 1. Since there is no physical alignment relationship between the continuous optical modulation layer and the light emitting unit array and the light receiving unit array, the sensor does not require one-to-one alignment between the optical modulation layer and the substrate layer during manufacturing. This eliminates the performance inconsistency problem caused by alignment deviation in current solutions, reduces manufacturing process difficulty and cost, and is conducive to large-area, flexible production. 2. The continuous optical modulation layer covers the entire sensing area, and each light receiving unit receives the comprehensive reflection of multiple texture or pattern features in the area above it. Therefore, when the sensor is subjected to force, the light intensity signal collected by any light receiving unit contains the comprehensive information of the deformation of multiple feature units. The local changes in texture or pattern caused by small deformation can be encoded into a more differentiated spatial change in light intensity over a larger range, which improves the detection sensitivity to tangential force and small slip.

[0017] 3. By setting preset fine textures or patterns on the sensing interface, the response characteristics of the sensor to different directions and types of forces can be customized by adjusting the texture type or pattern form of the modulation layer, without changing the overall structure of the sensor. This achieves design flexibility and adapts to the sensing needs of different application scenarios, from robot grasping to biomechanical monitoring. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall structure of the reflective tactile sensor of the present invention; Figure 2 This is a schematic diagram of one arrangement of the substrate light emitting unit and the light receiving unit of the present invention; Figure 3 This is a schematic diagram of another arrangement of the substrate light emitting unit and the light receiving unit of the present invention; Figure 4 This is a schematic diagram illustrating the design of the three-dimensional microstructure texture route for the optical modulation layer of this invention; Figure 5 This is a schematic diagram of the design of the optical modulation layer of the present invention using a patterned reflective layer route; Figure 6 This is a simulation diagram of the structural mechanical deformation of the strip pattern of the present invention under vertical pressure; Figure 7 This is a simulation diagram of the structural mechanical deformation of the strip pattern of the present invention under horizontal displacement; Figure 8 This is a schematic diagram comparing the light intensity distribution of the striped pattern and the uniform reflective layer of the present invention under normal indentation. Figure 9 This is a schematic diagram comparing the light intensity distribution of the striped pattern and the uniform reflective layer of the present invention under tangential slip. Figure 10 This is a schematic diagram comparing the light intensity distribution of the checkerboard pattern and the uniform reflective layer of the present invention under horizontal displacement. Figure 11 This is a flowchart illustrating the force field reconstruction method of the reflective tactile sensor of the present invention. Detailed Implementation

[0019] To address the technical challenges of current reflective optical tactile sensors, such as high manufacturing precision requirements, complex processes, low yields, and limited sensing resolution due to the need for individual physical alignment between elastic microstructures and photoelectric sensing units, as well as insensitivity to minute tangential deformations, a reflective tactile sensor and its force field reconstruction method are proposed. This sensor employs a finely textured or patterned layer continuously covering the entire sensing area above a transparent elastic layer as an optical modulation layer. This modulation layer is not individually aligned with the underlying arrays of light emitting and receiving units, ensuring that the light signal collected by any light receiving unit originates from the combined reflection of multiple textured or patterned features rather than the individual reflection of a single feature. Under stress, the modulation layer deforms along with the elastic layer, changing the spatial geometry of its texture or pattern. This encodes the surface deformation field into differentiated light intensity change signals between each light receiving unit. These signals are then input into a neural network via differential light intensity images to reconstruct the three-dimensional deformation field and invert the three-dimensional force field distribution. This scheme eliminates the alignment step in the manufacturing process, reduces the difficulty of the process and the manufacturing cost, and is conducive to large-area flexible production. At the same time, each receiving unit receives the comprehensive reflection signal of multiple feature units, so that the minute tangential deformation causes distinguishable changes in spatial light intensity, which improves the detection sensitivity of tangential force and minute slip, and provides a physical basis for realizing super-resolution force field reconstruction through algorithms.

[0020] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art.

[0021] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0022] Example 1 In this embodiment, a reflective tactile sensor is provided, referring to... Figure 1 The reflective tactile sensor includes at least a base layer 1, a transparent elastic layer 2, and an optical modulation layer 3.

[0023] A substrate 1 is provided, on which an array of light receiving units 11 and an array of light emitting units 12 are disposed. The light emitting units are used to emit probe light to the transparent elastic layer, and the light receiving units are used to receive the light signal reflected back by the optical modulation layer and convert it into an electrical signal for output. The arrangement of the light emitting units and light receiving units on the substrate can be carried out in various ways, and this embodiment does not impose constraints on the specific design of the substrate.

[0024] Optionally, refer to Figure 2 The light emitting unit and the light receiving unit constitute multiple sensing elements. Each sensing element contains a central light emitting unit and multiple light receiving units distributed around it. Each sensing unit consists of a central light-emitting diode and four photodiodes distributed around it, forming a dense array.

[0025] Optionally, refer to Figure 3 Furthermore, the light emitting unit and the light receiving unit can be sparsely arranged on the substrate layer, which is suitable for low-cost sensing of large-area deformation.

[0026] Optionally, the optical emitting unit and the optical receiving unit can be arranged in an alternating periodic pattern on the substrate.

[0027] Optionally, the light receiving unit of the substrate layer is not limited to photodiodes, but can be flexible optoelectronic devices such as organic photodiodes and perovskite photodetectors, or integrated with a flexible thin-film transistor array to achieve active matrix readout and improve signal readout speed and integration.

[0028] Optionally, the light emitting unit is not limited to a light-emitting diode, but may also use a vertical cavity surface-emitting laser or a micro laser diode as the light source.

[0029] A transparent elastic layer 2 is disposed on the base layer 1 and is used to transmit light signals and generate elastic deformation when subjected to force; Optionally, the transparent elastic layer is an optically transparent elastic material, such as silicone or other transparent elastomer materials.

[0030] Optionally, the transparent elastic layer serves as a light transmission channel, enabling the probe light to propagate from the light emitting unit of the substrate to the optical modulation layer, and allowing the modulated reflected light to return to the light receiving unit. When the sensor surface is subjected to force, the external load is transmitted to the transparent elastic layer through the optical modulation layer. This layer undergoes elastic deformation such as compression, shearing, or bending. This deformation further drives changes in the geometric shape or spatial position of the optical modulation layer, thereby altering the modulation state of the texture or pattern to the incident light. When the external force is removed, the layer elastically returns to its original shape, restoring the optical modulation layer to its unstressed state, ready for the next sensing operation.

[0031] An optical modulation layer 3 is disposed on or inside the transparent elastic layer. The optical modulation layer is a continuous layer with a preset fine texture or pattern that covers the entire sensing area. It is used to modulate the incident light into a light field with spatial coding characteristics, and the optical modulation layer covers the entire sensing area of ​​the sensor. As an alternative implementation, the lower surface or interior of the optical modulation layer is provided with fine micro-nano-scale textures or patterns, which are continuous and cover the entire sensing area. The optical modulation layer covers the entire sensing area of ​​the sensor and is used to modulate the incident light into a light field with spatial coding characteristics.

[0032] Optionally, there is no physical alignment between the optical modulation layer and the light emitting unit array and the light receiving unit array. The light signal received by any light receiving unit originates from the combined reflection of multiple texture or pattern features in the region above it. This mechanism enables the sensor to encode localized minute deformations into differentiated light intensity change signals over a larger range, providing a physical basis for super-resolution force field reconstruction through algorithms. Therefore, when the surface is subjected to force deformation, the optical modulation layer undergoes local deformation along with the transparent elastic layer, and the geometric features of its texture or pattern change spatially. Differentiated light intensity changes are formed between light receiving units at different locations, thereby encoding the continuous deformation field applied to the sensor surface into a spatially varying light intensity distribution.

[0033] Optionally, refer to Figure 4 The optical modulation layer can employ a three-dimensional microstructure texture to create periodic, fine three-dimensional structures on its surface. (Refer to...) Figure 5 The optical modulation layer adopts a patterned reflective layer approach, in which a reflective substrate is prepared on the lower surface of the modulation layer and a light-absorbing material pattern is set.

[0034] Optionally, the spacing between adjacent feature units of the pattern or texture is smaller than the spacing between the light emitting unit and the adjacent light receiving unit. Under this condition, each light receiving unit is covered with multiple feature units of the texture or pattern. When subjected to force, the light intensity signal collected by any light receiving unit is the comprehensive response of the deformation of the multiple feature units above it. This comprehensive response contains richer spatial coding information, providing a physical basis for realizing super-resolution force field reconstruction through algorithms.

[0035] As an alternative implementation, the optical modulation layer is a patterned reflective layer, which includes a high-reflection region and a low-reflection / light-absorbing region, and the high-reflection region and the low-reflection / light-absorbing region are arranged alternately in at least one direction.

[0036] Specifically, a continuous reflective substrate, such as a silver plating layer, can be prepared on the lower surface of the modulation layer. Then, a light-absorbing material is patterned on the reflective substrate so that some areas of the reflective substrate are covered by the light-absorbing material to form low-reflection / light-absorbing areas, while the remaining uncovered reflective substrate areas constitute high-reflection areas.

[0037] Optionally, the material of the high-reflectivity region can be at least one of a silver film, an aluminum film, a white diffuse reflection coating, or a high-reflectivity polymer layer. The material of the low-reflectivity / light-absorbing region can be at least one of a black ink, carbon black, a matte coating, or a near-infrared absorbing material. The patterned reflective layer can be formed by printing, spraying, transfer printing, laser etching, thin film lamination, or nanoimprinting.

[0038] Optionally, when the sensor surface is subjected to force, the transparent elastic layer undergoes elastic deformation such as compression, shearing or bending. The patterned reflective layer deforms along with the transparent elastic layer, and the relative spatial position and distribution pattern between its high reflective area and low reflective / light-absorbing area change accordingly. That is, the local pattern is stretched, compressed or twisted, which changes the reflectivity distribution of the local area and thus modulates the light intensity returned to the light receiving unit.

[0039] Optionally, the pattern of the patterned reflective layer can be a one-dimensional stripe pattern, with high-reflection areas and low-reflection / light-absorbing areas alternating along one direction to enhance the identification of tangential displacement and shear force in a certain direction.

[0040] Optionally, the pattern can also be a two-dimensional checkerboard pattern, dot matrix pattern, cross stripe pattern, ring pattern, or coded array pattern, used to simultaneously encode tangential displacements in both x and y directions and assist in reconstructing the local indentation topography.

[0041] To verify the light field modulation effect of the patterned reflective layer, this embodiment compares and analyzes the responses of striped and checkerboard patterns under stress through simulation. The simulation employs a simplified scalar light intensity calculation model: the emitted light from the light emitting unit is approximated as a Gaussian distribution, the light receiving unit is approximated as a finite receiving aperture, and the reflective layer is represented by a spatial reflectivity function. The comparison objects include a uniform planar reflective layer and a patterned reflective layer.

[0042] Reference Figure 6 and Figure 7 Under normal pressure, the elastic layer and patterned reflective layer of the striped pattern undergo compressive deformation; under tangential displacement, asymmetric deformation and shear response are generated. (Refer to...) Figure 8 During normal indentation, the striped reflection / absorption pattern results in more pronounced differential light intensity changes at different light receiver locations, while the uniform reflective layer primarily exhibits overall light intensity changes and has a weaker ability to distinguish local indentation locations and deformation ranges. (Refer to...) Figure 9During tangential slip, the strip-shaped reflection / absorption pattern converts the lateral displacement into a directional shift of the light spot at the receiving end, enhancing the photosensitive array's ability to resolve tangential forces. In contrast, the uniform reflective layer primarily causes slight changes in overall light intensity and lacks identifiable spatial shift characteristics. (Refer to...) Figure 10 The checkerboard-like reflective / absorbent two-dimensional pattern introduces periodic spatial coding in both the x and y directions, causing horizontal displacement to lead to two-dimensional position shift and local light intensity redistribution. In contrast, the uniform reflective layer lacks two-dimensional spatial coding features and has a weaker ability to distinguish tangential displacement and complex contact deformation.

[0043] Optionally, the optical modulation layer is a pseudo-random speckle pattern layer. When the transparent elastic layer is deformed in any direction by an external force, the pseudo-random speckle pattern generates a change in light intensity, and the deformation field of the sensor surface can be reconstructed using an image correlation algorithm. The image correlation algorithm calculates the displacement and strain of a sub-region by comparing the grayscale distribution of local sub-regions in the speckle images before and after deformation, thereby reconstructing the surface deformation field; specifically, a digital image correlation method can be used to obtain sub-pixel precision displacement field information by tracking the positional changes of feature regions in the speckle pattern before and after deformation.

[0044] As an optional implementation, the optical modulation layer is a textured layer with a three-dimensional microstructure, and the three-dimensional microstructure is arranged in an array on the surface of the optical modulation layer.

[0045] Specifically, periodic micro-three-dimensional structures can be fabricated on the surface of the optical modulation layer. These three-dimensional microstructures can be any one or a combination of pyramid arrays, inverted pyramid arrays, micropillar arrays, hemispherical dome arrays, sinusoidal gratings, or randomly rough surfaces. The feature size of the three-dimensional microstructure is less than 2 mm, preferably between 10 μm and 500 μm.

[0046] Optionally, when the sensor surface is subjected to force, the transparent elastic layer undergoes elastic deformation such as compression, shearing, or bending. The three-dimensional microstructure deforms along with the transparent elastic layer, and its geometric parameters, such as orientation, spacing, and depth, change. These microstructures, through their geometry, produce directional scattering, diffuse reflection, or diffraction of incident light. The changes in the geometric parameters of the deformed microstructure alter the direction and spatial distribution pattern of the scattered light, thereby modulating the light intensity returning to the light receiving unit.

[0047] Optionally, a highly reflective material, such as a silver, aluminum, or titanium dioxide reflective coating, can be sprayed or plated onto the surface of the three-dimensional microstructure to improve the signal-to-noise ratio. The three-dimensional microstructure can be fabricated on the elastomer using soft lithography, nanoimprinting, micro-injection molding, laser direct writing, or two-photon polymerization 3D printing.

[0048] In this embodiment, by setting textures or patterns, the sensor becomes highly sensitive to the in-plane shear strain of the transparent elastic layer. When subjected to force, the light intensity signal undergoes stretching, compression, or twisting along with the elastic layer, resulting in a recognizable spatial change. By adjusting the texture or pattern type of the optical modulation layer, such as regular stripes, speckle, three-dimensional microstructures, or multi-scale superimposed patterns, the sensor's sensing characteristics can be customized for different application scenarios.

[0049] As an alternative implementation, the texture or pattern of the optical modulation layer is a multi-scale superimposed pattern with coarse periodic components and fine periodic components superimposed.

[0050] Specifically, coarse and fine periodic patterns are superimposed in the same reflective layer, enabling the effective detection of both large-scale displacement and minute deformation, achieving multi-resolution readout. The coarse periodic pattern is used to detect large-scale displacement or pressure distribution, while the fine periodic pattern is used to detect minute local deformation or high-frequency vibration.

[0051] Optionally, the material of the optical modulation layer is not limited to silicone; it can be a photoelastic material, where stress directly causes a change in the material's refractive index, thereby modulating the optical path and enhancing sensitivity. Thermochromic or electrochromic materials can also be used, enabling the sensor to simultaneously possess force sensing and visual feedback functions.

[0052] Optionally, the optical modulation layer may also be doped with fluorescent particles that are sensitive to specific chemical substances, enabling the sensor to sense both force and chemical stimuli simultaneously; or, combined with thermally conductive and insulating designs, enabling the sensor to sense temperature distribution simultaneously.

[0053] Optionally, the reflective tactile sensor also includes a light-shielding protective layer 4 disposed above the optical modulation layer 3. The light-shielding protective layer is a flexible, opaque skin that provides wear resistance, stain resistance, and surface tactile feedback.

[0054] Optionally, the optical modulation layer can be disposed on the surface of the object under test, and the base layer can move near the object under test as an independent reading head to non-contactly measure the dynamic deformation of the object surface by analyzing the reflected light field.

[0055] In this embodiment, by employing an optical modulation layer that continuously covers the sensing area and does not require physical alignment with the light emitting and receiving unit arrays, the problems of complex manufacturing processes, high costs, and low yields caused by aligning microstructures with sensing units one by one are eliminated, which is beneficial for large-area, flexible production. Simultaneously, the overall sensor structure remains stable, and different sensing requirements can be adapted by changing the texture or pattern type of the modulation layer without redesigning the overall architecture, thus offering design flexibility. In summary, the sensor in this embodiment achieves high-sensitivity sensing of normal and tangential forces while maintaining simple manufacturing processes and controllable costs, providing a reliable signal acquisition foundation for high-precision force field reconstruction.

[0056] Based on the same inventive concept, this invention also provides a force field reconstruction method corresponding to the reflective tactile sensor in Embodiment 1, as shown in Embodiment 2.

[0057] Example 2 Based on Embodiment 1, another embodiment of the present invention is proposed, with reference to... Figure 11 The force field reconstruction methods for reflective tactile sensors include: Step S1: Obtain the baseline light intensity signal of each optical receiving unit in an unstressed state; In this embodiment, the baseline light intensity signal refers to the photocurrent value output by each light receiving unit when the sensor surface is not subjected to any external load, serving as a reference benchmark for subsequent stress state detection. Obtaining the baseline signal through a calibration process can eliminate the influence of factors such as individual sensor differences, ambient temperature changes, and power fluctuations of the light emitting unit on the measurement results.

[0058] As an optional implementation, before the sensor is put into use, it is placed in an initial state without external load, and the photocurrent value output by each optical receiving unit is read sequentially and recorded as a baseline light intensity signal. This calibration process can be completed once before the sensor leaves the factory, or it can be performed periodically at the field to compensate for baseline drift caused by environmental changes.

[0059] Step S2: Obtain the current light intensity signal of each optical receiving unit under stress, and calculate the differential light intensity signal of each optical receiving unit based on the baseline light intensity signal and the current light intensity signal. In this embodiment, when the sensor surface is subjected to external force, the optical modulation layer deforms along with the transparent elastic layer, and the geometric features of its texture or pattern change, causing the intensity of reflected light received by each light receiving unit to change relative to the unforced state. The differential light intensity signal is the normalized difference between the current light intensity signal and the baseline light intensity signal, used to characterize the degree of light intensity change of each light receiving unit before and after the force is applied.

[0060] As an optional implementation, the photocurrent value of each optical receiving unit in the current state is collected. Calculate the normalized relative change of each unit using the following formula:

[0061] in, Let be the differential light intensity signal of the i-th optical receiving unit. Let be the current photocurrent value of the i-th optical receiving unit under stress. This represents the baseline photocurrent value of the i-th optical receiving unit under unloaded conditions. Normalization eliminates the influence of output power fluctuations in the optical emitting units and individual sensor differences on the measurement results.

[0062] Step S3: Arrange the differential light intensity signals of each optical receiving unit according to their spatial positions on the substrate to form a two-dimensional differential light intensity image; In this embodiment, the optical receiving units are arranged in an array on the substrate, with each unit corresponding to a spatial coordinate. The differential light intensity signal of each unit is then... By filling in the corresponding image pixels according to their physical locations, a two-dimensional differential light intensity image D(x, y) can be obtained. This image reflects the spatial variation pattern of light intensity distribution after the sensor surface is subjected to force, and can be used as input data for subsequent neural network deformation field reconstruction.

[0063] As an alternative implementation, for optical receiving units arranged in a rectangular array, the differential light intensity signals of each unit can be directly mapped to pixel values ​​of a two-dimensional image according to the row and column positions of the array. For optical receiving units arranged in a non-rectangular array, such as sparse or ring arrays, the spatial coordinates of each unit can be determined first, and then the discrete differential light intensity signals can be mapped to a two-dimensional image on a regular grid using an interpolation method.

[0064] Optionally, after acquiring the two-dimensional differential light intensity image, the image can be directly input into a trained physical information neural network. The physical information neural network then performs a one-step mapping to output the three-dimensional force field distribution on the sensor surface. The physical information neural network is pre-trained using multiple sets of training data, each set containing the differential light intensity image and the corresponding true value of the three-dimensional force field. The training steps for this physical information neural network are described in subsequent steps S501 to S503.

[0065] Step S4: Input the two-dimensional differential light intensity image into the trained convolutional neural network, and output the three-dimensional deformation field of the optical modulation layer surface from the convolutional neural network. In this embodiment, the three-dimensional deformation field refers to the displacement of each point on the surface of the optical modulation layer in three-dimensional space, including the normal displacement and the tangential displacement in two orthogonal directions, denoted as... .

[0066] Alternatively, a deep convolutional encoder-decoder network can be used as the solver. Using the two-dimensional differential light intensity image D(x, y) as input, the output is the three-dimensional deformation field of the optical modulation layer surface. The encoder extracts spatial features from the differential intensity image through multi-layer convolution and pooling operations. The decoder restores the feature map to its original resolution through upsampling and skip connections, ultimately outputting a three-channel deformation field image with the same size as the input image, corresponding to the normal displacement. and two tangential displacements , .

[0067] As an optional implementation, the convolutional neural network needs to be trained in advance with a large amount of training data to learn the mapping relationship from differential light intensity images to three-dimensional deformation fields. The training data can be generated through multiphysics simulation.

[0068] Specifically, a mechanical model of the sensor structure can be established using finite element analysis software to simulate the three-dimensional deformation field of the transparent elastic layer and optical modulation layer under external forces of different positions, directions, and magnitudes. Simultaneously, an optical simulation software is used to establish a light field propagation model to simulate the light intensity distribution received by each light receiving unit under the corresponding deformation state. The simulated light intensity distribution data is processed into differential light intensity images according to steps S1 to S3, and paired with the corresponding ground truth values ​​of the three-dimensional deformation field to form a large-scale training dataset. This training dataset is used to supervise the training of the convolutional neural network, optimizing the network parameters by minimizing the error between the network's output deformation field and the simulated ground truth values ​​until the network converges.

[0069] Step S5: Based on the physical relationship between surface force and surface deformation in elasticity, the three-dimensional force field distribution on the sensor surface is obtained by inversion from the three-dimensional deformation field.

[0070] In this embodiment, in elasticity, there is a definite physical relationship between surface force and surface deformation. For a linear elastic body, the surface force f and the surface displacement u satisfy the convolution relationship expressed by the Green's function:

[0071] Where r = (x, y) are the position coordinates of the sensor surface, and S is the sensor surface area. This represents the deformation response of the sensor in the j direction when a unit point force is applied in the k direction.

[0072] By solving this inverse problem, the three-dimensional force field distribution on the sensor surface can be deduced from the three-dimensional deformation field obtained in step S4.

[0073] As an optional implementation, the step of obtaining the three-dimensional force field distribution by inverting the three-dimensional deformation field includes: constructing a linear relationship matrix between the three-dimensional deformation field and the three-dimensional force field, wherein the linear relationship matrix characterizes the deformation response generated at each position on the sensor surface under the action of a unit force; and introducing a smoothness constraint condition based on the linear relationship matrix to minimize the deviation between the reconstructed deformation field and the measured deformation field while suppressing the solution instability caused by measurement noise, thereby obtaining a stable three-dimensional force field distribution.

[0074] Specifically, the continuous convolution relationship is discretized into a system of linear equations. Where U is the known displacement vector obtained from step S4, F is the force vector to be determined, and G is the Green's function matrix.

[0075] Measurement noise or model errors can cause drastic oscillations in the solution to this inverse problem; therefore, regularization methods are needed to obtain stable numerical solutions. Specifically, Based on the linear relationship matrix, a smoothness constraint is introduced to minimize the deviation between the reconstructed deformation field and the measured deformation field, while suppressing the solution instability caused by measurement noise, thus obtaining a stable three-dimensional force field distribution. The solution objective of the regularization method is:

[0076] in, This is a data fitting term used to constrain the deformation generated by the inverted force field to be as consistent as possible with the deformation field measured in step S4. This is a regularization term used to constrain the spatial variation of the force field from becoming too drastic. This is a regularization parameter used to balance the weights between data fitting and smoothness constraints; It is a regularization operator.

[0077] As an optional implementation, a physical information neural network can be used to achieve a one-step, end-to-end direct mapping from differential light intensity images to a three-dimensional force field. This physical information neural network is a neural network model that incorporates physical constraints during training. Unlike purely data-driven neural networks, the loss function of the physical information neural network includes not only data fitting terms but also residual terms from the elasticity control equations, thereby constraining the force field output by the network to satisfy the fundamental equations of elasticity.

[0078] Specifically, the training steps for a physical information neural network include: Step S501: Obtain multiple sets of training data, each set containing the differential light intensity image and the corresponding three-dimensional force field true value. The training data can be generated through multiphysics simulation, and its specific generation method is the same as that of the training data generation method of the convolutional neural network in step S4, that is, by constructing a closed-loop data chain of force, deformation, light field change, and electrical signal through finite element analysis and optical simulation, a large amount of paired data is obtained.

[0079] Step S502: Construct a physical information neural network. During the training process of the physical information neural network, its loss function includes both a data fitting term and a residual term of the elasticity control equation. The data fitting term is used to measure the deviation between the force field output by the network and the true value of the force field in the training data. The residual term of the elasticity control equation is used to constrain the force field output by the network to satisfy the basic equation of elasticity. Among them, the physical information neural network The input is a two-dimensional differential light intensity image D(x, y), and the output is a three-dimensional force field distribution. The loss function of this network includes both data fitting terms and residual terms from the elasticity control equations:

[0080] in, This represents the residuals of the governing equations of elasticity. These are weighting coefficients used to balance the relative importance of data fitting and physical constraints.

[0081] Step S503: Iteratively train the physical information neural network using the training data until the loss function converges. After training, the three-dimensional force field distribution is directly mapped by the trained physical information neural network based on the two-dimensional differential light intensity image, without explicitly solving the deformation field first and then inverting the force field, achieving end-to-end one-step inverse solution.

[0082] Both the Green's function-based regularization method and the end-to-end method based on physical information neural networks can independently implement step S5, and the choice can be made based on specific needs in practical applications. When the sensor structure is relatively simple and the elasticity model is easy to establish, the Green's function regularization method can be used, as its physical meaning is clear and its computational efficiency is high. When the sensor structure is complex and it is difficult to establish an accurate analytical or numerical model, the physical information neural network method can be used, as it can learn the end-to-end mapping from light intensity to force field without explicit modeling. The two methods can also be used in combination, for example, using simulation data generated by the Green's function regularization method as the training data source for the physical information neural network.

[0083] Optionally, the force field reconstruction can also be implemented using other algorithms, including but not limited to analytical calibration, lookup table methods, and least squares methods. The analytical calibration method directly calculates the force field by establishing an analytical expression between the light intensity signal and the force field; the lookup table method pre-calibrates and stores a light intensity-force field mapping table, and obtains the force field distribution during actual measurement by looking up the table and interpolation; the least squares method estimates the force field distribution by minimizing the sum of squared errors between the light intensity signal and the model's predicted value. These algorithms are suitable for application scenarios with relatively simple sensor structures and relatively clear mapping relationships, and can be flexibly selected according to actual needs.

[0084] This embodiment decomposes the complex force field reconstruction problem into two sub-problems: deformation field estimation and force field inversion. The first part uses a data-driven deep convolutional neural network to estimate the deformation field from differential light intensity images. The second part uses an elasticity mechanics physical model to invert the force field from the deformation field. The combination of these two methods achieves a complete reconstruction from the sensor's distributed light intensity signal to a three-dimensional force field. This reconstruction method can effectively extract the spatially encoded light intensity change characteristics generated after the optical modulation layer is subjected to force, and stably and accurately invert the three-dimensional force field distribution on the sensor surface, providing reliable tactile perception data for applications such as robot dexterity and human-computer interaction.

[0085] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0089] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, third, etc., does not indicate any order. These words can be interpreted as names.

[0090] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0091] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A reflective tactile sensor, characterized in that, include: A substrate layer, on which an array of light emitting units and an array of light receiving units are disposed; A transparent elastic layer is disposed on the substrate layer to transmit optical signals and generate elastic deformation when subjected to force; An optical modulation layer is disposed on or inside the transparent elastic layer. The optical modulation layer is a continuous layer with a preset fine texture or pattern that covers the entire sensing area. It is used to modulate the incident light into a light field with spatial coding characteristics, and the optical modulation layer covers the entire sensing area of ​​the sensor. There is no physical alignment between the optical modulation layer and the optical emitting unit array and the optical receiving unit array.

2. The reflective tactile sensor as described in claim 1, characterized in that, The optical modulation layer is any one of the following: A patterned reflective layer comprising a high-reflectivity region and a low-reflectivity / light-absorbing region, wherein the high-reflectivity region and the low-reflectivity / light-absorbing region are alternately arranged along at least one direction; Pseudo-random speckle pattern layer; A textured layer with a three-dimensional microstructure, wherein the three-dimensional microstructure is arranged in an array on the surface of the optical modulation layer; A multi-scale superimposed pattern containing both coarse and fine periodic components.

3. The reflective tactile sensor as described in claim 1, characterized in that, It also includes a light-shielding protective layer disposed above the optical modulation layer.

4. The reflective tactile sensor as described in claim 1, characterized in that, The optical emitting unit array and the optical receiving unit array are arranged in an alternating pattern on the substrate layer.

5. The reflective tactile sensor as described in claim 1, characterized in that, The spacing between adjacent feature units of the texture or pattern of the optical modulation layer is smaller than the unit spacing of the light emitting unit array or the light receiving unit array.

6. A method for force field reconstruction of a reflective tactile sensor, characterized in that, The method includes: Acquire the baseline light intensity signal of each optical receiving unit under no-force conditions; Acquire the current light intensity signal of each optical receiving unit under stress, and calculate the differential light intensity signal of each optical receiving unit based on the baseline light intensity signal and the current light intensity signal; The differential light intensity signals of each optical receiving unit are arranged into a two-dimensional differential light intensity image according to their spatial positions on the substrate layer; The two-dimensional differential light intensity image is input into a trained convolutional neural network, which outputs the three-dimensional deformation field of the optical modulation layer surface. Based on the physical relationship between surface force and surface deformation in elasticity, the three-dimensional force field distribution on the sensor surface is obtained by inversion from the three-dimensional deformation field.

7. The method as described in claim 6, characterized in that, The step of obtaining the three-dimensional force field distribution of the sensor surface by inverting the three-dimensional deformation field based on the physical relationship between surface force and surface deformation in elasticity includes: A linear relationship matrix is ​​constructed between the three-dimensional deformation field and the three-dimensional force field. The linear relationship matrix characterizes the deformation response generated at each position on the sensor surface under the action of a unit force. Based on the linear relationship matrix, a smoothness constraint is introduced to minimize the deviation between the reconstructed deformation field and the measured deformation field, while suppressing the solution instability caused by measurement noise, thus obtaining a stable three-dimensional force field distribution.

8. The method as described in claim 6, characterized in that, The training data for the convolutional neural network is obtained through the following methods: A mechanical model of the sensor structure is established to simulate the three-dimensional deformation field of the transparent elastic layer and the optical modulation layer under the action of external forces of different positions, directions and magnitudes; A light field propagation model is established to simulate the light intensity distribution received by each light receiving unit corresponding to the three-dimensional deformation field. The simulated light intensity distribution is processed into a differential light intensity image, which is then paired with the corresponding three-dimensional deformation field ground value to form a training dataset.

9. The method as described in claim 6, characterized in that, After the step of arranging the differential light intensity signals of each optical receiving unit into a two-dimensional differential light intensity image according to their spatial positions on the substrate, the method includes: The two-dimensional differential light intensity image is directly input into a trained physical information neural network, and a one-step mapping is directly performed through the physical information neural network to output the three-dimensional force field distribution on the sensor surface. The physical information neural network is pre-trained using multiple sets of training data, each set of training data containing the differential light intensity image and the corresponding three-dimensional force field true value.

10. The method as described in claim 9, characterized in that, The training steps of the physical information neural network include: Multiple sets of training data are acquired, each set of training data including the differential light intensity image and the corresponding three-dimensional force field true value; During training, a loss function containing data fitting terms and residual terms of the elasticity control equations is used for iterative optimization; wherein, the data fitting terms are used to measure the deviation between the force field output by the network and the true value of the force field in the training data, and the residual terms of the elasticity control equations are used to constrain the force field output by the network to satisfy the basic equations of elasticity. The physical information neural network is iteratively trained using the training data until the loss function converges.

Images