Calculation method for contact edge and pressure distribution based on shear strain field

Abstract

This invention relates to the field of tactile sensing technology and discloses a method for calculating contact edges and pressure distribution based on shear strain fields. This method, based on mathematical analytical theory, achieves high-precision contact boundary identification and pressure distribution field reconstruction by calculating the divergence of the shear strain field. The method includes: calculating the shear strain field of a flexible skin surface using the real-time centroid position of marked points, and generating a divergence field by performing spatial differentiation on the shear strain field; determining the object contact boundary based on the boundary between the positive and negative regions of the divergence field; and calculating the pressure distribution field using the direct proportionality between the divergence field integral and the normal pressure. This invention achieves high spatial density measurement of contact areas and pressure through a purely mathematical analytical method, is independent of training data, has universality, and is applicable to hardware of different sizes and shapes. It can be used for contact edge and pressure distribution monitoring in robotic tactile sensing, as well as dynamic pressure distribution measurement in medical monitoring equipment.

Description

Technical Field

[0001] This invention relates to the field of tactile sensing technology, specifically to a method for real-time calculation of the contact edges and pressure distribution of multiple objects in complex scenarios based on the shear strain field of flexible skin markers of a visual tactile sensor. Background Technology

[0002] Skin is a crucial part of the human body that interacts with the external environment. It enables humans to perceive different shapes, textures, and contact pressures. In the field of robotics, perceiving the contact state, position, and area of ​​contact with objects is essential. In recent years, robotic tactile sensors based on capacitive, piezoresistive, optical, and magnetic principles have been extensively researched. For example, resistive sensing skin composed of an array of sensing units can perform wide-range sensing; however, such sensors are complex to manufacture, have numerous leads, high hysteresis, and poor stability. Benefiting from low cost, simple manufacturing process, real-time performance, and high stability, vision-based optical sensors have emerged as leaders in tactile sensors, offering relatively high spatial accuracy in locating contact areas. However, existing visual-tactile sensing technologies rely on machine learning for data analysis, resulting in large data processing volumes, which limits further performance improvements and cost reductions. For instance, most visual-tactile sensors acquire the shape of the contact object through machine learning, which involves complex algorithms, extensive training, and often relies on GPUs to accelerate processing when acquiring tactile information. Moreover, errors increase significantly when encountering data outside the training data range, requiring extensive retraining when the sensor's shape or size changes, resulting in poor versatility. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a method for real-time calculation of contact edges and pressure distribution of multiple objects in complex scenarios based on the shear strain field of flexible skin markers using visual-tactile sensors. This method solves the problems of traditional methods relying on machine learning, large data processing volume, and poor real-time performance, thus meeting the requirements for real-time measurement and feedback of edge detection and pressure distribution of multiple objects in complex environments such as robot grasping.

[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0005] This invention provides a method for calculating contact edge and pressure distribution based on shear strain field, comprising the following steps:

[0006] S1000: Converts the centroid displacement field of the flexible skin markers of the visual-touch sensor into a shear strain vector field that characterizes local deformation, and then performs spatial differentiation on the shear strain vector field to generate a divergence field.

[0007] S2000: Determine the contact boundary of the object based on the critical point of the positive and negative sources of the divergence field;

[0008] S3000: By utilizing the direct proportional relationship between divergence field characteristics and normal force, the proportionality coefficient is determined through calibration experiments, which enables the reconstruction of the pressure distribution field based on real-time divergence field characteristics.

[0009] In one embodiment, in S3000, the divergence field characteristic is the integral of the region with positive divergence values ​​in the divergence field.

[0010] In one embodiment, in step S3000, the mapping function between the divergence field characteristic quantity and the normal force is fitted by collecting calibration data, including the following steps:

[0011] S3001. Control the three-dimensional displacement stage to first make the visual-tactile sensor contact the calibration head, and maintain the visual-tactile sensor and the calibration head in constant force contact in the vertical direction;

[0012] S3002: Control the vertical movement of the three-dimensional displacement stage, and the calibration head causes the flexible skin to deform. Record the divergence field characteristic quantity corresponding to each position, and at the same time record the normal force reading of the force sensor on the three-dimensional displacement stage.

[0013] S3003. By collecting multiple sets of divergence field feature quantities and corresponding normal forces, fit the mapping function from divergence field feature quantities to normal forces;

[0014] S3004. Based on the scaling factor of the mapping function from the divergence field characteristic quantity to the normal force, each divergence element is converted into pressure.

[0015] In one embodiment, S3000, the reconstructing of the pressure distribution field based on the real-time divergence field characteristics further includes: dividing the divergence field into several divergence micro-elements, and independently calculating the local pressure based on the divergence field characteristics of each divergence micro-element.

[0016] In one embodiment, the divergence element in S3004 refers to the divergence scalar value or average divergence value corresponding to each grid cell after dividing the flexible skin surface of the visual-touch sensor into M rows and N columns of grid cells; the mapping function is expressed as... ,in Where is the local pressure, and k is the scaling factor of the mapping function. For the scalar value or average divergence value corresponding to the grid cell, when When it is negative, Set to zero.

[0017] In one embodiment, in S1000, the spatial differentiation operation employs numerical differentiation techniques to calculate the divergence field.

[0018] In a preferred embodiment, the numerical differentiation technique can employ the central difference method.

[0019] In one embodiment, S2000 further includes multi-object contact recognition: performing connected component analysis on the positive sources of the divergence field, and determining a multi-object contact scenario when multiple independent positive source connected regions are detected, and performing contact boundary detection on each independent positive source connected region.

[0020] Compared with the prior art, the beneficial technical effects of the present invention are:

[0021] 1. This invention uses a pure mathematical analytical method to process displacement data, which does not require complex iterative calculations. The algorithm has high computational efficiency and can meet the real-time response requirements of robot real-time operation or dynamic monitoring of medical equipment.

[0022] 2. The analytical algorithm of this invention is applicable to visual and tactile sensors of different sizes and structures (such as planar / curved surfaces), and can be quickly adapted to new scenarios by simply adjusting the calibration parameters.

[0023] 3. This invention establishes the mapping relationship between divergence field and normal force through pre-calibration, which reduces the intensive calibration steps required by traditional tactile sensors and lowers deployment costs and complexity.

[0024] In summary, this invention achieves tactile perception through a purely mathematical analytical method, which is highly real-time and versatile. Its analytical method is applicable to hardware of different sizes and shapes, and can be adapted to various application scenarios with only simple calibration of key parameters. It is expected to realize real-time edge detection and pressure distribution monitoring in robots, as well as dynamic pressure distribution measurement in medical monitoring equipment. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the analysis of multi-object edge detection and pressure distribution calculation based on a visual-tactile sensor according to the present invention.

[0026] Figure 2 This is a schematic diagram of the structure of a visual-tactile sensor according to the present invention;

[0027] Figure 3 The original image was captured by the visual-tactile sensor of this invention.

[0028] Figure 4 A vector field distribution diagram of shear strain generated when a spherical object compresses elastic skin;

[0029] Figure 5 (a) shows the shear strain vector field and corresponding divergence field distribution of the same calibration head at different pressing depths; Figure 5 (b) shows the shear strain vector field and corresponding divergence field distribution of calibration heads of different shapes at a pressing depth of 2.5 mm;

[0030] Figure 6 This is a schematic diagram illustrating the quantitative evaluation results of the contact area detection accuracy of objects with different pressing depths and shapes according to the present invention.

[0031] Figure 7 The graph shows the linear relationship between the divergence integral value and the normal force under the action of four cylindrical calibration heads of different diameters.

[0032] Figure 8 A pressure distribution diagram showing the contact between objects of different geometric shapes and elastic skin;

[0033] Figure 9 The instantaneous pressure distribution captured when a sphere falls freely and impacts an elastic skin.

[0034] Figure 10 A schematic diagram illustrating the evolution of pressure distribution during feather scraping;

[0035] Figure 11 Pressure distribution changes when a miniaturized visual-tactile sensor is mounted on a gripper to hold a dropper; Figure 11 (a) in the diagram is a schematic of the gripper holding the dropper. Figure 11 (b) is a comparison diagram of the pressure distribution of the visual tactile sensor in three states: static, holding the dropper, and squeezing the dropper.

[0036] Figure 12 For measuring dynamic pressure distribution in medical monitoring equipment; Figure 12 (a) in the diagram is a schematic diagram of the pressure distribution in the hospital bed. Figure 12 (b) is a schematic diagram of wheelchair pressure distribution. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0038] The following will refer to Figures 1 to 12 Embodiments of the present invention are described in detail below. While specific embodiments of the invention are shown in the accompanying drawings, it should be understood that these descriptions are exemplary only and not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0039] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0040] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0041] The following description, with reference to the accompanying drawings, illustrates a specific implementation of the present invention for multi-object edge detection and pressure distribution based on a visual-tactile sensor. One type of visual-tactile sensor in this invention employs a mathematical analytical method based on displacement field vector calculations, enabling real-time calculation of the edge contour and pressure distribution of contacting objects.

[0042] like Figure 2 As shown, in one embodiment, the present invention provides a visual-tactile sensor for multi-object edge detection and pressure distribution calculation. The visual-tactile sensor includes a black skin layer 1, white marker dots 2, a transparent silicone layer 3, an elastomer support frame 4, a uniformly emitting light fixture, a transparent acrylic plate 6, a camera support frame 7, and a camera 8. The transparent silicone layer 3 is embedded inside the elastomer support frame 4, and the transparent acrylic plate 6 is glued to the bottom of the elastomer support frame 4 to support the transparent silicone layer 3. The uniformly emitting light fixture includes an LED light group 5 and a transparent acrylic light guide plate, wherein the LED light group 5 consists of multiple LED beads arranged in a ring array along the outer edge of the transparent acrylic light guide plate. The LED light group 5 is fixedly connected to the transparent acrylic light guide plate 6 using adhesives such as epoxy resin. When the LED light assembly 5 is powered on and emits light, the light enters from the side of the transparent acrylic light guide plate 6. After multiple total internal reflections and scattering within the acrylic material, the light is uniformly emitted from the light-emitting surface of the transparent acrylic light guide plate 6, forming a uniform light emission effect without obvious dark areas. The optical marking structure includes a three-layer composite structure (see...). Figure 2The outermost black skin layer 1 is made of carbon nanotubes and liquid silicone, and is cast after vacuum stirring and degassing. The transparent silicone layer 3 is 1 cm thick, and the inner transparent acrylic sheet 6 is 5 mm thick. The transparent silicone layer 3 is bonded to the transparent acrylic sheet 6 without bubbles through a silicone vacuum degassing process. The white marker 2 is made of white pigment and silicone. The array of white marker 2 is formed by screen printing. An acrylic mask with 943 small holes after laser cutting is formed on the surface of the transparent silicone layer 3 through a perforated mask scraping process. The black skin layer 1 constitutes the high-contrast background layer of the white marker 2. The upper end of the camera support frame 7 is provided with a first connecting part, which is fixedly connected to the corresponding connecting part of the elastic body support frame 4 by screws or other first fasteners. The lower end of the camera support frame 7 is provided with a second connecting part, which is fixedly connected to the mounting interface of the camera body by screws or other second fasteners. The original image acquired by the sensor in this embodiment is as follows: Figure 3 As shown.

[0043] S1000: As Figure 2 and Figure 4 As shown, when the flexible skin of the visual-touch sensor in this embodiment (i.e., the composite structure formed by the black skin layer 1, white markers 2, and transparent silicone layer 3) comes into contact with an external object, the contact force F causes the elastic medium layer to undergo a corresponding deformation U, thereby causing the marker layer, which is tightly bonded to it, to also undergo a corresponding displacement, denoted as . , (In the following text, n refers to this range). Represents the shear strain vector field. This represents the displacement components of the nth marker point in the x and y directions. This represents the total number of marker points. Raw images of the marker point layer are captured at each time step using a high-speed camera. Compared to the previous frame image , These represent the (k-1)th and kth times, respectively, where the marker points of adjacent two frames are matched and sorted. The current frame's marker point position is used. Subtracting the contactless state (default initial time) Non-contact marker location The relative displacement of the current frame marker point is obtained, which is referred to as the shear strain vector field in this invention. Using this relationship, the magnitude and distribution of the contact force can be obtained from each frame of the image. The markers on the flexible skin have a diameter of 0.6 mm and a centroid spacing of 1.2 mm, totaling... A precise shear strain vector field can be obtained by tracking the displacement of embedded marker points within the flexible skin. In this invention, a small ball is placed at the center of the sensing area of ​​the visual-tactile sensor in this embodiment. Given a downward displacement, the shear strain vector field is obtained by tracking the marker points. Figure 4 The shear strain vector field is shown. Building upon the achievement of obtaining a real-time, high-precision shear strain vector field by tracking the displacement of the centroid of the marker point, this invention further establishes a mapping relationship between the shear strain vector field u, the contact area T, and the pressure distribution P. When a normal force is applied to the flexible skin of the visual-tactile sensor, the marker point exhibits a tendency to migrate towards the boundary of the contact area. For example... Figure 1 The distribution characteristics of the shear strain vector field u shown in (a) are highly consistent with the boundary zone between the converging and diverging flows in the vector field. Therefore, this invention proposes the concept of a shear strain divergence field, denoted as... This is used to characterize the spatial flux properties of the shear strain vector field u, that is, to describe the degree of convergence / divergence at each point in the field. For calculating the discretized... In this embodiment, the present invention uses the Delaunay triangulation method to divide the marker point matrix into several triangular regions. For a given query point, the triangle to which it belongs is first located, and then the interpolation surface parameters are calculated based on formula (1).

[0044] (1)

[0045] For a given interpolation grid This invention uses discrete linear interpolation described in interpolation equation (2) to obtain... :

[0046] (2)

[0047] in, , , Let a, b, and c represent the coordinates of the vertices of the triangle, and let a, b, and c be the undetermined parameters of the interpolation equation. This represents the maximum index of the grid along the x-axis. This represents the maximum index of the grid along the y-axis. Represents the set of natural numbers. Represents the non-negative integer indices of grid nodes in the x-axis and y-axis directions. This represents the interpolated shear strain vector field.

[0048] In this embodiment, the present invention uses the central difference method to calculate the discretized shear strain divergence field. The specific steps are as follows:

[0049] (3)

[0050] S2000: The sign of the divergence reflects the local stress state: a positive divergence value indicates that the surrounding material is under tensile stress, while a negative divergence value corresponds to compressive stress. For example... Figure 1 As shown in (b), the marked points on the outer side of the contact edge exhibit negative divergence characteristics due to compressive stress. The zero-value boundary zone between the positive and negative divergence regions can accurately mark the contact edge line, which is adjacent to the physical boundary of the contact surface. This proves that the divergence field can serve as a reliable indicator for contact boundary detection. Let... These are the interpolated shear strain vector fields. The x and y components, Represents the interpolation grid The spacing. Define the divergence threshold. , Represents the divergence field The standard deviation is satisfied by extracting the shear strain divergence field. The area is used to determine the contact range.

[0051] In one embodiment, the present invention applies normal pressure at different depths to the visual-tactile sensor using an 8mm diameter cylindrical calibration head, and records the contact area size extracted by the visual-tactile sensor using a divergence field at each depth. Figure 6 (a) shows the shear strain vector field of the 8mm cylindrical calibration head at different indentation depths (0.5-3mm). divergence field And the detection / actual contact area is compared; another embodiment of the present invention is to apply the same depth of positive pressure to the visual tactile sensor using calibration heads of different shapes, and record the actual area size of the calibration head and the contact area area extracted by the visual tactile sensor using the divergence field for each shape. Figure 6 (b) shows the test results of different calibrator shapes at a fixed depth of 2 mm; using area ratio Evaluation accuracy ( For the detection area, (The actual area is shown). The accuracy is optimal when the indentation depth is 2-2.5mm, and it maintains excellent accuracy for calibration heads of different shapes.

[0052] S3000: In one embodiment, the present invention applies a vertical displacement of 0-3mm (step size 0.5mm) to an 8 / 10 / 12 / 14mm diameter cylindrical calibration head and calculates the divergence integral value within the contact area. Simultaneously, the normal force of the force sensor used for calibration is recorded. Displaying a number. For example... Figure 7 As shown: Divergence integral With normal force The relationship is linear; and Following a unified constitutive relation:

[0053] (4)

[0054] parameter The divergence-force conversion coefficient is used to establish the shear strain divergence field. In the sensor image coordinate system of this embodiment, 33 pixels correspond to 1 mm in the world coordinate system, that is, the single pixel scale is approximately 0.03 mm. Single pixel area: The physical area of ​​a single pixel is approximately 9 × 10⁻⁶ mm. -10 m² (Calculation formula: The interpolation grid spacing is set to 10 pixels, and the physical area corresponding to each interpolation grid cell is... It is determined by the following formula: This parameter is used for area integration calculations when calculating pressure distribution.

[0055] The pressure distribution is established through the following derivation process. With divergence field Quantitative relationship between them:

[0056] ;

[0057] ;

[0058] ;

[0059] (5)

[0060] The pressure-divergence conversion coefficient It is used to establish a quantitative relationship between the divergence field and the pressure distribution. Figure 1 (c) in the figure gives the pressure distribution field transformed from the divergence field.

[0061] In one embodiment, such as Figure 8 As shown, this invention demonstrates the pressure distribution results when a visual-tactile sensor comes into contact with six different objects. Tests show that the pressure distribution exhibits a more pronounced concentration at the edges of the contact area.

[0062] In one embodiment, such as Figure 9 As shown, this invention releases a small ball freely from above the sensor, causing it to contact the black skin layer surface of the visual-touch sensor and bounce up, thereby detecting and capturing the pressure distribution data at the moment of contact in real time to verify the high dynamic response characteristics of the sensor.

[0063] In one embodiment, such as Figure 10As shown, the present invention uses a feather to contact the black skin layer surface of the visual-tactile sensor of the present invention. The sensor can accurately capture and record the minute pressure distribution changes generated during the stroking process to verify its high sensitivity detection capability for weak forces.

[0064] One example of a miniaturized application of this visual-tactile sensor is as follows: Figure 11 As shown, the visual-tactile sensor of the present invention is integrated into the robot gripper to monitor the pressure distribution changes in real time during the liquid suction operation of the squeezing dropper, and dynamically controls the opening and closing degree of the gripper based on the pressure data, so as to achieve accurate judgment and operation adjustment of the liquid volume.

[0065] like Figure 12 As shown, this invention is expected to enable dynamic pressure distribution measurement in medical monitoring equipment, allowing for dynamic measurement and visualization of the pressure distribution on the patient's body surface, providing important data support for clinical applications such as pressure ulcer prevention and rehabilitation treatment.

[0066] This invention primarily provides a method for edge detection and pressure distribution based on visual-tactile sensors. It achieves high-precision contact boundary recognition and pressure field reconstruction through a mathematical analytical method based on displacement vector operations. This invention achieves tactile perception through a purely mathematical analytical method, offering high real-time performance and good versatility. Its analytical method is applicable to hardware of different sizes and shapes, requiring only simple calibration of key parameters to adapt to various application scenarios. The method provided by this invention has the potential to help robots acquire information such as the shape and pressure distribution of contacted objects at the fingertips of robotic hands and the ends of end effectors, facilitating better interaction between these robots and their surroundings. It also holds promise for applications such as wheelchairs and hospital beds, enabling real-time detection of patient pressure distribution. The above description is merely a simple and effective implementation example of this invention.

[0067] The parts of this invention not disclosed in detail are well-known technologies in the field.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for calculating contact edge and pressure distribution based on shear strain field, characterized in that, Includes the following steps: S1000: Converts the centroid displacement field of the flexible skin markers of the visual-touch sensor into a shear strain vector field characterizing local deformation, and uses the central difference method to perform spatial differentiation on the shear strain vector field to generate a divergence field. S2000: Determine the contact boundary of the object based on the critical point of the positive and negative sources of the divergence field; perform connected component analysis on the positive sources of the divergence field; when multiple independent positive source connected regions are detected, it is determined to be a multi-object contact scene; and perform contact boundary detection on each independent positive source connected region separately. S3000: The integral of the region with positive divergence in the divergence field is used as the characteristic quantity of the divergence field. Based on the positive proportional relationship between the characteristic quantity of the divergence field and the normal force, the proportionality coefficient is determined through calibration experiments. The divergence field is divided into several divergence micro-elements. The local pressure is calculated independently according to the divergence field characteristic quantity of each divergence micro-element, and then the pressure distribution field is reconstructed. In step S3000, the mapping function between the divergence field characteristic quantity and the normal force is fitted by collecting calibration data. Includes the following steps: S3001. Control the three-dimensional displacement stage to first make the visual-tactile sensor contact the calibration head, and maintain the visual-tactile sensor and the calibration head in constant force contact in the vertical direction; S3002: Control the vertical movement of the three-dimensional displacement stage, and the calibration head causes the flexible skin to deform. Record the divergence field characteristic quantity corresponding to each position, and at the same time record the normal force reading of the force sensor on the three-dimensional displacement stage. S3003. By collecting multiple sets of divergence field feature quantities and corresponding normal forces, fit the mapping function from divergence field feature quantities to normal forces; S3004. Based on the scaling factor of the mapping function from the divergence field characteristic quantity to the normal force, each divergence element is converted into pressure.

2. The method for calculating contact edge and pressure distribution based on shear strain field as described in claim 1, characterized in that, The divergence element in S3004 refers to the divergence scalar value or average divergence value corresponding to each grid cell after dividing the flexible skin surface of the visual-touch sensor into M rows and N columns of grid cells; the mapping function is expressed as... ,in Where is the local pressure, and k is the scaling factor of the mapping function. For the scalar value or average divergence value corresponding to the grid cell, when When it is negative, Set to zero.

Images