Simulation method and device of visual tactile sensor, storage medium, program product and computing equipment

Abstract

A simulation method and device for a visual tactile sensor, a storage medium, a program product, and a computing device, the method comprising: acquiring optical simulation information and an expected value of an imaging quality index, the optical simulation information comprising: flexible layer feature information and optical element feature information; performing multiple times of optical imaging simulation based on the optical simulation information until the simulation result of the optical imaging simulation meets the expected value of the imaging quality index; when the simulation result of the optical imaging simulation meets the expected value of the imaging quality index, determining a three-dimensional model of the visual tactile sensor based on the current optical simulation information; and obtaining at least one mechanical simulation task, and for each mechanical simulation task, performing mechanical simulation on the three-dimensional model based on the material and external force loading conditions in the mechanical simulation task to obtain a mechanical simulation result of the mechanical simulation task. Accurate simulation of the visual tactile sensor can be realized, and the design efficiency of the visual tactile sensor is improved.

Description

Technical Field

[0001] This application relates to the field of sensor technology, specifically to a simulation method and apparatus for a visual-tactile sensor, a storage medium, a program product, and a computing device. Background Technology

[0002] With the rapid development of technologies such as robotics and human-computer interaction, acquiring high-precision, multi-dimensional tactile information has become one of the important ways for intelligent systems to perceive their environment. As an emerging technology that combines visual and tactile perception capabilities, the visual-tactile sensor has attracted much attention due to its simple structure, ease of integration, and ability to perceive rich information.

[0003] Currently, in the design process of visual-tactile sensors, designers typically select appropriate materials and structures based on experience, according to the application scenario's requirements for range, accuracy, etc., and produce sensor samples. Then, adjustments are made to the materials and structure of the samples based on the test results. This experience-based design is inefficient, resulting in long development cycles for visual-tactile sensors. Furthermore, it is prone to problems such as inaccurate material matching and unstable response performance, preventing visual-tactile sensors from meeting the demands of high-precision sensing. Summary of the Invention

[0004] One of the technical objectives of this application is to provide a simulation method for visual-tactile sensors, which can achieve accurate simulation of visual-tactile sensors and improve the design efficiency of visual-tactile sensors.

[0005] In a first aspect, embodiments of this application provide a simulation method for a visual-tactile sensor, characterized in that the visual-tactile sensor includes: a flexible layer and optical elements, the optical elements including a camera and a light source, the method including: acquiring optical simulation information and expected values ​​of imaging quality indicators, the optical simulation information including: flexible layer feature information and optical element feature information, the flexible layer feature information including the transparency of the flexible layer and the distribution and characteristics of markers in the flexible layer, the optical element feature information including the properties of the optical element and its layout in the visual-tactile sensor; performing multiple optical imaging simulations based on the optical simulation information until the simulation result of the optical imaging simulation is obtained. The process involves: satisfying the expected value of the imaging quality index; optimizing the optical simulation information if the expected value is not met; performing the next optical imaging simulation based on the optimized optical simulation information; determining the three-dimensional model of the visual-tactile sensor based on the current optical simulation information when the simulation result of the optical imaging simulation satisfies the expected value of the imaging quality index; acquiring at least one mechanical simulation task, each mechanical simulation task including: the material and external force loading conditions of the flexible layer; and performing a mechanical simulation on the three-dimensional model based on the material and external force loading conditions in each mechanical simulation task to obtain the mechanical simulation result of the mechanical simulation task.

[0006] Optionally, the optical imaging simulation includes: calling a pre-configured simulation file of a reference external force to simulate applying the reference external force to the flexible layer; generating a first simulation image based on the optical simulation information, wherein the first simulation image refers to the image captured by the camera on the flexible layer when the reference external force is applied to the flexible layer in the simulation; determining whether the deformation and / or displacement of the marker in the first simulation image matches the reference external force, and if the determination result is negative, optimizing the optical simulation information.

[0007] Optionally, the imaging quality index includes: target sensing area, and the optical imaging simulation includes: generating a second simulation image based on the optical simulation information; determining the effective imaging area of ​​the second simulation image, and judging whether the area of ​​the effective imaging area is greater than or equal to the target sensing area; if the judgment result is no, then optimizing the optical simulation information.

[0008] Optionally, optimizing the optical simulation information includes: using an optimization algorithm to optimize the optical simulation information.

[0009] Optionally, for each mechanical simulation task, the three-dimensional model is subjected to mechanical simulation based on the material and external force loading conditions in the mechanical simulation task, including: obtaining the constitutive model corresponding to the material; and performing finite element analysis on the three-dimensional model according to the constitutive model and the external force loading conditions to obtain the simulated stress information of the flexible layer.

[0010] Optionally, obtaining the constitutive model corresponding to the material includes: determining the constitutive model type; determining the constitutive model corresponding to the material from multiple constitutive models belonging to the constitutive model type; wherein, if the external force loading frequency in the external force loading condition is greater than or equal to a first threshold, then the constitutive model type is a hyperviscoelastic model, otherwise the constitutive model type is a viscoelastic model.

[0011] Optionally, during the finite element analysis of the three-dimensional model, the flexible layer is divided into a multi-layer array mesh, and the simulation force information of the flexible layer includes: the displacement of each mesh; for each mechanical simulation task, the three-dimensional model is mechanically simulated based on the material and external force loading conditions in the mechanical simulation task, which also includes: determining the displacement gradient of each mesh according to the displacement of the multi-layer array mesh; and determining the uniformity of the displacement field gradient of the flexible layer according to the displacement gradient of the multi-layer array mesh.

[0012] Optionally, the external force loading conditions include: loading conditions for normal force and loading conditions for tangential force, wherein the magnitude of the normal force is constant, and the magnitude of the tangential force gradually increases from zero over time, the tangential force is applied after the normal force and the tangential force continuously increases over time; finite element analysis is performed on the three-dimensional model based on the constitutive model and the external force loading conditions, including: performing finite element analysis on the three-dimensional model based on the constitutive model, the loading conditions for normal force and the loading conditions for tangential force to determine the strain energy change rate of the flexible layer after the tangential force is applied; the strain energy change rate of the flexible layer when the magnitude of the tangential force is a static friction threshold is determined as the slip detection sensitivity of the flexible layer.

[0013] Optionally, the external force loading conditions include multiple single-dimensional force loading conditions. Based on the constitutive model and the external force loading conditions, finite element analysis is performed on the three-dimensional model to obtain the simulated stress information of the flexible layer, including: for each single-dimensional force, finite element analysis is performed on the three-dimensional model based on the constitutive model and the loading conditions of that single-dimensional force to obtain the displacement field of the flexible layer when that single-dimensional force is applied independently; a force response matrix is ​​constructed, which includes the eigenvectors of the displacement field of the flexible layer under the action of each single-dimensional force; based on the force response matrix, the multiaxial load decoupling capability value of the flexible layer is determined.

[0014] Secondly, embodiments of this application provide a simulation device for a visual-tactile sensor. The device includes: a first acquisition module, configured to acquire optical simulation information and expected values ​​of imaging quality indicators. The optical simulation information includes flexible layer feature information and optical element feature information. The flexible layer feature information includes the transparency of the flexible layer and the distribution and characteristics of markers within the flexible layer. The optical element feature information includes the properties of the optical element and its layout within the visual-tactile sensor. An optical simulation module is configured to perform multiple optical imaging simulations based on the optical simulation information until the simulation result of the optical imaging simulation meets the expected value of the imaging quality indicator. Wherein, if the imaging quality indicator is not met, the simulation result is not considered. The optical simulation information is optimized to meet the expected value of the quality index, and the next optical imaging simulation is performed based on the optimized optical simulation information; a three-dimensional model determination module is used to determine the three-dimensional model of the visual-touch sensor based on the current optical simulation information when the simulation result of the optical imaging simulation meets the expected value of the imaging quality index; a second acquisition module is used to acquire at least one mechanical simulation task, each mechanical simulation task including: the material of the flexible layer and the external force loading conditions; a mechanical simulation module is used to perform mechanical simulation on the three-dimensional model based on the material and external force loading conditions in the mechanical simulation task for each mechanical simulation task, and obtain the mechanical simulation result of the mechanical simulation task.

[0015] Thirdly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when run by a processor, performs the steps of the simulation method for the visual-tactile sensor provided in the first aspect.

[0016] Fourthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the simulation method for the visual-tactile sensor provided in the first aspect.

[0017] Fifthly, embodiments of this application also provide a computing device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the steps of the simulation method for a visual-tactile sensor provided in the first aspect when running the computer program.

[0018] Compared with the prior art, the technical solution of this application embodiment has the following beneficial effects:

[0019] This application embodiment considers that the visual-tactile sensor does not directly output mechanical signals (such as displacement, strain, etc.), but indirectly infers tactile information (such as force, shape, texture, slip, etc.) through captured optical images. The same mechanical deformation will produce drastically different images under different optical conditions. Therefore, in the solution of this application embodiment, the optical configuration of the visual-tactile sensor is first determined through optical simulation optimization, then a three-dimensional model of the visual-tactile sensor is generated based on the determined optical configuration, and mechanical simulation is performed based on the generated three-dimensional model to obtain the mechanical performance indicators of the visual-tactile sensor, so that subsequent design optimization can be carried out based on the mechanical performance indicators obtained from the simulation. By adopting the above solution, fixing the optical configuration of the visual-tactile sensor through optical simulation first, the mechanical response of the flexible layer obtained in the subsequent mechanical simulation process can be closer to the mechanical response reflected by the optical image captured by the camera in the visual-tactile sensor. Therefore, the obtained mechanical simulation results are more realistic and reliable, which helps to ensure the accuracy of the simulation results, thereby improving the development efficiency of the visual-tactile sensor.

[0020] Furthermore, in the scheme of this application embodiment, a pre-configured simulation file of the reference external force is called during the optical simulation process to simulate the application of the reference external force to the flexible layer. Then, the deformation of the marker in the obtained first simulation image is used to determine whether the imaging quality index is met. By adopting the above scheme, it is beneficial to ensure that the optical configuration in the simulated visual-tactile sensor can ensure that when the flexible layer undergoes a small deformation due to the reference external force, the camera can capture clear and easily identifiable visual changes.

[0021] Furthermore, in the scheme of this application embodiment, the uniformity of the displacement field gradient is calculated during the mechanical simulation process in order to accurately measure whether the deformation of the surface of the flexible layer can spread smoothly when it is subjected to force. This is beneficial to ensure that the external force of the designed visual-tactile sensor will be uniformly distributed on the surface of the flexible layer during use.

[0022] Furthermore, in the scheme of this application embodiment, the multi-axis load decoupling capability value is calculated during the mechanical simulation process in order to accurately measure the multi-dimensional force decoupling capability of the visual-tactile sensor. This helps to ensure that the designed visual-tactile sensor can accurately separate and independently measure the forces in each dimension during use, thus ensuring the accuracy and reliability of the measurement results.

[0023] Furthermore, in the embodiments of this application, the slip detection sensitivity is calculated during the mechanical simulation process in order to accurately measure the response capability of the visual-tactile sensor to small tangential force changes or high-frequency vibrations while maintaining a certain normal force, thereby helping to enable the designed visual-tactile sensor to have better slip detection capability. Attached Figure Description

[0024] Figure 1This is a schematic diagram of the structure of a visual-tactile sensor;

[0025] Figure 2 This is a flowchart illustrating a simulation method for a visual-tactile sensor according to an embodiment of this application.

[0026] Figure 3 This is a schematic diagram of the structure of a simulation device for a visual-tactile sensor according to an embodiment of this application. Detailed Implementation

[0027] A visual-tactile sensor is a device that captures visual information using visual means and calculates tactile information from that visual information. In related technologies, visual-tactile sensors typically use a transparent, flexible material as the force-bearing object, and embed a set of markers on or within the surface of this material. When an external force is applied to the sensor surface, a camera tracks the displacement of these markers, and the sensor perceives and estimates the external force based on the tracked displacement.

[0028] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a visual-tactile sensor. (Example) Figure 1 As shown, the visual-tactile sensor 100 may include: a flexible layer 10, a marker 20, a camera 30, a light source 40, a housing 50, and a support layer 60.

[0029] The housing 50 provides mechanical support for the various components inside the visual-tactile sensor 100, thereby providing a stable working platform for the visual-tactile sensor 100. In specific implementations, the material of the housing 50 can be engineering plastics, metals, etc., but is not limited to these, and this document does not limit it.

[0030] Furthermore, the housing 50 has an opening, and the flexible layer 10 is disposed at the opening of the housing 50, so that the outer surface of the flexible layer 10 is directly exposed to the external environment, so that the flexible layer 10 can come into contact with an object. For example, the flexible layer 10 can be directly encapsulated or fixed to the opening of the housing 50 by means of adhesive, but is not limited thereto.

[0031] In practical implementation, the flexible layer 10 can be transparent and elastic. That is, the flexible layer 10 can be a transparent elastomer. It should be noted that this paper does not limit the shape of the flexible layer 10. The flexible layer 10 can be pointed, planar, curved (such as spherical), tubular, balloon-shaped, irregular curved, etc., to adapt to different robot end effectors or meet the needs of application scenarios.

[0032] Furthermore, a marker 20 is provided within the flexible layer 10. The marker 20 can refer to an object used for optical tracking that can undergo identifiable displacement and / or deformation when the flexible layer 10 is subjected to force. The "marker" can be a pattern, structure, component, etc. on the flexible layer 10, and this application is not limited thereto.

[0033] Furthermore, the camera 30 can be disposed at the bottom of the housing 50, and the camera 30 is used to acquire images of the flexible layer 10. In a specific implementation, the camera may include an image sensor and a lens, wherein the image sensor may be a complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) sensor, but is not limited to these.

[0034] Furthermore, such as Figure 1 As shown, the visual-tactile sensor 100 may also include a light source 40, which can be used to create a uniform optical environment for the visual sensor 100 to improve the image quality acquired by the camera 30. Specifically, the light source 40 can provide illumination for image acquisition by the camera 30. For example, the light source 40 can be used to illuminate the marker 20 in the flexible layer 10.

[0035] In some embodiments, the visual-touch sensor may further include a support layer 60 located between the flexible layer 10 and the camera 30, which can be used to adjust the overall stress transmission. In specific implementations, the support layer 60 may be transparent. Exemplarily, the support layer 60 may be a highly transparent silicone or acrylic or optical-grade silicon wafer, or a polyethylene terephthalate film, flexible printed circuit board, polyimide film, etc., but is not limited thereto.

[0036] In practical applications, visual-tactile sensors can be used in various scenarios such as medical rehabilitation, wearable sensing, and industrial grasping. Different application scenarios have different requirements for the mechanical properties of visual-tactile sensors, including range, accuracy, and sensitivity. Currently, sensors are typically designed by engineers. However, this method is inefficient, leading to long development cycles for visual-tactile sensors. Furthermore, it suffers from problems such as inaccurate material matching and unstable response performance, preventing visual-tactile sensors from meeting the demands of high-precision sensing.

[0037] This application provides an automated simulation method for visual-tactile sensors, which can perform realistic and reliable simulations of visual-tactile sensors, thereby improving the research and development efficiency of visual-tactile sensors.

[0038] Specifically, this application embodiment considers that the visual-tactile sensor does not directly output mechanical signals (such as displacement, strain, etc.), but indirectly infers tactile information (such as force, shape, texture, slip, etc.) through captured optical images. The same mechanical deformation will produce drastically different images under different optical conditions. Therefore, in the solution of this application embodiment, the optical configuration of the visual-tactile sensor is first determined through optical simulation optimization, then a three-dimensional model of the visual-tactile sensor is generated based on the determined optical configuration, and mechanical simulation is performed based on the generated three-dimensional model to obtain the mechanical performance indicators of the visual-tactile sensor, so that subsequent design optimization can be carried out based on the mechanical performance indicators obtained from the simulation. By adopting the above solution, fixing the optical configuration of the visual-tactile sensor through optical simulation first, the mechanical response of the flexible layer obtained in the subsequent mechanical simulation process can be closer to the mechanical response reflected by the optical image captured by the camera in the visual-tactile sensor. Therefore, the obtained mechanical simulation results are more realistic and reliable, which helps to ensure the accuracy of the simulation results.

[0039] It should be noted that the solutions provided in the embodiments of this application can be used to design, simulate, and optimize the visual-touch sensor before its manufacture. Through the solutions provided in the embodiments of this application, it is possible to determine... Figure 1 The structure and materials of the haptic sensor 100 are described. For example, the solution provided in this application embodiment can determine one or more of the following: the dimensions (e.g., thickness) of the flexible layer 10, the material of the flexible layer, the distribution of the marker 20 in the flexible layer 10, the properties of the camera 30, the position of the camera 30, the number of light sources 40, the model of the light sources 40, the position of the light sources 40, the dimensions of the housing 50, and the dimensions of the support layer 60. Compared to traditional methods, this effectively avoids the design path of relying on experience and repeated trial and error, significantly improving sensor development efficiency and performance matching.

[0040] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0041] Reference Figure 2 , Figure 2 This is a flowchart illustrating a simulation method for a visual-tactile sensor according to an embodiment of this application. Figure 2 The illustrated scheme can be executed by a computing device, which can be any suitable device with data receiving and data processing capabilities, such as a computer, tablet computer, etc. The embodiments of this application do not limit this. Figure 2 The methods shown may include S21, S22, S23 and S24.

[0042] S21, obtain the expected values ​​of optical simulation information and imaging quality indicators;

[0043] S22, Perform multiple optical imaging simulations based on the optical simulation information until the simulation results of the optical imaging simulation meet the expected value of the imaging quality index.

[0044] S23, when the simulation results of the optical imaging simulation meet the expected value of the imaging quality index, the three-dimensional model of the visual-tactile sensor is determined based on the current optical simulation information;

[0045] S24, Obtain at least one mechanical simulation task, each mechanical simulation task including: the material of the flexible layer and the external force loading conditions;

[0046] S25, For each mechanical simulation task, perform mechanical simulation on the three-dimensional model based on the material and external force loading conditions in the mechanical simulation task to obtain the mechanical simulation results of the mechanical simulation task.

[0047] In S21, optical simulation information and expected values ​​of imaging quality indicators can be obtained.

[0048] In practice, the expected values ​​of optical simulation information and imaging quality indicators can be input by the user or pre-configured. For example, the user can input the expected values ​​of optical simulation information and imaging quality indicators on a visual interactive interface.

[0049] Specifically, optical simulation information may include: flexible layer feature information and optical element feature information. Furthermore, optical simulation information may also include: the housing dimensions of the haptic sensor. In this paper, "optical elements" may include a camera and a light source.

[0050] The flexible layer feature information may include, but is not limited to, the transparency of the flexible layer, the distribution of markers in the flexible layer, and the characteristics of the markers.

[0051] The optical element characteristic information may include the attributes of the optical element. For example, the attributes of the optical element may include at least one of the following: camera model, camera parameters (such as signal strength, field of view, image sensor size, etc.), light source type, and light intensity. In one example, a camera model library can be pre-configured, from which the user can select a target model.

[0052] Furthermore, the optical element feature information may also include: the layout of the optical elements in the visual-touch sensor. Specifically, the layout of the optical elements in the visual-touch sensor may include one or more of the following: the number of cameras, the position of each camera in the visual-touch sensor, the relative positional relationship between the cameras and the flexible layer, the number of light sources, the position of each light source, the illumination direction of each light source, the relative positional relationship between each light source and the flexible layer, and the relative positional relationship between each light source and the cameras.

[0053] For example, the relative positional relationship between the camera and the flexible layer can be the distance between the camera and the flexible layer, or the allowable distance range between the camera and the flexible layer.

[0054] Furthermore, multiple imaging quality indicators can be pre-configured, and users can set the expected values ​​for multiple imaging quality indicators.

[0055] For example, imaging quality metrics may include at least one of the following: deformation feature contrast, target sensing area, illumination uniformity, and image resolution. Deformation feature contrast can refer to the magnitude of deformation of the marker in the image when a reference external force is applied to the flexible layer. Target sensing area can refer to the area of ​​the region on the flexible layer capable of responding to external forces. Illumination uniformity can refer to the uniformity of illuminance formed by the light source on the inner surface of the flexible layer.

[0056] It should be noted that, in the embodiments of this application, the deformation of the marker can also be understood as the displacement of the marker.

[0057] In S22, optical imaging simulation is performed through multiple iterations based on optical simulation information. Specifically, in S22, the expected value of the imaging quality index in S21 can be used as the target information to optimize and iterate the optical simulation information in S21.

[0058] More specifically, the optical simulation information in S21 can be used as the initial optical simulation information for the first optical imaging simulation. In each optical imaging simulation, if the simulation result does not meet the expected value of the imaging quality index, the optical simulation information is optimized, and then the next optical imaging simulation is performed based on the optimized optical simulation information. If the simulation result meets the expected value of the imaging quality index, then S23 can continue to be executed.

[0059] For example, optimization algorithms can be used to optimize optical simulation information. These algorithms could include genetic algorithms, particle swarm optimization algorithms, or Bayesian optimization algorithms.

[0060] The process of single-shot optical imaging simulation is described in detail below.

[0061] Specifically, an optical simulation engine can be invoked to generate a simulated image based on the current optical simulation information. This simulated image is obtained by simulating the image captured by a camera on the flexible layer. For example, the optical simulation engine can be any tool capable of optical simulation, such as the Zemax engine or LightTools; this application embodiment does not limit this. Furthermore, the actual value of the imaging quality index can be calculated based on the simulated image, and then compared with the expected value to determine whether the simulation result of the optical simulation information meets the expected value of the imaging quality index.

[0062] In one possible implementation, before invoking the optical simulation engine, a pre-configured simulation file of a reference external force can be invoked to simulate the application of a reference external force to the flexible layer. The reference external force can be normal pressure, shear force, etc., and this embodiment does not limit the type of reference external force. In specific implementations, the magnitude of the reference external force is less than or equal to a preset external force threshold. That is, the reference external force is a small external force, causing minor deformation and / or displacement of the markers in the flexible layer.

[0063] Furthermore, the optical simulation engine is invoked to generate the first simulated image. That is, the first simulated image is a simulated image captured by the camera on the flexible layer when an object applies a reference external force to the flexible layer in the visual-tactile sensor based on the current optical simulation information.

[0064] Furthermore, the deformation of the marker in the first simulation image is calculated based on the first simulation image, and it is determined whether the deformation of the marker in the first simulation image matches the reference external force. If the determination result is negative, it is determined that the simulation result of the current optical simulation information does not meet the deformation characteristic contrast.

[0065] For example, if the deformation of the marker in the first simulated image is greater than or equal to the expected value of the deformation feature contrast, it can be determined that the deformation of the marker in the first simulated image matches the reference external force; otherwise, it is determined that the deformation of the marker in the first simulated image does not match the reference external force. When the deformation of the marker in the first simulated image matches the reference external force, it means that the current optical simulation information can ensure that when the flexible layer undergoes a small deformation under the reference external force, the camera can capture a clear and easily identifiable visual change.

[0066] In the above scheme, using the deformation feature contrast as an imaging quality indicator is beneficial to ensure that the optical configuration in the simulated visual-touch sensor can ensure that when the flexible layer is subjected to a reference external force and undergoes a small deformation, the camera can capture a clear and easily identifiable visual change.

[0067] In another possible implementation, an optical simulation engine can be invoked to generate a second simulated image based on the current optical simulation information. Further, the effective imaging region of the second simulated image can be determined, and it can be judged whether the area of ​​the effective imaging region is greater than or equal to the target sensing area. If the judgment result is negative, it can be determined that the simulation result of the current optical simulation information does not meet the imaging quality index. The effective imaging region of the second simulated image can be determined based on various appropriate image processing algorithms, and this embodiment is not limited to this.

[0068] In other embodiments of this application, illumination uniformity can also be determined based on the generated simulation image. Specifically, the illumination intensity of the light source at various locations on the flexible layer can be determined based on the pixel values ​​of each pixel in the simulation image, thereby determining the uniformity of the illumination on the inner surface of the flexible layer. Further, if the illumination uniformity determined based on the simulation image is less than the expected value of the illumination uniformity obtained in S11, it indicates that the current optical simulation information cannot meet the imaging requirements of the visual-touch sensor. This approach helps to ensure that the optical configuration in the simulated visual-touch sensor avoids local overexposure or underexposure.

[0069] Therefore, in S22, starting with the optical simulation information input by the user, and with the expected value of the imaging quality index as the optimization target, automatic optimization is performed to obtain the optimal optical configuration in the visual-touch sensor.

[0070] In one example, when the simulation results of the optical imaging simulation meet the imaging quality indicators, an optical performance report can also be generated. In the optical performance report, the illumination effect of the light source and the imaging effect of the camera can be displayed to the user in the form of pseudo-color images, comparison images, etc.

[0071] In S23, when the simulation results of the optical imaging simulation meet the imaging quality index, a three-dimensional model of the visual-tactile sensor is generated based on the current optical simulation information.

[0072] In practical implementation, the optical configuration of the visual-tactile sensor can be determined through multiple iterations of optical simulation in S22. For example, the dimensions of the flexible layer, its transparency, the distribution of markers within the flexible layer, and the housing dimensions of the visual-tactile sensor can also be determined. Furthermore, the structure and configuration of the visual-tactile sensor determined by the optical simulation are transferred to subsequent mechanical simulation processes.

[0073] Furthermore, after generating the 3D model, a finite element model can be generated based on the 3D model for finite element analysis in S25. More specifically, the force transmission can be defined by determining the contact relationships between the components in the visual-tactile sensor. Additionally, the 3D model can be meshed into multiple meshes arranged in a multi-layered array. The mesh simulates the smallest force-bearing unit in an actual sensor. The contact relationships and mesh density can be pre-configured or set by the user.

[0074] In S24, the user can input mechanical parameter information for the mechanical simulation task. For example, the user can input the mechanical parameter information for the mechanical simulation task through a visual interactive interface.

[0075] Specifically, the mechanical parameter information for the mechanical simulation task may include the material of the flexible layer. For example, a material library may be pre-configured, from which the user can select one or more candidate materials. For example, materials in the material library may include: transparent silicone, polyurethane elastomer, transparent hydrogel, special transparent rubber, or other optically transparent polymer materials.

[0076] Furthermore, the mechanical parameter information for the mechanical simulation task can also include external force loading conditions. Specifically, external force loading conditions can include: load type and load magnitude. That is, the external force acting on the flexible layer can be defined through the external force loading conditions. The load type can be any of the following: normal loading, shear loading, dynamic loading, periodic loading, simultaneous loading at multiple contact points, etc. Dynamic loading and periodic loading can be used to simulate the flexible layer's response to vibration or frequency, while simultaneous loading at multiple contact points can be used to simulate the flexible layer's response to multi-finger pressure. In practical implementation, users can determine the load type in the mechanical simulation task based on the application scenario of the visual-tactile sensor. By setting multiple loading types, the mechanical performance of the visual-tactile sensor under various real-world force conditions (such as normal force, shear force, point pressure, surface pressure, mixed loading, etc.) can be simulated.

[0077] Furthermore, the external force loading conditions can also include boundary conditions. These boundary conditions can be constraints imposed by the applied external force.

[0078] In S25, mechanical simulations are performed for each mechanical simulation task. Specifically, finite element analysis is used to perform mechanical simulations for each task.

[0079] Specifically, the constitutive model corresponding to the flexible layer material can be obtained; then, based on the constitutive model corresponding to the flexible layer material and the external force loading conditions, finite element analysis is performed on the three-dimensional model of the visual-tactile sensor to obtain the simulated force information of the flexible layer.

[0080] In practice, for each material in the material library, a constitutive model corresponding to that material can be pre-configured. After obtaining the material selected by the user, the constitutive model corresponding to that material can be called.

[0081] In one possible implementation, each material can have multiple types of constitutive models. In practice, the constitutive model type can be determined first, and then the corresponding constitutive model for that material can be determined.

[0082] For example, various types of constitutive models may include: hyperelastic models, viscoelastic models, and hyperviscoelastic models.

[0083] In one example, the external force loading condition may include the load frequency. If the load frequency is greater than or equal to a preset frequency threshold, the constitutive model type can be determined to be a viscoelastic model or a hyperviscoelastic model; otherwise, the constitutive model type is determined to be a hyperelastic model.

[0084] Since hyperelastic models primarily describe the stress-strain relationship of materials under quasi-static loading, flexible materials exhibit viscoelastic effects such as stress relaxation and creep in dynamic loading scenarios like robot interaction and texture perception. Therefore, hyperelastic models cannot accurately characterize the relaxation behavior of flexible materials in dynamic responses. Viscoelastic or hyperviscoelastic models, by adding time-dependent terms to the hyperelastic model (e.g., using Proni series to define the relaxation behavior), can simulate and quantify the frequency response characteristics of flexible materials. Therefore, using viscoelastic or hyperviscoelastic models as constitutive models for simulation can predict the response amplitude and phase delay of the flexible layer of a visual-tactile sensor at different vibration frequencies, enabling the selection or design of materials and structures with the highest sensitivity within the target frequency band.

[0085] Furthermore, for each mechanical simulation task, the constitutive model of the material, the external force loading conditions, and the finite element model of the visual-tactile sensor can be input into a finite element solver (such as Abaqus) to obtain the simulated stress information of the flexible layer corresponding to the mechanical simulation task. Specifically, the simulated stress information of the flexible layer can include the stress information of each mesh of the flexible layer, wherein the stress information can include at least one of the following: displacement, stress, and strain.

[0086] Furthermore, for each mechanical simulation task, the actual values ​​of the mechanical performance indicators corresponding to the simulation task can be calculated based on the force information of each grid in the flexible layer.

[0087] For example, mechanical performance indicators may include: range. Specifically, the stress-strain curve of the flexible layer can be determined based on the stress information of the flexible layer. Furthermore, the actual value of the range can be determined based on the stress corresponding to the linear segment in the stress-strain curve.

[0088] For example, mechanical performance indicators may include sensitivity or Young's modulus. Sensitivity can be the slope of the linear segment of the stress-strain curve, i.e., Young's modulus. Thus, the actual value of the sensitivity can be determined.

[0089] For example, mechanical performance indicators may include displacement field gradient uniformity. For instance, if a visual-tactile sensor is used in scenarios requiring high spatial resolution or where it is necessary to avoid local signal saturation (such as pressure distribution imaging or complex shape recognition), displacement field gradient uniformity can be used as a mechanical performance indicator.

[0090] In practical implementation, the displacement field of the flexible layer can be determined first, and then the displacement field gradient of the flexible layer can be calculated. The uniformity of the displacement field gradient can then be determined based on this gradient. The displacement field of the flexible layer can include the displacements of multiple grids within the flexible layer, and the displacement field gradient can also include the displacement gradients of these multiple grids. These multiple grids are arranged in a multi-layer array. The displacement here can be a displacement in the normal direction. For example, the displacement of a grid can be represented as Uz(x,y), and the displacement gradient of the grid can be represented as ||Uz(x,y)|. Furthermore, the uniformity of the displacement field gradient can be determined based on the displacement gradients of the multiple grids within the flexible layer. In one example, the uniformity of the displacement field gradient can be determined based on the displacement gradients of multiple grids within the target sensing region of the flexible layer. For example, the uniformity of the displacement field gradient can be the standard deviation or coefficient of variation of the displacement gradients of the multiple grids. A smaller value for the uniformity of the displacement field gradient indicates a more uniform displacement gradient, a more dispersed stress distribution, and better mechanical properties of the flexible layer.

[0091] In the above scheme, the uniformity of the displacement field gradient is used as a mechanical performance index. This can accurately measure whether the deformation of the surface of the flexible layer can spread smoothly when it is under stress, rather than being concentrated at a certain point. This helps to ensure that the external force of the designed visual-touch sensor will be evenly distributed on the surface of the flexible layer during use.

[0092] For example, mechanical performance indicators may include multi-axis load decoupling capability values. Specifically, when visual-tactile sensors are used in dexterity applications, they need to simultaneously sense forces in multiple directions (such as normal and shear forces). Therefore, during mechanical calculations in the use of visual-tactile sensors, the forces in multiple directions need to be separated and calculated independently. To ensure the accuracy of the force calculation results, the visual-tactile sensor needs to have good multi-dimensional force decoupling capability.

[0093] Therefore, the external force loading conditions for mechanical simulation tasks can include loading conditions for multiple single-dimensional forces. For each single-dimensional force, a finite element analysis can be performed on the three-dimensional model of the visual-tactile sensor based on the constitutive model and the loading conditions of that single-dimensional force to obtain the displacement field of the flexible layer when that single-dimensional force is applied independently. The multiple single-dimensional forces can include any of the following: unit normal force, unit first shear force, unit second shear force, and unit normal torque. The directions of the first shear force and the second shear force are perpendicular to each other, and both the directions of the first shear force and the second shear force are perpendicular to the normal.

[0094] Furthermore, a force response matrix is ​​constructed, which includes the eigenvectors of the displacement field of the flexible layer under the action of various single-dimensional forces. The eigenvectors of the displacement field can be obtained by dimensionality reduction of the displacement field using principal component analysis.

[0095] Furthermore, the condition number of the force response matrix or the cosine similarity between row vectors can be used as a value for multi-axis load decoupling capability. A smaller multi-axis load decoupling capability value indicates better multi-axis load decoupling capability of the visual-tactile sensor.

[0096] In the above scheme, the multi-axis load decoupling capability value is used as a mechanical performance index, which enables the visual tactile sensor to have a better ability to decouple multi-dimensional forces. This allows for the precise separation and independent measurement of forces in each dimension during use, ensuring the accuracy and reliability of the measurement results.

[0097] For example, mechanical performance indicators may include slip detection sensitivity. Specifically, when a visual-haptic sensor is applied to a dexterous hand, in order for the dexterous hand to stably grasp objects, the visual-haptic sensor needs to detect the tendency to slide before the object begins to slide.

[0098] In view of this, in the scheme of the present application embodiment, the external force loading conditions include: normal force loading conditions and tangential force loading conditions, wherein the magnitude of the normal force is constant, and the magnitude of the tangential force gradually increases from zero over time or the tangential force is a high-frequency oscillating tangential force.

[0099] Furthermore, finite element analysis can be performed on the three-dimensional model based on the constitutive model, the loading conditions of the normal force, and the loading conditions of the tangential force to determine the strain energy change rate of the flexible layer. That is, the strain energy change rate of the flexible layer is determined during the application of the tangential force. Further, as the magnitude of the tangential force gradually increases, the strain energy change rate of the flexible layer when the magnitude of the tangential force is equal to the static friction threshold can be used as the slip detection sensitivity. A higher slip detection sensitivity indicates that the visual-tactile sensor can more easily detect impending slippage, and the slip detection performance is better. In other examples, the strain energy change rate of the flexible layer can also be replaced by the amplitude of a high-frequency stress / strain signal.

[0100] In the above scheme, the slip detection sensitivity is used as a mechanical performance index, which can accurately measure the response capability of the visual tactile sensor to small tangential force changes or high-frequency vibrations while maintaining a certain normal force. This is beneficial to enable the designed visual tactile sensor to have better slip detection capability.

[0101] In one embodiment of this application, the visual-tactile sensor will be subjected to tens of thousands or even millions of cyclic loading cycles during long-term use. This can lead to degradation of the material properties of the flexible layer (such as softening and stiffness changes) or microcrack propagation, ultimately affecting the accuracy and lifespan of the visual-tactile sensor. Therefore, in addition to the constitutive model, one embodiment of this application may also pre-configure a fatigue damage model for each material. The fatigue damage model can be pre-calibrated. For example, the fatigue damage model can be a fatigue life criterion based on the maximum principal strain amplitude.

[0102] Furthermore, the external force loading condition can be cyclic loading. Specifically, the external force loading condition defines the cyclic application of force to simulate the process of repeatedly applying pressure and then releasing it onto the flexible layer. Further, through finite element analysis, the strain of each grid in the flexible layer can be determined each time force is applied. Therefore, the damage value of each grid can be determined based on the change in strain of each grid over the number of cycles. Furthermore, indication information of fatigue-prone grids can be output to the user so that the user can reinforce these grids. For example, the thickness at the fatigue-prone grid can be increased. Here, fatigue-prone grids can be grids whose damage value is greater than or equal to a preset damage threshold. Using the above scheme, the long-term reliability simulation of the visual-tactile sensor can be achieved, which is beneficial for extending the sensor's service life.

[0103] In S25, the mechanical simulation results of each mechanical simulation task can be output, or the mechanical performance indicators obtained based on the mechanical simulation results can be output. Alternatively, the mechanical parameter information in the acquired mechanical simulation task can be iteratively optimized, and the optimized mechanical parameter information can be output.

[0104] In one embodiment of this application, multiple iterative optimizations can be performed based on the mechanical parameter information in the user-input mechanical simulation task. Specifically, the user-input mechanical parameter information can be used as the initial mechanical simulation task for the first mechanical simulation. In each mechanical simulation, if the actual value of the mechanical performance index obtained from the simulation does not meet the expected value, the current mechanical parameter information is optimized, and then the next mechanical simulation is performed based on the optimized mechanical parameter information. If the actual value of the mechanical performance index obtained from the simulation meets the expected value, the final visual-tactile sensor can be obtained based on the current mechanical parameter information. Optimizing the current mechanical parameter information can refer to optimizing the material of the flexible layer, the thickness of the flexible layer, etc., but is not limited to these. For example, an optimization algorithm can be used to optimize the current mechanical parameter information.

[0105] In another embodiment of this application, a user can create multiple mechanical simulation tasks, perform mechanical simulations based on each task, and obtain a sensor mechanical performance profile corresponding to that task. The sensor mechanical performance profile may include actual values ​​of multiple mechanical performance indicators. Furthermore, the sensor mechanical performance profiles for each mechanical simulation task can be output to the user.

[0106] For example, while outputting sensor mechanical performance profiles for various mechanical simulation tasks, the system can also indicate recommended mechanical parameter information to the user.

[0107] In another embodiment of this application, a mapping relationship between mechanical parameter information and mechanical performance indicators can also be established. For example, a machine learning model can be used to fit the mapping relationship between mechanical parameter information and mechanical performance indicators. As another example, interpolation methods can be used to fit the mapping relationship between mechanical parameter information and mechanical performance indicators.

[0108] From the above, Figure 2 In the proposed scheme, the optimal layout and parameters of the internal optical components of the sensor are determined through optical simulation before mechanical simulation. This provides the optimal optical configuration for subsequent mechanical analysis, making the mechanical response of the flexible layer obtained during mechanical simulation closer to the mechanical response reflected by the optical image captured by the camera in the visual-touch sensor. This ensures the accuracy of the simulation results and truly realizes the full-link design and simulation from "vision" to "touch", forming a complete closed-loop optimization design scheme for visual-touch sensors.

[0109] Furthermore, in the solution of this application embodiment, the user only needs to input optical simulation information, mechanical parameter information, etc., without having to manually create a finite element model. The modeling, simulation and analysis process is integrated in the background, which reduces the professional requirements for sensor designers, simplifies the operation process of designers, and helps to improve the development efficiency of sensors.

[0110] It should be noted that the various embodiments, possible implementations, possible methods of implementation, various examples, and solutions in various situations provided in this article can be used individually or in combination to achieve different technical effects.

[0111] It is understandable that, in specific implementation, the above method can be implemented using a software program that runs in the processor integrated inside the chip or chip module; or, the method can be implemented using hardware or a combination of hardware and software, such as using a dedicated chip or chip module, or using a dedicated chip or chip module combined with a software program.

[0112] Reference Figure 3 , Figure 3 This application provides a simulation device for a visual-tactile sensor, comprising:

[0113] The first acquisition module 31 is used to acquire optical simulation information and expected values ​​of imaging quality indicators. The optical simulation information includes: flexible layer feature information and optical element feature information. The flexible layer feature information includes the transparency of the flexible layer and the distribution and characteristics of markers in the flexible layer. The optical element feature information includes the properties of the optical element and its layout in the visual-tactile sensor.

[0114] The optical simulation module 32 is used to perform multiple optical imaging simulations based on the optical simulation information until the simulation result of the optical imaging simulation meets the expected value of the imaging quality index. In the case that the expected value of the imaging quality index is not met, the optical simulation information is optimized, and the next optical imaging simulation is performed based on the optimized optical simulation information.

[0115] The model generation module 33 is used to determine the three-dimensional model of the visual-tactile sensor based on the current optical simulation information when the simulation result of the optical imaging simulation meets the expected value of the imaging quality index.

[0116] The second acquisition module 34 is used to acquire at least one mechanical simulation task, each mechanical simulation task including: the material of the flexible layer and the external force loading conditions;

[0117] The mechanical simulation module 34 is used to perform mechanical simulation on the three-dimensional model based on the material and external force loading conditions in each mechanical simulation task, and obtain the mechanical simulation results of the mechanical simulation task.

[0118] For more information on the working principle, working method, and beneficial effects of the simulation device in the embodiments of this application, please refer to the relevant description of the simulation device above, which will not be repeated here.

[0119] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is run by a computer, the above-described method is executed. The storage medium may include read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. The storage medium may also include non-volatile memory or non-transitory memory, etc.

[0120] This application also provides a computing device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of the method described above when running the computer program.

[0121] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0122] It should be understood that in the embodiments of this application, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0123] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0124] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0126] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or in a combination of hardware and software functional units. For example, for various devices or products applied to or integrated into a chip, each module / unit can be implemented using hardware such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware such as circuits; for various devices or products applied to or integrated into a chip module, each module / unit can be implemented using hardware such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0127] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0128] In this application's embodiments, "multiple" refers to two or more. The descriptions of "first," "second," etc., appearing in this application's embodiments are merely illustrative and for distinguishing the described objects; they have no order and do not indicate a specific limitation on the number of devices in this application's embodiments, nor do they constitute any limitation on the embodiments of this application.

[0129] While this application discloses the above information, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application shall be determined by the scope defined in the claims.

Claims

1. A simulation method for a visual-tactile sensor, characterized in that, The visual-tactile sensor includes a flexible layer and optical elements, the optical elements including a camera and a light source, and the method includes: The optical simulation information and the expected values ​​of the imaging quality indicators are obtained. The optical simulation information includes: flexible layer feature information and optical element feature information. The flexible layer feature information includes the transparency of the flexible layer and the distribution and characteristics of the markers in the flexible layer. The optical element feature information includes the properties of the optical element and its layout in the visual-tactile sensor. Multiple optical imaging simulations are performed based on the optical simulation information until the simulation results meet the expected value of the imaging quality index. If the expected value of the imaging quality index is not met, the optical simulation information is optimized, and the next optical imaging simulation is performed based on the optimized optical simulation information. When the simulation results of the optical imaging simulation meet the expected value of the imaging quality index, the three-dimensional model of the visual-tactile sensor is determined based on the current optical simulation information. Obtain at least one mechanical simulation task, each mechanical simulation task including: the material of the flexible layer and the external force loading conditions; For each mechanical simulation task, the three-dimensional model is subjected to mechanical simulation based on the material and external force loading conditions in the mechanical simulation task to obtain the mechanical simulation results of the mechanical simulation task.

2. The simulation method according to claim 1, characterized in that, The optical imaging simulation includes: Call the pre-configured simulation file of the reference external force to simulate the application of the reference external force to the flexible layer; A first simulation image is generated based on the optical simulation information. The first simulation image refers to the image captured by the camera on the flexible layer when the reference external force is applied to the flexible layer in a simulation. Determine whether the deformation and / or displacement of the marker in the first simulation image matches the reference external force. If the determination result is negative, optimize the optical simulation information.

3. The simulation method according to claim 1, characterized in that, The imaging quality metrics include: target sensing area; the optical imaging simulation includes: A second simulated image is generated based on the optical simulation information; The effective imaging area of ​​the second simulated image is determined, and it is determined whether the area of ​​the effective imaging area is greater than or equal to the target sensing area. If the determination result is not, the optical simulation information is optimized.

4. The simulation method according to claim 1, characterized in that, Optimizing the optical simulation information includes: The optical simulation information is optimized using an optimization algorithm.

5. The simulation method according to claim 1, characterized in that, For each mechanical simulation task, a mechanical simulation is performed on the three-dimensional model based on the material and external force loading conditions in that task, including: Obtain the constitutive model corresponding to the material; Based on the constitutive model and the external force loading conditions, finite element analysis is performed on the three-dimensional model to obtain the simulated stress information of the flexible layer.

6. The simulation method according to claim 5, characterized in that, Obtaining the constitutive model corresponding to the material includes: Determine the constitutive model type; Determine the constitutive model corresponding to the material from multiple constitutive models belonging to the constitutive model type; Wherein, if the external force loading frequency in the external force loading condition is greater than or equal to the first threshold, the constitutive model type is a hyperviscoelastic model; otherwise, the constitutive model type is a viscoelastic model.

7. The simulation method according to claim 5, characterized in that, During the finite element analysis of the three-dimensional model, the flexible layer is divided into a multi-layer array mesh, and the simulation force information of the flexible layer includes the displacement of each mesh. For each mechanical simulation task, the mechanical simulation of the three-dimensional model is performed based on the material and external force loading conditions in that mechanical simulation task, and the process also includes: Based on the displacement of the multi-layer array mesh, determine the displacement gradient of each mesh; The uniformity of the displacement field gradient of the flexible layer is determined based on the displacement gradient of the multi-layer arrayed mesh.

8. The simulation method according to claim 5, characterized in that, The external force loading conditions include: loading conditions for normal force and loading conditions for tangential force. The magnitude of the normal force is constant, and the magnitude of the tangential force gradually increases from zero over time. The tangential force is applied after the normal force and continuously increases over time. Based on the constitutive model and the external force loading conditions, a finite element analysis is performed on the three-dimensional model, including: Based on the constitutive model, the loading conditions of the normal force and the loading conditions of the tangential force, a finite element analysis is performed on the three-dimensional model to determine the strain energy change rate of the flexible layer after the tangential force is applied. The strain energy change rate of the flexible layer when the magnitude of the tangential force is equal to the static friction threshold is determined as the slip detection sensitivity of the flexible layer.

9. The simulation method according to claim 5, characterized in that, The external force loading conditions include multiple single-dimensional force loading conditions. Based on the constitutive model and the external force loading conditions, finite element analysis is performed on the three-dimensional model to obtain the simulated stress information of the flexible layer, including: For each single-dimensional force, a finite element analysis is performed on the three-dimensional model according to the constitutive model and the loading conditions of the single-dimensional force to obtain the displacement field of the flexible layer when the single-dimensional force is applied independently. Construct a force response matrix, which includes the feature vectors of the displacement field of the flexible layer under the action of various single-dimensional forces; Based on the force response matrix, the multiaxial load decoupling capability value of the flexible layer is determined.

10. A simulation device for a visual-tactile sensor, characterized in that, The device includes: The first acquisition module is used to acquire optical simulation information and imaging quality indicators. The optical simulation information includes: flexible layer feature information and optical element feature information. The flexible layer feature information is used to describe at least the transparency of the flexible layer and the distribution and characteristics of the markers in the flexible layer. The optical element feature information is used to describe at least the properties of the optical element and the relative positional relationship between the optical element and the flexible layer. An optical simulation module is used to perform multiple optical imaging simulations based on the optical simulation information until the simulation results of the optical imaging simulation meet the imaging quality index. In the case where the imaging quality index is not met, the optical simulation information is optimized, and the next optical imaging simulation is performed based on the optimized optical simulation information. A 3D model determination module is used to determine the 3D model of the visual-tactile sensor based on the current optical simulation information when the simulation results of the optical imaging simulation meet the imaging quality index. The second acquisition module is used to acquire at least one mechanical simulation task, each mechanical simulation task including: the material of the flexible layer and the external force loading conditions; The mechanics simulation module is used to perform mechanics simulation on the three-dimensional model based on the material and external force loading conditions in each mechanics simulation task, and obtain the mechanics simulation results for that mechanics simulation task.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it performs the steps of the simulation method for the visual-tactile sensor as described in any one of claims 1 to 9.

12. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the simulation method for the visual-tactile sensor according to any one of claims 1 to 9.

13. A computing device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the simulation method for the visual-tactile sensor according to any one of claims 1 to 9.

Images