Simulation method and system of tactile sensor signal, electronic equipment and medium

Abstract

The invention provides a touch sensor signal simulation method and system, electronic equipment and a medium. Simulating a point cloud set of the touch sensor under the action of external force; wherein the point cloud set comprises three-dimensional coordinates of a plurality of deformation points on the touch sensor and corresponding stress values; based on the three-dimensional coordinate of each deformation point, determining the distance between each deformation point and each preset sensing position in the touch sensor; according to the distance between the deformation point and each sensing position, screening out a local point cloud set corresponding to each sensing position from the point cloud set, and carrying out weighting processing on stress values in each local point cloud set to obtain a comprehensive stress value of each sensing position; inputting each comprehensive stress value into an electric signal simulation model, and generating a simulated tactile electric signal of each sensing position under the external force; wherein the electric signal simulation model is a neural network model. According to the invention, the generation efficiency and accuracy of the tactile simulation data are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field, and particularly relates to a simulation method and system of a tactile sensor signal, an electronic device and a medium. BACKGROUND

[0002] At present, robots increasingly rely on tactile perception when performing fine tasks such as grasping and operating, especially in complex scenarios, tactile information plays a crucial role in improving the accuracy and stability of grasping. In order to improve such capabilities, a large amount of data is needed to train the robot, but training in a real environment not only has high cost and long time, but also easily causes wear and tear to the hardware. Therefore, more and more robot training tasks rely on simulation platforms, which can achieve low-cost and high-efficiency data collection by constructing virtual environments and running a large number of simulation processes in parallel. However, the current mainstream simulation platform mainly supports visual tactile sensors, and its simulation process needs to simulate soft body deformation and optical imaging, which has large computational overhead and is difficult to meet the demand of rapid training.

[0003] Although non-visual tactile sensors are more suitable for efficient simulation and large-scale parallel training because they directly output electrical signals, the current simulator does not have the signal-level modeling capability of such sensors, and lacks a complete physical-perception mapping mechanism from soft body deformation to electrical signal output, which limits its effective deployment and strategy training in simulation. Therefore, it is necessary to provide a simulation method, system, device and medium for tactile sensor signals. SUMMARY

[0004] The present application provides a simulation method, system, device and medium for tactile sensor signals to improve the technical problem that the existing simulation system cannot effectively simulate the signal response of the tactile sensor under the stress state.

[0005] The simulation method for tactile sensor signals provided by the present application comprises: simulating a point cloud set of a tactile sensor under external force; wherein the point cloud set comprises three-dimensional coordinates of a plurality of deformation points on the tactile sensor and corresponding stress values; based on the three-dimensional coordinates of each deformation point, the distance between each deformation point and each preset perception position in the tactile sensor is determined respectively; according to the distance between the deformation point and each perception position, the local point cloud set corresponding to each perception position is selected from the point cloud set, and the stress values in each local point cloud set are weighted to obtain the comprehensive stress value of each perception position; inputting each comprehensive stress value into an electrical signal simulation model to generate simulated tactile electrical signals of each perception position under external force; wherein the electrical signal simulation model is a neural network model.

[0006] In an embodiment of the present application, for each sensing position, a local point cloud set corresponding to the sensing position is screened from the point cloud set according to the distance between the morphing point and the sensing position, and the force values in the local point cloud set are weighted to obtain a comprehensive force value of the sensing position, including: selecting a preset number of point cloud points closest to the sensing position from the point cloud set to form a local point cloud set of the sensing position; for each point cloud point in the local point cloud set: according to the distance from the point cloud point to the sensing position, the force value corresponding to the point cloud point is weighted to obtain a weighted force value corresponding to the point cloud point; the weighted force values of all point cloud points are averaged to obtain the comprehensive force value of the sensing position.

[0007] In an embodiment of the present application, the neural network model is a multi-layer perceptron, and the comprehensive force values are input into the electrical signal simulation model to generate simulated haptic electrical signals of the simulated haptic sensor under external force, including: according to the arrangement position of the sensing position on the haptic sensor, the corresponding comprehensive force values are sequentially arranged to form a comprehensive force value sequence; the comprehensive force value sequence is input into the multi-layer perceptron to perform multi-layer feature extraction and nonlinear mapping on the comprehensive force value sequence to generate simulated haptic electrical signals of each sensing position of the haptic sensor under external force.

[0008] In an embodiment of the present application, the haptic sensor is a non-visual pneumatic haptic sensor.

[0009] In an embodiment of the present application, when the simulated haptic sensor is subjected to external force for a preset time length, a plurality of simulated haptic electrical signals of the preset time length are generated, and after the comprehensive force values are input into the electrical signal simulation model to generate simulated haptic electrical signals of the simulated haptic sensor under external force, the method further includes: inputting the plurality of simulated haptic electrical signals of the preset time length into a press head classification model to obtain a press head type corresponding to the external force applied on the simulated haptic sensor; wherein the press head classification model is a neural network model based on an attention mechanism.

[0010] In an embodiment of the present application, the plurality of simulated haptic electrical signals of the preset time length are input into the press head classification model to obtain a press head type corresponding to the external force applied on the simulated haptic sensor, including: for each simulated haptic electrical signal: inputting the simulated haptic electrical signal into a feature extraction module of the press head classification model to extract global electrical signal features corresponding to the simulated haptic electrical signal based on a spatial attention mechanism; inputting the global electrical signal features corresponding to each simulated haptic electrical signal into a pooling module of the press head classification model to perform global maximum pooling and global average pooling on each global electrical signal feature, and fusing the features after global maximum pooling and global average pooling to obtain fused electrical signal features; inputting the fused electrical signal features into a classifier of the press head classification model to obtain the press head type.

[0011] In an embodiment of the present application, the analog tactile electrical signal is input to the feature extraction module of the press head classification model, and the global electrical signal features corresponding to the analog tactile electrical signal are extracted based on the spatial attention mechanism, including: inputting the analog tactile electrical signal to the feature extraction network of the press head classification model, and extracting the local electrical signal features of the analog tactile electrical signal at each sensing position; inputting each local electrical signal feature to the spatial attention network of the press head classification model, and using the spatial attention mechanism to weight and fuse each local electrical signal feature to obtain the global electrical signal features corresponding to the analog tactile electrical signal.

[0012] The present application also provides a simulation system of a tactile sensor signal, which comprises: a point cloud acquisition module, configured to acquire a point cloud set of an analog tactile sensor under external force; wherein the point cloud set comprises three-dimensional coordinates and corresponding force values of a plurality of deformation points on the tactile sensor; a distance calculation module, configured to determine the distance between each deformation point and each preset sensing position in the tactile sensor based on the three-dimensional coordinates of each deformation point; a weighting module, configured to filter out a local point cloud set corresponding to each sensing position from the point cloud set according to the distance between the deformation point and each sensing position, and perform weighted processing on the force values in each local point cloud set to obtain a comprehensive force value of each sensing position; and an electrical signal simulation module, configured to input each comprehensive force value to an electrical signal simulation model to generate an analog tactile electrical signal of each sensing position under external force; wherein the electrical signal simulation model is a neural network model.

[0013] The present application also provides an electronic device, comprising: one or more processors; a storage device configured to store one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the simulation method of the tactile sensor signal of any one of the above.

[0014] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer, causes the computer to execute the simulation method of the tactile sensor signal of any one of the above.

[0015] The application provides a tactile sensor signal simulation method, a system, an electronic device and a medium. BRIEF DESCRIPTION OF DRAWINGS

[0016] The drawings incorporated in and forming a part of the specification, illustrate several aspects of the present application, and together with the description, serve to explain the principles of the application. It is apparent to those skilled in the art in light of the disclosure that the scope of the application encompasses more than the specific embodiments described herein. Thus, it is to be understood that the drawings are for illustration only and that the actual scope of the application is defined by the appended claims.

[0017] In the drawings:

[0018] Figure 1 A flowchart of a tactile sensor signal simulation method provided by an embodiment of the application is shown in the figure.

[0019] Figure 2 A structure diagram of a tactile sensor provided by an embodiment of the application is shown in the figure.

[0020] Figure 3 A whole flowchart of simulating tactile electric signal generation provided by an embodiment of the application is shown in the figure.

[0021] Figure 4 A flowchart of out-of-roll sensor data collection provided by an embodiment of the application is shown in the figure.

[0022] Figure 5 A schematic diagram and a confusion matrix of press head classification results provided by an embodiment of the application are shown in the figure.

[0023] Figure 6 A structure block diagram of a tactile sensor signal simulation system provided by an embodiment of the application is shown in the figure.

[0024] Figure 7 A structure diagram of an electronic device provided by an embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0025] The present application is herein described, by way of example only, with the assistance of the accompanying drawings detailed description of which is made below. Various modifications and adaptations to the embodiments of this application can be employed and design implementations to which the embodiments of this application are not limited, and are to be understood that this application is not limited to the embodiments described unless otherwise specified.

[0026] It is also to be understood that the following description is only illustrative of the principles of the present application, the purpose being to present preferred embodiments of the application in a simplified form as an aid to understanding the application. The present application is therefore not limited to the embodiments described herein, but can vary and be implemented in many different ways. The embodiments described are therefore to be understood only as non-limiting examples.

[0027] In the following description, numerous specific details are discussed in order to provide a thorough understanding of the embodiments of the application. However, those skilled in the relevant arts will recognize that the embodiments of the application can be practiced without these specific details. In other instances, well-known structures and devices are not described in detail in order to avoid obscuring the embodiments of the application.

[0028] The inventor finds that touch is an important means of perception for human interaction with the external environment, and has important application value in the fields of robots, virtual reality and medical devices. With the development of science and technology, the development of high-performance touch sensors has become the key to solving the needs of complex tasks in many fields. Touch information includes the size, distribution, direction of contact force, and the roughness and hardness of the object surface, which plays a crucial role in the completion of grasping and operation in complex scenarios by robots. For example, the end of a robot dexterous hand needs to achieve stable grasping and fine operation of objects through tactile perception, and medical rehabilitation equipment needs to detect the contact pressure of patients to provide personalized rehabilitation training programs. In robot training tasks, a large amount of data is needed for driving. However, due to the fragility and instability of real robots, it is a time and money consuming task to train robots in reality and let them learn strategies that can successfully perform tasks. Therefore, the mainstream approach is to build appropriate simulation scenarios through a simulator, and to achieve data collection with less expense through a large number of parallel environments in simulation. At present, there are few methods for simulating touch sensors in simulation, and the existing simulation methods are all for simulation of optical-based visual touch sensors. Simulating the soft contact deformation process and the light path reconstruction required by the photometric stereo method in simulation are computationally expensive tasks. The higher the precision, the more computing resources are needed, which is difficult to meet the needs of large-scale and rapid simulation.

[0029] To improve the above problems, the present application provides an analog method of tactile sensor signal, aiming to use a simulation engine to establish a model that can restore the response behavior of a soft tactile sensor in reality. The model can output an electrical signal similar to the real sensor when the same external stimulus (such as a press generated by a press head) is applied to the sensor in the simulation environment. In this way, a large number of simulated tactile electrical signals are generated without the need to repeatedly use the actual sensor, avoiding its wear or damage, thereby supporting the robot equipped with the simulation sensor model to perform task scenario construction and strategy training in a virtual environment, and the training results can be migrated to real-world applications. Especially for non-visual tactile sensors, the present application eliminates the dependence on ray tracing modeling in traditional simulation, which requires a large amount of computing resources. By finite element analysis to calculate the soft deformation and pressure distribution, the corresponding electrical signal can be predicted, which not only saves a lot of computing resources, but also improves the simulation speed and accuracy. This type of sensor is convenient to use in simulation tasks, and the electrical signals obtained by simulation are used to train the robot, and the trained model is migrated to actual applications to ensure task training without damaging the real sensor.

[0030] The following briefly introduces the content involved in the present application: The soft simulation adopted is based on the finite element method. This method discretizes soft materials (such as silicone) into a number of small units connected to each other (such as tetrahedral mesh) to calculate the stress, strain and displacement distribution during deformation, thereby accurately simulating the nonlinear mechanical properties of soft materials (such as large deformation, superelasticity, etc.). The finite element solver, as the core module of the simulation engine, is responsible for calculating the discrete model through numerical methods, outputting the physical quantity distribution results of the structural response, and building a bridge from the partial differential equation to the engineering available solution. The solving accuracy and computational efficiency directly determine the reliability of the simulation process. In this application, the finite element solver uses the built-in module in the mainstream robot simulation platform Isaac Gym to solve the physical response of the sensor soft part under external force. Since the tactile sensor used is a non-visual sensor based on a pressure sensor, its sensing process depends on the internal pressure change caused by the deformation of the soft part. However, existing simulation systems generally cannot accurately simulate the complete process from soft deformation to pressure change to pressure sensor electrical signal output. Therefore, the present application complements the simulation link from pressure distribution to electrical signal by building a neural network model, realizing the whole process simulation of the non-visual soft tactile sensor from external force to internal electrical signal generation.

[0031] It should be noted that the present application is not limited to the use of Isaac Gym and its built-in finite element module, and other simulation platforms supporting finite element calculation such as ANSYS can also be applicable. The key point of the present application is that the electric signal simulation neural network proposed in the present application can be used as a general module to receive the pressure distribution data obtained in the simulation and output the electric signal approximating the response of the real sensor, so that the high portability of soft haptic sensor modeling can be realized on different simulation platforms.

[0032] As shown in Figure 1 The simulation method of the haptic sensor signal includes the following steps:

[0033] S11, simulate the point cloud set of the haptic sensor under the action of external force; wherein the point cloud set includes the three-dimensional coordinates of a plurality of deformation points on the haptic sensor and the corresponding force values.

[0034] In order to simulate the haptic electric signal generated by the deformation of the internal soft part of the haptic sensor when it is subjected to external force in the simulation environment, a pre-set external force can be applied to the surface of the haptic sensor using the pressing head in the simulation environment, and the simulation engine calculates the deformation process of the sensor soft part in a finite element manner to generate a point cloud set composed of a plurality of deformation points, wherein the deformation point is the force point of the soft part of the haptic sensor that deforms under the action of external force and participates in force conduction. The haptic sensor of the present application is a non-visual pneumatic haptic sensor. Each point cloud point in the point cloud set contains its three-dimensional coordinate (x, y, z) and corresponding force value (such as tension value) s, i.e. the tensor form of the point cloud point is [M, N, 4], wherein the dimension M is the number of preset sensing positions (such as 8), N is the number of point cloud points selected in each sensing position, and 4 represents the three-dimensional coordinates and force value of each point cloud point.

[0035] It should be noted that, as Figure 2As shown, the tactile sensor used in the present application is composed of three parts: a sensor shell made of 3D printing material, a sensor gel body casted by EcoFlex silicone, and a circuit board installed with multiple pressure sensors. The three parts are tightly fitted in actual assembly to form a complete sensor structure. The working principle of the sensor is as follows: when the flexible silicone soft body contacts with an external object, it will deform and transmit the pressure received to the multiple pressure sensors distributed on the circuit board, and different sensors at different positions will thus generate different electrical signals, thereby realizing tactile perception. In actual use, the supporting shell hardly deforms, while the sensor gel body deforms due to the stimulation such as contact received by the sensor, and transmits the pressure to the pressure sensors, causing the reading of the sensor to change. In order to accurately reproduce the structure and behavioral response of the sensor in the simulation environment, the present application models one-to-one in SolidWorks based on the real sensor structure, and imports the model into the Isaac Gym simulator in STL format. Among them, the simulation model is divided into a rigid body part and a soft body part, which correspond to the real sensor shell and the silicone flexible gel body structure respectively. In order to achieve consistency in physical response, the parameters of the rigid body and soft body models in the Isaac Gym simulator are set as follows: the rigid body part sets the mass, density, and shape parameters related to the Mesh structure in 3D modeling. The soft body part sets the mass, density, spatial structure and attribute parameters of the finite element unit (the coordinates of each vertex in each tetrahedral unit), and the finite element deformation parameters (such as Young's modulus, Poisson's ratio, etc.). The initial values of the above parameters directly correspond to the physical parameters in the real sensor, and are fine-tuned through experimental comparison feedback, so that the deformation amplitude and mechanical performance such as reaction force of the gel body structure in the simulation under the same stimulation are as close to reality as possible, to ensure the authenticity of the simulation data generated subsequently.

[0036] S12, based on the three-dimensional coordinates of each deformation point, the distance between each deformation point and each preset perception position in the tactile sensor is determined respectively.

[0037] According to the actual structure of the tactile sensor, a plurality of fixed perception positions are preset in the simulation modeling, wherein each perception position represents the center of the installation area of the pressure sensing element in the real tactile sensor. For each point cloud point generated by simulation, the distance between the point cloud point and each perception position can be obtained according to the three-dimensional coordinates thereof, so as to subsequently perform point cloud screening.

[0038] S13, according to the distance between the deformation point and each perception position, the local point cloud set corresponding to each perception position is screened from the point cloud set, and the force values in each local point cloud set are weighted to obtain the comprehensive force value of each perception position.

[0039] According to the distance between each deformation point on each tactile sensor and the preset sensing position, a preset number of point cloud points closest to the sensing position are selected to form a local point cloud set of the sensing position. For each point cloud point in the local point cloud set, the force value is weighted and processed, and all the weighted force values are summed to obtain the comprehensive force value of the sensing position. The comprehensive force value represents the overall force of the sensing position under the current external force.

[0040] In an optional embodiment of the present application, for each sensing position, step S13 includes the following processing process: from the point cloud set, a preset number of point cloud points closest to the sensing position are selected to form a local point cloud set of the sensing position; for each point cloud point in the local point cloud set: according to the distance between the point cloud point and the sensing position, the force value corresponding to the point cloud point is weighted to obtain the weighted force value corresponding to the point cloud point; the weighted force values of all point cloud points are calculated by mean value to obtain the comprehensive force value of the sensing position.

[0041] For each sensing position, the following processing is performed: from the point cloud set output by the simulation engine, a preset number N i of point cloud points closest to the sensing position are selected to form a local point cloud set corresponding to the sensing position. For each point cloud point in the local point cloud set, the force value is weighted according to the distance between the point cloud point and the sensing position to obtain the weighted force value of the point cloud point, wherein the closer the point cloud point to the sensing position, the greater the weight, so as to reflect the contribution of the point cloud point to the force of the sensing position. After weighting all point cloud points in the local point cloud set, the weighted force values of all point cloud points are averaged to obtain the comprehensive force value of the sensing position, as shown in formula (1):

[0042]

[0043] Wherein, F i is the comprehensive force value of the i-th sensing position, N i is the number of local point clouds corresponding to the i-th sensing position, i.e. the N i closest points to the i-th sensing position are selected from the point cloud set, s i,k is the force value of the k-th point cloud point in the i-th local point cloud set, p i,k is the three-dimensional coordinates of the k-th point cloud point in the i-th local point cloud set, and x i is the three-dimensional coordinates of the i-th sensing position. Through the above weighted average method, the mapping process from the three-dimensional finite element data tensor [M, N, 4] to the sensing position comprehensive force value tensor [M, 1] is realized. Through this mapping, the response information of the sensor soft area in the simulation under the force process can be accurately captured, which is used for subsequent simulation of more accurate generation of electrical signals.

[0044] S14, input each comprehensive stress value to an electric signal simulation model to generate a simulated haptic electric signal of each sensing position under external force; wherein the electric signal simulation model is a neural network model.

[0045] It is considered that in mainstream simulators, for a precisely defined soft body, deformation occurs under specified external stimulation, and the internal pressure distribution changes during the entire deformation process can be solved by the finite element method. However, the current simulation platform is difficult to accurately simulate the complete process of external pressure being transmitted to the embedded pressure sensor through the soft body structure and further converted into an electric signal output. Therefore, the electric signal consistent with the output of the real sensor under the simulation condition cannot be directly obtained. To improve this problem, the present application proposes an electric signal simulation method based on a neural network, which learns the mapping relationship between the comprehensive stress value of the sensing position and the sensor output electric signal by training an electric signal simulation model. Specifically, after completing the calculation of the comprehensive stress values of all sensing positions, these comprehensive stress values can be sequentially arranged to form a comprehensive stress value sequence and input into the pre-trained electric signal simulation model. By analyzing the characteristic information of the comprehensive stress value, a simulated haptic electric signal vector corresponding to the haptic sensor structure is generated, wherein each dimension in the simulated haptic electric signal vector represents the electric signal of the corresponding sensing position. Through the above process, the electric signal generation of the non-visual haptic sensor under the action of the preset external force can be realized in the simulation environment.

[0046] In an optional embodiment of the present application, the neural network model is a multilayer perceptron, and step S14 includes the following processes: according to the arrangement position of the sensing position in the haptic sensor, the corresponding comprehensive stress value is sequentially arranged to form a comprehensive stress value sequence; the comprehensive stress value sequence is input into the multilayer perceptron to perform multilayer feature extraction and nonlinear mapping on the comprehensive stress value sequence, and generate a simulated haptic electric signal of each sensing position of the haptic sensor under external force.

[0047] In order to efficiently and accurately simulate the electrical signal generated by the tactile sensor under the action of external force in the simulation environment, in the embodiment, a multi-layer perceptron (MLP) is used as an electrical signal simulation model. The MLP includes a preset number (for example, 3) of cascaded fully connected layers, each of which includes a preset dimension (for example, 128) of hidden layer neurons, a batch normalization layer, and a ReLU activation layer. The batch normalization layer can improve the training stability and convergence rate, and the ReLU activation layer can improve the fitting ability of the model to complex mapping relationships. Specifically, according to the spatial arrangement order of the sensing positions on the tactile sensor, the integrated force values of the corresponding sensing positions are sequentially arranged to form an integrated force value sequence of a fixed length, and the integrated force value sequence is input into the electrical signal simulation model of the multi-layer perceptron structure. Through layer-by-layer feature conversion and non-linear mapping of the input information, a simulated tactile electrical signal sequence is obtained, and each value in the simulated tactile electrical signal sequence represents a simulated tactile response of a sensing position under the action of an external force, which is used to simulate the electrical signal generation process of a real tactile sensor under the action of the same external force.

[0048] As shown in Figure 3 , the overall process of simulating the generation of a tactile electrical signal and the neural network structure are shown. Specifically, the soft tactile sensor generated in the simulation is composed of a plurality of tetrahedral finite element units, each unit including three-dimensional spatial coordinates (x ( t ) ,y ( t ) ,z ( t ) ) and corresponding force values (such as stress) s ( t ) . All 8 sensing positions form a [8, N, 4] tensor, where 8 represents 8 sensing positions, and N is the number of point clouds around each sensing position. The local force characteristics of each sensing position can be compressed into an integrated force value F i by using the distance weighted average method, forming a [8, 1] tensor. Through a multi-layer perceptron (such as three layers), each layer includes a fully connected layer, a batch normalization layer, and a ReLU activation function in turn, and a Dropout of 0.2 is used to prevent overfitting. A simulated tactile electrical signal vector with a dimension of [8, 1] corresponding to the sensing positions is obtained.

[0049] In an optional embodiment of the present application, when the simulated tactile sensor is subjected to an external force for a preset time length, a plurality of simulated tactile electrical signals of the preset time length are generated, and after step S15, the method further comprises: inputting the plurality of simulated tactile electrical signals of the preset time length into a press head classification model to obtain a press head type corresponding to the external force applied on the simulated tactile sensor; wherein the press head classification model is a neural network model based on an attention mechanism.

[0050] In step S15, the analog tactile electrical signals of each sensing position of the generated tactile sensor under the action of external force are generated. In order to further realize the identification of the source of external force, in the embodiment, the analog tactile electrical signals at multiple time points are continuously obtained according to a preset sampling frequency during the continuous action of the external force, and an analog tactile electrical signal sequence with a preset time length is formed. The analog tactile electrical signal sequence is input into the pressing head classification model, the spatial features of the electrical signals at different sensing positions and the time sequence features at different time steps are extracted based on the attention mechanism, and the action mode of the external force is comprehensively analyzed, so that the type of the pressing head applied to the analog tactile sensor is identified.

[0051] In an optional embodiment of the present application, a plurality of analog tactile electrical signals with a preset time length are input into the pressing head classification model to obtain the type of the pressing head corresponding to the external force applied to the analog tactile sensor, including steps S16 to S18:

[0052] S16, for each analog tactile electrical signal: input the analog tactile electrical signal into the feature extraction module of the pressing head classification model, and extract the global electrical signal features corresponding to the analog tactile electrical signal based on the spatial attention mechanism.

[0053] For each time step of the analog tactile electrical signal, it is input into the feature extraction module in the pressing head classification model, wherein the feature extraction module is a neural network based on the spatial attention mechanism. In the feature extraction module, the input analog tactile electrical signal is embedded and mapped and the attention weight is calculated, so as to obtain the attention coefficient of each sensing position. The analog tactile electrical signal of the corresponding sensing position is weighted using the attention coefficient, and the weighted sum result of all sensing positions is the global electrical signal feature of the current time step. The global electrical signal feature is a group of fixed dimension vectors, and each dimension corresponds to each sensing position on the tactile sensor one by one, and is used to represent the contribution degree of the response feature of each sensing position of the tactile sensor under the action of the external force at the current time step.

[0054] S17, input the global electrical signal features corresponding to each analog tactile electrical signal into the pooling module of the pressing head classification model, respectively perform global maximum pooling and global average pooling on each global electrical signal feature, and fuse the features after the global maximum pooling and the global average pooling to obtain the fused electrical signal feature.

[0055] To further extract stable time-series electrical signal features, the global electrical signal features of all time steps can be jointly input into a pooling module in the pressing head classification model. The pooling module includes a global maximum pooling layer and a global average pooling layer in parallel, which are used to extract the feature responses of each perception position in the time dimension. Specifically, for each feature dimension corresponding to a perception position, the global maximum pooling layer extracts the maximum response value in the entire time series, representing the strongest tactile signal perception. The global average pooling layer calculates the average response value of the perception position in the time series, reflecting the overall force trend. The results of the two pooling operations are concatenated in the feature dimension. After all feature dimensions are processed, the fused electrical signal features are formed. The fused electrical signal features can retain both the peak information in the time series and the overall trend of the electrical signal changes, enhancing the model's perception ability of different pressing head features.

[0056] In an optional embodiment of the present application, step S17 includes the following data processing process: inputting the analog tactile electrical signal into the feature extraction network of the pressing head classification model to extract the local electrical signal features of the analog tactile electrical signal at each perception position; inputting each local electrical signal feature into the spatial attention network of the pressing head classification model to obtain the global electrical signal feature corresponding to the analog tactile electrical signal by using the spatial attention mechanism to weight and fuse each local electrical signal feature.

[0057] Specifically, for the analog tactile electrical signal of each time step: inputting the analog tactile electrical signal into the feature extraction network in the pressing head classification model to obtain the local electrical signal features of each perception position by embedding and encoding the analog tactile electrical signal at each perception position. To further highlight the feature contribution of key perception positions, the local electrical signal features are jointly input into the spatial attention network, the attention weight of each perception position is calculated based on the spatial attention mechanism, and the local electrical signal features are weighted and fused based on the attention weight to obtain the global electrical signal feature of the current time step.

[0058] S18, inputting the fused electrical signal features into the classifier of the pressing head classification model to obtain the pressing head type.

[0059] The fused electrical signal features obtained by the pooling module are input into the classifier in the pressing head classification model. Through multiple fully connected layers, the prediction probability of each pressing head category is generated based on the tactile response patterns of different perception positions and time series included in the fused electrical signal features, and the category corresponding to the maximum prediction probability is taken as the recognized pressing head type.

[0060] As Figure 4As shown below, the following is a further description of the above-mentioned press head classification task: in order to verify the actual utility of the simulation method of the tactile sensor, the present application constructs a press head classification task to evaluate its performance. The specific task is: to distinguish five kinds of press heads with different geometric shapes when they press on the surface of the tactile sensor. By inputting the collected tactile electrical signal sequence into the classification neural network, the accurate identification of the press head type is successfully realized. In order to complete the task, a multi-source data set integrating simulation and real environment is constructed, and the ratio of simulation data to real data is about 10:1, in order to simulate the situation of limited data collection in reality. Among them, the simulation data is obtained by mapping the soft state sequence generated by finite element simulation through the electrical signal simulation network of the present application. The real data is derived from the actual sampling of the physical sensor. All sampling processes output electrical signal time series and their corresponding press head category labels. In order to ensure the effectiveness of the classification task, mainly using geometrically distinct medium and large press heads, and controlling the diversity of motion types, the distribution characteristics of the data in simulation and reality are ensured. In addition, boundary effects are avoided in the sampling process to ensure that the pressing action always occurs within the effective sensing area of the sensor. The above task verification results show that the simulation sensor model proposed by the present application has good practicability, and the non-visual tactile sensing system can be applied or deployed in complex tasks.

[0061] It can be understood that, in order to accurately simulate the electrical signal response behavior of the non-visual soft tactile sensor under the action of external force in the simulation environment, the electrical signal simulation model trained by the present application can obtain the electrical signal of the tactile sensor according to the point cloud data generated by the simulation engine by establishing the mapping relationship between the local deformation stress data obtained from simulation and the simulated electrical signal.

[0062] Specifically, continue as Figure 4As shown, when the object (such as the pressing head) in the virtual environment comes into contact with the simulated sensor, the sensor gel body deforms continuously, and the simulation engine outputs a series of finite element data in the deformation process in the form of time steps, which is a point cloud set. According to the point cloud set, the comprehensive stress value at different perception positions can be obtained, and a fixed-length stress value sequence is generated according to the spatial arrangement order. Inputting the sequence into the pre-trained multi-layer perception neural network can predict a simulated haptic electrical signal sequence similar to the real sensor electrical signal. The electrical signal simulation model of the present application is trained, and during training, the loss function between the real electrical signal and the simulated signal is minimized, and the gradient descent method is used to continuously iterate and optimize the network weight. During training, the performance of the validation set is also combined to manually adjust the hyperparameters of the model, such as the learning rate, Dropout probability, etc., to improve the convergence effect and generalization ability of the model. The data set used in training is obtained in a one-to-one paired manner from the real environment and the simulation environment. Specifically, each training data comes from the same pressing head, the same motion trajectory, and the same pressing behavior in the real and simulation environments, and is strictly aligned in the time step dimension. The data sampling strategy includes saturated sampling and random sampling. Saturated sampling refers to traversing all small pressing head types, common motion patterns, and covering multiple regions of the sensor gel body surface to obtain as rich data distribution as possible. Through multi-angle and multi-position dense coverage sampling, a training set is constructed to ensure that the model can learn comprehensive pressing response features. Random sampling refers to randomly selecting pressing heads, motion trajectories, and pressing positions for validation set construction and generalization ability improvement.

[0063] As shown in FIG. 6, Figure 5 As shown in FIG. 6, Figure 5 The geometric appearance and number of seven pressing heads are given, and the confusion matrix on the left corresponds to the model trained using real sensor data, and the overall classification accuracy is only 39.29%, and there is classification confusion, for example, almost all class 3 is misjudged as class 0. It can be seen that in the case of limited sample size, the model is easy to confuse the categories, and the classification performance is unstable. The confusion matrix on the right is trained based on the simulation data generated by the present scheme, and the classification accuracy reaches 95.00%, and almost all pressing heads can be accurately identified, indicating that a large number of continuous high-quality simulated haptic electrical signals obtained by simulation can significantly improve the discrimination ability of the model.

[0064] As shown in FIG. 6, Figure 6As shown, the analog system 600 of the tactile sensor signal includes a point cloud acquisition module 610, a distance calculation module 620, a weighting module 630, and an electrical signal simulation module 640. The point cloud acquisition module 610 is configured to simulate a point cloud set of the tactile sensor under the external force; the point cloud set includes three-dimensional coordinates of a plurality of deformation points on the tactile sensor and corresponding force values. The distance calculation module 620 is configured to determine the distance between each deformation point and each preset sensing position in the tactile sensor based on the three-dimensional coordinates of each deformation point. The weighting module 630 is configured to filter out a local point cloud set corresponding to each sensing position from the point cloud set according to the distance between the deformation point and each sensing position, and to perform weighting processing on the force values in each local point cloud set to obtain a comprehensive force value of each sensing position. The electrical signal simulation module 640 is configured to input each comprehensive force value into an electrical signal simulation model to generate an analog tactile electrical signal of each sensing position under the external force; the electrical signal simulation model is a neural network model.

[0065] The specific limitations of the analog system of the tactile sensor signal can be referred to the limitations of the analog method of the tactile sensor signal in the foregoing, which will not be repeated here. Each module in the analog system of the tactile sensor signal can be realized by software, hardware, and a combination thereof, in whole or in part. Each module can be embedded in or independent of the processor in the computer device in hardware format, or stored in the memory in the computer device in software format, so that the processor can call the corresponding operations of each module.

[0066] It should be noted that, in order to highlight the innovative part of the present application, modules not closely related to solving the technical problems proposed in the present application are not introduced in this embodiment, but this does not mean that there are no other modules in this embodiment.

[0067] As shown in Figure 7 The electronic device 7 can include a memory 71, a processor 72, and a bus, and can further include a computer program stored in the memory 71 and executable on the processor 72, such as a tactile sensor signal simulation program.

[0068] The memory 71 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or a DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 71 can be an internal storage unit of the electronic device 7 in some embodiments, such as a mobile hard disk of the electronic device 7. The memory 71 can also be an external storage device of the electronic device 7 in other embodiments, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 7. Further, the memory 71 can include both an internal storage unit and an external storage device of the electronic device 7. The memory 71 can be used to store application software installed on the electronic device 7 and various data, such as the analog code of the tactile sensor signal, and can also be used to temporarily store data that has been output or will be output.

[0069] The processor 72 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 72 is a control unit of the electronic device 7, which connects various components of the entire electronic device 7 through various interfaces and lines, and executes various functions and processes data of the electronic device 7 by running or executing programs or modules stored in the memory 71 (such as the analog program of the tactile sensor signal, etc.) and calling data stored in the memory 71.

[0070] The processor 72 executes an operating system of the electronic device 7 and various installed application programs. The processor 72 executes the application programs to implement the steps in the above-described analog method of the tactile sensor signal.

[0071] For example, the computer program can be divided into one or more modules stored in the memory 71 and executed by the processor 72 to complete the present application. One or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device 7. For example, the computer program can be divided into a point cloud acquisition module 610, a distance calculation module 620, a weighting module 630, and an electrical signal simulation module 640.

[0072] The integrated unit in the form of the software function module can be stored in a computer readable storage medium, which can be non-volatile or volatile. The software function module is stored in a storage medium, and includes a plurality of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute part of the functions of the method for simulating a tactile sensor signal according to various embodiments of the present application.

[0073] In summary, considering that collecting real sensor data in reality can cause hardware wear and tear and cannot be collected in parallel, which is relatively inefficient, for this purpose, the present application realizes efficient data generation in a virtual environment by constructing a tactile sensor electrical signal simulation model that can restore real behavior. This simulation system supports multi-task parallel operation and can quickly collect large-scale training data without damaging the hardware. The data generated by the simulation environment can be used to train decision strategy models and migrate them to real robot systems, thereby reducing training costs and improving the adaptability and usability of the strategy in real-world scenarios.

[0074] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not intended to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical ideas disclosed by the present application should be covered by the claims of the present application.

Claims

1. A method for simulating tactile sensor signals, characterized in that, The method includes: A point cloud set simulating a tactile sensor under external force; wherein the point cloud set includes the three-dimensional coordinates and corresponding force values ​​of multiple deformation points on the tactile sensor; Based on the three-dimensional coordinates of each deformation point, the distance between each deformation point and each preset sensing position in the tactile sensor is determined. Based on the distance between the deformation point and each sensing location, local point cloud sets corresponding to each sensing location are selected from the point cloud set, and the force values ​​in each local point cloud set are weighted to obtain the comprehensive force value of each sensing location. Each comprehensive force value is input into the electrical signal simulation model to generate simulated tactile electrical signals at each sensing position under the external force; wherein, the electrical signal simulation model is a neural network model.

2. The method for simulating tactile sensor signals according to claim 1, characterized in that, For each sensing location, based on the distance between the deformation point and the sensing location, a local point cloud set corresponding to the sensing location is selected from the point cloud set. The force values ​​in the local point cloud set are then weighted to obtain the comprehensive force value of the sensing location, including: From the point cloud set, select a preset number of point cloud points that are closest to the sensing location to form a local point cloud set for the sensing location; For each point in the local point cloud set: the force value corresponding to the point is weighted according to the distance from the point to the sensing position to obtain the weighted force value of the point. The weighted force values ​​of all point cloud points are averaged to obtain the comprehensive force value at the sensing location.

3. The method for simulating tactile sensor signals according to claim 1, characterized in that, The neural network model is a multilayer perceptron. The step of inputting various combined force values ​​into an electrical signal simulation model to generate simulated tactile electrical signals from the simulated tactile sensor under the external force includes: Based on the position of the sensing location, the corresponding comprehensive force values ​​are arranged in sequence to form a comprehensive force value sequence. The comprehensive force value sequence is input into the multilayer sensor, and multilayer feature extraction and nonlinear mapping are performed on the comprehensive force value sequence to generate simulated tactile electrical signals of each sensing position of the tactile sensor under the external force.

4. The method for simulating tactile sensor signals according to claim 1, characterized in that, The tactile sensor is a non-visual barometric tactile sensor.

5. The method for simulating tactile sensor signals according to claim 1, characterized in that, When the simulated tactile sensor is subjected to an external force for a preset duration, it generates multiple simulated tactile electrical signals for a preset duration. After inputting each comprehensive force value into the electrical signal simulation model to generate the simulated tactile electrical signals of the simulated tactile sensor under the external force, the method further includes: inputting the multiple simulated tactile electrical signals for a preset duration into a press head classification model to obtain the press head type corresponding to the external force applied to the simulated tactile sensor; wherein, the press head classification model is a neural network model based on an attention mechanism.

6. The method for simulating tactile sensor signals according to claim 5, characterized in that, The step involves inputting multiple simulated tactile electrical signals of a preset duration into a press head classification model to obtain the press head type corresponding to the external force applied to the simulated tactile sensor, including: For each simulated tactile electrical signal: the simulated tactile electrical signal is input into the feature extraction module of the press head classification model, and the global electrical signal features corresponding to the simulated tactile electrical signal are extracted based on the spatial attention mechanism; The global electrical signal features corresponding to each simulated tactile electrical signal are input into the pooling module of the press head classification model. Global max pooling and global average pooling are performed on each global electrical signal feature respectively. The features after global max pooling and global average pooling are then fused to obtain fused electrical signal features. The fused electrical signal features are input into the classifier of the press head classification model to obtain the press head type.

7. The method for simulating tactile sensor signals according to claim 6, characterized in that, The simulated tactile electrical signal is input to the feature extraction module of the pressure head classification model. Based on the spatial attention mechanism, the global electrical signal features corresponding to the simulated tactile electrical signal are extracted, including: The simulated tactile electrical signal is input into the feature extraction network of the press head classification model to extract the local electrical signal features of the simulated tactile electrical signal at each sensing location; Each local electrical signal feature is input into the spatial attention network of the press head classification model, and the spatial attention mechanism is used to perform weighted fusion of each local electrical signal feature to obtain the global electrical signal feature corresponding to the simulated tactile electrical signal.

8. A simulation system for tactile sensor signals, characterized in that, The system includes: A point cloud acquisition module is used to simulate the point cloud set of a tactile sensor under the action of external force; wherein, the point cloud set includes the three-dimensional coordinates of multiple deformation points on the tactile sensor and the corresponding force values; The distance calculation module is used to determine the distance between each deformation point and each preset sensing position in the tactile sensor based on the three-dimensional coordinates of each deformation point. The weighting module is used to filter out the local point cloud sets corresponding to each sensing position from the point cloud set based on the distance between the deformation point and each sensing position, and to weight the force values ​​in each local point cloud set to obtain the comprehensive force value of each sensing position. An electrical signal simulation module is used to input various comprehensive force values ​​into an electrical signal simulation model to generate simulated tactile electrical signals at each sensing location under the external force; wherein, the electrical signal simulation model is a neural network model.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs that, when executed by the one or more processors, cause the electronic device to implement the method for simulating tactile sensor signals as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by the computer's processor, causes the computer to perform a method for simulating the tactile sensor signal as described in any one of claims 1 to 7.

Images