Texture recognition method and system, medium, equipment and touch sensor

Abstract

The invention discloses a texture recognition method and system, a medium, equipment and a tactile sensor in the technical field of tactile perception, and aims to solve the technical problems existing when a robot perceives fine textures. The method comprises the following steps: acquiring change data of magnetic field intensity; performing data preprocessing on the change data of the magnetic field intensity, inputting the data into a pre-trained texture recognition model, and performing texture recognition to obtain a recognition result; the training of texture recognition comprises the steps of inputting a model input sample into a texture recognition model, performing random cutting on the input sample by using a Dropout layer, performing feature and attention weighted fusion by using an LSTM network layer and an attention module, learning a nonlinear relationship between features by using a KAN linear layer, and finally performing label prediction through a SoftMax layer to obtain a prediction result; and calculating the loss between the prediction result and the actual label, and optimizing the texture recognition model parameters by minimizing the loss. According to the invention, a fine texture sensing capability is provided for the robot.

Description

Technical Field

[0001] The present invention relates to the technical field of tactile perception, and particularly relates to a texture recognition method, system, medium, device and tactile sensor. Background Art

[0002] With the popularization of robots in various fields such as industry and medicine, tactile perception ability has gradually become the key for robots to achieve precise operation of physical objects. Although many research results have been achieved for tactile sensors at present, the research and development of tactile sensors for perceiving and identifying fine surface features (such as the texture or roughness of an object) still faces the following challenges: First, in order to improve the accuracy of identifying features as much as possible, the tactile sensor needs to have a high spatial resolution. Second, the tactile sensor needs to have ultra-high sensitivity so that the sensor can respond to weak stimuli when interacting with tiny surface features. Third, the miniaturization of the sensor helps to expand its application scenarios, reduce costs and energy consumption, etc., and at the same time improve the integrability, but this will also increase the complexity in aspects such as structural design and circuit design.

[0003] The tactile sensor collects a large amount of tactile data, but the processing of these tactile data still has certain challenges. Extracting relevant features from the tactile data obtained from the sensor and performing effective identification requires designing and optimizing complex algorithms and models. These algorithms need to be able to distinguish different texture types and have high accuracy. Summary of the Invention

[0004] The purpose of the present invention is to solve the above-mentioned technical problems existing in the robot's perception of fine textures, and provide a texture recognition method, system, medium, device and tactile sensor. The tactile sensor combined with the texture recognition method provides the robot with fine texture perception ability.

[0005] To solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0006] On the one hand, the present invention provides a tactile sensor, including: a magnetorheological elastomer 1, a silicone layer 2, a base 3, a PCB 4 and a cover plate 5;

[0007] Both sides of the base 3 are provided with grooves, the magnetorheological elastomer 1 is fixed in one of the grooves through the silicone layer 2, and the PCB 4 is placed in the other groove;

[0008] The PCB 4 is arranged with a Hall sensor array for detecting the magnetic field change brought by the deformation after the magnetorheological elastomer 1 contacts the texture surface;

[0009] One end of the base 3 where the PCB 4 is located is connected to the cover plate 5, and the cover plate 5 is used to bear the externally applied force;

[0010] The PCB4 is built with a texture recognition module for texture recognition to obtain a recognition result.

[0011] Optionally, the raw materials of the magnetorheological elastomer 1 include neodymium iron boron permanent magnet particles, silicone rubber, and silicone oil.

[0012] Optionally, the cover plate 5 is connected to the base hole 301 of the base 3 through a cover plate pin 501.

[0013] In a second aspect, the present invention provides a texture recognition method, including:

[0014] Obtaining the change data of the magnetic field intensity;

[0015] Performing data preprocessing on the change data of the magnetic field intensity to obtain model input data;

[0016] Inputting the model input data into a pre-trained texture recognition model for texture recognition to obtain a recognition result;

[0017] The training of the texture recognition model includes:

[0018] Obtaining the change data of the magnetic field intensity as training samples and performing data preprocessing to obtain model input samples;

[0019] Building a texture recognition model according to the long short-term memory model LSTM, the attention mechanism, and the Kolmogorov-Arnold network KAN, including a Dropout layer, an LSTM network layer, an Attention module, a KAN linear layer, and a SoftMax layer;

[0020] Inputting the model input samples into the texture recognition model, randomly cropping the input samples using the Dropout layer, respectively inputting the cropped features into the LSTM network layer and the attention module for feature and attention weighted fusion, using the KAN linear layer to learn the non-linear relationship between features, and finally performing label prediction through the SoftMax layer to obtain a prediction result;

[0021] Calculating the loss between the prediction result and the actual label, and optimizing the texture recognition model parameters by minimizing the loss.

[0022] Optionally, each single sample in the training samples records the x-y-z three-dimensional magnetic field intensity values of 4 magnetorheological elastomer tactile sensors; the data preprocessing includes filtering out sequences with unchanged magnetic field intensity.

[0023] Optionally, the training of the texture recognition further includes data packing, which inputs multiple groups of samples as a batch into the model, thereby accelerating the training of the model.

[0024] Optionally, the loss between the calculated prediction result and the actual label is implemented by the following cross-entropy loss function:

[0025] (1)

[0026] where represents the loss between the prediction result and the actual label, represents the number of samples, represents the input tensor the score of the 𝑖-th sample corresponding to class 𝑗 in represents the input tensor the score corresponding to the true class 𝑦𝑖 of the 𝑖-th sample in

[0027] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, the steps of any of the texture recognition methods described in the second aspect are implemented.

[0028] In a fourth aspect, the present invention provides a computer device / equipment / system, including:

[0029] a memory for storing computer programs / instructions;

[0030] a processor for executing the computer programs / instructions to implement the steps of any of the texture recognition methods described in the second aspect.

[0031] In a fifth aspect, the present invention provides a computer program product, including computer programs / instructions, characterized in that when the computer programs / instructions are executed by a processor, the steps of any of the texture recognition methods described in the second aspect are implemented.

[0032] Compared with the prior art, the beneficial effects achieved by the present invention:

[0033] 1. The tactile sensor proposed by the present invention has the characteristics of small volume, high sensitivity, low power consumption, high signal-to-noise ratio and high spatial resolution. It can accurately capture subtle magnetic field signals when contacting the texture, and provide high-quality texture recognition output through the texture recognition module, with a texture recognition accuracy of 0.1 mm;

[0034] 2. The texture recognition method proposed by the present invention combines the attention mechanism and KAN on the basis of LSTM. The linear layer improved by KAN enhances the memory ability of the model, effectively improves the estimation accuracy for long time series, and thus improves the precision of texture recognition;

[0035] 3. The computer-readable storage medium, computer device / equipment / system, and computer program product provided by the present invention can execute the steps of the texture recognition method provided by the present invention. Description of the Drawings

[0036] Figure 1 is a flowchart of the texture recognition method according to an embodiment of the present invention;

[0037] Figure 2 is a structural diagram of the texture recognition model according to an embodiment of the present invention;

[0038] Figure 3 is a schematic structural diagram of the tactile sensor according to an embodiment of the present invention;

[0039] Figure 4 is a schematic structural diagram of the base according to an embodiment of the present invention;

[0040] Figure 5 is a schematic structural diagram of the PCB board according to an embodiment of the present invention;

[0041] Figure 6 is an actual diagram of 20 kinds of convex textures according to an embodiment of the present invention;

[0042] Figure 7 is an actual diagram of 34 kinds of concave textures according to an embodiment of the present invention.

[0043] Among them: 1 - magnetorheological elastomer; 2 - silicone rubber layer; 3 - base; 301 - base hole; 4 - PCB; 5 - cover plate; 501 - cover plate pin. Detailed Embodiments

[0044] The technical solution of the present invention will be described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present application and the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0045] It should be noted that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0046] Embodiment 1:

[0047] An embodiment of the present invention discloses a tactile sensor. Referring to Figure 3 as shown, it includes: a magnetorheological elastomer 1, a silicone rubber layer 2, a base 3, a PCB 4, and a cover plate 5.

[0048] Reference Figure 4 As shown, grooves are provided on both sides of the base 3 for placing the magnetorheological elastomer 1 and the PCB 4 respectively; the magnetorheological elastomer 1 is fixed in one of the grooves through the silicone layer 2, and the PCB 4 is placed in the other groove; the cover plate 5 is connected to the base hole 301 of the base 3 through the cover plate pin 501, and the cover plate 5 is used to bear the externally applied force. When the tactile sensor is subjected to an externally applied force, the externally applied force is transmitted through the magnetorheological elastomer 1, the silicone layer 2, the base 3 and the PCB 4 in sequence, and finally acts on the cover plate 5.

[0049] Specifically, the magnetorheological elastomer 1 is made of neodymium iron boron permanent magnet particles, silicone rubber and silicone oil in proportion, and has a high magnetic permeability and good ductility; among them, the content of the neodymium iron boron permanent magnet particles accounts for 50%, the proportion of the silicone oil accounts for 3%, the particle size of the neodymium iron boron permanent magnet particles is 5 µm, 2000 mesh; the silicone rubber selected is Ecoflex™00-30, its elastic modulus is 0.0552 MPa, the tensile strength is 1.1032 MPa, and the dynamic viscosity is 3 Pa⋅s; in this embodiment, the silicone layer 2 is coated on the grooves of the magnetorheological elastomer 1 and the base 3 when it is liquid, and forms the silicone layer 2 after curing.

[0050] Reference Figure 5 As shown, the size of the PCB 4 is 16 mm × 20 mm, and a Hall sensor array is arranged thereon for detecting the magnetic field change caused by the deformation after the magnetorheological elastomer 1 contacts the texture surface; the size of the Hall sensor array is 7 mm × 10 mm, the Hall sensor selected is MLX90393, the size is 3 mm × 3 mm × 1 mm, the Hall sensor array uses 4 MLX90393, and 12-dimensional magnetic field data can be obtained. The MLX90393 integrates an analog-to-digital converter inside and can directly convert the Hall voltage into a digital signal; the Hall sensor array with a small range and high density enables the tactile sensor to have an ultra-high spatial resolution, and the texture recognition accuracy reaches 0.1 mm; the PCB 4 is built-in with a texture recognition module for texture recognition to obtain the recognition result.

[0051] In order to evaluate the effect of the tactile sensor, a data experiment was carried out in this embodiment. A tactile texture data set was established by using the data of the tactile sensor contacting the texture surface for training the texture recognition module built in the PCB 4.

[0052] The texture data set contains 4050 groups of data, and the data is collected by the tactile sensor from 54 kinds of textures, and 75 groups of data are collected for each kind of texture. Reference Figures 6 - 7, the 54 textures are composed of 20 convex textures and 34 concave textures. Through experiments, it is proved that the tactile sensor combined with the texture recognition module has a 100% recognition accuracy for the 54 textures.

[0053] In summary, the tactile sensor proposed in this embodiment uses magnetic field information to realize the recognition of object textures; when the tactile sensor contacts the texture surface, the magnetorheological elastomer 1 deforms, resulting in a corresponding change in the magnetic field of the elastomer; this change can be detected by the Hall effect sensor array installed on the PCB 4; subsequently, through the texture recognition module to analyze and algorithmically process the collected data, the relationship between the magnetic field change and the surface texture can be established, so as to realize the recognition of the texture.

[0054] Embodiment 2:

[0055] The embodiment of the present invention discloses a texture recognition method, as shown in reference Figure 1 shown, including:

[0056] S1, obtaining the change data of the magnetic field intensity;

[0057] S2, performing data preprocessing on the change data of the magnetic field intensity to obtain model input data;

[0058] S3, inputting the model input data into a pre-trained texture recognition model for texture recognition to obtain a recognition result.

[0059] Specifically,

[0060] As shown in reference Figure 2 is the structure diagram of the texture recognition model. In step S3, the training of the texture recognition model includes:

[0061] S3.1, obtaining the change data of the magnetic field intensity as training samples and performing data preprocessing to obtain model input samples;

[0062] S3.2, building a texture recognition model according to the Long Short-Term Memory (LSTM) model, attention mechanism and Kolmogorov–Arnold Network (KAN), including a Dropout layer, an LSTM network layer, an Attention module, a KAN linear layer and a SoftMax layer;

[0063] S3.3. Input the input samples of the model into the texture recognition model. Use the Dropout layer to randomly crop the input samples, and input the cropped features into the LSTM network layer and the attention module respectively for feature and attention weighted fusion. Then use the KAN linear layer to learn the non-linear relationship between features. Finally, perform label prediction through the SoftMax layer to obtain the prediction result;

[0064] S3.4. Calculate the loss between the prediction result and the actual label, and optimize the parameters of the texture recognition model by minimizing the loss.

[0065] In this embodiment, the input of the texture recognition model is N groups of data on the change of magnetic field intensity. Each group of data records the x-y-z three-dimensional magnetic field intensity values of 4 sensors, and the length of the data is L. Then the ideal data dimension is N×L×12. Since the lengths of the data sequences between samples are different, it is necessary to preprocess and package the input data so that the model can adapt to data with different sequence lengths. Specifically, the purpose of data preprocessing is to filter out sequences with unchanged magnetic field intensity caused by reasons such as stopping pressing the sensor. Data packaging is to input multiple groups of data into the model as a batch, thereby accelerating the training of the model. For data with different sequence lengths, the model uses the LSTM network as the classification network.

[0066] In this embodiment, the Dropout function is used to randomly crop the packaged data to prevent overfitting during training. The cropped features are respectively input into the LSTM network and the attention module. The unique gating mechanism of the LSTM network can better capture the long-term dependencies in the long sequence. The memory unit in the LSTM network can store and update information at different time steps. Therefore, the LSTM network can well extract the texture features of the long sequence data. At the same time, the long sequence data reduces the influence of the flexible sensor caused by environmental and usage condition changes, so as to more accurately identify the texture features.

[0067] After the feature and attention weighted fusion, the KAN linear layer is used to increase the model's ability to learn non-linear relationships. The improved linear layer of KAN enhances the model's memory ability and effectively improves the estimation accuracy for long time sequences. Finally, the prediction probabilities of all labels are estimated through the SoftMax layer, and the label with the highest probability is the final output result.

[0068] In this embodiment, the loss function of the texture recognition model is the cross-entropy loss function. The cross-entropy loss calculates the negative log-likelihood of the model's prediction probability for the correct class. Optimizing this loss function can make the output probability distribution of the model closer to the true label distribution.

[0069] The calculation formula of the cross-entropy loss function is as follows:

[0070] (1)

[0071] Wherein, represents the loss between the predicted result and the actual label, represents the number of samples, represents the input tensor the score of the 𝑖-th sample corresponding to the category 𝑗 in represents the input tensor the score corresponding to the true category 𝑦𝑖 of the 𝑖-th sample in

[0072] In summary, the texture recognition method proposed in this embodiment combines the attention mechanism and KAN on the basis of LSTM. The linear layer improved by KAN enhances the memory ability of the model, effectively improves the estimation accuracy for long time series, thereby improving the precision of texture recognition and having good practicability.

[0073] Embodiment 3:

[0074] This embodiment provides a computer-readable storage medium, on which computer programs / instructions are stored. When the computer programs / instructions are executed by a processor, the steps of the texture recognition method described in any one of Embodiment 2 are implemented.

[0075] Embodiment 4:

[0076] This embodiment provides a computer device / system, including:

[0077] a memory for storing computer programs / instructions;

[0078] a processor for executing the computer programs / instructions to implement the steps of the texture recognition method described in any one of Embodiment 2.

[0079] Embodiment 5:

[0080] This embodiment provides a computer program product, including computer programs / instructions, characterized in that when the computer programs / instructions are executed by a processor, the steps of the texture recognition method described in any one of Embodiment 2 are implemented.

[0081] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0082] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0083] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0085] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope protected by the present invention and the claims. All of these are within the protection scope of the present invention.

Claims

1. A tactile sensor, characterized in that: include: Magnetorheological elastomer (1), silicone layer (2), base (3), PCB (4) and cover plate (5); Grooves are provided on both sides of the base (3); the magnetorheological elastomer (1) is fixed in one of the grooves through the silicone layer (2); and the PCB (4) is placed in the other groove; A Hall sensor array is arranged on the PCB (4) for detecting changes in the magnetic field caused by deformation of the magnetorheological elastomer (1) after contact with the textured surface; One end of the base (3) on which the PCB (4) is arranged is connected to the cover plate (5), and the cover plate (5) is used to withstand an externally applied force; The PCB (4) has a built-in texture recognition module for performing texture recognition to obtain a recognition result.

2. The tactile sensor according to claim 5, characterized in that: The raw materials of the magnetorheological elastomer (1) include neodymium iron boron permanent magnet particles, silicone rubber and silicone oil.

3. The tactile sensor according to claim 5, characterized in that: The cover plate (5) is connected to the base hole (301) of the base (3) via a cover plate latch (501).

4. A texture recognition method, characterized in that: include: Obtaining the change data of magnetic field strength; Performing data preprocessing on the change data of the magnetic field intensity to obtain model input data; Inputting the model input data into a pre-trained texture recognition model to perform texture recognition and obtain a recognition result; The training of the texture recognition model includes: Obtaining the changing data of magnetic field intensity as training samples, and performing data preprocessing to obtain model input samples; A texture recognition model is built based on the long short-term memory model LSTM, attention mechanism and Kolmogorov-Arnold network KAN, including Dropout layer, LSTM network layer, Attention module, KAN linear layer and SoftMax layer; The model input sample is input into the texture recognition model, the input sample is randomly cropped using the Dropout layer, the cropped features are respectively input into the LSTM network layer and the attention module, the features and attention weighted fusion are performed, and the nonlinear relationship between the features is learned using the KAN linear layer, and finally the label prediction is performed through the SoftMax layer to obtain the prediction result; The loss between the predicted results and the actual labels is calculated, and the texture recognition model parameters are optimized by minimizing the loss.

5. The texture recognition method according to claim 4, characterized in that: Each individual sample in the training sample records the values ​​of the xyz three-dimensional magnetic field strength of four magnetorheological elastomer tactile sensors; the data preprocessing includes filtering out sequences with constant magnetic field strength.

6. The texture recognition method according to claim 4, characterized in that: The texture recognition training also includes data packaging, which inputs multiple groups of samples into the model as a batch, thereby speeding up the training of the model.

7. The encrypted network traffic classification method based on the structured state space dual model according to claim 4 is characterized in that: The loss between the calculated prediction result and the actual label This is achieved through the following cross entropy loss function: （1） in, Represents the loss between the predicted result and the actual label, represents the number of samples, Represents the input tensor The score of the 𝑖th sample in the corresponding category 𝑗, Represents the input tensor The score corresponding to the true category 𝑦𝑖 of the 𝑖th sample in .

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the texture recognition method described in any one of claims 4 to 7 are implemented.

9. A computer device / equipment / system, characterized in that: include: Memory, for storing computer programs / instructions; A processor, configured to execute the computer program / instructions to implement the steps of the texture recognition method according to any one of claims 4 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the texture recognition method described in any one of claims 4 to 7 are implemented.

Images