Tactile sensing device and method

Abstract

The invention provides a tactile sensing device and a tactile sensing method, which can be applied to the technical field of tactile sensing. The tactile sensing device comprises a tactile sensor which comprises a plurality of neodymium iron boron magnetic structures; the first end of the measuring circuit is electrically connected with the first end of the tactile sensor, the second end of the measuring circuit is electrically connected with the second end of the tactile sensor, and the measuring circuit is used for continuously detecting the resistance value of the tactile sensor to obtain a resistance sequence under the condition that the tactile sensor is subjected to external acting force; and the processor is electrically connected with the output end of the measuring circuit and is used for determining a target area where an external acting force acts on the tactile sensor according to the resistance sequence and determining the magnitude of the external acting force acting on the target area of the tactile sensor according to the target area, the resistance sequence and the initial three-dimensional shape of the tactile sensor so as to obtain a force sequence.

Description

Technical Field

[0001] The present invention relates to the field of tactile sensing technology, and more specifically, to a tactile sensing device and method. Background Art

[0002] In related technologies, tactile sensor design usually expands the coverage area of ​​the sensing array by increasing the number of sensors to achieve high sensitivity and multi-parameter detection capabilities similar to biological skin. These tactile sensors mainly sense regional mechanics, temperature, surface texture, vibration and other information by using multiple physically isolated channels. With the help of multi-channel design of hardware, independent addressing of each channel is achieved to obtain the distribution characteristics of the signal. However, this method of increasing the number of physical sensors, although it can enhance the perception capability, also introduces a series of technical challenges and bottlenecks.

[0003] First of all, simply increasing the number and density of hardware channels will not only fail to surpass biological tactile receptors in the short term, but will also significantly increase the complexity and cost of hardware design and manufacturing. Secondly, to achieve the simultaneous measurement of multiple physical characteristics, it is also necessary to introduce multiple types of sensors and use a complex structure of multi-layer sensor stacking. These increased complexities have severely limited the actual promotion and popularization of tactile sensors in application scenarios such as portable devices, wearable devices, and robots. Therefore, the current design of tactile sensors faces significant bottlenecks in terms of cost, integration, reliability, etc., and an innovative solution is urgently needed to improve its overall performance and economy. Summary of the invention

[0004] In view of this, the present invention provides a tactile sensing device and method.

[0005] According to one aspect of the present invention, a tactile sensing device is provided, comprising: a tactile sensor, wherein the tactile sensor comprises a plurality of neodymium iron boron magnetic structures; a measuring circuit, wherein a first end of the measuring circuit is electrically connected to a first end of the tactile sensor, and a second end of the measuring circuit is electrically connected to a second end of the tactile sensor, and the measuring circuit is used for continuously detecting a resistance value of the tactile sensor when the tactile sensor is subjected to an external force, so as to obtain a resistance sequence; and a processor, wherein the processor is electrically connected to an output end of the measuring circuit, and is used for determining, based on the resistance sequence, a target area of ​​the tactile sensor where the external force acts on the tactile sensor, and determining, based on the target area, the resistance sequence and an initial three-dimensional morphology of the tactile sensor, a magnitude of the external force acting on the target area of ​​the tactile sensor, so as to obtain a force sequence, wherein the multiple force values ​​included in the force sequence correspond one-to-one to the multiple resistance values ​​included in the resistance sequence.

[0006] According to an embodiment of the present invention, the processor determines, based on the resistance sequence, a target area where the external force acts on the tactile sensor, including: extracting peak features for the multiple resistance values; processing the peak features using a first prediction model, a second prediction model, and a third prediction model, respectively, to obtain a first prediction result, a second prediction result, and a third prediction result output by the first prediction model, the second prediction model, and the third prediction model, respectively, wherein the first prediction model, the second prediction model, and the third prediction model include different network structures; and determining, based on the first prediction result, the second prediction result, and the third prediction result, the target area where the external force acts on the tactile sensor.

[0007] According to an embodiment of the present invention, the processor determines the magnitude of the external force acting on the target area of ​​the tactile sensor according to the target area, the resistance sequence and the initial three-dimensional topography of the tactile sensor, and obtains the force sequence, including: determining the target three-dimensional topography corresponding to the target area according to the target area and the initial three-dimensional topography; and obtaining the force sequence according to the target three-dimensional topography and the resistance sequence.

[0008] According to an embodiment of the present invention, the processor obtains the force sequence based on the target three-dimensional morphology and the resistance sequence, including: extracting features from the resistance sequence using a long short-term memory network to obtain resistance features; extracting features from the target three-dimensional morphology using a convolutional neural network to obtain morphology features; fusing the resistance features and the morphology features to obtain a first fused feature; and mapping the first fused feature to obtain the force sequence.

[0009] According to an embodiment of the present invention, the initial three-dimensional shape is obtained by measuring the shape of the tactile sensor in an initial state using a white light interferometer.

[0010] According to an embodiment of the present invention, the tactile sensor includes multiple preset areas, adjacent preset areas in the multiple preset areas overlap, and the initial three-dimensional morphology is obtained by cyclically performing the following operations: for the i-th preset area among the multiple preset areas, applying a test pressure to the i-th preset area of ​​the tactile sensor at a preset frequency to obtain multiple pressure values, wherein i is a positive integer; mapping the multiple pressure values ​​to obtain a test resistance sequence, wherein the multiple test resistance values ​​included in the test resistance sequence correspond one-to-one to the multiple pressure values; performing feature fusion on the test resistance sequence and the preset noise sequence to obtain a second fusion feature; and using a conditional adversarial generative network to process the second fusion feature to obtain an initial three-dimensional morphology corresponding to the i-th preset area.

[0011] According to an embodiment of the present invention, the first prediction model includes a support vector machine model; the second prediction model includes a random forest model; and the third prediction model includes a decision tree model.

[0012] According to an embodiment of the present invention, the target region includes at least one NdFeB magnetic microstructure.

[0013] According to an embodiment of the present invention, the tactile sensor is obtained according to the following operations: forming a conductive nano-silver coating on the surface of a plurality of NdFeB magnetic micron particles to obtain a silver / NdFeB core-shell structure; mixing the silver / NdFeB core-shell structure with silica gel and magnetizing it to obtain a magnetized mixture; applying a preset magnetic field to the magnetized mixture using a permanent magnet to obtain the plurality of NdFeB magnetic structures, and then obtaining the tactile sensor, wherein the plurality of NdFeB magnetic structures have different sizes and the plurality of NdFeB magnetic structures have different morphologies.

[0014] According to another aspect of the present invention, a tactile sensing method is provided, comprising: when the tactile sensor is subjected to an external force, continuously detecting the resistance value of the tactile sensor to obtain a resistance sequence, wherein the tactile sensor includes a plurality of neodymium iron boron magnetic structures; determining a target area of ​​the tactile sensor on which the external force acts according to the resistance sequence, and determining the magnitude of the external force acting on the target area of ​​the tactile sensor according to the target area, the resistance sequence and an initial three-dimensional morphology of the tactile sensor to obtain a force sequence, wherein the plurality of force values ​​included in the force sequence correspond one-to-one to the plurality of resistance values ​​included in the resistance sequence.

[0015] According to an embodiment of the present invention, since the tactile sensor includes a plurality of NdFeB magnetic structures, the tactile sensor can respond to changes in pressure, shear force, temperature and magnetic field. By using the first end of the measuring circuit to be electrically connected to the first end of the tactile sensor, the second end of the measuring circuit to be electrically connected to the second end of the tactile sensor, the measuring circuit continuously detects the resistance value of the tactile sensor when the tactile sensor is subjected to an external force, and obtains a resistance sequence, so that when the tactile sensor is subjected to an external force, a signal acquisition channel of the measuring circuit is used to collect the resistance value corresponding to the tactile sensor, and a resistance sequence is obtained, and the structure of the measuring circuit is relatively simple. Then, by using the processor according to the resistance sequence, the target area of ​​the tactile sensor is determined to be acted on by the external force, and the magnitude of the external force acting on the target area of ​​the tactile sensor is determined according to the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor, and the force sequence is obtained, so that the signal acquisition channel of the processor is used to collect the resistance sequence corresponding to the target area, and at the same time, since a target area represents a soft channel, the physical channel of hardware isolation is replaced by such a virtual soft channel, and when the external force is determined to act on the target area of ​​the tactile sensor according to the resistance sequence, the corresponding soft channel can be determined. According to the target area, resistance sequence and initial three-dimensional morphology of the tactile sensor, the magnitude of the external force acting on the target area of ​​the tactile sensor is determined, and when the force sequence is obtained, the force sequence corresponding to the soft channel can be obtained. Thus, while achieving high sensitivity and multi-parameter detection capabilities similar to biological skin, the sensor hardware complexity and manufacturing cost are reduced, and the limitations of traditional sensor design are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings.

[0017] Figure 1 A schematic structural diagram of a tactile sensing device according to an embodiment of the present invention is shown.

[0018] Figure 2 A schematic diagram showing the resistance change of a tactile sensor when a single NdFeB magnetic structure is subjected to external forces in different directions according to an embodiment of the present invention is shown.

[0019] Figure 3A The schematic diagram shows the macroscopic change of the resistance of the tactile sensor over time when the tactile sensor is subjected to an external downward force according to an embodiment of the present invention.

[0020] Figure 3B The schematic diagram shows the microscopic change of the resistance of the tactile sensor over time when the tactile sensor is subjected to an external downward force according to an embodiment of the present invention.

[0021] Figure 4A A schematic diagram showing the distribution of preset areas including NdFeB magnetic structures in a tactile sensor according to an embodiment of the present invention is shown.

[0022] Figure 4B A schematic diagram showing peak values ​​of multiple resistance values ​​included in a resistance sequence according to an embodiment of the present invention is shown.

[0023] Figure 5 A schematic structural diagram of a tactile sensor according to an embodiment of the present invention is shown.

[0024] Figure 6 A flowchart of determining a target area of ​​a tactile sensor where an external force acts is shown according to an embodiment of the present invention.

[0025] Figure 7 A flowchart of determining the magnitude of an external force acting on a target area of ​​a tactile sensor and obtaining a force sequence is shown in accordance with a processor of an embodiment of the present invention.

[0026] Figure 8 A schematic diagram showing an initial three-dimensional appearance of a tactile sensor according to an embodiment of the present invention is shown.

[0027] Fig. 9 A flowchart of a processor determining an initial three-dimensional shape of a tactile sensor according to an embodiment of the present invention is shown.

[0028] Fig.10 A flow chart of a tactile sensing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0029] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0030] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0031] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0032] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0033] Tactile sensing technology based on physically isolated sensor arrays is the main method used for tactile perception in current humanoid robots and human-computer interaction systems. This method imitates the high-density distribution of human tactile receptors, and its main development goal is to eventually reach and exceed the density and number of human tactile receptors.

[0034] In order to sense multiple physical quantities, such as multi-dimensional force signals, temperature signals, vibration signals, etc., it is also necessary to develop targeted technologies that integrate different types of sensors into the same sensor array. These sensors often use measurement principles such as piezoresistance, piezoelectricity, and capacitance. Some high-density sensors even need to introduce a driving backplane containing switching transistors to isolate different channels and prevent signal crosstalk. Some sensors introduce surface microstructures to enhance the response to tiny forces or to adjust the linear range and sensitivity of the sensor. These sensors often only focus on changes in macroscopic signals, but ignore the microscopic signal response of the microstructure, and the difference in the microscopic signal of each microstructure after the introduction of random microstructures. Faced with the needs of humanoid robots, embodied intelligence technology, and human-computer interaction technology for multi-dimensional force sensing and large-area tactile detection. Current tactile sensors are too complex, have high processing costs, and are still far behind the performance of biological tactile receptors. The main shortcomings of the current tactile sensor array are as follows.

[0035] First, the sensor array based on physical channel isolation has a high degree of complexity in hardware design. Depending on the piezoelectric, piezoresistive or piezocalytic measurement method, different physical isolation methods need to be adopted, such as passive matrix or active matrix driving method, so as to achieve independent addressing. These methods undoubtedly increase the cost of sensor design and manufacturing. It is not suitable for large-scale application and large-scale promotion; secondly, the existing sensor arrays can rarely realize the measurement of multi-dimensional force. The tactile sensor based on the planar structure only has a good perception ability for the force in the vertical direction. The measurement of tangential force requires the introduction of a three-dimensional structure, but the sensor with a three-dimensional structure is difficult to realize arraying. Finally, the processing of existing sensor signals mainly focuses on the processing of macroscopic signals, while the high-frequency components contained in the macroscopic signals are often simply regarded as environmental interference and noise, and little attention is paid to their connection with the microscopic structure of the sensor.

[0036] In view of this, embodiments of the present invention provide a tactile sensing device and method, which can be applied to the field of tactile sensing technology.

[0037] Figure 1 A schematic structural diagram of a tactile sensing device according to an embodiment of the present invention is shown.

[0038] like Figure 1 As shown, the tactile sensing device 100 may include a tactile sensor 110 , a measurement circuit 120 , and a processor 130 .

[0039] The touch sensor 110 may include a plurality of NdFeB magnetic structures 111 .

[0040] According to an embodiment of the present invention, the NdFeB magnetic structure 111 is a three-dimensional microstructure, has electrical conductivity and can respond to three-dimensional forces.

[0041] According to the embodiment of the present invention, each NdFeB magnetic structure 111 is different in size and morphology, and can respond to changes in pressure, shear force, temperature and magnetic field.

[0042] According to an embodiment of the present invention, each NdFeB magnetic structure 111 can be used as a random surface microstructure of the tactile sensor 110. The tactile sensor 110 can be a random surface microstructure array with conductive and piezoresistive effects, which are on the same conductive substrate and are connected to each other mechanically and electrically.

[0043] The first end of the measuring circuit 120 may be electrically connected to the first end of the tactile sensor 110, and the second end of the measuring circuit 120 may be electrically connected to the second end of the tactile sensor 110. The measuring circuit 120 may be used to continuously detect the resistance value of the tactile sensor 110 to obtain a resistance sequence when the tactile sensor 110 is subjected to an external force.

[0044] According to an embodiment of the present invention, the measurement circuit 120 is a measurement circuit with a high sampling rate.

[0045] According to an embodiment of the present invention, the sampling frequency of the measurement circuit 120 is 1000 Hz or above. The sampling frequency of the measurement circuit 120 can be selected according to actual conditions and is not limited here. For example, the sampling frequency of the measurement circuit 120 can be 1000 Hz, 1200 Hz, or 1500 Hz.

[0046] According to an embodiment of the present invention, when the NdFeB magnetic structure 111 responds to an external force, the tactile sensor 110 can not only have macroscopic changes but also use a high sampling rate measurement circuit 120 to record the response of each NdFeB magnetic structure 111 to the external force.

[0047] According to the embodiment of the present invention, the measurement circuit 120 can not only measure the macroscopic response of the electrical signal of the tactile sensor 110 to the tactile sensation, but also collect the microscopic signal related to the single NdFeB magnetic structure 111 .

[0048] Figure 2 A schematic diagram showing the resistance change of a tactile sensor when a single NdFeB magnetic structure is subjected to external forces in different directions according to an embodiment of the present invention is shown.

[0049] like Figure 2 As shown, the magnitude of the external force acting on the single NdFeB magnetic structure 111 is the same, but the direction is different, for example, the direction is Figure 2 In the case of directions 1-8, the resistance change caused by the microstructural deformation of a single NdFeB magnetic structure 111 has clear characteristic differences. Therefore, the direction and magnitude of the external force can be identified through an artificial intelligence algorithm.

[0050] According to an embodiment of the present invention, when an external force acts on a preset area including a plurality of NdFeB magnetic structures 111, the resistance change caused by the deformation of the plurality of NdFeB magnetic structures 111 included in the preset area is Figure 2 The resistance change caused by the deformation of a single NdFeB magnetic structure 111 in FIG. 1 is similar and will not be described in detail here.

[0051] The processor 130 may be electrically connected to the output terminal of the measurement circuit 120. The processor 130 may be used to determine the target area of ​​the tactile sensor 110 where the external force acts on the tactile sensor 110 according to the resistance sequence, and determine the magnitude of the external force acting on the target area of ​​the tactile sensor 110 according to the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor 110, to obtain a force sequence. The multiple force values ​​included in the force sequence correspond one-to-one to the multiple resistance values ​​included in the resistance sequence.

[0052] According to an embodiment of the present invention, when the tactile sensor 110 includes multiple preset areas, the target area may be any one of the multiple preset areas.

[0053] For example, the number of NdFeB magnetic structures 111 included in the target area can be selected according to actual conditions and is not limited here.

[0054] For example, the tactile sensor 110 may include 49 preset areas, adjacent preset areas among the 49 preset areas may overlap, and each preset area may include 8 NdFeB magnetic structures 111. The tactile sensor 110 may include 49 preset areas, adjacent preset areas among the 49 preset areas may overlap, and each preset area may include 9 NdFeB magnetic structures 111.

[0055] Figure 3A The schematic diagram shows the macroscopic change of the resistance of the tactile sensor over time when the tactile sensor is subjected to an external downward force according to an embodiment of the present invention. Figure 3B The schematic diagram shows the microscopic change of the resistance of the tactile sensor over time when the tactile sensor is subjected to an external downward force according to an embodiment of the present invention.

[0056] exist Figure 3A and Figure 3B In the figure, the horizontal axis is time and the vertical axis is resistance.

[0057] In related technologies, piezoresistive sensors usually only focus on macroscopic changes and rarely consider the data value of microscopic changes.

[0058] Depend on Figure 3B It can be seen that the microscopic changes obtained by the high sampling rate measurement circuit 120 of the tactile sensor 110 of the embodiment of the present invention correspond to the characteristics of the NdFeB magnetic structure 111 on the surface of the tactile sensor 110. The microscopic changes in the resistance of the tactile sensor over time have a convex structure, and its appearance time, maintenance time, shape, etc. are closely related to the height, profile, and size of the NdFeB magnetic structure 111.

[0059] Figure 4A A schematic diagram showing the distribution of preset areas including NdFeB magnetic structures in a tactile sensor according to an embodiment of the present invention is shown. Figure 4B A schematic diagram showing peak values ​​of multiple resistance values ​​included in a resistance sequence according to an embodiment of the present invention is shown. Figure 4B The horizontal axis is time and the vertical axis is resistance.

[0060] Figure 4A and Figure 4BThe figure shows the relationship between a preset area including a NdFeB magnetic structure and peak values ​​of a plurality of resistance values ​​included in a resistance sequence in a touch sensor according to an embodiment of the present invention.

[0061] Figure 4B The peak values ​​of the multiple resistance values ​​included in the displayed resistance sequence are obtained according to the following operation: after extracting the baseline of the resistance change curve of the tactile sensor 110 during the process of being pressed down by the external force, after retaining the microscopic components, the resistance value of each protrusion is obtained as the peak value.

[0062] For example, according to Figure 4A and Figure 4B It can be seen that as time increases, peak values ​​associated with preset areas 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 and 21 can be obtained in sequence.

[0063] According to an embodiment of the present invention, although the multiple resistance values ​​included in the resistance sequence are output externally through one channel, since the peak values ​​of the multiple resistance values ​​included in the resistance sequence appear at different times, maintain different lengths, and have different shapes, a correspondence between the three-dimensional microstructure morphology characteristics and these peak characteristics can be established through an artificial intelligence algorithm.

[0064] According to an embodiment of the present invention, since the tactile sensor includes a plurality of NdFeB magnetic structures, the tactile sensor can respond to changes in pressure, shear force, temperature and magnetic field. By using the first end of the measuring circuit to be electrically connected to the first end of the tactile sensor, the second end of the measuring circuit to be electrically connected to the second end of the tactile sensor, the measuring circuit continuously detects the resistance value of the tactile sensor when the tactile sensor is subjected to an external force, and obtains a resistance sequence, so that when the tactile sensor is subjected to an external force, a signal acquisition channel of the measuring circuit is used to collect the resistance value corresponding to the tactile sensor, and a resistance sequence is obtained, and the structure of the measuring circuit is relatively simple. Then, by using the processor according to the resistance sequence, the target area of ​​the tactile sensor is determined to be acted on by the external force, and the magnitude of the external force acting on the target area of ​​the tactile sensor is determined according to the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor, and the force sequence is obtained, so that the signal acquisition channel of the processor is used to collect the resistance sequence corresponding to the target area, and at the same time, since a target area represents a soft channel, the physical channel of hardware isolation is replaced by such a virtual soft channel, and when the external force is determined to act on the target area of ​​the tactile sensor according to the resistance sequence, the corresponding soft channel can be determined. According to the target area, resistance sequence and initial three-dimensional morphology of the tactile sensor, the magnitude of the external force acting on the target area of ​​the tactile sensor is determined, and when the force sequence is obtained, the force sequence corresponding to the soft channel can be obtained. Thus, while achieving high sensitivity and multi-parameter detection capabilities similar to biological skin, the sensor hardware complexity and manufacturing cost are reduced, and the limitations of traditional sensor design are improved.

[0065] According to an embodiment of the invention, the target region comprises at least one NdFeB magnetic microstructure.

[0066] For example, the number of the NdFeB magnetic structures 111 included in the target area may be 1, 5, 7, 8, 9, 10, 12 or 15.

[0067] Figure 5 A schematic structural diagram of a tactile sensor according to an embodiment of the present invention is shown.

[0068] Figure 1 The tactile sensor 110 and Figure 5 The tactile sensors in have similar structures and functions.

[0069] Figure 5The tactile sensor is obtained according to the following operations: forming a conductive nano-silver coating on the surface of a plurality of NdFeB magnetic micron particles to obtain a silver / NdFeB core-shell structure; mixing the silver / NdFeB core-shell structure with silica gel and magnetizing it to obtain a magnetized mixture; applying a preset magnetic field to the magnetized mixture using a permanent magnet to obtain a plurality of NdFeB magnetic structures, and then obtaining a tactile sensor, wherein the sizes of the plurality of NdFeB magnetic structures are different from each other, and the morphologies of the plurality of NdFeB magnetic structures are different from each other.

[0070] For example, the preset electric field can be Figure 5 The magnetized mixture, under the action of the external magnetic field, will generate a NdFeB magnetic structure according to the characteristics of the combination of micron particles and the magnetic field distribution. The NdFeB magnetic structure has conductivity and piezoresistive effect, and can respond to external multi-axis forces to form a unique resistance characteristic signal. Since these NdFeB magnetic structures have different physical characteristics such as height and size, their individual resistance characteristic signals also have corresponding characteristics. These characteristics can provide training data for the artificial intelligence algorithm when pressing different areas of the tactile sensor and different NdFeB magnetic structures. Thus, by constructing an artificial intelligence algorithm to identify the area of ​​action of the external force and regenerate the external force, it is also possible to establish a large number of relationships between structures and electrical signals, such as resistance signals, and infer the structural characteristics of the tactile sensor through the electrical signals of the tactile sensor.

[0071] The processor 130 determines, based on the resistance sequence, a target area where the external force acts on the tactile sensor 110, which may include: extracting peak features for multiple resistance values; processing the peak features using a first prediction model, a second prediction model, and a third prediction model, respectively, to obtain a first prediction result, a second prediction result, and a third prediction result output by the first prediction model, the second prediction model, and the third prediction model, respectively, wherein the first prediction model, the second prediction model, and the third prediction model include different network structures; and determining, based on the first prediction result, the second prediction result, and the third prediction result, a target area where the external force acts on the tactile sensor 110.

[0072] According to an embodiment of the present invention, before extracting peak features from a plurality of resistance values, the plurality of resistance values ​​may be preprocessed to filter out noise and perform normalization.

[0073] According to an embodiment of the present invention, each preset area will generate different peak characteristics when under pressure. The peak characteristics may be, for example, peak height, width, peak value, and distance between peaks. These peak characteristics may be input into an integrated learning model after preprocessing, so as to determine the target area from multiple preset areas through the integrated learning model.

[0074] According to an embodiment of the present invention, the first prediction model includes a support vector machine model; the second prediction model includes a random forest model; and the third prediction model includes a decision tree model.

[0075] For example, when the tactile sensor 110 includes 49 preset areas, the first prediction result may include 49 first probability values ​​corresponding to the 49 preset areas, and the sum of the 49 first probability values ​​is 1, the second prediction result may include 49 second probability values ​​corresponding to the 49 preset areas, and the sum of the 49 second probability values ​​is 1, and the third prediction result may include 49 third probability values ​​corresponding to the 49 preset areas, and the sum of the 49 third probability values ​​is 1. The first probability value, the second probability value, and the third probability value corresponding to each of the 49 preset areas are weighted and summed to obtain the weighted probability value corresponding to each of the 49 preset areas. The preset area corresponding to the maximum weighted probability value among the 49 weighted probability values ​​is determined as the target area. In this way, accurate judgment of different target areas is achieved through a weighted voting mechanism.

[0076] According to an embodiment of the present invention, a target area represents a soft channel, and peak features are extracted from multiple resistance values; the peak features are processed using a first prediction model, a second prediction model, and a third prediction model, respectively, to obtain a first prediction result, a second prediction result, and a third prediction result output by the first prediction model, the second prediction model, and the third prediction model, respectively; based on the first prediction result, the second prediction result, and the third prediction result, the target area where the external force acts on the tactile sensor 110 is determined, so that the powerful ability of machine learning can be used to simulate multiple virtual channels, thereby improving the resolution and perception diversity of the tactile sensing device.

[0077] According to the embodiment of the present invention, by simulating multiple physical channels through such a virtual channel, the present invention significantly improves the resolution and perception sensitivity of the system while reducing the dependence on the number of physical sensors.

[0078] Figure 6 A flowchart of determining a target area of ​​a tactile sensor where an external force acts is shown according to an embodiment of the present invention.

[0079] like Figure 6As shown, the processor 130 can extract peak features 620 for multiple resistance values ​​610; use the support vector machine model 630, the random forest model 640 and the decision tree model 650 to process the peak features 620 respectively, and obtain the first prediction result 660, the second prediction result 670 and the third prediction result 680 output by the support vector machine model 630, the random forest model 640 and the decision tree model 650 respectively; according to the first prediction result 660, the second prediction result 670 and the third prediction result 680, determine the target area 690 where the external force acts on the tactile sensor 110.

[0080] According to an embodiment of the present invention, a deep learning algorithm can be used to use the processor 130 to determine the magnitude of the external force acting on the target area of ​​the tactile sensor 110 according to the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor 110 to obtain a force sequence.

[0081] The processor 130 determines the magnitude of the external force acting on the target area of ​​the tactile sensor 110 according to the target area, the resistance sequence and the initial three-dimensional shape of the tactile sensor 110. Obtaining the force sequence may include: determining the target three-dimensional shape corresponding to the target area according to the target area and the initial three-dimensional shape; and obtaining the force sequence according to the target three-dimensional shape and the resistance sequence.

[0082] The processor 130 may obtain the force sequence according to the target three-dimensional morphology and the resistance sequence, including: using a long short-term memory network (LSTM) to extract features of the resistance sequence to obtain resistance features; using a convolutional neural network (CNN) to extract features of the target three-dimensional morphology to obtain morphology features; performing feature fusion on the resistance features and the morphology features to obtain a first fused feature; and mapping the first fused feature to obtain a force sequence.

[0083] According to an embodiment of the present invention, channel splicing may be performed on resistance features and morphology features to obtain a first fusion feature.

[0084] Figure 7 A flowchart of determining the magnitude of an external force acting on a target area of ​​a tactile sensor and obtaining a force sequence is shown in accordance with a processor of an embodiment of the present invention.

[0085] like Figure 7As shown, the processor 130 can use the long short-term memory network 720 to extract features from the resistance sequence 710 to obtain a resistance feature 730. After determining the target three-dimensional shape corresponding to the target area according to the target area and the initial three-dimensional shape, the convolutional neural network 750 is used to extract features from the target three-dimensional shape 740 to obtain a shape feature 760. The resistance feature 730 and the shape feature 760 are feature fused to obtain a first fused feature 770. The first fused feature 770 is mapped to obtain a force sequence 780.

[0086] According to an embodiment of the present invention, CNN is used to analyze the morphological image data collected from the tactile sensor 110, such as the three-dimensional morphology of the target, and extract key visual features, such as morphological features. LSTM processes serialized resistance data, such as a resistance series, to capture the time-dependent characteristics and dynamic changes of multiple resistors included in the resistance series. The outputs of the two networks are then fused, and the corresponding force magnitude is predicted through a fully connected layer to obtain an accurate force calibration result. This method enables the tactile sensing device 100 to automatically learn and calibrate the force under different morphologies and resistance changes, thereby improving the accuracy and real-time performance of perception. This method ensures the adaptability and accuracy of the tactile sensing device 100 in the face of complex changes in the external environment, and significantly improves the system's response sensitivity to different contact forces.

[0087] According to an embodiment of the present invention, the initial three-dimensional shape is obtained by measuring the shape of the tactile sensor in an initial state using a white light interferometer.

[0088] Figure 8 A schematic diagram showing an initial three-dimensional appearance of a tactile sensor according to an embodiment of the present invention is shown.

[0089] Depend on Figure 8 It can be seen that the sizes of the multiple NdFeB magnetic structures are different from each other, and the morphologies of the multiple NdFeB magnetic structures are different from each other.

[0090] According to an embodiment of the present invention, based on the same preparation operation Figure 8 When the multiple tactile sensors 110 are shown, the multiple NdFeB magnetic structures included in each of the multiple tactile sensors 110 are all consistent. The manufacturing process of the tactile sensor 110 has high repeatability and stability, ensuring the consistency of the microstructure and good tactile performance. Therefore, before using the tactile sensor device 100, a reference tactile sensor can be prepared based on the same operation as preparing the tactile sensor 110, and a reference tactile sensor device can be formed based on the reference tactile sensor.

[0091] Then, based on the test resistance sequence and the preset noise sequence corresponding to the reference tactile sensor, the second fusion feature corresponding to the reference tactile sensor is obtained, and the second fusion feature corresponding to the reference tactile sensor is used as input, and the initial three-dimensional shape scanned by the white light interferometer corresponding to the reference tactile sensor is used as a label to train the initial conditional adversarial generation network to obtain the final conditional adversarial generation network. Therefore, when using the tactile sensing device 100, the initial three-dimensional shape corresponding to the tactile sensor 110 can be generated based on the conditional adversarial generation network, without the need to use the white light interferometer to measure the shape of the tactile sensor 110 in the initial state, thereby improving the efficiency of obtaining the initial three-dimensional shape of the tactile sensor 110.

[0092] According to an embodiment of the present invention, the conditional adversarial generation network attempts to create an image that conforms to the actual tactile sensor morphology. Through continuous training, the conditional adversarial generation network can learn how to more accurately reproduce the actual morphology of the tactile sensor under different resistance sequence conditions, thereby providing more accurate morphological features for input to other decoding algorithms.

[0093] According to an embodiment of the present invention, the morphological features of a microstructure sensor can be predicted using a conditional adversarial generative network without actual manufacturing, thereby providing a rapid verification and optimization means for sensor design.

[0094] According to an embodiment of the present invention, the tactile sensor 110 may include multiple preset areas, and adjacent preset areas in the multiple preset areas overlap. The initial three-dimensional shape is obtained by the processor 130 by cyclically performing the following operations: for the i-th preset area among the multiple preset areas, a test pressure is applied to the i-th preset area of ​​the tactile sensor at a preset frequency to obtain multiple pressure values, where i is a positive integer; the multiple pressure values ​​are mapped to obtain a test resistance sequence, where the test resistance sequence includes multiple test resistance values ​​corresponding to multiple pressure values ​​one by one; the test resistance sequence and the preset noise sequence are feature fused to obtain a second fused feature; the second fused feature is processed using a conditional generative adversarial network (CGAN) to obtain an initial three-dimensional shape corresponding to the i-th preset area.

[0095] According to the embodiment of the present invention, the magnitude and direction of the test pressure can be selected according to actual conditions and are not limited here. For example, the direction of the test pressure can be vertically downward or obliquely downward.

[0096] According to an embodiment of the present invention, the test resistance sequence and the preset noise sequence have the same dimension. The test resistance sequence and the preset noise sequence may be subtracted to obtain a second fusion feature.

[0097] Fig. 9 A flowchart of a processor determining an initial three-dimensional shape of a tactile sensor according to an embodiment of the present invention is shown.

[0098] like Fig. 9 As shown, for the i-th preset area, after obtaining the test resistance sequence 910 corresponding to the i-th preset area, the test resistance sequence 910 and the preset noise sequence 920 are feature fused to obtain a second fused feature 930. The second fused feature 930 is processed using a conditional adversarial generation network 940 to obtain an initial three-dimensional shape 950 corresponding to the i-th preset area.

[0099] According to the embodiments of the present invention, the artificial intelligence-based tactile sensing device provided by the embodiments of the present invention can solve the following four problems: 1) reduce the dependence on the number of physical sensors and reduce the hardware manufacturing and maintenance costs; 2) empower sensors through intelligent algorithms to realize the signal feature extraction and recognition of a single microstructure or multiple microstructures, realize multi-channel perception from the software, and reduce the complexity of the system; 3) construct the relationship between the sensor's resistance signal and force signal through a neural network to reduce the sensor calibration process; 4) through the training of a large number of sensor mechanical and electrical signal characteristics and the relationship between the morphological characteristics, the distribution law and morphological characteristics of the sensor surface microstructure are inferred.

[0100] According to an embodiment of the present invention, the artificial intelligence-based tactile sensing device provided by the embodiment of the present invention can identify the force acting on the sensor and the force magnitude through an artificial intelligence algorithm, and can ultimately reversely infer the surface morphology of the sensor through the sensor electrical signal.

[0101] According to an embodiment of the present invention, compared with the traditional hardware decoupling method, the present invention uses a deep learning algorithm to perform virtual decoupling and soft channel labeling on tactile signals, for example, multiple resistance signals included in the resistance sequence, which significantly improves the flexibility and accuracy of tactile information processing. By introducing a deep neural network, efficient decoding and multi-parameter perception of complex tactile signals are achieved. Through the concept of virtual channels, the dependence on hardware physical channels is reduced, the functional integration and perception capabilities of the system are improved, and different application scenarios and requirements can be flexibly responded to. Through a data-driven deep learning model, the present invention has significantly improved the real-time performance of signal processing and decoding, especially after localized deployment in an embedded system, the system can achieve efficient and real-time tactile perception. Due to the reduced dependence on high-density physical sensors, the present invention significantly reduces manufacturing costs and energy consumption. At the same time, through the enhanced adaptability of intelligent algorithms, the system can work stably in complex and changeable environments, and is particularly suitable for application fields such as robots and wearable devices.

[0102] Based on the above tactile sensing device, an embodiment of the present invention further provides a tactile sensing method.

[0103] Fig.10 A flow chart of a tactile sensing method according to an embodiment of the present invention is shown.

[0104] like Fig.10 As shown, the touch sensing method may include operations S1001 to S1002.

[0105] In operation S1001 , when the tactile sensor is subjected to an external force, a resistance value of the tactile sensor is continuously detected to obtain a resistance sequence, wherein the tactile sensor includes a plurality of NdFeB magnetic junctions.

[0106] In operation S1002, based on the resistance sequence, it is determined that the external force acts on the target area of ​​the tactile sensor, and based on the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor, the magnitude of the external force acting on the target area of ​​the tactile sensor is determined to obtain a force sequence, wherein the multiple force values ​​included in the force sequence correspond one-to-one to the multiple resistance values ​​included in the resistance sequence.

[0107] It should be noted that the tactile sensing method part in the embodiment of the present invention corresponds to the tactile sensing device part in the embodiment of the present invention. The description of the tactile sensing method part specifically refers to the tactile sensing device part, which will not be repeated here.

[0108] It should be noted that, unless it is explicitly stated that there is a sequence of execution between different operations shown in the flowchart in the embodiments of the present invention, or different operations have a sequence of execution in technical implementation, otherwise, the execution order of multiple operations may not be particular, and multiple operations may also be executed simultaneously.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the apparatus, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0110] The embodiments of the present invention have been described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment has been described above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. The scope of the present invention is limited by the attached embodiments and their equivalents. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A tactile sensing device, characterized in that: The device comprises: A tactile sensor, wherein the tactile sensor comprises a plurality of neodymium iron boron magnetic structures; a measuring circuit, wherein a first end of the measuring circuit is electrically connected to a first end of the tactile sensor, a second end of the measuring circuit is electrically connected to a second end of the tactile sensor, and the measuring circuit is used to continuously detect a resistance value of the tactile sensor when the tactile sensor is subjected to an external force to obtain a resistance sequence; A processor is electrically connected to the output end of the measurement circuit, and is used to determine, based on the resistance sequence, the target area of ​​the tactile sensor where the external force acts, and to determine, based on the target area, the resistance sequence, and an initial three-dimensional topography of the tactile sensor, the magnitude of the external force acting on the target area of ​​the tactile sensor, to obtain a force sequence, wherein the multiple force values ​​included in the force sequence correspond one-to-one to the multiple resistance values ​​included in the resistance sequence.

2. The device according to claim 1, characterized in that The processor determining, according to the resistance sequence, a target area of ​​the tactile sensor on which the external force acts includes: extracting peak features from the plurality of resistance values; Using a first prediction model, a second prediction model, and a third prediction model to process the peak feature respectively, to obtain a first prediction result, a second prediction result, and a third prediction result outputted by the first prediction model, the second prediction model, and the third prediction model respectively, wherein the first prediction model, the second prediction model, and the third prediction model include different network structures; A target area of ​​the tactile sensor on which the external force acts is determined according to the first prediction result, the second prediction result, and the third prediction result.

3. The device according to claim 1 or 2, characterized in that: The processor determines the magnitude of the external force acting on the target area of ​​the tactile sensor according to the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor, and obtains the force sequence including: Determining a target three-dimensional shape corresponding to the target area according to the target area and the initial three-dimensional shape; The force sequence is obtained according to the target three-dimensional morphology and the resistance sequence.

4. The device according to claim 3, characterized in that The processor obtains the force sequence according to the target three-dimensional morphology and the resistance sequence, including: Extracting features of the resistance sequence using a long short-term memory network to obtain resistance features; Using a convolutional neural network to extract features of the target three-dimensional shape to obtain shape features; Performing feature fusion on the resistance feature and the morphology feature to obtain a first fusion feature; The first fusion feature is mapped to obtain the force sequence.

5. The device according to claim 4, characterized in that The initial three-dimensional shape is obtained by measuring the shape of the tactile sensor in the initial state using a white light interferometer.

6. The device according to claim 4, characterized in that The tactile sensor includes a plurality of preset areas, adjacent preset areas of the plurality of preset areas overlap, and the initial three-dimensional shape is obtained by cyclically performing the following operations: For an i-th preset area among the multiple preset areas, applying a test pressure to the i-th preset area of ​​the tactile sensor at a preset frequency to obtain multiple pressure values, wherein i is a positive integer; Mapping the multiple pressure values ​​to obtain a test resistance sequence, wherein the multiple test resistance values ​​included in the test resistance sequence correspond one-to-one to the multiple pressure values; Performing feature fusion on the test resistance sequence and the preset noise sequence to obtain a second fusion feature; The second fused features are processed using a conditional generative adversarial network to obtain an initial three-dimensional shape corresponding to the i-th preset area.

7. The device according to claim 2, characterized in that The first prediction model includes a support vector machine model; The second prediction model includes a random forest model; The third prediction model includes a decision tree model.

8. The device according to claim 1 or 2, characterized in that: The target region includes at least one NdFeB magnetic microstructure.

9. The device according to claim 1 or 2, characterized in that: The tactile sensor is obtained according to the following operations: forming a conductive nano-silver coating on the surface of a plurality of NdFeB magnetic micron particles to obtain a silver / NdFeB core-shell structure; The silver / neodymium iron boron core-shell structure is mixed with silica gel and magnetized to obtain a magnetized mixture; A preset magnetic field is applied to the magnetized mixture using a permanent magnet to obtain the multiple NdFeB magnetic structures, and then the tactile sensor is obtained, wherein the multiple NdFeB magnetic structures have different sizes and different morphologies.

10. A tactile sensing method, characterized in that: The method comprises: When the tactile sensor is subjected to an external force, the resistance value of the tactile sensor is continuously detected to obtain a resistance sequence, wherein the tactile sensor includes a plurality of NdFeB magnetic structures; According to the resistance sequence, the target area of ​​the tactile sensor on which the external force acts is determined, and according to the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor, the magnitude of the external force acting on the target area of ​​the tactile sensor is determined to obtain a force sequence, wherein the multiple force values ​​included in the force sequence correspond one-to-one to the multiple resistance values ​​included in the resistance sequence.

Images