Tactile Sensing Device and Method

Abstract

The present invention provides a tactile sensing device and method, which can be applied to the field of tactile sensing technology. The tactile sensing device includes: a tactile sensor, wherein the tactile sensor includes a plurality of neodymium iron boron magnetic structures; a measurement circuit, the first end of the measurement circuit is electrically connected to the first end of the tactile sensor, the second end of the measurement circuit is electrically connected to the second end of the tactile sensor, and the measurement circuit is configured to continuously detect the resistance value of the tactile sensor when the tactile sensor is subjected to an external force, so as to obtain a resistance sequence; a processor, the processor is electrically connected to the output end of the measurement circuit, and is configured to determine the target area where the external force acts on the tactile sensor according to the resistance sequence, and determine the magnitude of the external force acting on the target area of the tactile sensor according to the target area, the resistance sequence and the initial three-dimensional shape of the tactile sensor, so as to obtain a force sequence.

Description

Technical Field

[0001] The present invention relates to the field of tactile sensing technology, and more particularly, to a tactile sensing device and method. Background Art

[0002] In related technologies, the design of tactile sensors usually expands the coverage area of the sensing array by increasing the number of sensors to achieve high sensitivity and multi-parameter detection capabilities similar to biological skin. These tactile sensors mainly use multiple physically isolated channels to sense various information such as regional mechanics, temperature, surface texture, and vibration. With the multi-channel design of the hardware, independent addressing of each channel is achieved, and the distribution characteristics of the signals are obtained. However, this method of increasing the number of physical sensors, although it can enhance the sensing ability, also introduces a series of technical challenges and bottlenecks.

[0003] First of all, simply relying on increasing the number and density of hardware channels not only cannot exceed biological tactile receptors in the short term, but also significantly increases the complexity and cost of hardware design and manufacturing. Secondly, to achieve synchronous measurement of multiple physical characteristics, multiple types of sensors need to be introduced, and a complex structure of multi-layer sensor stacking is used. These increases in complexity severely limit the practical popularization and application of tactile sensors in application scenarios such as portable devices, wearable devices, and robots. Therefore, the current tactile sensor design has significant bottlenecks in terms of cost, integration, reliability, etc., and an innovative solution is urgently needed to improve its overall performance and economy. Summary of the Invention

[0004] In view of this, the present invention provides a tactile sensing device and method.

[0005] According to one aspect of the present invention, there is provided a tactile sensing device, including: a tactile sensor, wherein the tactile sensor includes a plurality of neodymium iron boron magnetic structures; a measurement circuit, a first end of the measurement circuit is electrically connected to a first end of the tactile sensor, a second end of the measurement circuit is electrically connected to a second end of the tactile sensor, and the measurement circuit is configured to continuously detect a resistance value of the tactile sensor when the tactile sensor is subjected to an external force, so as to obtain a resistance sequence; a processor, the processor is electrically connected to an output end of the measurement circuit, and is configured to determine a target area on which the external force acts on the tactile sensor according to the resistance sequence, and determine a magnitude of the external force acting on the target area of the tactile sensor according to the target area, the resistance sequence, and an initial three-dimensional shape of the tactile sensor, so as to obtain a force sequence, wherein a plurality of force values included in the force sequence correspond one-to-one to a plurality of resistance values included in the resistance sequence.

[0006] According to an embodiment of the present invention, the processor determines the target area of the external force acting on the tactile sensor according to the resistance sequence, including: extracting peak features from the plurality of resistance values; respectively processing the peak features by using a first prediction model, a second prediction model, and a third prediction model to obtain a first prediction result, a second prediction result, and a third prediction result respectively output by the first prediction model, the second prediction model, and the third prediction model, wherein the network structures included in the first prediction model, the second prediction model, and the third prediction model are different; determining the target area of the external force acting on the tactile sensor according to the first prediction result, the second prediction result, and the third prediction result.

[0007] According to an embodiment of the present invention, the processor determines the magnitude of the external force acting on the target area of the tactile sensor according to the target area, the resistance sequence, and the initial three-dimensional shape of the tactile sensor, and obtains a force sequence, including: determining a target three-dimensional shape corresponding to the target area according to the target area and the initial three-dimensional shape; obtaining the force sequence according to the target three-dimensional shape and the resistance sequence.

[0008] According to an embodiment of the present invention, the processor obtains the force sequence according to the target three-dimensional shape and the resistance sequence, including: extracting resistance features from the resistance sequence by using a long short-term memory network; extracting shape features from the target three-dimensional shape by using a convolutional neural network; performing feature fusion on the resistance features and the shape features to obtain a first fusion feature; and mapping the first fusion feature to obtain the force sequence.

[0009] According to an embodiment of the present invention, the initial three-dimensional shape is obtained by measuring the shape of the tactile sensor in an initial state by using a white light interferometer.

[0010] According to an embodiment of the present invention, the tactile sensor includes a plurality of preset areas, and adjacent preset areas in the plurality of preset areas overlap. The initial three-dimensional shape is obtained by repeatedly performing the following operations: for the i-th preset area in the plurality of preset areas, applying a test pressure to the i-th preset area of the tactile sensor at a preset frequency to obtain a plurality of pressure values, where i is a positive integer; mapping the plurality of pressure values to obtain a test resistance sequence, where the plurality of test resistance values included in the test resistance sequence correspond one-to-one to the plurality of pressure values; performing feature fusion on the test resistance sequence and a preset noise sequence to obtain a second fusion feature; and processing the second fusion feature by using a conditional adversarial generation network to obtain the initial three-dimensional shape corresponding to the i-th preset area.

[0011] According to an embodiment of the present invention, the first prediction model includes a support vector machine model; the second prediction model includes a random forest model; and the third prediction model includes a decision tree model.

[0012] According to an embodiment of the present invention, the target area includes at least one neodymium iron boron magnetic microstructure.

[0013] According to an embodiment of the present invention, the tactile sensor is obtained by the following operations: forming a conductive silver nanocoating on the surface of a plurality of neodymium iron boron magnetic microparticles to obtain a silver / neodymium iron boron core-shell structure; mixing the silver / neodymium iron boron core-shell structure with silica gel and then magnetizing to obtain a magnetized mixture; applying a preset magnetic field to the magnetized mixture by a permanent magnet to obtain the plurality of neodymium iron boron magnetic structures, and further obtaining the tactile sensor, wherein the sizes of the plurality of neodymium iron boron magnetic structures are different from each other, and the morphologies of the plurality of neodymium iron boron magnetic structures are different from each other.

[0014] According to another aspect of the present invention, there is provided a tactile sensing method, including: continuously detecting the resistance value of the tactile sensor to obtain a resistance sequence when the tactile sensor is subjected to an external force, wherein the tactile sensor includes a plurality of neodymium iron boron magnetic structures; determining the target area of the external force acting on the tactile sensor according to the resistance sequence, and determining the magnitude of the external force acting on the target area of the tactile sensor according to the target area, the resistance sequence, and the initial three-dimensional morphology of the tactile sensor to obtain a force sequence, wherein the plurality of force values included in the force sequence correspond one-to-one to the plurality of resistance values included in the resistance sequence.

[0015] According to an embodiment of the present invention, since the tactile sensor includes a plurality of neodymium iron boron magnetic structures, the tactile sensor can respond to changes in pressure, shear force, temperature, and magnetic field. By using the first end of the measurement circuit electrically connected to the first end of the tactile sensor and the second end of the measurement circuit electrically connected to the second end of the tactile sensor, the measurement circuit continuously detects the resistance value of the tactile sensor when the tactile sensor is subjected to an external force, obtains a resistance sequence, and realizes that when the tactile sensor is subjected to an external force, a resistance value corresponding to the tactile sensor is collected through one signal acquisition channel of the measurement circuit to obtain a resistance sequence, and the structure of the measurement circuit is relatively simple. Then, by using the processor to determine the target area where the external force acts on the tactile sensor according to the resistance sequence, and determine the magnitude of the external force acting on the target area of the tactile sensor according to the target area, the resistance sequence, and the initial three-dimensional shape of the tactile sensor, a force sequence is obtained, and it is realized that a resistance sequence corresponding to the target area is collected through one signal acquisition channel of the processor. At the same time, since one target area represents a soft channel, such a virtual soft channel replaces the physically isolated physical channel. When determining the target area where the external force acts on the tactile sensor according to the resistance sequence, the corresponding soft channel can be determined. When determining the magnitude of the external force acting on the target area of the tactile sensor according to the target area, the resistance sequence, and the initial three-dimensional shape of the tactile sensor to obtain a force sequence, a force sequence corresponding to the soft channel can be obtained. Furthermore, while realizing the high sensitivity and multi-parameter detection ability similar to biological skin, the hardware complexity and manufacturing cost of the sensor are reduced, and the limitations of traditional sensor design are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features, and advantages of the present invention will become more apparent from the following description of the embodiments of the present invention with reference to the accompanying drawings.

[0017] Figure 1 FIG. shows a schematic structural diagram of a tactile sensing device according to an embodiment of the present invention.

[0018] Figure 2 FIG. shows a schematic diagram of the resistance change of a tactile sensor when a single neodymium iron boron magnetic structure is subjected to external forces in different directions according to an embodiment of the present invention.

[0019] Figure 3A FIG. shows a schematic diagram of the macroscopic change of the resistance of a tactile sensor over time when the tactile sensor is subjected to a downward external force according to an embodiment of the present invention.

[0020] Figure 3B FIG. shows a schematic diagram of the microscopic change of the resistance of a tactile sensor over time when the tactile sensor is subjected to a downward external force according to an embodiment of the present invention.

[0021] Figure 4A The schematic diagram of the preset area distribution including the neodymium iron boron magnetic structure in the tactile sensor according to an embodiment of the present invention is shown.

[0022] Figure 4B The schematic diagram of the peak values of multiple resistance values included in the resistance sequence according to an embodiment of the present invention is shown.

[0023] Figure 5 The schematic structural diagram of the tactile sensor according to an embodiment of the present invention is shown.

[0024] Figure 6 The flowchart of the processor according to an embodiment of the present invention for determining the target area of the external force acting on the tactile sensor is shown.

[0025] Figure 7 The flowchart of the processor according to an embodiment of the present invention for determining the magnitude of the external force acting on the target area of the tactile sensor and obtaining the force sequence is shown.

[0026] Figure 8 The schematic diagram of the initial three-dimensional topography of the tactile sensor according to an embodiment of the present invention is shown.

[0027] Figure 9 The flowchart of the processor for determining the initial three-dimensional topography of the tactile sensor according to an embodiment of the present invention is shown.

[0028] Figure 10 The flowchart of the tactile sensing method according to an embodiment of the present invention is shown. Detailed implementation manners

[0029] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present invention. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present invention.

[0030] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0031] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0032] In cases where expressions similar to "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0033] The tactile sensing technology based on a physically isolated sensor array is the main method currently used for tactile perception in humanoid robots and human-machine interaction systems. This method mimics the high-density distribution state of human tactile receptors and aims mainly to achieve and exceed the density and quantity of human tactile receptors.

[0034] In order to sense multiple physical quantities, such as multi-dimensional force signals, temperature signals, vibration signals, etc., it is also necessary to specifically develop technologies for integrating different types of sensors in the same sensor array. These sensors often adopt measurement principles such as piezoresistive, piezoelectric, and capacitive. Some high-density sensors even need to introduce a driving backplane containing switching transistors to isolate different channels and prevent signal crosstalk. Some sensors introduce surface microstructures to enhance the response to small forces or to adjust the linear range and sensitivity of the sensors. These sensors often only focus on the changes in macroscopic signals, while ignoring the microscopic signal responses of the microstructures and the differences in the microscopic signals of each microstructure if random microstructures are introduced. Facing the requirements of humanoid robots, embodied intelligence technologies, and human-machine interaction technologies for multi-dimensional force sensing and large-area tactile detection, the current tactile sensors are too complex, have high processing costs, and there is still a large gap in performance compared to biological tactile receptors. The main deficiencies of the current tactile sensor arrays are as follows.

[0035] First, the sensor array based on physical channel isolation has a high level of complexity in hardware design. Depending on piezoelectric, piezoresistive, or piezocapacitive measurement methods, different physical isolation methods are required, such as using passive matrix or active matrix driving methods to achieve independent addressing. These methods undoubtedly increase the cost of sensor design and manufacturing and are not suitable for large-scale applications and wide-range promotion. Secondly, existing sensor arrays rarely can achieve multi-dimensional force measurement. Tactile sensors based on planar structures only have good sensing ability for forces in the vertical direction. For the measurement of tangential forces, a three-dimensional structure needs to be introduced, but it is difficult to array the three-dimensional structure sensors. Finally, the processing of existing sensor signals mainly focuses on the processing of macroscopic signals, and the high-frequency components contained in the macroscopic signals are often simply regarded as environmental interference and noise, with little attention paid to their connection with the microscopic structure of the sensors.

[0036] In view of this, embodiments of the present invention provide a tactile sensing device and method, which can be applied to the field of tactile sensing technology.

[0037] Figure 1 The structural schematic diagram of the tactile sensing device according to an embodiment of the present invention is shown.

[0038] As Figure 1 shown, the tactile sensing device 100 may include a tactile sensor 110, a measurement circuit 120, and a processor 130.

[0039] The tactile sensor 110 may include a plurality of neodymium iron boron magnetic structures 111.

[0040] According to an embodiment of the present invention, the neodymium iron boron magnetic structure 111 is a three-dimensional microstructure, has conductivity, and can respond to three-dimensional forces.

[0041] According to an embodiment of the present invention, each neodymium iron boron magnetic structure 111 is different in size and morphology and can respond to changes in pressure, shear force, temperature, and magnetic field.

[0042] According to an embodiment of the present invention, each neodymium iron boron magnetic structure 111 can be used as a random surface microstructure of the tactile sensor 110. The tactile sensor 110 can be an array of random surface microstructures with conductive and piezoresistive effects, and these random surface microstructure arrays are interconnected mechanically and electrically on the same conductive substrate.

[0043] The first end of the measurement circuit 120 can be electrically connected to the first end of the tactile sensor 110, and the second end of the measurement circuit 120 can be electrically connected to the second end of the tactile sensor 110. The measurement circuit 120 can be used to continuously detect the resistance value of the tactile sensor 110 when the tactile sensor 110 is subjected to an external force to obtain a resistance sequence.

[0044] According to an embodiment of the present invention, the measurement circuit 120 is a high-sampling-rate measurement circuit.

[0045] According to an embodiment of the present invention, the sampling frequency of the measurement circuit 120 is 1000 HZ or above. The sampling frequency of the measurement circuit 120 can be selected according to actual situations and is not limited herein. For example, the sampling frequency of the measurement circuit 120 can be 1000 HZ, 1200 HZ, 1500 HZ, etc.

[0046] According to an embodiment of the present invention, when the neodymium iron boron magnetic structure 111 responds to an external force, in addition to the macroscopic changes, the tactile sensor 110 can also use the high-sampling-rate measurement circuit 120 to record the response of each neodymium iron boron magnetic structure 111 to the external force.

[0047] According to an embodiment of the present invention, the measurement circuit 120 can not only measure the macroscopic response of the electrical signal of the tactile sensor 110 to touch, but also collect the microscopic signals related to a single neodymium iron boron magnetic structure 111.

[0048] Figure 2 The schematic diagram of the resistance change of the tactile sensor is shown when a single neodymium iron boron magnetic structure is under external forces in different directions according to an embodiment of the present invention.

[0049] As Figure 2 shown, when the magnitude of the external force acting on a single neodymium iron boron magnetic structure 111 is the same but the directions are different, for example, the directions are the directions 1-8 in Figure 2 , the resistance changes caused by the microstructure deformation of the single neodymium iron boron magnetic structure 111 have clear characteristic differences. Therefore, the acting direction and magnitude of the external force can be identified through an artificial intelligence algorithm.

[0050] According to an embodiment of the present invention, when the external force acts on a preset area including a plurality of neodymium iron boron magnetic structures 111, the resistance changes caused by the deformation of the plurality of neodymium iron boron magnetic structures 111 included in the preset area are similar to the resistance changes caused by the deformation of a single neodymium iron boron magnetic structure 111 in Figure 2 and will not be elaborated herein.

[0051] The processor 130 can be electrically connected to the output end of the measurement circuit 120. The processor 130 can be used to determine the target area where the external force acts on the tactile sensor 110 according to the resistance sequence, and determine the magnitude of the external force acting on the target area of the tactile sensor 110 according to the target area, the resistance sequence, and the initial three-dimensional shape of the tactile sensor 110, so as to obtain a force sequence. Among them, the multiple force values included in the force sequence correspond one by one to the multiple resistance values included in the resistance sequence.

[0052] According to an embodiment of the present invention, when the tactile sensor 110 includes a plurality of preset regions, the target region may be any one of the plurality of preset regions.

[0053] For example, the number of neodymium iron boron magnetic structures 111 included in the target region can be selected according to the actual situation and is not limited herein.

[0054] For example, the tactile sensor 110 may include 49 preset regions, adjacent preset regions among the 49 preset regions may overlap, and each preset region may include 8 neodymium iron boron magnetic structures 111. The tactile sensor 110 may include 49 preset regions, adjacent preset regions among the 49 preset regions may overlap, and each preset region may include 9 neodymium iron boron magnetic structures 111.

[0055] Figure 3A Shows a schematic diagram of the macroscopic change of the resistance of the tactile sensor over time when the tactile sensor according to the embodiment of the present invention is subjected to an external pressing force. Figure 3B Shows a schematic diagram of the microscopic change of the resistance of the tactile sensor over time when the tactile sensor according to the embodiment of the present invention is subjected to an external pressing force.

[0056] In Figure 3A and Figure 3B , the abscissa is time and the ordinate is resistance.

[0057] In the related art, generally, piezoresistive sensors only focus on macroscopic changes and rarely consider the data value of microscopic changes.

[0058] From Figure 3B it can be seen that the microscopic changes obtained by the tactile sensor 110 of the embodiment of the present invention through the high-sampling-rate measurement circuit 120 correspond to the characteristics of the neodymium iron boron magnetic structures 111 on the surface of the tactile sensor 110. The microscopic change of the resistance of the tactile sensor over time has a convex structure, and its appearance time, maintenance time, shape, etc. are greatly related to the height, contour, and size of the neodymium iron boron magnetic structures 111.

[0059] Figure 4A Shows a schematic diagram of the distribution of preset regions including neodymium iron boron magnetic structures in the tactile sensor according to the embodiment of the present invention. Figure 4B Shows a schematic diagram of the peaks of a plurality of resistance values included in the resistance sequence according to the embodiment of the present invention. Figure 4B The abscissa of

[0060] Figure 4A and Figure 4BShows the relationship between a preset area including a neodymium iron boron magnetic structure in a tactile sensor according to an embodiment of the present invention and the peaks of a plurality of resistance values included in a resistance sequence.

[0061] Figure 4B The peaks of the plurality of resistance values included in the displayed resistance sequence are obtained by the following operations: After extracting the baseline from the resistance change curve of the tactile sensor 110 during the process of being pressed by an external force, and retaining the microscopic components, the resistance value of each protrusion obtained is the peak value.

[0062] For example, according to Figure 4A and Figure 4B it can be known that as time increases, the peak values respectively related to the preset areas 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, and 21 can be obtained in sequence.

[0063] According to an embodiment of the present invention, although the plurality of resistance values included in the resistance sequence are output externally through one channel, since the peak values of the plurality of resistance values included in the resistance sequence appear at different times, have different maintained lengths, and different morphologies, the corresponding relationship between the three-dimensional microstructure topography features and these peak features can be established through an artificial intelligence algorithm.

[0064] According to an embodiment of the present invention, since the tactile sensor includes a plurality of neodymium iron boron magnetic structures, the tactile sensor can respond to changes in pressure, shear force, temperature, and magnetic field. By electrically connecting the first end of the measurement circuit to the first end of the tactile sensor and the second end of the measurement circuit to the second end of the tactile sensor, the measurement circuit continuously detects the resistance value of the tactile sensor when the tactile sensor is subjected to an external force, obtaining a resistance sequence, thereby realizing that when the tactile sensor is subjected to an external force, a resistance value corresponding to the tactile sensor is collected through one signal acquisition channel of the measurement circuit, obtaining a resistance sequence, and the structure of the measurement circuit is relatively simple. Then, by using the processor to determine the target area where the external force acts on the tactile sensor according to the resistance sequence, and determining the magnitude of the external force acting on the target area of the tactile sensor according to the target area, the resistance sequence, and the initial three-dimensional shape of the tactile sensor, obtaining a force sequence, it is realized that a resistance sequence corresponding to the target area is collected through one signal acquisition channel of the processor. At the same time, since one target area represents a soft channel, such a virtual soft channel replaces the physically isolated physical channel. When determining the target area where the external force acts on the tactile sensor according to the resistance sequence, the corresponding soft channel can be determined. When determining the magnitude of the external force acting on the target area of the tactile sensor according to the target area, the resistance sequence, and the initial three-dimensional shape of the tactile sensor, obtaining a force sequence, a force sequence corresponding to the soft channel can be obtained. Furthermore, while realizing the high sensitivity and multi-parameter detection ability similar to biological skin, the hardware complexity and manufacturing cost of the sensor are reduced, and the limitations of traditional sensor design are improved.

[0065] According to an embodiment of the present invention, the target area includes at least one neodymium iron boron magnetic microstructure.

[0066] For example, the number of neodymium iron boron magnetic structures 111 included in the target area can be 1, 5, 7, 8, 9, 10, 12, or 15, etc.

[0067] Figure 5 The structural schematic diagram of the tactile sensor according to an embodiment of the present invention is shown.

[0068] Figure 1 The tactile sensor 110 in Figure 5 The tactile sensor in has a similar structure and function.

[0069] Figure 5The tactile sensor in [it] is obtained according to the following operations: forming a conductive nano - silver coating on the surface of multiple neodymium - iron - boron magnetic micron particles to obtain a silver / neodymium - iron - boron core - shell structure; mixing the silver / neodymium - iron - boron core - shell structure with silica gel and then magnetizing to obtain the magnetized mixture; applying a preset magnetic field to the magnetized mixture by using a permanent magnet to obtain multiple neodymium - iron - boron magnetic structures, and further obtaining the tactile sensor, wherein the sizes of the multiple neodymium - iron - boron magnetic structures are different from each other, and the morphologies of the multiple neodymium - iron - boron magnetic structures are different from each other.

[0070] For example, the preset electric field can be Figure 5 the external magnetic field in [it]. Under the action of the external magnetic field, the magnetized mixture will generate neodymium - iron - boron magnetic structures according to the characteristics of micron - particle combination and magnetic - field distribution. The neodymium - iron - boron magnetic structure has conductivity and piezoresistive effect, can respond to external multi - axis forces, and form a unique resistance characteristic signal. Since these neodymium - iron - boron magnetic structures have different physical characteristics such as height and size, their individual resistance characteristic signals also have corresponding characteristics. These characteristics can provide training data for the artificial - intelligence algorithm when pressing different regions of the tactile sensor and different neodymium - iron - boron magnetic structures. Thus, by constructing an artificial - intelligence algorithm, the acting region of the external force can be identified, the external force can be regenerated, and also by establishing a large number of mutual relationships between the structure and electrical signals, such as resistance signals, the structural characteristics of the tactile sensor can be deduced from the electrical signals of the tactile sensor.

[0071] The processor 130 determines the target region where the external force acts on the tactile sensor 110 according to the resistance sequence, which can include: extracting peak characteristics from multiple resistance values; respectively processing the peak characteristics by using the first prediction model, the second prediction model and the third prediction model to obtain the first prediction result, the second prediction result and the third prediction result respectively output by the first prediction model, the second prediction model and the third prediction model, wherein the network structures included in the first prediction model, the second prediction model and the third prediction model are different; determining the target region where the external force acts on the tactile sensor 110 according to the first prediction result, the second prediction result and the third prediction result.

[0072] According to an embodiment of the present invention, before extracting peak characteristics from multiple resistance values, the multiple resistance values can be pre - processed to filter out noise and normalize.

[0073] According to an embodiment of the present invention, each preset region will generate different peak characteristics when being pressed. The peak characteristics can be, for example: the height, width, peak value and distance between peaks of the peak, etc. These peak characteristics can be input into an ensemble - learning model after pre - processing to determine the target region from multiple preset regions through the ensemble - learning model.

[0074] According to an embodiment of the present invention, the first prediction model includes a support vector machine model; the second prediction model includes a random forest model; the third prediction model includes a decision tree model.

[0075] For example, when the tactile sensor 110 includes 49 preset regions, the first prediction result may include 49 first probability values corresponding to the 49 preset regions respectively, and the sum of the 49 first probability values is 1. The second prediction result may include 49 second probability values corresponding to the 49 preset regions respectively, and the sum of the 49 second probability values is 1. The third prediction result may include 49 third probability values corresponding to the 49 preset regions respectively, and the sum of the 49 third probability values is 1. The first probability values, second probability values, and third probability values corresponding to the 49 preset regions are weighted and summed to obtain weighted probability values corresponding to the 49 preset regions respectively. The preset region corresponding to the maximum weighted probability value among the 49 weighted probability values is determined as the target region. Thus, accurate judgment of different target regions is achieved through a weighted voting mechanism.

[0076] According to an embodiment of the present invention, a target region represents a soft channel. By extracting peak features from multiple resistance values, and using the first prediction model, the second prediction model, and the third prediction model to process the peak features respectively, the first prediction result, the second prediction result, and the third prediction result output by the first prediction model, the second prediction model, and the third prediction model are obtained. According to the first prediction result, the second prediction result, and the third prediction result, the target region where the external force acts on the tactile sensor 110 is determined, which can utilize the powerful ability of machine learning to simulate multiple virtual channels, improving the resolution and perception diversity of the tactile sensing device.

[0077] According to an embodiment of the present invention, by means of simulating multiple physical channels through such virtual channels, the present invention reduces the dependence on the number of physical sensors while significantly improving the resolution and perception sensitivity of the system.

[0078] Figure 6 Shows a flowchart of a processor according to an embodiment of the present invention for determining the target region where an external force acts on a tactile sensor.

[0079] Such as Figure 6As shown, the processor 130 can extract peak features 620 from multiple resistance values 610; process the peak features 620 using a support vector machine model 630, a random forest model 640, and a decision tree model 650 respectively to obtain a first prediction result 660, a second prediction result 670, and a third prediction result 680 output by the support vector machine model 630, the random forest model 640, and the decision tree model 650 respectively; and determine a target area 690 where an external force acts on the tactile sensor 110 according to the first prediction result 660, the second prediction result 670, and the third prediction result 680.

[0080] According to an embodiment of the present invention, a deep learning algorithm can be used to implement determining the magnitude of the external force acting on the target area of the tactile sensor 110 by the processor 130 based on the target area, the resistance sequence, and the initial three-dimensional topography of the tactile sensor 110, and obtaining a force sequence.

[0081] The processor 130 determines the magnitude of the external force acting on the target area of the tactile sensor 110 based on the target area, the resistance sequence, and the initial three-dimensional topography of the tactile sensor 110, and obtaining a force sequence may include: determining a target three-dimensional topography corresponding to the target area according to the target area and the initial three-dimensional topography; and obtaining a force sequence according to the target three-dimensional topography and the resistance sequence.

[0082] The processor 130 obtaining a force sequence according to the target three-dimensional topography and the resistance sequence may include: extracting features of the resistance sequence using a long short-term memory network (LSTM); obtaining topography features by extracting features of the target three-dimensional topography using a convolutional neural network (CNN); performing feature fusion on the resistance features and the topography features to obtain a first fusion feature; and mapping the first fusion feature to obtain a force sequence.

[0083] According to an embodiment of the present invention, channel splicing can be performed on the resistance features and the topography features to obtain a first fusion feature.

[0084] Figure 7 The flowchart shows a processor according to an embodiment of the present invention for determining the magnitude of the external force acting on the target area of the tactile sensor and obtaining a force sequence.

[0085] As Figure 7As shown, the processor 130 can utilize the long short-term memory network 720 to perform feature extraction on the resistance sequence 710 to obtain the resistance feature 730. After determining the target three-dimensional topography corresponding to the target area based on the target area and the initial three-dimensional topography, the convolutional neural network 750 is utilized to perform feature extraction on the target three-dimensional topography 740 to obtain the topography feature 760. The resistance feature 730 and the topography feature 760 are subjected to feature fusion to obtain the first fusion feature 770. The first fusion feature 770 is mapped to obtain the force sequence 780.

[0086] According to an embodiment of the present invention, the CNN is used to analyze the topography image data collected from the tactile sensor 110, such as the target three-dimensional topography, and extract key visual features, such as the topography feature. The LSTM processes the serialized resistance data, such as the resistance sequence, and captures the time-dependent characteristics and dynamic changes of multiple resistances included in the resistance sequence. The outputs of these two networks are then fused, and a fully connected layer is used to predict the corresponding force magnitude, and an accurate force calibration result can be obtained. This method enables the tactile sensing device 100 to automatically learn and calibrate the force under different topographies and resistance changes, improving the accuracy and real-time performance of perception. This method ensures the adaptability and accuracy of the tactile sensing device 100 in the face of complex changes in the external environment, and significantly improves the response sensitivity of the system to different contact forces.

[0087] According to an embodiment of the present invention, the initial three-dimensional topography is obtained by measuring the topography of the tactile sensor in the initial state using a white light interferometer.

[0088] Figure 8 A schematic diagram of the initial three-dimensional topography of the tactile sensor according to an embodiment of the present invention is shown.

[0089] It can be seen from Figure 8 that the sizes of the multiple neodymium iron boron magnetic structures are different from each other, and the topographies of the multiple neodymium iron boron magnetic structures are different from each other.

[0090] According to an embodiment of the present invention, when the multiple tactile sensors 110 shown in Figure 8 are obtained based on the same preparation operation, the multiple neodymium iron boron magnetic structures included in each of the multiple tactile sensors 110 are all consistent. The manufacturing process of the tactile sensor 110 has high repeatability and stability, ensuring the consistency of the microstructures and good tactile performance. Therefore, before using the tactile sensing device 100, a reference tactile sensor can be prepared based on the same operation for preparing the tactile sensor 110, and a reference tactile sensing device can be formed based on the reference tactile sensor.

[0091] Then, based on the test resistance sequence corresponding to the reference tactile sensor and the preset noise sequence, a second fusion feature corresponding to the reference tactile sensor is obtained. Using the second fusion feature corresponding to the reference tactile sensor as the input and the initial three-dimensional topography scanned by the white light interferometer corresponding to the reference tactile sensor as the label, the initial conditional adversarial generation network is trained to obtain the final conditional adversarial generation network. Thus, when using the tactile sensing device 100, the initial three-dimensional topography corresponding to the tactile sensor 110 can be generated based on the conditional adversarial generation network, without the need to use a white light interferometer to measure the topography of the tactile sensor 110 in the initial state, improving the efficiency of obtaining the initial three-dimensional topography of the tactile sensor 110.

[0092] According to an embodiment of the present invention, the conditional adversarial generation network attempts to create an image that conforms to the topography of a real tactile sensor. Through continuous training, the conditional adversarial generation network can learn how to more accurately reproduce the actual topography of the tactile sensor under different resistance sequences, thereby providing more accurate topography features for input to other decoding algorithms.

[0093] According to an embodiment of the present invention, without actual manufacturing, the conditional adversarial generation network can be used to predict the topography features of the microstructural sensor, providing a fast means of verification and optimization for sensor design.

[0094] According to an embodiment of the present invention, the tactile sensor 110 may include a plurality of preset regions, and adjacent preset regions among the plurality of preset regions overlap. The initial three-dimensional topography is obtained by the processor 130 by repeatedly performing the following operations: for the i-th preset region among the plurality of preset regions, applying a test pressure to the i-th preset region of the tactile sensor at a preset frequency to obtain a plurality of pressure values, where i is a positive integer; mapping the plurality of pressure values to obtain a test resistance sequence, where the plurality of test resistance values included in the test resistance sequence correspond one-to-one to the plurality of pressure values; performing feature fusion on the test resistance sequence and the preset noise sequence to obtain a second fusion feature; and using a conditional adversarial generation network (CGAN, Conditional Generative Adversarial Network) to process the second fusion feature to obtain the initial three-dimensional topography corresponding to the i-th preset region.

[0095] According to an embodiment of the present invention, the magnitude and direction of the test pressure can be selected according to the actual situation and are not limited herein. For example, the direction of the test pressure can be vertically downward or obliquely downward.

[0096] According to an embodiment of the present invention, the test resistance sequence and the preset noise sequence have the same dimension. The test resistance sequence can be subtracted from the preset noise sequence to obtain the second fusion feature.

[0097] Figure 9 The flowchart shows how a processor determines the initial three - dimensional topography of a tactile sensor according to an embodiment of the present invention.

[0098] As Figure 9 shown, for the i - th preset region, after obtaining the test resistance sequence 910 corresponding to the i - th preset region, feature fusion is performed on the test resistance sequence 910 and the preset noise sequence 920 to obtain the second fusion feature 930. The conditional adversarial generation network 940 is used to process the second fusion feature 930 to obtain the initial three - dimensional topography 950 corresponding to the i - th preset region.

[0099] According to an embodiment of the present invention, the artificial - intelligence - based tactile sensing device provided by the embodiment of the present invention can solve the following four problems: 1) Reduce the dependence on the number of physical sensors, and lower the hardware manufacturing and maintenance costs; 2) Empower the sensors through intelligent algorithms to realize signal feature extraction and recognition of a single microstructure or multiple microstructures, and achieve multi - channel perception from software, reducing the system complexity; 3) Establish the relationship between the resistance signal and the force signal of the sensor through a neural network, reducing the sensor calibration process; 4) Through the training of the mutual relationship between a large number of mechanical and electrical signal features and topography features of the sensors, inversely deduce the distribution law and topography features of the microstructures on the sensor surface.

[0100] According to an embodiment of the present invention, the artificial - intelligence - based tactile sensing device provided by the embodiment of the present invention identifies the acting area and the magnitude of the acting force of the force on the sensor through an artificial - intelligence algorithm. And finally, it can inversely calculate the surface topography features of the sensor through the electrical signals of the sensor.

[0101] According to an embodiment of the present invention, compared with the traditional hardware decoupling method, the present invention uses a deep - learning algorithm to perform virtual decoupling and soft - channel annotation on tactile signals, such as multiple resistance signals included in the resistance sequence, significantly improving the flexibility and accuracy of tactile information processing. By introducing a deep neural network, efficient decoding and multi - parameter perception of complex tactile signals are realized. Through the concept of virtual channels, the dependence on hardware physical channels is reduced, the functional integration degree and perception ability of the system are improved, and it can flexibly respond to different application scenarios and requirements. Through the data - driven deep - learning model, the present invention has significantly improved the real - time performance of signal processing and decoding. Especially after being locally deployed in an embedded system, the system can achieve efficient and real - time tactile perception. Since the dependence on high - density physical sensors is reduced, the present invention significantly reduces the manufacturing cost and energy consumption. At the same time, through the enhanced adaptability of intelligent algorithms, the system can stably operate in complex and changeable environments, and is particularly suitable for application fields such as robots and wearable devices.

[0102] Based on the above tactile sensing device, an embodiment of the present invention further provides a tactile sensing method.

[0103] Figure 10 A flow chart of a tactile sensing method according to an embodiment of the present invention is shown.

[0104] like Figure 10 As shown, the touch sensing method may include operations S1001 to S1002.

[0105] In operation S1001 , when the tactile sensor is subjected to an external force, a resistance value of the tactile sensor is continuously detected to obtain a resistance sequence, wherein the tactile sensor includes a plurality of NdFeB magnetic junctions.

[0106] In operation S1002, based on the resistance sequence, it is determined that the external force acts on the target area of ​​the tactile sensor, and based on the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor, the magnitude of the external force acting on the target area of ​​the tactile sensor is determined to obtain a force sequence, wherein the multiple force values ​​included in the force sequence correspond one-to-one to the multiple resistance values ​​included in the resistance sequence.

[0107] It should be noted that the tactile sensing method part in the embodiment of the present invention corresponds to the tactile sensing device part in the embodiment of the present invention. The description of the tactile sensing method part specifically refers to the tactile sensing device part, which will not be repeated here.

[0108] It should be noted that, unless it is explicitly stated that there is a sequence of execution between different operations shown in the flowchart in the embodiments of the present invention, or different operations have a sequence of execution in technical implementation, otherwise, the execution order of multiple operations may not be particular, and multiple operations may also be executed simultaneously.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of apparatuses, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions. Those skilled in the art can understand that the features described in various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0110] The embodiments of the present invention have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present invention is defined by the appended embodiments and their equivalents. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.

Claims

1. A tactile sensing device, characterized in that: The device comprises: A tactile sensor, wherein the tactile sensor comprises a plurality of neodymium iron boron magnetic structures; a measuring circuit, wherein a first end of the measuring circuit is electrically connected to a first end of the tactile sensor, a second end of the measuring circuit is electrically connected to a second end of the tactile sensor, and the measuring circuit is used to continuously detect a resistance value of the tactile sensor when the tactile sensor is subjected to an external force to obtain a resistance sequence; A processor is electrically connected to the output end of the measurement circuit, and is used to determine, based on the resistance sequence, the target area of ​​the tactile sensor where the external force acts, and to determine, based on the target area, the resistance sequence, and an initial three-dimensional topography of the tactile sensor, the magnitude of the external force acting on the target area of ​​the tactile sensor, to obtain a force sequence, wherein the multiple force values ​​included in the force sequence correspond one-to-one to the multiple resistance values ​​included in the resistance sequence.

2. The device according to claim 1, characterized in that The processor determining, according to the resistance sequence, a target area of ​​the tactile sensor on which the external force acts includes: extracting peak features from the plurality of resistance values; Using a first prediction model, a second prediction model, and a third prediction model to process the peak feature respectively, to obtain a first prediction result, a second prediction result, and a third prediction result outputted by the first prediction model, the second prediction model, and the third prediction model respectively, wherein the first prediction model, the second prediction model, and the third prediction model include different network structures; A target area of ​​the tactile sensor on which the external force acts is determined according to the first prediction result, the second prediction result, and the third prediction result.

3. The device according to claim 1 or 2, characterized in that: The processor determines the magnitude of the external force acting on the target area of ​​the tactile sensor according to the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor, and obtains the force sequence including: Determining a target three-dimensional shape corresponding to the target area according to the target area and the initial three-dimensional shape; The force sequence is obtained according to the target three-dimensional morphology and the resistance sequence.

4. The device according to claim 3, characterized in that The processor obtains the force sequence according to the target three-dimensional morphology and the resistance sequence, including: Extracting features of the resistance sequence using a long short-term memory network to obtain resistance features; Using a convolutional neural network to extract features of the target three-dimensional shape to obtain shape features; Performing feature fusion on the resistance feature and the morphology feature to obtain a first fusion feature; The first fusion feature is mapped to obtain the force sequence.

5. The device according to claim 4, characterized in that The initial three-dimensional shape is obtained by measuring the shape of the tactile sensor in the initial state using a white light interferometer.

6. The device according to claim 4, characterized in that The tactile sensor includes a plurality of preset areas, adjacent preset areas of the plurality of preset areas overlap, and the initial three-dimensional shape is obtained by cyclically performing the following operations: For an i-th preset area among the multiple preset areas, applying a test pressure to the i-th preset area of ​​the tactile sensor at a preset frequency to obtain multiple pressure values, wherein i is a positive integer; Mapping the multiple pressure values ​​to obtain a test resistance sequence, wherein the multiple test resistance values ​​included in the test resistance sequence correspond one-to-one to the multiple pressure values; Performing feature fusion on the test resistance sequence and the preset noise sequence to obtain a second fusion feature; The second fused features are processed using a conditional generative adversarial network to obtain an initial three-dimensional shape corresponding to the i-th preset area.

7. The device according to claim 2, characterized in that The first prediction model includes a support vector machine model; The second prediction model includes a random forest model; The third prediction model includes a decision tree model.

8. The device according to claim 1 or 2, characterized in that: The target region includes at least one NdFeB magnetic microstructure.

9. The device according to claim 1 or 2, characterized in that: The tactile sensor is obtained according to the following operations: forming a conductive nano-silver coating on the surface of a plurality of NdFeB magnetic micron particles to obtain a silver / NdFeB core-shell structure; The silver / neodymium iron boron core-shell structure is mixed with silica gel and magnetized to obtain a magnetized mixture; A preset magnetic field is applied to the magnetized mixture using a permanent magnet to obtain the multiple NdFeB magnetic structures, and then the tactile sensor is obtained, wherein the multiple NdFeB magnetic structures have different sizes and different morphologies.

10. A tactile sensing method, characterized in that: The method comprises: When the tactile sensor is subjected to an external force, the resistance value of the tactile sensor is continuously detected to obtain a resistance sequence, wherein the tactile sensor includes a plurality of NdFeB magnetic structures; According to the resistance sequence, the target area of ​​the tactile sensor on which the external force acts is determined, and according to the target area, the resistance sequence and the initial three-dimensional morphology of the tactile sensor, the magnitude of the external force acting on the target area of ​​the tactile sensor is determined to obtain a force sequence, wherein the multiple force values ​​included in the force sequence correspond one-to-one to the multiple resistance values ​​included in the resistance sequence.

Images