Multi-mode tactile sensor, tactile sensing system and detection method thereof

Abstract

The invention relates to the technical field of sensors, and discloses a multi-mode tactile sensor, a tactile sensing system and a detection method thereof, the sensor comprises a base, a printed circuit board, a sponge body, a permanent magnet, a colloidal silica, a resistance measuring instrument and a magnetic sensing circuit; the printed circuit board is arranged at the bottom of the base; the sponge body is arranged above the base; the silica gel body covers the sponge body; the permanent magnet is embedded into the top of the inner side of the silica gel body; the resistance measuring instrument is arranged on the printed circuit board, is connected with the sponge body and is used for detecting a resistance signal of the sponge body and sending the resistance signal to the upper computer; the magnetic sensing circuit is arranged on the printed circuit board and used for detecting air pressure signals in the silica gel body and magnetic field signals around the permanent magnet and sending the air pressure signals and the magnetic field signals to an upper computer. According to the multi-mode tactile sensor provided by the invention, the pressure accurate measurement precision is improved according to collaborative analysis of air pressure change, resistance fluctuation and magnetic field displacement, and the multi-mode tactile sensor can adapt to detection requirements of different pressing environments through the advantages of different mode signals.

Description

Technical Field

[0001] The present invention relates to the field of sensor technology, and in particular to a multimodal tactile sensor, a tactile sensing system and a detection method thereof. Background Art

[0002] Tactile sensors are devices that sense physical contact, pressure, temperature, and other mechanical or environmental parameters. They are used in robotics, medical equipment, consumer electronics, and other fields. Tactile sensors convert mechanical signals into electrical signals through various physical mechanisms. They include piezoresistive, capacitive, piezoelectric, optical, and magnetic tactile sensors.

[0003] Piezoresistive tactile sensors measure pressure by exploiting the change in resistance of a material under pressure. However, existing piezoresistive tactile sensors are generally single-mode, with a limited sensing range that cannot meet the needs of diverse environments. They also suffer from low measurement accuracy in complex pressure environments.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, an object of the present invention is to provide a multimodal tactile sensor, a tactile sensing system and a detection method thereof, so as to solve the problem that the existing piezoresistive tactile sensors cannot meet the requirements of diversified pressing environments, resulting in low detection accuracy.

[0006] The technical solutions of the present invention are as follows:

[0007] In a first aspect, the present invention provides a multimodal tactile sensor for connecting to a host computer, comprising: a base, a printed circuit board, a sponge, a permanent magnet, a silicone body, a resistance meter, and a magnetic sensing circuit;

[0008] The printed circuit board is arranged at the bottom of the base;

[0009] The sponge is arranged above the base; the silicone cover is arranged on the sponge; and the permanent magnet is embedded in the top of the inner side of the silicone;

[0010] The resistance measuring instrument is arranged on the printed circuit board and connected to the sponge, and is used to detect the resistance signal of the sponge and send it to the host computer;

[0011] The magnetic sensing circuit is arranged on the printed circuit board, and is used to detect the air pressure signal inside the silicone body and the magnetic field signal around the permanent magnet, and send the air pressure signal and the magnetic field signal to the host computer.

[0012] According to a further configuration of the present invention, the resistance measuring instrument includes a multivibrator circuit and a first microprocessor; wherein,

[0013] The multivibrator circuit is connected to the sponge, and is used to detect the resistance change of the sponge and output a rectangular square wave signal to the first microprocessor;

[0014] The first microprocessor is connected to the multivibrator circuit and is used to obtain a resistance signal of the corpus cavernosum according to the rectangular square wave signal and send the resistance signal to a host computer.

[0015] According to a further configuration of the present invention, the magnetic sensing circuit includes a three-dimensional Hall sensor, an air pressure sensor and a second microprocessor; wherein,

[0016] The three-dimensional Hall sensor is connected to the second microprocessor and is used to detect the magnetic field signal around the permanent magnet;

[0017] The air pressure sensor is connected to the second microprocessor and is used to detect the air pressure signal inside the silicone body;

[0018] The second microprocessor is used to send the magnetic field signal and the air pressure signal to a host computer.

[0019] According to a further configuration of the present invention, a pressing plate is provided on the base; the silicone body is fixed to the base by the pressing plate.

[0020] According to a further configuration of the present invention, a wire hole is provided on the base, a wire is connected to the inner side of the sponge, and the wire is connected to the bottom layer of the printed circuit board through the wire hole.

[0021] According to a further configuration of the present invention, the sponge is made by soaking in conductive ink; the conductive ink is obtained by mixing carbon powder and polysilicone solution.

[0022] In a second aspect, the present invention provides a tactile sensing system, which includes a host computer and a multimodal tactile sensor as described above; the host computer is respectively connected to the resistance measuring instrument and the magnetic sensing circuit, and the host computer is used to fuse the resistance signal, the air pressure signal and the magnetic field signal to obtain the pressing force and the pressing position.

[0023] In a third aspect, the present invention provides a tactile sensing detection method based on the above-mentioned tactile sensing system, which comprises the steps of:

[0024] Obtaining the resistance signal collected by the resistance measuring instrument and the air pressure signal and magnetic field signal collected by the magnetic sensing circuit;

[0025] Performing data preprocessing on the resistance signal, the air pressure signal, and the magnetic field signal based on a neural network and then outputting the preprocessed data;

[0026] Divide the pressing area into the central area and the edge area;

[0027] For the central area, a linear regression algorithm is used to obtain the relationship model between air pressure and pressing intensity. For the edge area, a neural network is used for training and modeling to obtain the relationship model between the fusion feature value and pressing intensity.

[0028] The magnetic field signal is fused with the fusion features of resistance and air pressure to obtain the pressing intensity and pressing position.

[0029] According to a further configuration of the present invention, the step of using a neural network to train and model the edge region to obtain a relationship model between the fusion characteristic value and the pressure includes:

[0030] The pressure signal and resistance signal are partitioned and fused based on four independent blocks in the edge area of the tactile sensor. Each block dynamically adjusts the weight of the pressure signal and resistance signal according to the force range.

[0031] Normalize the pressure signal and the resistance signal, multiply them by the corresponding weights respectively, and add them together to obtain the fusion feature value;

[0032] A relationship model between the fused feature value and the pressing intensity is established according to the fused feature value.

[0033] According to a further configuration of the present invention, the step of performing data preprocessing on the resistance signal, the air pressure signal, and the magnetic field signal based on a neural network and then outputting the data comprises:

[0034] performing outlier detection on the resistance signal, the air pressure signal, and the magnetic field signal to remove outliers;

[0035] Performing data cleaning on the resistance signal, the air pressure signal, and the magnetic field signal to remove rows containing NaN or missing values;

[0036] Extract features and target variables from the cleaned data and fuse them into a whole;

[0037] The extracted features and target variables are standardized.

[0038] The present invention provides a multimodal tactile sensor, a tactile sensing system and a detection method thereof. The multimodal tactile sensor is used to connect to a host computer and includes: a base, a printed circuit board, a sponge, a permanent magnet, a silicone body, a resistance meter and a magnetic sensing circuit; the printed circuit board is arranged at the bottom of the base; the sponge is arranged above the base; the silicone body cover is arranged on the sponge; the permanent magnet is embedded in the top of the inner side of the silicone body; the resistance meter is arranged on the printed circuit board and connected to the sponge, the resistance meter is used to detect the resistance signal of the sponge and send it to the host computer; the magnetic sensing circuit is arranged on the printed circuit board, the magnetic sensing circuit is used to detect the air pressure signal inside the silicone body and the magnetic field signal around the permanent magnet, and the air pressure signal and the magnetic field signal are sent to the host computer. The multimodal tactile sensor provided by the present invention can improve the pressure measurement accuracy based on the coordinated analysis of air pressure changes, resistance fluctuations and magnetic field displacements, and can adapt to the detection requirements of different pressing environments through the advantages of different modal signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary personnel in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0040] Figure 1 It is a structural schematic diagram of the multimodal tactile sensor in the present invention.

[0041] Figure 2 It is a schematic diagram of the internal structure of the multimodal tactile sensor in the present invention.

[0042] Figure 3 It is a principle block diagram of the tactile sensing system in the present invention.

[0043] Figure 4 It is a flow chart of the multi-tactile sensing detection method of the present invention.

[0044] Figure 5 It is a schematic diagram of the multimodal fusion algorithm in the present invention.

[0045] Figure 6 It is the optimal hyperplane graph of SVM in this invention.

[0046] Figure 7 Schematic diagram of the region division of the hyperplane in the present invention.

[0047] The marks in the accompanying drawings are: 1. base; 2. printed circuit board; 3. sponge; 4. permanent magnet; 5. silicone body; 6. resistance meter; 61. multivibrator circuit; 62. first microprocessor; 7. magnetic sensing circuit; 71. three-dimensional Hall sensor; 72. air pressure sensor; 73. second microprocessor; 8. host computer; 9. pressure plate; 10. wire hole. DETAILED DESCRIPTION

[0048] The present invention provides a multimodal tactile sensor, a tactile sensing system, and a detection method thereof. To clarify the objectives, technical solutions, and effects of the present invention, the present invention is further described below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0049] In the embodiments and patent claims, unless otherwise specified herein, the words "a," "an," "the," and "the" may include plural forms. If the embodiments of the present invention include descriptions of "first," "second," etc., such descriptions are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly specifying the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features.

[0050] It should be further understood that the term "comprising" used in the description of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when an element is said to be "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or there can be intermediate elements. In addition, "connected" or "coupled" as used herein can include wireless connections or wireless couplings. The term "and / or" as used herein includes all or any units and all combinations of one or more of the items listed in association.

[0051] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0052] In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0053] Please also see Figures 1 to 3 , the present invention provides a preferred embodiment of a tactile sensing system.

[0054] In some embodiments, as Figures 1 to 3 As shown, the present invention provides a tactile sensing system, which includes a host computer 8 and a multimodal tactile sensor. The multimodal tactile sensor includes: a base 1, a printed circuit board 2, a sponge 3, a permanent magnet 4, a silicone body 5, a resistance meter 6, and a magnetic sensing circuit 7; the host computer 8 is connected to the resistance meter 6 and the magnetic sensing circuit 7 respectively; the printed circuit board 2 is arranged at the bottom of the base 1; the sponge 3 is arranged above the base 1; the silicone body 5 is covered on the sponge 3; the permanent magnet 4 is embedded in the top of the inner side of the silicone body 5; the resistance meter 6 is arranged on the printed circuit board 2 and connected to the sponge 3, and is used to detect the resistance signal of the sponge 3 and send it to the host computer 8; the magnetic sensing circuit 7 is arranged on the printed circuit board 2, and is used to detect the air pressure signal inside the silicone body 5 and the magnetic field signal around the permanent magnet 4, and send the air pressure signal and the magnetic field signal to the host computer 8. The host computer 8 is used to fuse the resistance signal, the air pressure signal and the magnetic field signal to obtain the pressing intensity and the pressing position.

[0055] In this embodiment, the base 1 is approximately cylindrical, the printed circuit board 2 is mounted on the bottom of the base 1, and the sponge 3 is mounted above the base 1. The silicone body 5 is mounted above the base 1 and covers the sponge 3, with a space between the sponge 3 and the silicone body 5. The permanent magnet 4 is embedded in the top of the inner side of the silicone body 5, and is located at the center of the silicone body 5 and spaced apart from the sponge 3.

[0056] The printed circuit board 2 has two layers, namely a top layer and a bottom layer. The magnetic sensing circuit 7 is integrated in the top layer of the printed circuit board to facilitate the detection of changes in the air pressure in the air cavity of the silicone body. The resistance measuring instrument 6 is integrated in the bottom layer of the printed circuit board and is connected to the sponge 3 through a wire. The resistance measuring instrument 6 and the magnetic sensing circuit 7 are integrated on the printed circuit board 2. The resistance measuring instrument 6 is connected to the sponge 3, which is a sponge resistor with good electrical properties and high elasticity obtained by soaking in conductive ink. When the sponge 3 is compressed by pressure, the conductive network inside the sponge 3 will change, thereby affecting its own conductivity, and the resistance length will change significantly. When the resistance of the sponge 3 changes, the resistance measuring instrument 6 can collect the resistance signal of the sponge 3 and output it.

[0057] Because the silicone body 5 is an airbag structure with space inside, when the silicone body 5 is pressed by the outside world, the air inside the silicone airbag is discharged, causing the air pressure inside the silicone body 5 to change instantaneously. According to the definition formula of pressure, when the force-bearing area is constant, the pressure will also change due to the change in pressure, so the magnitude of the pressing force can be derived from the change in the internal gas. In addition, because the permanent magnet 4 is installed in the silicone body 5, when the silicone body 5 is pressed, it will drive the permanent magnet 4 to deviate accordingly, thereby causing the magnetic field distribution around the permanent magnet 4 to change. The magnetic sensing circuit 7 can detect the air pressure signal inside the silicone body 5 and the magnetic field signal around the permanent magnet 4. In some embodiments, multiple silicone airbags can be integrated into the sensor device to achieve multi-region / multi-directional air pressure detection, that is, multiple air pressure detection, to improve the overall detection accuracy.

[0058] The resistance measuring instrument 6 and the magnetic sensing circuit 7 are respectively connected to the host computer 8, and can feed back the detected resistance signal, air pressure signal and magnetic field signal to the host computer 8. The host computer 8 can fuse the resistance signal, the air pressure signal and the magnetic field signal based on a fusion algorithm to obtain the pressing force and pressing position.

[0059] In the above technical solution, the multimodal tactile sensor provided by the present invention can improve the pressure measurement accuracy based on the coordinated analysis of air pressure changes, resistance fluctuations, and magnetic field displacements; through the advantages of different modal signals, it can adapt to the detection needs of pressing environments in different ranges and directions; compensate for the sudden changes or abnormal interference that may exist in single-modal sensing; at the same time, adopt a multimodal fusion algorithm to improve the robustness of the sensor; combine multiple sensing mechanisms and approaches to enhance the durability of the sensor and improve manufacturing efficiency.

[0060] In some embodiments, as Figure 3 As shown, the resistance measuring instrument 6 includes a multivibrator circuit 61 and a first microprocessor 62; wherein, the multivibrator circuit 61 is connected to the corpus spongiosum 3, for detecting the resistance change of the corpus spongiosum 3 and outputting a rectangular square wave signal to the first microprocessor 62; the first microprocessor 62 is connected to the multivibrator circuit 61, for obtaining the resistance signal of the corpus spongiosum 3 according to the rectangular square wave signal, and sending the resistance signal to the host computer 8.

[0061] In this embodiment, the resistance measuring instrument 6 is equipped with 16 sponge detection channels. The multivibrator circuit 61 utilizes a 555 oscillator, and the first microprocessor 62 can be an STM32C8T6 microprocessor. Both the multivibrator circuit 61 and the first microprocessor 62 are powered by a lithium battery. When the resistance of the corpus sponge 3 changes, the multivibrator circuit 61 utilizes deep positive feedback, using resistor-capacitor coupling to alternately turn on and off two electronic components, thereby self-exciting and generating a square wave output corresponding to the resistance value, i.e., a rectangular wave pulse signal. The first microprocessor 62 converts the multivibrator frequency output by the multivibrator circuit 61 into the corresponding resistance value of the conductive sponge, and outputs the resistance signal to the host computer 8 via the UART serial port. The serial port assistant displays the numerical change in the resistance of the corpus sponge 3 during the compression process.

[0062] In some embodiments, as Figure 3 As shown, the magnetic sensing circuit 7 includes a three-dimensional Hall sensor 71, an air pressure sensor 72, and a second microprocessor 73. The three-dimensional Hall sensor 71 is connected to the second microprocessor 73 to detect the magnetic field signal around the permanent magnet 4; the air pressure sensor 72 is connected to the second microprocessor 73 to detect the air pressure signal inside the silicone body 5; and the second microprocessor 73 is used to send the magnetic field signal and the air pressure signal to the host computer 8.

[0063] In this embodiment, air pressure data inside the silicone body 5 is collected via an air pressure sensor 72. In one implementation, the air pressure sensor 72 can be a WF183D air pressure sensor. When the silicone body 5 is pressed, the permanent magnet 4 on top of the silicone body 5 shifts, causing the surrounding magnetic field to change. The three-dimensional Hall effect sensor 71 can detect the magnetic field information around the permanent magnet 4 and convert it into an electrical signal, i.e., a magnetic field signal. The pressure sensor 72 and the second microprocessor 73 communicate via a UART serial port, and the three-dimensional Hall sensor 71 communicates with the second microprocessor 73 via an IIC bus. The second microprocessor 73 can send sensor configuration information and sensing information requests to the pressure sensor 72 and the three-dimensional Hall sensor 71. The pressure sensor 72 and the three-dimensional Hall sensor 71 can upload sensing information to the second microprocessor 73, which receives and processes the sensor information and sends the final acquired pressure signal and magnetic field signal to the mainboard of the host computer 8 via wireless communication (such as Bluetooth, Wi-Fi, ZigBee, etc.), thereby realizing remote data transmission and monitoring. At the same time, the serial port assistant can be used to display the real-time changes of the internal air pressure and the air cavity magnetic field. In one implementation, the three-dimensional Hall sensor 71 can use a Hall sensor of the MLX90393 model, and the second microprocessor 73 can use a microprocessor of the STM32C8T6 model. In some embodiments, multiple magnetic field sensors can be arranged in the sensing device to achieve more accurate magnetic field change detection in three-dimensional space.

[0064] In some embodiments, as Figure 1 As shown, a pressing plate 9 is provided on the base 1 , and a portion of the silicone body 5 is fixed on the base 1 by the pressing plate 9 , so that the silicone body 5 will not leak when deformed by force.

[0065] In some embodiments, as Figure 1 As shown, a wire hole 10 is provided on the base 1 , a wire is connected to the inner side of the sponge 3 , and the wire is connected to the bottom layer of the printed circuit board 2 through the wire hole 10 .

[0066] In this embodiment, wire holes 10 are arranged at intervals on the side of the base 1. The wires connected to the sponge 3 pass through the wire holes 10 and are connected to the bottom layer of the printed circuit board 2 at the bottom of the base 1. In this way, the resistance measuring instrument 6 can measure the resistance change of the sponge 3.

[0067] In some embodiments, the sponge 3 is made by soaking in conductive ink; the conductive ink is obtained by mixing carbon powder and silicone solution.

[0068] In this embodiment, the sponge 3 is a conductive, elastic sponge 3, which is made by soaking it in conductive ink. When compressed, the conductive grid inside the sponge 3 changes. The conductive ink can be prepared by mixing carbon powder and a silicone solution, or other materials with good conductivity and stability, such as graphene and carbon nanotubes. In this embodiment, the conductive ink used is a mixture of carbon powder and a silicone solution.

[0069] In some embodiments, as Figure 4 As shown, the present invention provides a tactile sensing detection method based on the above-mentioned tactile sensing system, which includes the steps of:

[0070] S100, obtaining a resistance signal collected by the resistance measuring instrument and / or obtaining an air pressure signal and a magnetic field signal collected by the magnetic sensing circuit;

[0071] Specifically, please combine Figure 5 The host computer communicates with the resistance measuring instrument and the magnetic sensing circuit to receive the resistance signal collected by the resistance measuring instrument and the magnetic field signal and air pressure signal collected by the magnetic sensing circuit.

[0072] S200, performing data preprocessing on the resistance signal, the air pressure signal, and the magnetic field signal based on a neural network and outputting the preprocessed data;

[0073] Specifically, a neural network based on the PaddlePaddle framework is used to fuse the collected resistance signals, air pressure signals, and magnetic field signals.

[0074] Please combine Figure 5 In the case of single mode, resistance, pressure and magnetic field are completed separately. Resistance uses nonlinear fitting (power function form), pressure uses linear fitting, and magnetic field uses support vector machine (SVM).

[0075] In the case of multimodality, this embodiment uses an algorithm based on the PaddlePaddle framework to fuse the three variables together for output. First, the collected resistance signal, air pressure signal, and magnetic field signal are preprocessed and the three signals are fused into a whole.

[0076] In some embodiments, the step of performing data preprocessing on the resistance signal, the air pressure signal, and the magnetic field signal based on a neural network and then outputting the data includes:

[0077] S210, performing outlier detection on the resistance signal, the air pressure signal, and the magnetic field signal to remove outliers;

[0078] S220, performing data cleaning on the resistance signal, the air pressure signal, and the magnetic field signal to remove rows containing NaN or missing values;

[0079] S230, extracting features and target variables from the cleaned data and integrating them into a whole;

[0080] S240: Standardize the extracted features and target variables.

[0081] Specifically, please combine Figure 5 During data preprocessing of the received resistance, pressure, and magnetic field signals, an outlier check is first performed. This embodiment uses the quartile detection method, which assumes that normal data is typically concentrated within the middle 50% of the data, and values outside this range are likely outliers. Data cleaning is then performed to confirm the presence of relevant data for all three signals and remove rows containing NaN or missing values to prevent program execution failures due to invalid data. Features and target variables are then extracted from the cleaned data and integrated into a single entity. Finally, the features and target variables are normalized to ensure greater numerical stability, facilitating subsequent model training.

[0082] S300, dividing the pressed area into a central area and a peripheral area;

[0083] Specifically, please combine Figures 5 to 7 In the fused feature space, the SVM algorithm is used for model training. SVM maximizes the interval between different categories of data by finding the optimal hyperplane, thereby achieving high-precision classification. The hyperplane is divided into five regions (such as Figure 5 A, B, C, D, E in the figure), the central permanent magnet area is defined as the central area (i.e. Figure 5 A in the figure), the surrounding area is the edge area, and the surrounding edge plane area (such as Figure 5 B, C, D, E), each area occupies 90 degrees. After being divided into two parts, the other data are processed.

[0084] S400: For the central area, a linear regression algorithm is used to obtain a relationship model between air pressure and pressing intensity. For the edge area, a neural network is used for training and modeling to obtain a relationship model between fusion feature values and pressing intensity.

[0085] Specifically, please combine Figure 5After completing the division of the pressing area, in the subsequent data processing, the data is divided into two parts: the central area and the edge area according to the different pressing areas for targeted processing. For the central area, considering that there is a relatively stable and significant linear relationship between the air pressure change and the pressing intensity in this area, a linear regression algorithm is used for modeling. Through the data pairs of air pressure characteristics and corresponding pressing intensity, a best fit straight line is fitted using the least squares method to establish a quantitative relationship model between air pressure and pressing intensity. The quantitative relationship model between air pressure and pressing intensity can accurately infer the corresponding pressing intensity based on the measured air pressure value, providing strong support for subsequent force detection.

[0086] For edge regions, the compression behavior is more complex, influenced not only by changes in air pressure but also by changes in corpus cavernosum resistance. Therefore, the combined effects of these two modalities on compression intensity need to be comprehensively considered. In this case, a neural network model based on the PaddlePaddle framework is used for training and modeling. This model is based on a multilayer perceptron architecture. The input layer receives normalized feature data, which is then transformed nonlinearly by multiple hidden layers. The output layer ultimately generates a predicted compression intensity value. The hidden layers utilize a combination of linear layers, batch normalization layers, and ReLU activation functions to capture complex nonlinear relationships. To improve generalization and prevent overfitting, this model uses K-fold cross-validation to partition the dataset. Dropout layers and L2 regularization are introduced during training. Training is optimized using a combination of linear warmup and cosine annealing learning rate schedulers. The MSE loss function is selected, and gradient descent is performed with the AdamW optimizer. This automatically adjusts network weights and bias parameters to gradually learn the optimal mapping between input features and target outputs.

[0087] In some embodiments, the step of using a neural network to train and model the edge region to obtain a relationship model between the fusion feature value and the pressure includes:

[0088] S410, partitioning and fusing the air pressure signal and the resistance signal based on four independent blocks in the edge area of the tactile sensor, wherein each block dynamically adjusts the weight of the air pressure signal and the resistance signal according to the force range;

[0089] S420, normalizing the air pressure signal and the resistance signal, multiplying them by corresponding weights respectively, and adding them to obtain a fusion feature value;

[0090] S430: Establish a relationship model between the fused feature value and the pressing intensity according to the fused feature value.

[0091] Specifically, please combine Figure 5Because different pressure ranges and directions exhibit different patterns in the modal signals, the PaddlePaddle framework's algorithm is designed with different weight divisions to achieve higher-precision pressure detection under multimodal data fusion. When weighting the pressure and resistance signals by region, the dataset is first divided into three intervals: low-force, medium-force, and high-force based on the tertiles of force. For the data in each interval, the correlation coefficients between the resistance and pressure features and the pressure are calculated. By comparing the absolute values of the two correlation coefficients, the relative importance of the two within that interval is determined. Features with larger absolute values of the correlation coefficient have higher weights within that interval, meaning that the feature has a greater influence on the prediction of pressure within that force range. Finally, the weights of sponge resistance and pressure within each interval are calculated based on the correlation coefficients, ensuring that the contribution of each modal data during the fusion process matches their actual correlation.

[0092] After completing the interval weighting of pressure and resistance, the interval-weighted fusion phase begins. For each pressure data point, a corresponding interval weight is selected based on the force range to which it belongs. The normalized resistance and pressure values are multiplied by the corresponding weights and then added together to obtain the fused feature value. This weighted fusion method ensures that the model can dynamically adjust its reliance on different modal data within different force ranges, fully leveraging the advantages of each modal data in a specific force range and providing more accurate fusion features for subsequent predictions.

[0093] The features and pressure data obtained by the above-mentioned weighted fusion are used to further establish the relationship curve between the fusion features and the force. The combination relationship is constructed by two methods: linear regression and neural network modeling. In linear regression modeling, there is a linear relationship between the air pressure features and the pressure. The model parameters are estimated using the least squares method to obtain a linear relationship curve between the fusion features and the force. In neural network modeling, the PressureNet model with a multi-layer perceptron architecture (built based on the PaddlePaddle framework) is used to learn the complex nonlinear relationship between the fusion features and the pressure through training. After the model is compiled and configured, it is trained using a training data set, and the network weights are adjusted through an optimization algorithm to minimize the difference between the predicted value and the true value. After the training is completed, the PressureNet model can predict the pressure based on the input fusion feature value, thereby achieving accurate prediction of the pressure. It should be noted that in addition to using a neural network based on the PaddlePaddle framework, this embodiment can also use other learning or deep learning algorithms, such as convolutional neural networks (CNNs), long short-term memory networks (LSTMs), etc.

[0094] S500: Fusing the magnetic field signal with the fusion features of resistance and air pressure to obtain a pressing intensity and a pressing position.

[0095] Specifically, please combine Figure 5 In different environments, for example, when resistance signals are not needed, the collected air pressure signals and magnetic field signals can be fused according to the divided areas to obtain the pressing force and pressing position. When the detection environment requires the use of resistance signals, air pressure signals, and magnetic field signals, the fused features are fused with the magnetic field signals according to the divided areas to obtain the pressing force and pressing position.

[0096] In summary, the multimodal tactile sensor, tactile sensing system, and detection method thereof provided by the present invention have the following beneficial effects:

[0097] It can realize multimodal pressure sensing based on the coordinated analysis of air pressure changes, resistance fluctuations, and magnetic field displacement, combined with relevant algorithms of machine learning and deep learning, and at the same time achieve high-precision detection of pressing force and direction; through the advantages of different modal signals, it can adapt to the detection needs of pressing environments in different ranges and directions; it realizes the integrated and intelligent collection and analysis of multimodal signals, and effectively reduces the mutual interference between each modal signal, promoting the diversification of pressure detection environments; the magnetic field sensor can sense extremely small magnetic field changes, and provide real-time feedback on the detection force and direction to achieve high-precision measurement, and does not require experimental calibration compared with traditional ones; the air pressure can sense smaller pressure changes, and the sensor has good durability. No additional calibration steps are required during use, and it can be plug-and-play; a multimodal fusion algorithm is used to improve the robustness of the sensor; a combination of multiple sensing mechanisms and approaches is used to enhance the durability of the sensor and improve production efficiency.

[0098] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A multimodal tactile sensor for connecting to a host computer, characterized in that: include: Base, printed circuit board, sponge, permanent magnet, silicone body, resistance meter and magnetic sensing circuit; The printed circuit board is arranged at the bottom of the base; The sponge is arranged above the base; the silicone cover is arranged on the sponge; and the permanent magnet is embedded in the top of the inner side of the silicone; The resistance measuring instrument is arranged on the printed circuit board and connected to the sponge, and is used to detect the resistance signal of the sponge and send it to the host computer; The magnetic sensing circuit is arranged on the printed circuit board, and is used to detect the air pressure signal inside the silicone body and the magnetic field signal around the permanent magnet, and send the air pressure signal and the magnetic field signal to the host computer.

2. The multimodal tactile sensor according to claim 1, wherein: The resistance measuring instrument includes a multivibrator circuit and a first microprocessor; wherein, The multivibrator circuit is connected to the sponge, and is used to detect the resistance change of the sponge and output a rectangular square wave signal to the first microprocessor; The first microprocessor is connected to the multivibrator circuit and is used to obtain a resistance signal of the corpus cavernosum according to the rectangular square wave signal and send the resistance signal to a host computer.

3. The multimodal tactile sensor according to claim 1, wherein: The magnetic sensing circuit includes a three-dimensional Hall sensor, an air pressure sensor and a second microprocessor; wherein, The three-dimensional Hall sensor is connected to the second microprocessor and is used to detect the magnetic field signal around the permanent magnet; The air pressure sensor is connected to the second microprocessor and is used to detect the air pressure signal inside the silicone body; The second microprocessor is used to send the magnetic field signal and the air pressure signal to a host computer.

4. The multimodal tactile sensor according to claim 1, wherein: A pressing plate is provided on the base; the silicone body is fixed on the base by the pressing plate.

5. The multimodal tactile sensor according to claim 1, wherein: The base is provided with a wire hole, the inner side of the sponge is connected with a wire, and the wire is connected to the bottom layer of the printed circuit board through the wire hole.

6. The multimodal tactile sensor according to claim 1, wherein: The sponge is made by soaking in conductive ink; the conductive ink is obtained by mixing carbon powder and polysilicone solution.

7. A tactile sensing system, characterized in that: It comprises a host computer and a multimodal tactile sensor as described in any one of claims 1 to 6; the host computer is connected to the resistance measuring instrument and the magnetic sensing circuit respectively, and the host computer is used to fuse the resistance signal, the air pressure signal and the magnetic field signal to obtain the pressing intensity and pressing position.

8. A tactile sensing detection method based on the tactile sensing system according to claim 7, characterized in that: Including steps: Obtaining the resistance signal collected by the resistance measuring instrument and the air pressure signal and magnetic field signal collected by the magnetic sensing circuit; Performing data preprocessing on the resistance signal, the air pressure signal, and the magnetic field signal based on a neural network and then outputting the preprocessed data; Divide the pressing area into the central area and the edge area; For the central area, a linear regression algorithm is used to obtain the relationship model between air pressure and pressing intensity. For the edge area, a neural network is used for training and modeling to obtain the relationship model between the fusion feature value and pressing intensity. The magnetic field signal is fused with the fusion features of resistance and air pressure to obtain the pressing intensity and pressing position.

9. The tactile sensing detection method according to claim 8, characterized in that: The step of using a neural network to train and model the edge area to obtain a relationship model between the fusion characteristic value and the pressure includes: The pressure signal and resistance signal are partitioned and fused based on four independent blocks in the edge area of the tactile sensor. Each block dynamically adjusts the weight of the pressure signal and resistance signal according to the force range. Normalize the pressure signal and the resistance signal, multiply them by the corresponding weights respectively, and add them together to obtain the fusion feature value; A relationship model between the fused feature value and the pressing intensity is established according to the fused feature value.

10. The tactile sensing detection method according to claim 8, characterized in that: The step of performing data preprocessing on the resistance signal, the air pressure signal, and the magnetic field signal based on a neural network and then outputting the data comprises: performing outlier detection on the resistance signal, the air pressure signal, and the magnetic field signal to remove outliers; Performing data cleaning on the resistance signal, the air pressure signal, and the magnetic field signal to remove rows containing NaN or missing values; Extract features and target variables from the cleaned data and fuse them into a whole; The extracted features and target variables are standardized.

Images