Perception feedback method of bionic electronic skin, bionic electronic skin and medium

Abstract

The present application provides a bionic electronic skin perception feedback method, bionic electronic skin and medium, and relates to the field of bionic intelligent tactile perception. The present application acts on the magnetic field through an external gesture action, generates a corresponding target magnetic field disturbance signal in the magnetic field disturbance monitoring timing curve based on the Hall effect, and then captures the original data of the target magnetic field disturbance signal in the magnetic field disturbance monitoring timing curve through a timing sliding window, and then processes the original data of the target magnetic field disturbance signal through an interference decay algorithm and feature extraction to obtain the target magnetic field disturbance signal target magnetic field disturbance signal characteristic data, and finally predicts and analyzes the target magnetic field disturbance signal characteristic data through an MLP multi-layer perception prediction model to obtain the corresponding target gesture action perception feedback result, thereby reducing the high-density integration requirements for sensors, improving the accuracy of interactive perception, and reducing production costs.

Description

Technical Field

[0001] The present application relates to the field of bionic intelligent tactile perception, and in particular to a bionic electronic skin perception feedback method, bionic electronic skin and medium. Background Art

[0002] With the rapid development of information technology, bionic intelligent tactile sensing technology has gradually become a key driving force in the field of human-machine integration and bionic robots. As an important part of this technology, flexible bionic tactile sensing devices have a physical form that is highly compatible with biological organisms and have attracted much attention in recent years.

[0003] However, the existing technologies are generally based on capacitive, inductive, photoelectric, piezoresistive and piezoelectric bionic electronic skins. These bionic electronic skins still have problems such as large sensor integration area, high density and low perception accuracy. At the same time, the preparation process of electronic skin sensors is very complicated, the sensor production cost is high, and it is difficult to mass produce. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a perception feedback method of bionic electronic skin, bionic electronic skin and medium, which sense gesture movements through a magnetic field skin device based on the Hall effect, thereby reducing the high-density integration requirements for sensors, improving the accuracy of interactive perception, and reducing production costs.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a bionic electronic skin perception feedback method, comprising:

[0006] In response to an external gesture action acting on the magnetic field sensing range of the bionic electronic skin, a corresponding target magnetic field disturbance signal is generated in a magnetic field disturbance monitoring timing curve based on the Hall effect;

[0007] Capturing the original data of the target magnetic field disturbance signal in the magnetic field disturbance monitoring time series curve based on the time series sliding window;

[0008] According to the interference decay algorithm, the original data of the target magnetic field disturbance signal is subjected to interference filtering to obtain the target magnetic field disturbance signal filtered data;

[0009] Extracting features from the target magnetic field disturbance signal filtering data to obtain target magnetic field disturbance signal feature data;

[0010] Based on the MLP multi-layer perception prediction model, the characteristic data of the target magnetic field disturbance signal is predicted and analyzed to obtain the target gesture action perception feedback result.

[0011] Furthermore, in some embodiments, before predicting and analyzing the target magnetic field disturbance signal filtering data based on the MLP multi-layer perception prediction model to obtain the target gesture action perception feedback result, the perception feedback method further includes:

[0012] Acquire a sample data set of magnetic field disturbance signals;

[0013] According to the interference decay algorithm, interference filtering is performed on the magnetic field disturbance signal sample data set to obtain the magnetic field disturbance signal sample filtered data set;

[0014] Extracting features from the magnetic field disturbance signal sample filtering data set to obtain a magnetic field disturbance signal sample feature data set;

[0015] Initialize the MLP multi-layer perception original model;

[0016] According to the magnetic field disturbance signal sample preprocessing data set, the MLP multi-layer perception original model is trained to obtain the MLP multi-layer perception prediction model.

[0017] Furthermore, in some embodiments, the MLP multi-layer perception original model is trained according to the magnetic field disturbance signal sample preprocessing data set to obtain the MLP multi-layer perception prediction model, including:

[0018] According to the activation function, the parameters of the magnetic field disturbance signal sample preprocessing data set are predicted in the MLP multi-layer perception original model to obtain the training output data of the MLP multi-layer perception original model;

[0019] According to the binary cross entropy loss function, the loss of the training output data is calculated to obtain the gradient loss data of the original MLP multi-layer perception model;

[0020] According to the Adam optimization algorithm, the parameters of the original MLP multi-layer perception model corresponding to the gradient loss data are reversely iterated and optimized to obtain the MLP multi-layer perception prediction model.

[0021] Furthermore, in some embodiments, after the MLP multi-layer perception original model is trained according to the magnetic field disturbance signal sample preprocessing data set to obtain the MLP multi-layer perception prediction model, the perception feedback method further includes:

[0022] Calculate the model parameters of the MLP multi-layer perception prediction model and match them with the remaining memory space;

[0023] When the model parameter amount is less than the remaining storage space in the memory, the multi-layer network weight values ​​of the MLP multi-layer perception prediction model are decomposed and encapsulated to obtain the network architecture of the MLP multi-layer perception prediction model, and the network architecture is stored in the memory;

[0024] In response to calling the MLP multi-layer perception prediction model, a network architecture is called from the memory based on the multi-layer perception machine forward propagation function to obtain a working process based on the MLP multi-layer perception prediction model;

[0025] Based on modifying the pointer address of the working process, the working process is connected to the shared memory of the external process.

[0026] Further, in some embodiments, capturing the target magnetic field disturbance signal raw data in the magnetic field disturbance monitoring timing curve based on the timing sliding window includes:

[0027] According to the preset time interval, the timing sliding window is translated to the next time period based on the current position of the magnetic field disturbance monitoring timing curve and the corresponding signal data is identified to obtain the target capture signal data;

[0028] When the peak drop value in the target capture signal data is greater than or equal to the preset peak drop value, the target capture signal data is used as the original data of the target magnetic field disturbance signal, and the timing sliding window is stopped from shifting to the next time period;

[0029] When the peak drop value in the target capture signal data is less than the preset peak drop value, the timing sliding window continues to shift to the next time period and identifies the corresponding target capture signal data until the peak drop value in the target capture signal data is greater than or equal to the preset peak drop value.

[0030] Further, in some embodiments, according to the interference decay algorithm, interference filtering is performed on the original data of the target magnetic field disturbance signal to obtain the target magnetic field disturbance signal filtered data, including:

[0031] According to the interference decay algorithm, the real value of the current environment without disturbance signal is obtained;

[0032] Perform difference calculation between the actual value and the preset relative zero point to obtain the current environmental deviation value;

[0033] According to the current environmental deviation value, the original data of the target magnetic field disturbance signal is differentially compensated to obtain the target magnetic field disturbance signal compensation data;

[0034] According to the time series discrete derivative, the target magnetic field disturbance signal compensation data is nonlinearly smoothed to obtain the target magnetic field disturbance signal filtering data.

[0035] Furthermore, in some embodiments, the target magnetic field disturbance signal filtering data is subjected to feature extraction to obtain target magnetic field disturbance signal feature data, including:

[0036] Calculating the mean of the target magnetic field disturbance signal filtering data to obtain first magnetic field disturbance signal characteristic data;

[0037] Performing variance calculation on the target magnetic field disturbance signal filtering data to obtain second magnetic field disturbance signal characteristic data;

[0038] Performing a summation calculation of the absolute values ​​of discrete derivatives of the target magnetic field disturbance signal filtering data to obtain third magnetic field disturbance signal characteristic data;

[0039] Performing peak drop calculation on the target magnetic field disturbance signal filtering data to obtain fourth magnetic field disturbance signal characteristic data;

[0040] Calculating the number of zero crossing points of the discrete derivative of the target magnetic field disturbance signal filtering data to obtain fifth magnetic field disturbance signal characteristic data;

[0041] The target magnetic field disturbance signal filtering data is subjected to a summation calculation of products of adjacent invariant sign values ​​of discrete derivatives to obtain sixth magnetic field disturbance signal characteristic data;

[0042] The first magnetic field disturbance signal characteristic data, the second magnetic field disturbance signal characteristic data, the third magnetic field disturbance signal characteristic data, the fourth magnetic field disturbance signal characteristic data, the fifth magnetic field disturbance signal characteristic data, and the sixth magnetic field disturbance signal characteristic data are merged in parameter dimensions to obtain target magnetic field disturbance signal characteristic data.

[0043] To achieve the above-mentioned purpose, the second aspect of the embodiment of the present application proposes a bionic electronic skin, wherein the bionic electronic skin comprises a flexible epidermal structure, wherein the flexible epidermal structure comprises: a flexible material, a Hall effect sensor and a magnetic sensing mold, wherein the flexible material comprises solid silicone and a micro magnet array, wherein the flexible material is used to construct an induced magnetic field, wherein the Hall effect sensor is used to sense and transmit a magnetic field disturbance signal, wherein the flexible material and the Hall effect sensor are both located inside the magnetic sensing mold;

[0044] A perception processing module, the perception processing module is connected to the Hall effect sensor, and the perception processing module is used to implement the perception feedback method of the bionic electronic skin of the first aspect embodiment when executed.

[0045] Further, in some embodiments, the bionic electronic skin is prepared by the following steps:

[0046] According to the Hall effect, magnetic field modeling and analysis are performed in three-dimensional space to obtain spatial magnetic field analysis results;

[0047] Based on the results of spatial magnetic field analysis, 3D printing modeling is performed to obtain a magnetic sensing mold;

[0048] Casting liquid silicone into a magnetic sensing mold, placing a micro magnet array and a Hall effect sensor into the liquid silicone, and obtaining a flexible skin structure after cooling and solidification;

[0049] The flexible epidermal structure and the sensing processing module are assembled into the shell to obtain a bionic electronic skin.

[0050] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, the perceptual feedback method of the bionic electronic skin of the above-mentioned first aspect embodiment is implemented.

[0051] The embodiments of the present application have the following beneficial effects: the present application acts on the magnetic field through an external gesture action, generates a corresponding target magnetic field disturbance signal in the magnetic field disturbance monitoring timing curve based on the Hall effect, and then captures the original data of the target magnetic field disturbance signal in the magnetic field disturbance monitoring timing curve through a timing sliding window, and then processes the original data of the target magnetic field disturbance signal through an interference decay algorithm and feature extraction to obtain target magnetic field disturbance signal target magnetic field disturbance signal characteristic data, and finally predicts and analyzes the target magnetic field disturbance signal characteristic data through an MLP multi-layer perception prediction model to obtain corresponding target gesture action perception feedback results, thereby reducing the requirements for high-density integration of sensors, improving the accuracy of interactive perception, and reducing production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flow chart of a perception feedback method of a bionic electronic skin provided in an embodiment of the present application;

[0053] Figure 2 is a schematic diagram of measuring magnetic field signals in different environments under the influence of thermal drift provided by an embodiment of the present application;

[0054] Figure 3 The embodiment of the present application provides Figure 1 The flowchart before step S105 in FIG.

[0055] Figure 4 Schematic diagram of the LeakyReLU activation function provided in the embodiment of the present application;

[0056] Figure 5 Schematic diagram of the Sigmoid activation function provided in the embodiment of the present application;

[0057] Figure 6 It is a schematic diagram of the structure of the MLP multi-layer perception original model provided in the embodiment of the present application;

[0058] Figure 7 The embodiment of the present application provides Figure 3 The flowchart of step S305 in ;

[0059] Figure 8It is a scatter plot of the accuracy of various prediction model training provided in the embodiments of the present application;

[0060] Fig. 9 The embodiment of the present application provides Figure 3 The flowchart after step S305 in FIG.

[0061] Fig.10 The embodiment of the present application provides Figure 1 The flowchart of step S102 in FIG.

[0062] Fig.11 is a statistical diagram of the duration of a magnetic field disturbance signal provided in an embodiment of the present application;

[0063] Fig.12 is a sliding schematic diagram of a timing sliding window provided in an embodiment of the present application;

[0064] Fig.13 The embodiment of the present application provides Figure 1 The flowchart of step S103 in FIG.

[0065] Fig.14 is a schematic diagram of a derivative curve of a magnetic field disturbance signal provided in an embodiment of the present application;

[0066] Fig.15 It is a schematic diagram of the characteristics of the magnetic field disturbance signal derivative signal corresponding to different gesture actions provided in the embodiment of the present application;

[0067] Fig.16 The embodiment of the present application provides Figure 1 The flowchart of step S104 in FIG.

[0068] Fig.17 is a schematic diagram of the structure of a bionic electronic skin provided in an embodiment of the present application;

[0069] Fig.18 is a structural schematic diagram of a flexible skin structure provided in an embodiment of the present application;

[0070] Fig.19 This is a flow chart of the preparation of bionic electronic skin provided in an embodiment of the present application;

[0071] Fig. 20 is a line graph of the spatial magnetic field analysis results provided by an embodiment of the present application;

[0072] Reference numerals:

[0073] Bionic electronic skin 100 , flexible epidermal structure 110 , flexible material 111 , Hall effect sensor 112 , magnetic sensing mold 113 , solid silicone 114 , micro magnet array 115 , and sensing processing module 120 . DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0075] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technical personnel in the technical field of this application. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0077] First, some nouns involved in this application are analyzed:

[0078] Bionic electronic skin is an artificial intelligence technology that mimics the perception and response of human skin. It simulates the human skin's ability to perceive external stimuli such as touch, pressure, and temperature by integrating sensors, electronic components, and artificial intelligence algorithms. Such technology has a wide range of application prospects, including robotics, prosthetics, and wearable devices.

[0079] The Multilayer Perceptron (MLP) model is a basic feedforward neural network model. It consists of an input layer, a hidden layer (can have multiple layers), and an output layer, and each layer contains multiple neurons. MLP models the complex relationship between input and output by learning weights.

[0080] Binary Cross Entropy Loss, also known as Logarithmic Loss, is commonly used for binary classification problems. For each sample, it calculates the difference between the model's output and the actual label, expressing the model's uncertainty for each sample.

[0081] The Adam algorithm (Adaptive Moment Estimation) is an adaptive learning rate optimization algorithm, commonly used in the training of deep learning models. Adam combines the first-order moment estimation (average gradient) and the second-order moment estimation (average gradient square) of the gradient to improve the effect of model training by adaptively adjusting the learning rate of each parameter.

[0082] With the rapid development of information technology, bionic intelligent tactile sensing technology has gradually become a key driving force in the field of human-machine integration and bionic robots. As an important part of this technology, flexible bionic tactile sensing devices have a physical form that is highly compatible with biological organisms and have attracted much attention in recent years.

[0083] However, the existing technologies are generally based on capacitive, inductive, photoelectric, piezoresistive and piezoelectric bionic electronic skins. These bionic electronic skins still have problems such as large sensor integration area, high density and low perception accuracy. At the same time, the preparation process of electronic skin sensors is very complicated, the sensor production cost is high, and it is difficult to mass produce.

[0084] The bionic electronic skin perception feedback method, bionic electronic skin and medium provided in the embodiments of the present application relate to the field of bionic intelligent tactile perception.

[0085] Based on this, the embodiments of the present application provide a perception feedback method of a bionic electronic skin, a bionic electronic skin and a medium, which acts on the magnetic field through external gesture actions, and generates a corresponding target magnetic field disturbance signal in the magnetic field disturbance monitoring timing curve based on the Hall effect; then the original data of the target magnetic field disturbance signal is captured in the magnetic field disturbance monitoring timing curve through a timing sliding window, and then the original data of the target magnetic field disturbance signal is processed through an interference decay algorithm and feature extraction to obtain the target magnetic field disturbance signal target magnetic field disturbance signal characteristic data, and finally the target magnetic field disturbance signal characteristic data is predicted and analyzed through an MLP multi-layer perception prediction model to obtain the corresponding target gesture action perception feedback result, thereby reducing the high-density integration requirements for sensors, improving the accuracy of interactive perception, and reducing production costs.

[0086] It should be noted that the perception feedback method of a bionic electronic skin provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements a perception feedback method of a bionic electronic skin, etc., but is not limited to the above forms.

[0087] See also Figure 1 , Figure 1 This is a flow chart of a bionic electronic skin perception feedback method provided in an embodiment of the present application, which may include but is not limited to steps S101 to S104.

[0088] Step S101: in response to an external gesture action acting on the magnetic field sensing range of the bionic electronic skin, a corresponding target magnetic field disturbance signal is generated in a magnetic field disturbance monitoring timing curve based on the Hall effect;

[0089] It should be noted that in order to realize the perception function of the bionic electronic skin, different gestures are applied to the bionic electronic skin, and the gestures are respectively "patting", "stroking" and "scratching" to sense the magnetic field signals and read the corresponding timing magnetic field signals according to the Hall effect to obtain the corresponding target magnetic field disturbance signals.

[0090] Step S102: capturing the original data of the target magnetic field disturbance signal in the magnetic field disturbance monitoring timing curve based on the timing sliding window;

[0091] It should be noted that the original data of the target magnetic field disturbance signal is obtained by sliding the magnetic field disturbance monitoring timing curve downward in time sequence through the timing sliding window to identify the samples.

[0092] Step S103: performing interference filtering on the original data of the target magnetic field disturbance signal according to the interference decay algorithm to obtain filtered data of the target magnetic field disturbance signal;

[0093] It should be noted that the magnetic field measurement of the Hall sensing element is easily affected by temperature. When the ambient magnetic field remains unchanged, the reading of the Hall effect magnetic field sensor will change with the change of temperature, that is, thermal drift. The influence of thermal drift on the sensor may have the following reasons: First, the performance of electronic components is affected by temperature. For example, the resistivity of resistors will change with temperature, which will affect the current and voltage in the circuit, and ultimately reflected in the change of readings; second, the increase in temperature causes the wrapped flexible material to expand, which causes the position of the magnet and the chip to change, and the sensor reading also changes. The problem of thermal drift causes obvious deviations in the magnetic field components measured at different times and places under the wrapping of the same magnetic skin, such as Figure 2 As shown, Figure 2 It is a schematic diagram of the magnetic field signal measurement under the influence of thermal drift in different environments provided by the embodiment of the present application. Therefore, it is necessary to perform interference filtering on the original data of the target magnetic field disturbance signal according to the interference decay algorithm to obtain the target magnetic field disturbance signal filtered data.

[0094] Step S104: extracting features from the target magnetic field disturbance signal filtering data to obtain target magnetic field disturbance signal feature data;

[0095] It should be noted that, in order to further consider the accuracy of subsequent model prediction, feature extraction is further performed on the target magnetic field disturbance signal filtering data features to obtain target magnetic field disturbance signal feature data.

[0096] Step S105: Predict and analyze the target magnetic field disturbance signal feature data based on the MLP multi-layer perception prediction model to obtain the target gesture action perception feedback result.

[0097] It should be noted that the target magnetic field disturbance signal characteristic data is input into the MLP multi-layer perception prediction model for data recognition and judgment to obtain the target gesture action perception feedback result.

[0098] Figure 3 The embodiment of the present application provides Figure 1 The flowchart before step S105 in the flowchart may include but is not limited to steps S301 to S305.

[0099] Step S301: Acquire a magnetic field disturbance signal sample data set;

[0100] It should be noted that, in the collection of the magnetic field disturbance signal sample data set, the corresponding magnetic field disturbance data is obtained through three kinds of artificial gesture stimulation. We observe the sampled data, artificially select the signal generated by each stimulation, capture the signal with a window size of 8 sampling points and record it as the magnetic field disturbance signal sample data set, as shown in Table 1. Table 1 is a 512-group magnetic field disturbance signal sample data set "eskin-3-512" provided in an embodiment of the present application.

[0101] Table 1: Example of gesture stimulus data samples

[0102]

[0103] Step S302: performing interference filtering on the magnetic field disturbance signal sample data set according to the interference decay algorithm to obtain a magnetic field disturbance signal sample filtered data set;

[0104] Specifically, the magnetic field disturbance signal sample data set is interference filtered by using an interference decay algorithm to obtain a magnetic field disturbance signal sample filtered data set.

[0105] Step S303: extracting features from the magnetic field disturbance signal sample filtering data set to obtain a magnetic field disturbance signal sample feature data set;

[0106] It should be noted that the data set is filtered for the magnetic field disturbance signal samples and then the features are extracted in eight dimensions, including variance, mean, force depth, peak difference, severity, and intensity.

[0107] Step S304: Initialize the MLP multi-layer perception original model;

[0108] Specifically, each layer of the original MLP multi-layer perception model has 32 neuron nodes. Figure 4 and Figure 5 As shown, Figure 4 is a schematic diagram of the LeakyReLU activation function provided in an embodiment of the present application, Figure 5 It is a schematic diagram of the Sigmoid activation function provided in the embodiment of the present application. In order to perform gradient propagation and model optimization on the MLP multi-layer perception original model later, LeakyReLU is selected as the activation function of the hidden layer, and the slope of LeakyReLU is set to 0.08. Since there is a classification problem in the output result, the activation function of the output layer of the MLP multi-layer perception original model is set to the Sigmoid activation function, as shown in Table 2, which is the parameter details of the MLP multi-layer perception original model provided in the embodiment of the present application.

[0109] Table 2: Details of the original MLP model parameters

[0110]

[0111] Step S205: training the MLP multi-layer perception original model according to the magnetic field disturbance signal sample preprocessing data set to obtain the MLP multi-layer perception prediction model.

[0112] Specifically, according to Figure 6 As shown, Figure 6 It is a structural diagram of the MLP multi-layer perception original model provided in the embodiment of the present application. The magnetic field disturbance signal sample preprocessing data set is input into the MLP multi-layer perception original model for training, and finally the MLP multi-layer perception prediction model is obtained according to the comparison between the preset iteration number threshold and the training iteration number. In the gradient loss process of training, the binary cross entropy loss function (BCEloss) loss function is used for gradient loss, and the Adam optimization algorithm is used for parameter adjustment in reverse optimization.

[0113] Figure 7 The embodiment of the present application provides Figure 3 The flowchart in step S305 in may include but is not limited to steps S701 to S703.

[0114] Step S701: According to the activation function, the parameters of the magnetic field disturbance signal sample preprocessing data set are predicted in the MLP multi-layer perception original model to obtain the training output data of the MLP multi-layer perception original model;

[0115] It should be noted that in the original MLP multi-layer perception model, the magnetic field disturbance signal sample preprocessing data set is first processed by each hidden layer through Output the hidden layer data results; finally, pass through the output layer The results are classified by threshold to obtain the training output data of the original MLP multi-layer perception model.

[0116] Step S702: According to the binary cross entropy loss function, the training output data is subjected to loss calculation to obtain the gradient loss data of the MLP multi-layer perception original model;

[0117] It should be noted that the binary cross entropy loss function is: BCELoss(p, y) = -[y·log(p)+(1-y)·log(1-p)], where p is the output probability of the model, indicating the probability of belonging to the positive category; y is the actual label, which takes a value of 0 or 1, indicating the true category of the sample; log is the natural logarithm. The goal of this loss function is to minimize the cross entropy between the model output and the actual label.

[0118] Step S703: According to the Adam optimization algorithm, reverse iterative optimization is performed on the parameters of the original MLP multi-layer perception model corresponding to the gradient loss data to obtain the MLP multi-layer perception prediction model.

[0119] It should be noted that the Adam parameters are initialized first, θ is the parameter of the original MLP multi-layer perception model, β1 and β2 are the exponential decay rates of the first-order matrix and the second-order matrix, α is the learning rate, and m and v are the initialization zero vectors, which are used to save the estimated values ​​of the first-order moment and the second-order moment of the gradient respectively. Then in each iteration, for each parameter θ i The gradient g t , perform the following steps:

[0120] Calculate the first-order matrix m t And the second-order matrix:

[0121] m t =β1·m t-1 +(1-β1)·g t ;

[0122]

[0123] Correct the deviation between the first-order and second-order matrices:

[0124]

[0125] Update parameters:

[0126]

[0127] Among them, t represents the number of iterations, and ∈ is a small constant to prevent the denominator from being zero.

[0128] It should also be noted that in the embodiment of the present application, a variety of prediction models are used to predict and analyze the magnetic field disturbance signal sample preprocessing data set and compare them, as shown in Table 3 and Figure 8 As shown in Table 3, the accuracy results of the training of various prediction models provided in the embodiments of the present application are as follows: Figure 8 It is a scatter plot of the accuracy of the training of various prediction models provided in the embodiments of the present application.

[0129] Table 3: Accuracy results of various prediction model training

[0130]

[0131] It can be seen that the MLP multi-layer perception model is used as the optimal model for prediction.

[0132] Fig. 9 The embodiment of the present application provides Figure 3The flowchart after step S305 in the flowchart may include but is not limited to steps S901 to S904.

[0133] Step S901: Calculate the model parameter quantity of the MLP multi-layer perception prediction model, and match the model parameter quantity with the remaining storage space quantity of the memory;

[0134] It should be noted that the matching of the model parameters with the remaining memory space is to avoid the subsequent storage of the MLP multi-layer perception prediction model in the memory of the Arduino uno microcontroller, which may cause a memory stack overflow and lead to model operation crash and data corruption.

[0135] Step S902: when the model parameter amount is less than the remaining storage space amount in the memory, decompose and encapsulate the multi-layer network weight values ​​of the MLP multi-layer perception prediction model, obtain the network architecture of the MLP multi-layer perception prediction model, and store the network architecture in the memory;

[0136] It should be noted that since the program of Arduino uno microcontroller is a unique compilation environment of Arduino, cpp and h header files can be called. Therefore, if you want to deploy the model, you first need to convert the trained model weights into cpp format files. The network of the MLP multi-layer perception prediction model is essentially the edge weights of the network and the threshold of each node, that is, its reasoning process is essentially a matrix operation, and each layer of weights in the MLP multi-layer perception prediction model network can be easily exported. Therefore, the multi-layer network weight values ​​of the MLP multi-layer perception prediction model are decomposed and encapsulated to obtain the network architecture of the MLP multi-layer perception prediction model, and its network architecture is stored in the memory of the Arduino uno microcontroller.

[0137] Step S903: in response to calling the MLP multi-layer perception prediction model, calling the network architecture from the memory based on the multi-layer perception machine forward propagation function to obtain a working process based on the MLP multi-layer perception prediction model;

[0138] Specifically, in response to an external call to the MLP multi-layer perception prediction model, the network architecture is called from the memory based on the multi-layer perceptron forward propagation function (i.e., activation function) to obtain a working process based on the MLP multi-layer perception prediction model, and the current working process is in a suspended state.

[0139] Step S904: Based on modifying the pointer address of the working process, the working process is connected to the shared memory of the external process.

[0140] Specifically, the working process is connected to the shared memory of the external process based on modifying the pointer address of the working process, so that the current working process is in an active ready state.

[0141] Fig.10 The embodiment of the present application provides Figure 1 The flowchart in step S102 may include but is not limited to steps S1001 to S1004.

[0142] Step S1001: according to a preset time interval, the timing sliding window is translated to the next time period based on the current position of the magnetic field disturbance monitoring timing curve and the corresponding signal data is identified to obtain the target capture signal data;

[0143] It should be noted that through 512 groups of experiments, the magnetic field disturbance signals generated by the three actions of "kissing", "stroking" and "scratching" were recorded and statistically analyzed, such as Fig.11 As shown, Fig.11 This is a statistical diagram of the duration of the magnetic field disturbance signal provided by the embodiment of the present application. Most of the durations of the magnetic field disturbances are within 2 seconds, which actually shows that the durations of the three actions are all within 2 seconds. The magnetic field disturbance generated by the applied action is a time series signal. In order to capture the magnetic field disturbance characteristics generated by each action, a time series sliding window is used.

[0144] Step S1002: when the peak drop value in the target capture signal data is greater than or equal to the preset peak drop value, the target capture signal data is used as the target magnetic field disturbance signal original data, and the time sequence sliding window is stopped from shifting to the next time period;

[0145] Step S1003: When the peak drop value in the target capture signal data is less than the preset peak drop value, the timing sliding window continues to shift to the next time period and identifies the corresponding target capture signal data until the peak drop value in the target capture signal data is greater than or equal to the preset peak drop value.

[0146] In step S1002 and step S1003, if Fig.12 As shown, Fig.12 It is a sliding schematic diagram of the timing sliding window provided in an embodiment of the present application. The sliding window is a technology for processing data streams or timing arrays. It performs a series of calculations by defining a fixed-size window and moving the window on the data stream or array. The size of this window is usually pre-set and remains unchanged during the processing. At each step, the window slides forward a certain distance to process the next data fragment. According to the above, the duration of the three actions is less than 2s, the timing size of the timing sliding window is set to 2s, and since the sampling rate of the Hall sensor element is 4 sample points per second, the sample size of the timing sliding window is 8 sample points.

[0147] Fig.13 The embodiment of the present application provides Figure 1The flowchart in step S103 in the flowchart may include but is not limited to steps S1301 to S1304.

[0148] Step S1301: According to the interference decay algorithm, obtain the real value in the current environment without disturbance signal;

[0149] Specifically, the true value under the current environment without disturbance signals refers to the true reading of the sensor when the measurement is stable and there is no external disturbance.

[0150] Step S1302: performing a difference operation between the actual value and the preset relative zero point to obtain a current environment deviation value;

[0151] Specifically, the value of the preset relative zero point is set to 0 on the x-axis component, -200 on the y-axis component, and -550 on the z-axis component, and then the actual value is differenced with the preset relative zero point to obtain the current environmental deviation value.

[0152] Step S1303: performing difference compensation on the original data of the target magnetic field disturbance signal according to the current environmental deviation value to obtain target magnetic field disturbance signal compensation data;

[0153] Specifically, each data in the original data of the target magnetic field disturbance signal is added with the current environmental deviation value as difference compensation to obtain the target magnetic field disturbance signal compensation data.

[0154] Step S1304: nonlinearly smooth the target magnetic field disturbance signal compensation data according to the time series discrete derivative to obtain the target magnetic field disturbance signal filtering data.

[0155] It should also be noted that, since the influence of thermal drift on magnetic field measurement is discrete, a small amount of error will still be generated after relative zero point difference compensation, and these errors will directly affect the quality of the sample signal. Therefore, it is also necessary to perform discrete derivation of the target magnetic field disturbance signal compensation data based on the magnetic field disturbance monitoring timing curve to obtain the magnetic field disturbance signal corresponding to each action. Fig.14 and Fig.15 As shown, Fig.14 is a schematic diagram of a derivative curve of a magnetic field disturbance signal provided in an embodiment of the present application, Fig.15 is a schematic diagram of the characteristics of the magnetic field disturbance signal derivative signal corresponding to different gesture actions provided in the embodiment of the present application. Let x1, x2, ..., x n is a sampling sequence within the sliding window. Since the sampling interval is the same each time, the discrete derivative of the sequence is defined as d(x i )=x i -x i-1 , and define d(x1)=0, then perform discrete derivative on the sequence to obtain the corresponding derivative smooth curve.

[0156] Fig.16 The embodiment of the present application provides Figure 1 The flowchart in step S104 may include but is not limited to steps S1601 to S1607.

[0157] Step S1601: performing mean calculation on the target magnetic field disturbance signal filtering data to obtain first magnetic field disturbance signal characteristic data;

[0158] Specifically, according to the above x1, x2, ..., x n is a sampling sequence within a sliding window, and the discrete derivative is defined as d(x i )=x i -x i-1 , then the calculation formula of the mean is: Among them, S is the abbreviation of sequence.

[0159] Step S1602: performing variance calculation on the target magnetic field disturbance signal filtering data to obtain second magnetic field disturbance signal characteristic data;

[0160] Specifically, the formula for calculating the variance is:

[0161] Step S1603: performing a summation calculation of the absolute values ​​of discrete derivatives on the target magnetic field disturbance signal filtering data to obtain third magnetic field disturbance signal characteristic data;

[0162] Specifically, the calculation formula for the sum of the absolute values ​​of discrete derivatives is

[0163] Step S1604: performing peak drop calculation on the target magnetic field disturbance signal filtering data to obtain fourth magnetic field disturbance signal characteristic data;

[0164] Specifically, the calculation formula for the peak drop is:

[0165] Step S1605: Calculating the number of zero crossings of the discrete derivative of the target magnetic field disturbance signal filtering data to obtain fifth magnetic field disturbance signal characteristic data;

[0166] Specifically, the formula for calculating the number of zero crossings of the discrete derivative is: in is a symbolic function.

[0167] Step S1606: performing a summation calculation of products of adjacent invariant sign values ​​of discrete derivatives on the target magnetic field disturbance signal filtering data to obtain sixth magnetic field disturbance signal characteristic data;

[0168] Specifically, the formula for calculating the number of zero crossings of the discrete derivative is: in is the indicator function.

[0169] Step S1607: Merge the first magnetic field disturbance signal characteristic data, the second magnetic field disturbance signal characteristic data, the third magnetic field disturbance signal characteristic data, the fourth magnetic field disturbance signal characteristic data, the fifth magnetic field disturbance signal characteristic data, and the sixth magnetic field disturbance signal characteristic data in parameter dimensions to obtain target magnetic field disturbance signal characteristic data, as shown in Table 4, which is a definition description of the six types of characteristic data provided in an embodiment of the present application.

[0170] Table 4: Definition and description of six types of characteristic data

[0171]

[0172] It should be noted that the target magnetic field disturbance signal characteristic data and the target magnetic field disturbance signal original data are respectively input into different prediction models to obtain the accuracy evaluation results of different prediction models, as shown in Table 5. Table 5 is the accuracy evaluation results of inputting the characteristic data and the original data into different prediction models provided in the application embodiment.

[0173] Table 5: Accuracy evaluation results of different prediction models with feature data and original data input

[0174]

[0175] It can be seen that using the target magnetic field disturbance signal characteristic data as the input data of the model can significantly improve the prediction accuracy of the model.

[0176] See also Fig.17 and Fig.18 , Fig.17 is a schematic diagram of the structure of a bionic electronic skin provided in an embodiment of the present application, Fig.18 1 is a structural schematic diagram of a flexible epidermal structure provided in an embodiment of the present application, and the above-mentioned perception feedback method based on bionic electronic skin can be realized. The bionic electronic skin 100 includes: a flexible epidermal structure 110, and the flexible epidermal structure 110 includes: a flexible material 111, a Hall effect sensor 112 and a magnetic sensing mold 113. The flexible material 111 includes solid silicone 114 and a micro magnet array 115. The flexible material 111 is used to construct an induced magnetic field, and the Hall effect sensor 112 is used to sense and transmit a magnetic field disturbance signal. The flexible material 111 and the Hall effect sensor 112 are both located inside the magnetic sensing mold 113;

[0177] The perception processing module 120 is connected to the Hall effect sensor 112, and the perception processing module 120 is used to implement the perception feedback method of the bionic electronic skin of the first embodiment when executed.

[0178] See also Fig.19 , Fig.19 This is a flow chart for preparing bionic electronic skin provided in an embodiment of the present application, which may include but is not limited to steps S1901 to S1907.

[0179] Step S1901: Based on the Hall effect, magnetic field modeling analysis is performed in three-dimensional space to obtain spatial magnetic field analysis results;

[0180] Specifically, see Fig. 20 , Fig. 20 It is a line graph of the spatial magnetic field analysis results provided in the embodiment of the present application, and the corresponding spatial magnetic field analysis results can be known.

[0181] Step S1902: Perform 3D printing modeling based on the spatial magnetic field analysis results to obtain a magnetic sensor mold;

[0182] Step S1903: pouring liquid silicone into the magnetic sensor mold, placing a micro magnet array and a Hall effect sensor into the liquid silicone, and obtaining a flexible skin structure after cooling and solidification;

[0183] Step S1904: Assemble the flexible epidermal structure and the sensing processing module into the shell to obtain a bionic electronic skin.

[0184] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned perception feedback method of the bionic electronic skin.

[0185] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0186] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Technical personnel in this field can know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0187] It will be understood by those skilled in the art that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0188] A person of ordinary skill in the art can understand that all or some steps in the method disclosed above, and the functional modules / units in the system and device can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0189] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0190] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0191] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0192] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a random access memory (Random Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), an electrically erasable programmable read only memory (Electrically Erasable Programmable read only memory, EEPROM) flash memory or other memory technology, an optical drive (Compact Disc-Read Only Memory, CD-ROM), a digital versatile disk (Digital Video Disc, DVD) or other optical disk storage, a magnetic box, a tape, a magnetic disk, a magnetic disk storage or other magnetic storage device, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0195] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modifications, equivalent substitutions and improvements made by technical staff in this field without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A bionic electronic skin perception feedback method, characterized in that: include: In response to an external gesture action acting on the magnetic field sensing range of the bionic electronic skin, a corresponding target magnetic field disturbance signal is generated in a magnetic field disturbance monitoring timing curve based on the Hall effect; Capturing the target magnetic field disturbance signal raw data in the magnetic field disturbance monitoring timing curve based on a timing sliding window; According to the interference decay algorithm, the original data of the target magnetic field disturbance signal is subjected to interference filtering to obtain the target magnetic field disturbance signal filtered data; Extracting features from the target magnetic field disturbance signal filtering data to obtain target magnetic field disturbance signal feature data; Based on the MLP multi-layer perception prediction model, the target magnetic field disturbance signal characteristic data is predicted and analyzed to obtain the target gesture action perception feedback result; Wherein, the process of capturing the original data of the target magnetic field disturbance signal in the magnetic field disturbance monitoring timing curve based on the timing sliding window includes the following steps: According to a preset time interval, the timing sliding window is translated to the next time period based on the current position of the magnetic field disturbance monitoring timing curve and the corresponding signal data is identified to obtain the target capture signal data; When the peak drop value in the target capture signal data is greater than or equal to the preset peak drop value, the target capture signal data is used as the target magnetic field disturbance signal original data, and the timing sliding window is stopped from shifting to the next time period; When the peak drop value in the target capture signal data is less than the preset peak drop value, the time sequence sliding window is further shifted to the next time period and the corresponding target capture signal data is identified. Until the peak drop value in the target capture signal data is greater than or equal to a preset peak drop value; Furthermore, the process of extracting features from the target magnetic field disturbance signal filtering data to obtain target magnetic field disturbance signal feature data includes the following steps: Performing mean calculation on the target magnetic field disturbance signal filtering data to obtain first magnetic field disturbance signal characteristic data; Performing variance calculation on the target magnetic field disturbance signal filtering data to obtain second magnetic field disturbance signal characteristic data; Performing a summation calculation of the absolute values ​​of discrete derivatives of the target magnetic field disturbance signal filtering data to obtain third magnetic field disturbance signal characteristic data; Performing peak drop calculation on the target magnetic field disturbance signal filtering data to obtain fourth magnetic field disturbance signal characteristic data; Calculating the number of zero crossing points of the discrete derivative of the target magnetic field disturbance signal filtering data to obtain fifth magnetic field disturbance signal characteristic data; The target magnetic field disturbance signal filtering data is subjected to a summation calculation of products of adjacent invariant sign values ​​of discrete derivatives to obtain sixth magnetic field disturbance signal characteristic data; The first magnetic field disturbance signal characteristic data, the second magnetic field disturbance signal characteristic data, the third magnetic field disturbance signal characteristic data, the fourth magnetic field disturbance signal characteristic data, the fifth magnetic field disturbance signal characteristic data, and the sixth magnetic field disturbance signal characteristic data are merged in parameter dimensions to obtain the target magnetic field disturbance signal characteristic data.

2. The bionic electronic skin sensory feedback method according to claim 1, characterized in that: Before performing prediction analysis on the target magnetic field disturbance signal filtering data based on the MLP multi-layer perception prediction model to obtain the target gesture action perception feedback result, the perception feedback method further includes: Acquire a sample data set of magnetic field disturbance signals; According to the interference decay algorithm, interference filtering is performed on the magnetic field disturbance signal sample data set to obtain a magnetic field disturbance signal sample filtered data set; Extracting features from the magnetic field disturbance signal sample filtering data set to obtain a magnetic field disturbance signal sample feature data set; Initialize the MLP multi-layer perception original model; The MLP multi-layer perception original model is trained according to the magnetic field disturbance signal sample preprocessing data set to obtain the MLP multi-layer perception prediction model.

3. The bionic electronic skin sensory feedback method according to claim 2, characterized in that: The method of training the MLP multi-layer perception original model according to the magnetic field disturbance signal sample preprocessing data set to obtain the MLP multi-layer perception prediction model includes: According to the activation function, parameter prediction is performed on the magnetic field disturbance signal sample preprocessing data set in the MLP multi-layer perception original model to obtain training output data of the MLP multi-layer perception original model; According to the binary cross entropy loss function, the training output data is subjected to loss calculation to obtain the gradient loss data of the MLP multi-layer perception original model; According to the Adam optimization algorithm, reverse iterative optimization is performed on the parameters of the MLP multi-layer perception original model corresponding to the gradient loss data to obtain the MLP multi-layer perception prediction model.

4. The bionic electronic skin sensory feedback method according to claim 2, characterized in that: After the MLP multi-layer perception original model is trained according to the magnetic field disturbance signal sample preprocessing data set to obtain the MLP multi-layer perception prediction model, the perception feedback method further includes: Calculating the model parameter quantity of the MLP multi-layer perception prediction model, and matching the model parameter quantity with the remaining storage space quantity of the memory; When the model parameter amount is less than the remaining storage space amount of the memory, decomposing and encapsulating the multi-layer network weight values ​​of the MLP multi-layer perception prediction model, obtaining the network architecture of the MLP multi-layer perception prediction model, and storing the network architecture in the memory; In response to calling the MLP multi-layer perception prediction model, calling the network architecture from the memory based on the multi-layer perception machine forward propagation function to obtain a working process based on the MLP multi-layer perception prediction model; Based on modifying the pointer address of the working process, the working process is connected to the shared memory of the external process.

5. The bionic electronic skin sensory feedback method according to claim 1, characterized in that: The method of performing interference filtering on the original data of the target magnetic field disturbance signal according to the interference decay algorithm to obtain the target magnetic field disturbance signal filtered data includes: According to the interference decay algorithm, the real value of the current environment without disturbance signal is obtained; Performing a difference operation between the actual value and the preset relative zero point to obtain a current environmental deviation value; According to the current environment deviation value, difference compensation is performed on the original data of the target magnetic field disturbance signal to obtain target magnetic field disturbance signal compensation data; According to the time series discrete derivative, the target magnetic field disturbance signal compensation data is nonlinearly smoothed to obtain the target magnetic field disturbance signal filtering data.

6. A bionic electronic skin, characterized in that: include: A flexible epidermis structure, the flexible epidermis structure comprising: a flexible material, a Hall effect sensor and a magnetic sensing mold, the flexible material comprising solid silicone and a micro magnet array, the flexible material being used to construct an induction magnetic field, the Hall effect sensor being used to sense and transmit a magnetic field disturbance signal, the flexible material and the Hall effect sensor being both located inside the magnetic sensing mold; A perception processing module, wherein the perception processing module is connected to the Hall effect sensor, and wherein the perception processing module is used to implement the perception feedback method of the bionic electronic skin as claimed in any one of claims 1 to 5 when executed.

7. The bionic electronic skin according to claim 6, characterized in that: The bionic electronic skin is prepared by the following steps: According to the Hall effect, magnetic field modeling and analysis are performed in three-dimensional space to obtain spatial magnetic field analysis results; Perform 3D printing modeling based on the spatial magnetic field analysis result to obtain the magnetic sensing mold; Casting liquid silicone into the magnetic sensing mold, placing the micro magnet array and the Hall effect sensor into the liquid silicone, and obtaining the flexible skin structure after cooling and solidification; The flexible epidermal structure and the sensing processing module are assembled into the interior of the shell to obtain the bionic electronic skin.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the perception feedback method of the bionic electronic skin as claimed in any one of claims 1 to 5 is implemented.

Images