A pulse graph neural network haptic object recognition method based on graph learning

Abstract

The application provides a haptic object recognition method based on a pulse graph neural network, and the algorithm comprises the following steps: obtaining a haptic graph, constructing an M-tree haptic graph or a Z-tree haptic graph based on the haptic graph; establishing an object recognition model, training the object recognition model by using the M-tree haptic graph or the Z-tree haptic graph, and the object recognition model comprising an LIF pulse neuron, a topological adaptive graph convolution layer, a full connection layer and a final voting layer; optimizing the object recognition model by using a Gaussian prior distribution loss and an object recognition model reverse propagation training loss; and judging the category of the haptic object by using the optimized object recognition model. The Gaussian prior distribution loss and the object recognition model reverse propagation training loss are weighted, so that the accuracy and stability of the object recognition model for object recognition are improved. Seven approximate functions are used to approximate the partial derivative of the LIF pulse neuron activation function, so that the accuracy of the algorithm is improved.

Description

Technical Field

[0001] This invention relates to the field of object recognition technology, and in particular to a pulse graph neural network-based tactile object recognition method based on graph learning. Background Technology

[0002] Spiking neural networks (object recognition models) are advanced neural network models inspired by biological nervous systems. As a type of deep learning, they possess a solid biological foundation. In object recognition models, the activation mechanism of neurons differs from that of traditional multilayer perceptron networks. Specifically, neurons are not activated in every iteration of propagation, but rather when their membrane potential reaches a specific threshold. When a neuron is activated, it sends signals to other neurons, thereby adjusting their membrane potentials.

[0003] The asynchronous, discrete, and sparsity characteristics of object recognition models enable them to directly process event-based tactile sensor data, achieving efficient information transmission and processing. This characteristic makes object recognition models promising for broad applications in the field of tactile sensing. However, existing technologies still face some challenges. The sparsity of tactile data and the spatiotemporal complexity of high-dimensional space pose challenges to the generalization ability of the models. Furthermore, existing tactile map construction methods have limitations in the spatial connectivity of high-dimensional tactile data, and the non-differentiable nature of impulse activity also affects the perception ability of existing object recognition models in complex tactile situations, thereby reducing the accuracy of the models in predicting object categories. Summary of the Invention

[0004] This invention aims to solve the problem of low accuracy in predicting object categories in existing technologies, and innovatively proposes a tactile object recognition method based on graph learning and pulse graph neural network.

[0005] To achieve the above-mentioned objectives of this invention, this invention provides a method for tactile object recognition based on a pulse graph neural network using graph learning, the algorithm comprising:

[0006] S100. Obtain a tactile map, and construct an M-tree tactile map or a Z-tree tactile map based on the tactile map;

[0007] S200. Establish an object recognition model and train the object recognition model using the M-tree tactile map or Z-tree tactile map. The object recognition model includes LIF spiking neurons, topological adaptive graph convolutional layers, fully connected layers, and a final voting layer.

[0008] S300. The object recognition model is optimized by weighting the Gaussian prior distribution loss and the backpropagation training loss of the LIF spiking neuron object recognition model.

[0009] S400. Use the optimized object recognition model to determine the category of the tactile object.

[0010] As an optional embodiment of the present invention, the calculation steps of the Gaussian prior distribution loss may include:

[0011] S301. Suppose the mean μ of the Gaussian prior distribution is 0 and the variance is σ. 2 The weight parameters of the object recognition model are encoded using the Gaussian prior distribution.

[0012] Its expression is:

[0013]

[0014] Among them, P(C m ) represents the m-th parameter after encoding, C m Let m represent the m-th parameter, and σ represent the standard deviation of the Gaussian prior distribution;

[0015] S302. Using the Gaussian distribution as the probability mass function, and iterating through the parameters of the object recognition model, calculate the probability value of each parameter under the Gaussian prior distribution.

[0016] Its expression is:

[0017]

[0018] Among them, L(C m ) represents the probability value of the m-th parameter, and log() represents taking the logarithm;

[0019] S303. Calculate the Gaussian prior distribution loss based on the probability value of each parameter;

[0020] Its expression is:

[0021]

[0022] Among them, L Gauss Let l represent the Gaussian prior distribution loss, l represent the total number of model parameters, and m represent the number of parameters.

[0023] As an optional embodiment of the present invention, the calculation steps of the backpropagation training loss of the LIF spiking neuron object recognition model may include:

[0024] S304, The expression of the LIF spiking neuron is:

[0025]

[0026] fire a spike&P t =Prest ,P t ≥P th

[0027] Where τ represents the membrane time constant, d represents the infinitesimal element symbol, and P t P represents the membrane potential of a spiking neuron at time t. rest Y represents the resting potential of a neuron. t P represents the influence of external input received by the neuron at time t. th Indicates the neuron's trigger threshold;

[0028] S305. Obtain the pulse generated by the neuron at time t;

[0029] Its expression is:

[0030]

[0031] Among them, E t O(P) represents the impulse generated by the neuron at time t. t ) represents the neuron's spiking activation function, when P t ≥P th The neuron generates a pulse when P t <P th The neuron does not generate pulses at this time;

[0032] S306. Set the resting potential of the neuron to zero, control the frequency and period of pulse firing when the neuron is stimulated by using a pulse firing mechanism, control the state of the membrane potential after the neuron is stimulated by using a membrane potential reset mechanism, and set the initial conditions.

[0033] Its expression is:

[0034]

[0035]

[0036] in, Y represents the neuron's membrane potential at time t preceding time t. t ′ represents external input, w n This represents the weight vector of the nth layer. This represents the pulse generation status of the k-th neuron in the (n-1)-th layer at time t, b n The bias vector of the nth layer;

[0037] S307. The membrane potential P of the spiking neuron at time t is based on the number of layers of the neuron. t Perform the conversion;

[0038] Its expression is:

[0039]

[0040]

[0041] Among them, P t n This represents the membrane potential of the nth layer spiking neuron at time t. This represents the membrane potential of the nth layer spiking neuron at time t-1. Simplified This indicates the pulse generation status of the k-th neuron in the n-th layer at time t-1;

[0042] S308. Calculate the backpropagation training loss of the LIF spiking neuron-based object recognition model using a spiking neural network model.

[0043] Its expression is:

[0044]

[0045] Where L represents the backpropagation training loss of the LIF spiking neuron object recognition model, S represents the total number of haptic map training samples, and R... s O represents the label vector of the s-th tactile data sample. s This represents the output vector of the output layer of the spiking neural network model of the LIF spiking neuron.

[0046] As an optional embodiment of the present invention, the algorithm may further include:

[0047] S500: Use a continuous and differentiable function to approximate the partial derivatives of LIF neuron impulse activity during backpropagation.

[0048] Its expression is:

[0049]

[0050] in, This represents finding the partial derivative, E. t This represents the pulse generated by the neuron at time t. This represents taking the partial derivative with respect to the neuron's impulse activation function, f(P). t ) represents an approximate function of the impulse activation function.

[0051] As an optional embodiment of the present invention, the approximation function may optionally include the Sigmoid function, the Rectangular function, the Piecewise LeakyReLU function, the S2NN substitution impulse function, the Piecewise Exponential function, the Piecewise Quadratic function, and the NonzeroSignLogAbs function.

[0052] As an optional embodiment of the present invention, the backpropagation function of the Sigmoid function may be:

[0053]

[0054] Wherein, f1(P t ) represents the Sigmoid function, and α represents the parameter that controls the smoothness of the gradient during backpropagation;

[0055] The backpropagation function of the Rectangular function is:

[0056]

[0057] Wherein, f2(P) t ) represents the Rectangular function, and b represents the peak width parameter;

[0058] The backpropagation function of the PiecewiseLeakyReLU function is:

[0059]

[0060] Among them, f3(P t ) represents the Piecewise LeakyReLU function, and W and c represent parameters that control the magnitude of the backpropagation gradient;

[0061] The backpropagation function of the S2NN substitution impulse function is:

[0062]

[0063] Among them, f4(P t ) represents the S2NN substitution impulse function, α represents the parameter controlling the smoothness of the gradient during backpropagation, and β represents the gradient smoothing coefficient;

[0064] The backpropagation function of the PiecewiseExponential function is:

[0065]

[0066] Among them, f5(P t) represents the PiecewiseExponential function. This indicates that the PiecewiseExponential function is applied to P. t Feature representation;

[0067] The backpropagation function of the PiecewiseQuadratic function is:

[0068]

[0069] Among them, f6(P t ) represents the PiecewiseQuadratic function;

[0070] The backpropagation function of the NonzeroSignLogAbs function is:

[0071]

[0072] Among them, f7(P) t ) represents the NonzeroSignLogAbs function.

[0073] As an optional embodiment of the present invention, the expression for approximating the partial derivatives of LIF neuron impulse activity during backpropagation using a continuous and differentiable function is approximated by the following function:

[0074]

[0075] Among them, f i (P t ) represents an approximate function.

[0076] As an optional embodiment of the present invention, the weighted sum of the Gaussian prior distribution loss and the backpropagation training loss of the LIF spiking neuron object recognition model is expressed as follows:

[0077] L tot =L Gauss +L

[0078] Among them, L tot L represents the total loss function of the object recognition model. Gauss Let represent the Gaussian prior distribution loss, and L represent the backpropagation training loss of the LIF spiking neuron object recognition model.

[0079] The beneficial effects of this invention are as follows: By weighting the Gaussian prior distribution loss and the backpropagation training loss of the LIF spiking neuron object recognition model, the accuracy and stability of the object recognition model are improved. By employing seven approximation functions to approximate the partial derivatives of the LIF spiking neuron activation function, the convergence speed and accuracy of the algorithm are improved. Simultaneously, the use of Gaussian prior distribution loss enhances the model's generalization ability. Incorporating Gaussian prior distribution loss into the loss function of the object recognition model also limits the amplitude of the model's parameters, avoiding overfitting of the training data. By integrating Gaussian prior distribution loss, the object recognition model not only needs to optimize data fitting but also needs to approximate the expected value of the prior distribution as closely as possible, which helps enhance the generalization ability and improve the stability of the object recognition model.

[0080] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0081] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0082] Figure 1 This is a flowchart of the pulse graph neural network tactile object recognition algorithm based on graph learning of the present invention.

[0083] Figure 2 This is a schematic diagram of the object recognition model of the present invention.

[0084] Figure 3 This invention presents a tactile map of 39 tactile nodes constructed using different methods. Detailed Implementation

[0085] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0086] A method for tactile object recognition based on pulse graph neural networks using graph learning, the algorithm comprising:

[0087] S100. Obtain a tactile map, and construct an M-tree tactile map or a Z-tree tactile map based on the tactile map;

[0088] like Figure 3As shown, in this embodiment, the NeuTouch sensor collects data using 39 radially arranged taxels. To enable the object recognition model to process the data collected by the NeuTouch sensor, this embodiment connects the spatial arrangement of the 39 nodes using different methods to construct a haptic map. This represents the information perceived by the NeuTouch sensor in graph form, allowing the object recognition model to be trained using convolutional methods similar to GCN. The haptic map in this embodiment is represented as G = (V, E), where V represents a set of n nodes and a set of undirected edges. Figure 3 Figure 1 shows haptic maps of 39 haptic nodes constructed using different methods. Figure 2a shows the spatial layout of the 39 haptic nodes under the NeuTouch sensor; Figure 3b shows the M-tree haptic map constructed using the M-tree method; and Figure 4c shows the Z-tree haptic map constructed using the Z-tree method. The haptic map defines the connections between neurons, which are crucial for information transmission and processing. Furthermore, the haptic map not only describes the connections between neurons but also explicitly defines the weights of these connections, which determine the transmission of information between neurons. This embodiment employs M-tree and Z-tree haptic map construction methods, enabling the object recognition model to capture the complex spatiotemporal information in event-driven haptic data and better simulate the information transmission methods of biological systems. For the 39 nodes of the NeuTouch sensor, each node contains multi-dimensional data, such as position coordinates and haptic signal values. Faced with the challenges of such multi-dimensional data, the introduction of the M-tree provides a feasible solution for effective indexing and management. In the construction of tactile graphs in spiking neural networks, M-trees, as a data structure, have unique advantages in processing multidimensional data, performing range queries, nearest neighbor queries, adaptability, and high performance.

[0089] This embodiment proposes a haptic map construction method based on M-trees. This method constructs an M-tree haptic map based on the structural features of the M-tree and the spatial arrangement features of the 39 nodes of the NeuTouch sensor. The construction result is as follows: Figure 3 As shown in b, the number of nodes k is set to 2. By employing the M-tree haptic graph construction method, an efficient, multi-dimensional, and real-time responsive M-tree haptic graph can be built for event-driven haptic data. This construction method has significant advantages for object recognition models performing event-driven haptic object recognition. In addition to efficiency and support for multi-dimensional data, the M-tree haptic graph construction method also demonstrates significant advantages in real-time performance and computational efficiency. By avoiding linear search of the entire dataset, this method reduces the computational complexity of the object recognition model, thereby improving the performance of haptic data processing and event recognition. This makes the object recognition model more suitable for real-time and efficient event perception and object recognition tasks, bringing significant performance improvements to the field of haptic perception applications.

[0090] Event-driven haptic maps contain rich temporal and spatial information. The spatial indexing structure built using Z-order curve trees offers advantages in range queries within multidimensional haptic data spaces, helping to preserve the spatial relationships between nodes in the haptic map. The Z-tree method, with its unique spatial mapping, maps adjacent multidimensional data points to adjacent positions in a one-dimensional sequence, thus cleverly maintaining the spatial continuity of the data. This characteristic is crucial for maintaining the relative positional relationships of data when processing haptic data, especially improving the spatial perception and processing of haptic information. Simultaneously, the Z-tree method enables real-time encoding of adjacent positions of haptic nodes, allowing data to be rapidly transmitted to spiking neural networks, thereby achieving instantaneous haptic object recognition. This is particularly important for object recognition models requiring low-latency responses.

[0091] This embodiment proposes a tactile map construction method based on Z-trees. This method combines the structural features of Z-trees with their spatial mapping method to construct a tactile map based on the spatial arrangement features of the 39 nodes of the NeuTouch sensor. The construction result is as follows: Figure 3 As shown in c, the spatial continuity of the original data is still maintained by the mapping of adjacent positions in the figure through the Z-tree, which provides stronger support for event-driven haptic learning. The haptic maps are connected after being sorted according to the Z-order curves. The connection order in the example figure is: [4,11,13,7,2,12,9,16,25,26,18,30,35,20,29,23,33,37,1,3,6,10,15,17,14,5,8,21,27,24,34,31,28,22,39,38,19,32,36].

[0092] S200. Establish an object recognition model and train the object recognition model using the M-tree tactile map or Z-tree tactile map. The object recognition model includes LIF spiking neurons, topological adaptive graph convolutional layers, fully connected layers, and a final voting layer.

[0093] It should be noted that since M-tree tactile maps and Z-tree tactile maps are constructed using different methods, they cannot both be simultaneously input into the object recognition model for training. In this embodiment, M-tree tactile maps are preferred for training the object recognition model; for example... Figure 2As shown, in this embodiment, the object recognition model consists of three LIF spiking neurons, one topology adaptive graph convolutional layer (TAGConv), two fully connected layers (FC), and a final voting layer (Voting) for recognition. The input M-tree tactile map is fed into the topology adaptive graph convolutional layer at different time points. The topology adaptive graph convolutional layer performs graph convolution on the M-tree tactile map to extract its features and generate a feature topology map. The feature topology map is then fed into the first LIF spiking neuron, which filters the signal values ​​of the feature topology map. The filtered signal values ​​are then mapped through the first fully connected layer. The second LIF spiking neuron filters the signal values ​​output by the first fully connected layer. This process is repeated through the second fully connected layer and the third LIF spiking neuron, and finally, the voting layer identifies the object category.

[0094] S300. The object recognition model is optimized by weighting the Gaussian prior distribution loss and the backpropagation training loss of the LIF spiking neuron object recognition model.

[0095] It should be noted that, in order to enhance the object recognition model's learning ability of event-driven haptic maps and alleviate the generalization phenomenon caused by the sparsity and spatiotemporal complexity of haptic maps, this invention introduces a Gaussian prior distribution parameter estimation method into the object recognition model. By introducing a Gaussian prior assumption into the parameter estimation process of the object recognition model, the weights and bias parameters of the network are constrained, thereby improving the generalization performance of the object recognition model. Since the object recognition model involves a large number of weight parameters used to connect neurons, these parameters need to be learned during training. To introduce the Gaussian prior, this invention uses a Gaussian distribution to encode the weight parameters. The Gaussian distribution is typically used to express the uncertainty of parameters; its characteristic is that the parameter values ​​are concentrated around the mean while allowing a certain degree of variation.

[0096] S400. Use the optimized object recognition model to determine the category of the tactile object.

[0097] like Figure 1 and Figure 2As shown, this invention improves the accuracy and stability of the object recognition model by weighting the Gaussian prior distribution loss and the backpropagation training loss of the LIF spiking neuron object recognition model. By employing seven approximation functions to approximate the partial derivatives of the LIF spiking neuron activation function, the convergence speed and accuracy of the algorithm are improved. Simultaneously, the use of Gaussian prior distribution loss enhances the model's generalization ability. Adding Gaussian prior distribution loss to the loss function of the object recognition model also limits the amplitude of the model's parameters, avoiding overfitting of the training data. By incorporating Gaussian prior distribution loss, the object recognition model not only needs to optimize data fitting but also needs to approximate the expected value of the prior distribution as closely as possible, which helps enhance the generalization ability and improve the stability of the object recognition model.

[0098] As an optional embodiment of the present invention, the calculation steps of the Gaussian prior distribution loss may include:

[0099] S301. Suppose the mean μ of the Gaussian prior distribution is 0 and the variance is σ. 2 The weight parameters of the object recognition model are encoded using the Gaussian prior distribution.

[0100] Its expression is:

[0101]

[0102] Among them, P(C m () represents the m-th parameter after encoding;

[0103] C m This represents the m-th parameter;

[0104] σ represents the standard deviation of the Gaussian prior distribution;

[0105] exp() represents an exponential function with base e.

[0106] w m This represents the m-th parameter.

[0107] It should be noted that the mean μ and variance σ of the Gaussian prior distribution 2 It can be preset or dynamically adjusted based on training data. Encoding the weight parameters using a Gaussian prior distribution can effectively incorporate prior data into the model, improving the model's generalization ability.

[0108] S302. Using the Gaussian distribution as the probability mass function, and iterating through the parameters of the object recognition model, calculate the probability value of each parameter under the Gaussian prior distribution.

[0109] Its expression is:

[0110]

[0111] Among them, L(C m () represents the probability value of the m-th parameter;

[0112] log() represents taking the logarithm;

[0113] It should be noted that when calculating probability values, a Gaussian distribution should be used as the probability mass function, and the logarithmic function should be used for calculation. This ensures the stability and accuracy of the calculation process. Weight parameter updates are based on weight values. During model training, the weight parameters are adjusted in real time based on changes in weight values. This allows the model to be continuously optimized during training, improving its accuracy and generalization ability.

[0114] S303. Calculate the Gaussian prior distribution loss based on the probability value of each parameter;

[0115] Its expression is:

[0116]

[0117] Among them, L Gauss This represents the Gaussian prior distribution loss;

[0118] l represents the total number of parameters in the model;

[0119] m represents the number of parameters.

[0120] It's important to note that the Gaussian prior distribution loss is calculated based on the probability value of each parameter. This allows the model to focus more on parameters that significantly influence the data distribution during training, resulting in a better fit to the data. Furthermore, using the log-likelihood function to calculate the loss makes the calculation more stable and accurate. Incorporating this Gaussian prior distribution loss into the model's loss function limits the amplitude of model parameters, preventing overfitting of the training data. By integrating the Gaussian prior distribution loss, the model not only needs to optimize data fitting but also needs to approximate the expected value of the prior distribution as closely as possible, which helps enhance the model's generalization ability and improves its stability.

[0121] As an optional embodiment of the present invention, the calculation steps of the backpropagation training loss of the LIF spiking neuron object recognition model may include:

[0122] S304, The expression of the LIF spiking neuron is:

[0123]

[0124] fire a spike&P t =P rest ,Pt ≥P th

[0125] Where τ represents the membrane time constant;

[0126] d represents the infinitesimal element symbol;

[0127] P t This represents the membrane potential of the spiking neuron at time t;

[0128] P rest This represents the resting potential of a neuron;

[0129] Y t This represents the influence of external input received by the neuron at time t;

[0130] P th Indicates the neuron's trigger threshold;

[0131] "Fire a spike" indicates a pulse firing state.

[0132] The '&' indicates that the membrane potential changes simultaneously with the pulse being emitted.

[0133] It should be noted that LIF (Leaky Integrate-and-Fire) neurons are a type of biological neuron model. LIF neuron models possess advantages in tactile recognition spiking neural networks, including high biological plausibility, excellent temporal accuracy, low energy consumption, good robustness, and ease of implementation. These characteristics make LIF neurons an effective tool for simulating biological nervous systems and processing tactile information. In this embodiment, the data input to the LIF neuron model is precisely filtered based on its neuron triggering threshold using the LIF neuron model.

[0134] S305. Obtain the pulse generated by the neuron at time t;

[0135] Its expression is:

[0136]

[0137] Among them, E t This represents the pulse generated by the neuron at time t;

[0138] O(P t ) represents the neuron's spiking activation function;

[0139] When P t ≥P th The neuron generates a pulse at that time;

[0140] When P t <P th The neuron does not generate pulses at this time;

[0141] It should be noted that the neuronal spiking activation function can be selected according to actual needs. Commonly used activation functions include, but are not limited to, the step function, the sigmoid function, and the ReLU function. In this embodiment, the step function is used as the neuronal spiking activation function. When the membrane potential exceeds a threshold, the neuron generates a spiking signal; when the membrane potential does not exceed the threshold, the neuron does not generate a spiking signal. By utilizing E... t This represents the impulses generated by the neuron at time t, and is determined by O(P). t ) is a step function, and it can be seen that the LIF neuron model has the discrete characteristics of binary pulses.

[0142] S306. Set the resting potential of the neuron to zero, control the frequency and period of pulse firing when the neuron is stimulated by using a pulse firing mechanism, control the state of the membrane potential after the neuron is stimulated by using a membrane potential reset mechanism, and set the initial conditions.

[0143] Its expression is:

[0144]

[0145]

[0146] in, This represents the neuron's membrane potential at time t, preceding the previous time step.

[0147] t i-1 This represents the time preceding time t.

[0148] t represents time.

[0149] Y t ' indicates external input;

[0150] w n This represents the weight vector of the nth layer;

[0151] This indicates the pulse generation status of the k-th neuron in the (n-1)-th layer at time t;

[0152] b n The bias vector of the nth layer;

[0153] It should be noted that by setting initial conditions, neurons can fire pulses and reset membrane potentials according to preset parameters and algorithms when stimulated, thereby controlling the frequency and period of pulse firing when stimulated, as well as the state of the membrane potential after stimulation. In this embodiment, the initial condition is set to the resting potential of the neuron being zero, that is, the membrane potential of the neuron is zero when it is not stimulated. By setting initial conditions, neurons can better adapt to input data during training, improving the accuracy and stability of the model.

[0154] S307. The membrane potential P of the spiking neuron at time t is based on the number of layers of the neuron. t Perform the conversion;

[0155] Its expression is:

[0156]

[0157]

[0158] Among them, P t n This represents the membrane potential of the nth layer spiking neuron at time t;

[0159] This represents the membrane potential of the nth layer spiking neuron at time t-1;

[0160] Simplified

[0161] This indicates the pulse generation status of the k-th neuron in the n-th layer at time t-1;

[0162] It should be noted that different conversion methods can be selected according to actual needs when converting the membrane potential of neurons. In this embodiment, a method based on the number of neuron layers is used to transmit and calculate the membrane potential of the spiking neuron layer by layer, thereby obtaining the membrane potential of the neuron at different times. The membrane potential P of the spiking neuron at time t is obtained by converting the membrane potential P of the spiking neuron at time t based on the number of neuron layers. t Transformation can make neurons have better hierarchy and logic when processing input data, thereby improving the accuracy and processing efficiency of the model.

[0163] S308. Calculate the backpropagation training loss of the LIF spiking neuron-based object recognition model using a spiking neural network model.

[0164] Its expression is:

[0165]

[0166] Where L represents the backpropagation training loss of the LIF spiking neuron object recognition model;

[0167] S represents the total number of training samples for the haptic map;

[0168] R s Represents the label vector of the s-th tactile data sample;

[0169] O s This represents the output vector of the output layer of the spiking neural network model of the LIF spiking neuron;

[0170] T represents the total time step;

[0171] When using an object recognition model, for a training sample labeled l, the neuron representing class l in the object recognition model has the highest output value.

[0172] It should be noted that calculating the backpropagation training loss is a crucial step in object recognition models based on LIF spiking neurons. By calculating the backpropagation training loss, the accuracy and error of the model in recognizing objects can be determined, thereby allowing for model optimization and improvement. In this embodiment, using a spiking neural network model based on LIF spiking neurons to calculate the backpropagation training loss allows for a better evaluation of the model's performance and recognition effectiveness, providing important reference for subsequent model training and applications.

[0173] As an optional embodiment of the present invention, the algorithm may further include:

[0174] S500: Use a continuous and differentiable function to approximate the partial derivatives of LIF neuron impulse activity during backpropagation.

[0175] Its expression is:

[0176]

[0177] in, This indicates the partial derivative;

[0178] E t This represents the pulse generated by the neuron at time t;

[0179] This represents taking the partial derivative with respect to the neuron's spiking activation function;

[0180] f(P t ) represents an approximate function of the impulse activation function.

[0181] Based on the calculation formula for the backpropagation training loss of the LIF spiking neuron object recognition model mentioned above, the relationship between the loss function and spiking activity and neuron membrane potential can be obtained as follows:

[0182]

[0183]

[0184] in, This indicates the partial derivative;

[0185] L represents the backpropagation training loss of the LIF spiking neuron object recognition model;

[0186] This represents the pulse generated by the neuron in the nth layer at time t;

[0187] This represents the pulse generated by the neuron in layer n+1 at time t;

[0188] P t n This represents the membrane potential of the nth layer spiking neuron at time t;

[0189] This represents the pulse generated by the neuron in the nth layer at time t+1;

[0190] Using the chain rule of composite functions, we obtain the backpropagation training L and weights w. n Bias b n Backpropagation relationship in object recognition models:

[0191]

[0192]

[0193] in, This represents the pulse generated by the neuron in layer n-1 at time t;

[0194] The above analysis shows that impulse activity has binary and discrete characteristics, and the membrane potential is affected by both the time and spatial domains, which prevents object recognition models from using the backpropagation method in traditional network models to update parameters. Therefore, this invention solves this problem by using a method that approximates the derivative of the neuron's activation function.

[0195] As an optional embodiment of the present invention, the approximation function may optionally include the Sigmoid function, the Rectangular function, the Piecewise LeakyReLU function, the S2NN substitution impulse function, the Piecewise Exponential function, the Piecewise Quadratic function, and the NonzeroSignLogAbs function.

[0196] It should be noted that using the derivative of the neuron activation function can effectively solve the backpropagation problem in object recognition models using spiking neurons, thereby better optimizing and improving the model. In this embodiment, various approximation functions can be optionally used, such as the sigmoid function, the rectangular function, the piecewise LeakyReLU function, the S2NN substitute spiking function, the piecewise Exponential function, the piecewise Quadratic function, and the NonzeroSignLogAbs function. These functions can be selected and combined according to specific application scenarios and needs to meet different model optimization requirements. By using these approximation functions, the derivative of the neuron activation function can be better approximated, thereby more accurately calculating the backpropagation training loss and improving the model's accuracy and processing efficiency. At the same time, these approximation functions can also reduce computational complexity, increase the speed of model training and application, and provide better support for practical applications.

[0197] As an optional embodiment of the present invention, the backpropagation function of the Sigmoid function may be:

[0198]

[0199] Wherein, f1(P t ) represents the Sigmoid function;

[0200] α represents a parameter that controls the smoothness of the gradient during backpropagation;

[0201] P t This represents the membrane potential of the spiking neuron at time t;

[0202] e represents the natural base;

[0203] The backpropagation function of the Rectangular function is:

[0204]

[0205] Wherein, f2(P) t ) represents the Rectangular function;

[0206] b represents the parameter indicating the peak width;

[0207] sign() represents a sign function;

[0208] 'a' represents the peak width coefficient;

[0209] The backpropagation function of the PiecewiseLeakyReLU function is:

[0210]

[0211] Among them, f3(P t ) represents the Piecewise LeakyReLU function, and W and c represent parameters that control the magnitude of the backpropagation gradient;

[0212] The backpropagation function of the S2NN substitution impulse function is:

[0213]

[0214] Among them, f4(P t ) represents the S2NN substitution impulse function, α represents the parameter controlling the smoothness of the gradient during backpropagation, and β represents the gradient smoothing coefficient;

[0215] The backpropagation function of the PiecewiseExponential function is:

[0216]

[0217] Among them, f5(P t ) represents the PiecewiseExponential function. This indicates that the PiecewiseExponential function is applied to P. t Feature representation;

[0218] The backpropagation function of the PiecewiseQuadratic function is:

[0219]

[0220] Among them, f6(P t ) represents the PiecewiseQuadratic function;

[0221] |P t | represents the absolute value of the membrane potential of the spiking neuron at time t;

[0222] The backpropagation function of the NonzeroSignLogAbs function is:

[0223]

[0224] Among them, f7(P) t ) represents the NonzeroSignLogAbs function.

[0225] It should be noted that the same parameter settings are maintained consistently across the seven approximation functions during model training. These backpropagation functions can be effectively used to update model parameters, thereby optimizing and improving the model. In this invention, different backpropagation functions can be selected based on specific application scenarios and requirements to meet different model optimization needs. By using these backpropagation functions, training loss can be calculated more accurately, improving model accuracy and processing efficiency. Simultaneously, these backpropagation functions can reduce computational complexity, increase the speed of model training and application, and provide better support for practical applications.

[0226] As an optional embodiment of the present invention, the expression for approximating the partial derivatives of LIF neuron impulse activity during backpropagation using a continuous and differentiable function is approximated by the following function:

[0227]

[0228] Among them, f i (P t ) represents an approximate function.

[0229] It should be noted that, in order to overcome the non-differentiability of impulses during the model training process, this embodiment uses seven approximation functions to be introduced into the model training. The impact of these functions on the recognition performance and time complexity of the object recognition model is comprehensively compared and evaluated. This is to gain a deeper understanding of the impact of different approximation functions on the performance of event-driven tactile object recognition, and to provide guidance for selecting approximation functions suitable for tactile learning object recognition models, thereby overcoming the non-differentiability of impulses.

[0230] As an optional embodiment of the present invention, the weighted sum of the Gaussian prior distribution loss and the backpropagation training loss of the LIF spiking neuron object recognition model is expressed as follows:

[0231] L tot =L Gauss +L

[0232] Among them, L tot L represents the total loss function of the object recognition model. Gauss Let represent the Gaussian prior distribution loss, and L represent the backpropagation training loss of the LIF spiking neuron object recognition model.

[0233] It should be noted that the identification process of this invention includes the following steps:

[0234] Step 1: Tactile Data Input: Input the data collected by the NeuTouch tactile sensor;

[0235] Step 2: Construction of the pulse tactile map: Construct the pulse tactile map from the input tactile data using a tactile map construction method based on M-tree or Z-tree;

[0236] Step 3: Topological Convolution: Perform topological adaptive graph convolution on the impulse haptic map to output the feature matrix of the haptic map;

[0237] Step 4: Pulse firing: Activate the pulses in the pulse tactile data using the LIF neuron model to achieve pulse firing;

[0238] Step 5: Feature Mapping: The feature matrices are merged into column vectors, and the features are mapped to the next pulse firing layer through a fully connected layer. In this embodiment, it has a 256-sample label space.

[0239] Step 6: Pulse firing: Activate the pulses in the pulse tactile data using the LIF neuron model to achieve pulse firing;

[0240] Step 6: Feature mapping: Features are mapped through a fully connected layer, which in this embodiment has a space of 128 sample labels;

[0241] Step 7: Pulse firing: Activate the pulses in the pulse tactile data using the LIF neuron model to achieve pulse firing;

[0242] Step 8: Output Recognition Results: Output the tactile object recognition results predicted by the model. Although embodiments of the present invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for tactile object recognition based on pulse graph neural networks using graph learning, characterized in that, The method includes: S100. Obtain a tactile map, and construct an M-tree tactile map or a Z-tree tactile map based on the tactile map; S200. Establish an object recognition model and train the object recognition model using the M-tree tactile map or Z-tree tactile map. The object recognition model includes LIF spiking neurons, topological adaptive graph convolutional layers, fully connected layers, and a final voting layer. S300. The object recognition model is optimized by weighting the Gaussian prior distribution loss and the backpropagation training loss of the LIF spiking neuron object recognition model. S400. Use the optimized object recognition model to determine the category of the tactile object; The calculation steps for the Gaussian prior distribution loss include: S301. Let the mean of the Gaussian prior distribution be... The variance is 0. The weight parameters of the object recognition model are encoded using the Gaussian prior distribution. Its expression is: ， in, Indicates the encoded number One parameter, Indicates the first One parameter, This represents the standard deviation of the Gaussian prior distribution. This represents an exponential function with base e. Indicates the first One parameter; S302. Using the Gaussian prior distribution as the probability mass function, and iterating through the parameters of the object recognition model, calculate the probability value of each parameter under the Gaussian prior distribution. Its expression is: ， in, Indicates the first The probability values ​​of each parameter. Indicates taking the logarithm; S303. Calculate the Gaussian prior distribution loss based on the probability value of each parameter; Its expression is: ， in, This represents the Gaussian prior distribution loss. This indicates the total number of parameters in the model. Indicates the number of parameters; The method further includes: S500: Use a continuous and differentiable function to approximate the partial derivatives of LIF neuron impulse activity during backpropagation. Its expression is: ， in, This indicates finding the partial derivative. express The pulses generated by the neuron at any given moment This represents taking the partial derivative with respect to the neuron's spiking activation function. An approximation of the impulse activation function; The approximation functions include the Sigmoid function, the Rectangular function, the Piecewise LeakyReLU function, the S2NN substitution impulse function, the Piecewise Exponential function, the Piecewise Quadratic function, and the NonzeroSignLogAbs function.

2. The method for tactile object recognition based on graph learning and pulse graph neural network as described in claim 1, characterized in that, The calculation steps for the backpropagation training loss of the LIF spiking neuron object recognition model include: S304, The expression of the LIF spiking neuron is: ， ， in, Indicates the membrane time constant. Represents the infinitesimal element symbol. Represents spiking neurons Membrane potential at time t, This represents the resting potential of a neuron. express The influence of external input received by a neuron at any given time. Indicates the neuron's trigger threshold. Indicates the pulse delivery status. This indicates that the membrane potential changes simultaneously with the pulse delivery; S305, Obtain The pulses generated by neurons at any given moment; Its expression is: ， in, express The pulses generated by the neuron at any given moment Represents the neuron's spiking activation function, when The neuron generates a pulse when The neuron does not generate pulses at this time; S306. Set the resting potential of the neuron to zero, control the frequency and period of pulse firing when the neuron is stimulated by using a pulse firing mechanism, control the state of the membrane potential after the neuron is stimulated by using a membrane potential reset mechanism, and set the initial conditions. Its expression is: ， ， in, express The neuron membrane potential at the moment preceding time. express The moment before the moment, Indicates external input. Indicates the first Layer weight vector, express Time of the first The first layer The generation of impulses in each neuron No. Layer bias vector; S307. Based on the number of layers of the neurons, the spiking neurons... membrane potential at time 1 Perform the conversion; Its expression is: ， in, express Time of the first Membrane potential of layer spiking neurons express Time of the first Membrane potential of layer spiking neurons Simplified , express Time of the first The first layer The generation of impulses in each neuron; S308. Calculate the backpropagation training loss of the LIF spiking neuron-based object recognition model using a spiking neural network model. Its expression is: ， in, This represents the backpropagation training loss of the LIF spiking neuron object recognition model. This represents the total number of training samples for the haptic map. Indicates the first The label vector of each tactile data sample, Indicates the total time step. Indicates time, This represents the output vector of the output layer of the spiking neural network model of the LIF spiking neuron.

3. The method for tactile object recognition based on graph learning and pulse graph neural network as described in claim 1, characterized in that, The backpropagation function of the Sigmoid function is: ， in, This represents the Sigmoid function. Represents the natural base. A parameter that controls the smoothness of the gradient during backpropagation; The backpropagation function of the Rectangular function is: ， in, Represents the Rectangular function. The parameter representing the peak width, Represents a symbolic function. Indicates the peak width coefficient; The backpropagation function of the PiecewiseLeakyReLU function is: ， in, This represents the Piecewise Leaky ReLU function. and This represents the parameter that controls the magnitude of the backpropagation gradient; The backpropagation function of the S2NN substitution impulse function is: ， in, This indicates that S2NN replaces the impulse function. The parameter represents the smoothness of the gradient during backpropagation. Represents the gradient smoothing coefficient; The backpropagation function of the PiecewiseExponential function is: ， in, This represents the PiecewiseExponential function. This indicates that the PiecewiseExponential function is paired with... Feature representation; The backpropagation function of the PiecewiseQuadratic function is: ， in, This represents the PiecewiseQuadratic function. Represents spiking neurons The absolute value of the membrane potential at any given time; The backpropagation function of the NonzeroSignLogAbs function is: ， in, This refers to the NonzeroSignLogAbs function.

4. The method for tactile object recognition based on graph learning and pulse graph neural network as described in claim 3, characterized in that, The expression for the partial derivatives of LIF neuron impulse activity during backpropagation, which uses a continuous and differentiable function, is approximately expressed as follows: ， in, It represents an approximate function.

5. The method for tactile object recognition based on graph learning and pulse graph neural network as described in claim 1, characterized in that, The weighted sum of the Gaussian prior distribution loss and the backpropagation training loss of the LIF spiking neuron object recognition model is expressed as follows: ， in, This represents the total loss function of the object recognition model. This represents the Gaussian prior distribution loss. This represents the backpropagation training loss of the LIF spiking neuron object recognition model.

Images