Anti-interference target identification method, program, equipment and storage medium combining spiking neural network and capsule network

Abstract

The invention belongs to the technical field of image target recognition, and particularly relates to an anti-interference target recognition method combining a pulse neural network and a capsule network, a program, equipment and a storage medium. A capsule structure is introduced into a pulse brain-like model, a self-attention mechanism is introduced, the protocol degree between capsules can be learned in single forward propagation, a capsule attention map is generated through a self-attention layer, and then the capsule attention map is mapped to a classification layer through a one-dimensional convolution layer, so that the model structure and the calculation complexity are simplified, and the accuracy of the model is improved. And irrelevant capsule voting can be selected and inhibited by utilizing a self-attention mechanism, so that information useful for entity recognition is reserved, and the capabilities of long-distance dependence and feature selection are improved. The method has good robustness under the conditions of different noises and different intensities, has good generalization and invariance for affine transformation, and improves the anti-interference capability of target identification.

Description

Technical Field

[0001] The present invention belongs to the technical field of image target recognition, and specifically relates to an anti-interference target recognition method, program, device and storage medium combining a pulse neural network and a capsule network. Background Art

[0002] Convolutional neural networks (CNNs) have achieved great success in various fields such as object classification, visual tracking, and object segmentation due to their powerful feature representation capabilities. However, there are still some limitations associated with CNNs. First, high-level features are obtained by weighting low-level features, while ignoring their spatial relationships. A face is just a simple combination of features of eyes, nose, and mouth. Even if the positions of these features change relatively, the classifier will still regard it as a face. Second, the commonly used pooling operation discards the precise location information of the target and is destructive to the spatial relationship. To solve these problems, capsule networks use a group of neurons (vectors, matrices) to represent more attributes, which are called capsules. These attributes represent the properties of the corresponding features, such as position, color, and texture. The length of the vector represents the probability of these attributes. In addition to the capsule concept, a special routing algorithm is also used to ensure that the output of the lower-level capsules to the closely related upper-level capsules.

[0003] Artificial neural networks have poor generalization and migration capabilities and are easily interfered with. In intelligent recognition applications, only independent single-frame data is generally used, and the mutual information of the data is not utilized. By studying the spatiotemporal correlation information of the pulse brain model, the target's anti-interference ability can be improved.

[0004] Although the capsule network mimics the multi-layer visual system with a parsed tree-like structure, it is still far from the information processing mechanism of the human brain. As the third generation of artificial neural networks, spiking neural networks (SNNs) use discrete spikes to transmit information, which is more biologically reasonable and energy-efficient. SNNs have strong sparsity, and neurons that do not emit spikes will not update network weights. In recent years, SNNs have greatly promoted the development of event-based neuromorphic hardware platforms, showing ideal low latency and high efficiency. On the other hand, due to the spike coding mechanism, SNNs are very robust to input interference and show excellent performance in noise resistance. Many traditional SNN structures are inspired by the connections between brain regions to mimic certain brain region functions. However, due to its complex network connections, it is difficult to expand to deeper network structures. In order to solve the above problems, the capsule structure is introduced in the spiking neural network to make full use of its spatial and temporal characteristics.

[0005] In the field of capsule neural networks, many works are integrated into the modeling of routing mechanisms, which can be roughly divided into two categories, supervised and unsupervised. However, no matter which method is used, it generally only considers the spatial relationship between capsules, but ignores the temporal relationship. The dynamic routing algorithm uses biologically reasonable spike time-dependent plasticity (STDP) for dynamic interaction between spike capsules, taking into account the causal relationship between capsules, but the calculation parameters are very large and require multiple iterations. Therefore, a new non-iterative, highly parallel routing algorithm is used instead of dynamic routing. The algorithm can easily handle the reduced number of capsules, and due to the introduction of the self-attention mechanism, it can well learn meaningful information and increase the ability of long-distance dependence and feature selection. Summary of the invention

[0006] The purpose of the present invention is to provide an anti-interference target recognition method, program, device and storage medium combining a pulse neural network and a capsule network, which takes into account the intrinsic spatiotemporal information of continuous frames, has strong anti-interference ability and low energy consumption.

[0007] An anti-interference target recognition method combining a pulse neural network and a capsule network comprises the following steps:

[0008] Step 1: Build a capsule network based on spiking neurons and perform pre-training;

[0009] The capsule network based on pulse neurons includes a convolution layer, a deep convolution layer, a main capsule layer and a digital capsule layer;

[0010] The convolution layer is used to extract basic features in the input image and map the input image to a higher dimensional space;

[0011] The deep convolutional layer is used to create a vector representation of the basic features of the image;

[0012] The main capsule layer is used to generate a combination of basic features of the image to generate impulse capsules;

[0013] The digital capsule layer uses self-attention routing to select features and update routing weights, and outputs recognition results and confidence levels;

[0014] Step 2: Input the image to be recognized into the pre-trained capsule network based on spiking neurons, and take the classification result with the highest confidence in the digital capsule layer as the recognition result of the target.

[0015] Furthermore, the capsule network based on pulse neurons adopts LIF neurons as basic neurons, and LIF neurons continuously receive pulses δ from presynaptic neurons. j (t), and dynamically changes its membrane potential V i (t); when the membrane potential Vi (t) is greater than the threshold V th When the LIF neuron fires a pulse, the membrane potential V i (t) is greater than the threshold V th When the membrane potential V i (t) will be minus the threshold value V th , as shown below:

[0016]

[0017] Where C and R are the membrane capacitance and membrane resistance respectively, w ij is the synaptic weight between the i-th LIF neuron and the j-th LIF neuron, N is the number of presynaptic neurons, and dt is V i (t) is the minimum simulation time step; V reset represents the reset potential, δ i (t) represents the pulse emitted by the i-th LIF neuron;

[0018] Convert the above formula into discrete form as shown below:

[0019]

[0020] Here, λ is the attenuation factor of membrane potential.

[0021] The convolution layer extracts the features of the input image through convolution operations and generates a feature map. Each value in each feature map represents the degree of feature activation of the input image in a specific area, which is the input signal of the LIF neuron. The membrane potential of the LIF neuron will be updated according to the size and weight of the input feature value according to the above formula; at each time step, the LIF neuron will receive multiple values ​​in the feature map as input signals until the membrane potential exceeds the threshold, the neuron will emit a pulse and reset the membrane potential.

[0022] Furthermore, the capsule network based on pulse neurons includes multiple convolutional layers, each of which performs a conventional convolution operation on the input image, extracts the basic features of the image, and activates the LIF neurons before inputting them to the next layer. The specific process is represented by the following formula:

[0023] F r+1 (X r )=LIF(Conv kxk (X r ))

[0024] Among them, X r Represents the input image data matrix of the rth layer; F r+1 (X) represents the output image data matrix of the rth layer, that is, the input image data matrix of the r+1th layer; Conv kxk (Xr ) represents the feature map obtained after the input image data matrix is ​​convolved with a convolution kernel of size kxk.

[0025] Furthermore, the deep convolution layer of the capsule network based on pulse neurons uses a deep separable convolution with linear activation, and first performs channel-by-channel convolution, where one convolution kernel is responsible for one channel, and one channel is convolved by only one convolution kernel. The number of channels of the feature map generated by this process is exactly the same as the number of channels of the input, and the feature map cannot be expanded. Moreover, this operation performs convolution operations on each channel of the input layer independently, and cannot effectively utilize the feature information of different channels at the same spatial position. Therefore, it is necessary to perform point-by-point convolution again, and perform weighted combination of the feature map of the previous step in the depth direction to generate a new feature map. The deep separable convolution can simplify and reduce the number of parameters required for the capsule creation process, reduce computational complexity, make it easier to learn the correlation between features, and better capture the information in the input image, making full use of the filtering function of the image to intelligently extract all capsule features.

[0026] Furthermore, in the main capsule layer of the capsule network based on spiking neurons, the weights are determined using a self-attention dynamic routing algorithm; let u i and v j Represent the i-th low-level capsule and the j-th high-level capsule, respectively, w ij is the weight matrix of all affine transformations between two adjacent capsule layers, through w ij Each lower-level capsule has a prediction vector of a higher-level capsule c ij is a matrix containing all coupling coefficients generated by the self-attention algorithm, d ij is the logarithmic weight prior matrix of all other weights obtained through discriminant learning at the same time; finally, the prediction results are weighted summed to obtain s j , as shown in the following formula:

[0027]

[0028] The coupling coefficient is calculated starting from the self-attention tensor using the following formula:

[0029]

[0030] Each capsule in a higher layer contains a symmetric matrix A r ,parameter It helps to stabilize training and helps maintain a balance between coupling coefficients and logarithmic prior probabilities. a is a set parameter and r represents the rth layer. Each self-attention matrix contains the score consistency of each capsule prediction combination in the lower layer. The matrix of all coupling coefficients is calculated by the following formula:

[0031]

[0032] The sum of the coupling coefficients between the capsule in the rth layer and all the capsules in the next r+1 layer is 1. The initial logarithmic prior probability is added to the coupling coefficients in turn to obtain the effective routing weights. This process remains the same when multiple capsule layers are stacked on each other to create a deeper level.

[0033] Furthermore, in the digital capsule layer of the capsule network based on spiking neurons, all information is represented by a vector of binary pulses, and the original self-attention dynamic routing is extended to SNN as shown in the following formula:

[0034]

[0035] In order to consider the activities of pre- and post-synaptic neurons and the relative order of their spike firings and to update the network weights in a shorter time, the synaptic trace x is used. pre To record the peak information of the neuron; the arrival of each presynaptic pulse will leave a trace. When there is a pulse, this trace will be updated to 1. When there is no pulse, it will decay exponentially, as shown in the following formula, where τ is the time constant;

[0036]

[0037] The generation of pulses actually selects and emphasizes a feature in the feature map. Only those features with sufficient intensity to exceed the neuron threshold will be transmitted by the neuron in the form of pulses. Through the emission and transmission of pulses, information is further propagated and processed in the network. These pulses will be received by subsequent neurons and used as their inputs to continue feature extraction, classification or recognition tasks.

[0038] The salient trace encodes the temporal relationship of the synapses between presynaptic and postsynaptic neurons within a positive time window. If the presynaptic spike triggers the postsynaptic neuron within the time window, the connection will be strengthened; otherwise, it will be weakened, as determined by a threshold bias x. offset Determine; when the presynaptic pulse arrives, the intensity of the change depends on the time difference between the presynaptic pulse and the postsynaptic pulse, which is calculated as follows:

[0039]

[0040] The output layer is no longer represented by a scalar, but by a vector; the capsule of the digital capsule layer represents not only the confidence of the existence of a certain object category, but also all the attributes extracted from its parts; the length of the instantiation vector represents the confidence of the individual existence of a capsule, and its length is close to 1 if and only if the individual it represents is the only individual present in the image; therefore, in order to allow multiple categories, the following formula is used to calculate each class represented by each capsule of the digital capsule layer:

[0041] L r =T r max(0,m + -||u r ||) 2 +λ(1-T r )max(0,||u r ||-m - ) 2

[0042] Among them, if all capsules in each layer exist, and m + , m - and λ is the hyperparameter to be tuned, then T r is equal to 1.

[0043] Furthermore, when the capsule network based on spiking neurons is pre-trained, the output of the correct digital capsule layer is used as the input of the decoder, and the image is reconstructed. The loss function is the Euclidean distance between the reconstructed image and the input image. The margin loss function is used to convert each individual margin loss L r Add together to calculate the final score of the training phase; use a reconstruction regularizer to encourage all output capsules to learn meaningful features and improve anti-interference capabilities. The average value of the L2 loss between the input image and the decoder output is calculated by a special factor and added to the margin loss; only the correct digital capsule layer vectors are used during training, and incorrect vectors are ignored; the decoder forces the capsule to learn features that are useful for reconstructing the original image, and the reconstructed image is as close to the input image as possible.

[0044] A computer device / equipment / system comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-mentioned anti-interference target recognition method combining a pulse neural network and a capsule network.

[0045] A computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the above-mentioned anti-interference target recognition method combining a pulse neural network and a capsule network.

[0046] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned anti-interference target recognition method combining a pulse neural network and a capsule network.

[0047] The beneficial effects of the present invention are:

[0048] The present invention introduces capsule structure into the pulse brain model, combines their rich spatiotemporal information processing capabilities, fully considers the spatial relationship between the part and the whole and the causal relationship of the spike sequence in the time domain, and better considers the dynamic spatiotemporal information between consecutive frames. Under the dual effects of capsule network and pulse network, the present invention has good robustness in the face of different noises and conditions of different intensities, and also has good generalization and invariance to affine transformation, thereby improving the anti-interference ability of target recognition. By introducing the self-attention mechanism, the present invention can learn the degree of agreement between capsules in a single forward propagation, thereby avoiding the iterative process; a capsule attention map is generated through the self-attention layer, and then a one-dimensional convolutional layer is used to map it to the classification layer, thereby simplifying the model structure and computational complexity, and can use the self-attention mechanism to select and suppress irrelevant capsule votes, thereby retaining information useful for entity recognition and increasing the ability of long-distance dependence and feature selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a diagram of the LIF neuron model in the present invention.

[0050] Figure 2 This is the structural diagram of a common capsule network.

[0051] Figure 3 The figure is an overall framework diagram of the anti-interference target recognition method combining pulse neural network and capsule network in the present invention.

[0052] Figure 4 This is a structural diagram of the depth-wise separable convolutional layer in the present invention.

[0053] Figure 5 This is a structural diagram of the self-attention routing in the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described below in conjunction with the accompanying drawings.

[0055] The present invention provides an anti-interference target recognition method, program, device and storage medium combining a pulse neural network and a capsule network, constructs a pulse neural network model using capsule networks and LIF pulse neurons, gives the neural network better biological characteristics, and can more accurately simulate the information transfer process of the brain; uses a convolutional layer to extract the two-dimensional features of the input image; uses a deep convolutional layer to intelligently extract the characteristics of all capsules; uses a main capsule layer to further extract features and generate pulse capsules; uses self-attention routing to better select features and update routing weights; uses a digital capsule layer to output recognition results and confidence; uses a fully connected layer to reconstruct images and encourage the model to learn meaningful features. The present invention has strong anti-interference ability and a small number of parameters, and can also dynamically consider spatiotemporal information.

[0056] like Figure 1 As shown, the LIF neuron is selected as the basic neuron. The LIF neuron model continuously receives pulses δ from the presynaptic neuron. j (t), and dynamically changes its membrane potential V i (t). g(·) is the pulse emission function, when the membrane potential is greater than the threshold V th When , it controls the neuron to fire pulses. In order to reduce information loss and ensure the convergence of the network, a soft reset method is used. When the membrane potential exceeds the threshold, the membrane potential will be subtracted from the threshold, as shown in the following equation:

[0057]

[0058] Where C and R are membrane capacitance and membrane resistance respectively, w ij is the synaptic weight between the i-th LIF neuron and the j-th LIF neuron, N is the number of presynaptic neurons, and dt is V i (t) is the minimum simulation time step, usually 1ms; V reset represents the reset potential, δ i (t) represents the pulse emitted by the i-th LIF neuron;

[0059] For the convenience of calculation and modeling, the equation is converted into a discrete form, as shown in the following equation, where λ is the attenuation factor of the membrane potential.

[0060]

[0061] The convolution layer extracts the features of the input image through convolution operations and generates feature maps. Each value in each feature map represents the degree of feature activation in a specific area of ​​the input image, which is the input signal of the LIF neuron. The membrane potential of the LIF neuron will be updated according to the size and weight of the input feature value according to the above equation. At each time step, the LIF neuron will receive multiple values ​​in the feature map as input signals until the membrane potential exceeds the threshold, the neuron will emit a pulse, and reset the membrane potential.

[0062] like Figure 2 As shown in Figure 1, in general, the pulse capsule neural network consists of two parts: encoder and decoder. According to different specific tasks, its structure is expanded and improved. The first 6 layers are encoders and the last 3 layers are decoders.

[0063] like Figure 3 As shown in the figure, in the framework of the entire network, the encoder includes 6 layers, the first three layers are convolutional layers, the fourth layer is a deep convolutional layer, the fifth layer is a main capsule layer, the sixth layer is a digital capsule layer, and the decoder consists of 3 fully connected layers. The first three convolutional layers detect the basic features in the 2D image and map the input image to a higher-dimensional space to facilitate the creation of capsules; the fourth deep convolutional layer is used by the main capsule to create a vector representation of the features they represent; the fifth main capsule layer obtains the basic features detected by the convolutional layer and produces a combination of features; the sixth digital capsule layer, composed of 32 pulse capsules, is used to encode spatial information and perform final recognition; the following three layers are all fully connected layers, which decode the image for reconstruction.

[0064] Input a picture, go through three conventional convolution operations with a step size of 1, extract the basic features of the picture, activate it with LIF neurons, and input it to the next layer, as shown in the following equation:

[0065] F r+1 (X r )=LIF(Conv kxk (X r ))

[0066] Among them, X r Represents the input image data matrix of the rth layer; F r+1 (X) represents the output image data matrix of the rth layer, that is, the input image data matrix of the r+1th layer; Conv kxk (X r ) represents the feature map obtained after the input image data matrix is ​​convolved with a convolution kernel of size kxk.

[0067] like Figure 4As shown in the figure, the deep convolution layer uses a deep separable convolution with linear activation. It first performs channel-by-channel convolution, with one convolution kernel responsible for one channel, and one channel is convolved by only one convolution kernel. The number of channels of the feature map generated by this process is exactly the same as the number of channels of the input, and the feature map cannot be expanded. Moreover, this operation performs convolution operations on each channel of the input layer independently, and does not effectively utilize the feature information of different channels at the same spatial position. Therefore, it is necessary to perform point-by-point convolution again to perform weighted combination of the feature map of the previous step in the depth direction to generate a new feature map. Deep separable convolution can simplify and reduce the number of parameters required for the capsule creation process, reduce computational complexity, make it easier to learn the correlation between features, and better capture the information in the input image, making full use of the filtering function of the image to intelligently extract all capsule features.

[0068] Each capsule in the main capsule layer is composed of 8 pulse units. Each capsule applies eight convolution kernels with a step size of 2 to the input, further extracts features, cuts the last dimension of the output, and converts all the intermediate dimensions to obtain a pulse capsule with a length of 8, which is input to the next layer. Each capsule in the digital capsule layer takes the tensor of the previous layer as input, uses the transfer matrix W for matrix multiplication, discards the last dimension, and then sets the initial weight to obtain a weighted cumulative vector. The degree of consistency between the calculation result and each original vector is accumulated, and finally the L2 norm of each type of membrane potential is used as the final output of the recognition result. The length of the vector is the confidence of the recognition.

[0069] In the main capsule layer, the weights are determined using the self-attention dynamic routing algorithm; let u i and v j Represent the i-th low-level capsule and the j-th high-level capsule, respectively, w ij is the weight matrix of all affine transformations between two adjacent capsule layers, through w ij Each lower-level capsule has a prediction vector of a higher-level capsule c ij is a matrix containing all coupling coefficients generated by the self-attention algorithm, d ij is the logarithmic weight prior matrix of all other weights obtained through discriminant learning at the same time; finally, the prediction results are weighted summed to obtain s j , as shown in the following formula:

[0070]

[0071] The prior helps to create more connected capsules. The self-attention routing dynamically assigns the detected shapes to the whole represented in the specific instance considered. By introducing the self-attention mechanism, the model is able to learn the agreement between capsules in a single forward propagation, thus avoiding the iterative process, and can be used to select and suppress irrelevant capsule votes, thereby retaining information useful for entity recognition and increasing the ability of long-distance dependencies and feature selection. The coupling coefficient is calculated starting from the self-attention tensor using the following equation:

[0072]

[0073] Each capsule in a higher layer contains a symmetric matrix A r ,parameter It helps to stabilize the training and helps maintain the balance between the coupling coefficient and the logarithmic prior probability. a is a set parameter and r represents the rth layer. Each self-attention matrix contains the score consistency of each capsule prediction combination of the lower layer. Therefore, they can be used to calculate all coupling coefficients, and the matrix of all coupling coefficients can be calculated by the following equation:

[0074]

[0075] From the above formula, we can get that the sum of the coupling coefficients between the capsule in the rth layer and all the capsules in the next r+1 layer is 1. The initial logarithmic prior probability is added to the coupling coefficient in turn to obtain the effective routing weight. This process remains unchanged when there are multiple capsule layers, stacked on each other to create a deeper level.

[0076] In the digital capsule layer, all information is represented by a vector of binary pulses. Compared with real-valued artificial neurons, in addition to the partial and overall relationships between low-level capsules and high-level capsules in the spatial domain, there is also pulse time information to describe the causal relationship between pulse sequences in the time domain. The original self-attention dynamic routing is extended to SNN as shown in the following equation:

[0077]

[0078] In order to consider the activities of pre- and post-synaptic neurons and the relative order of their spike firings and to update the network weights in a shorter time, the synaptic trace x is used. pre To record the peak information of the neuron; the arrival of each presynaptic pulse will leave a trace. When there is a pulse, this trace will be updated to 1. When there is no pulse, it will decay exponentially, as shown in the following formula, where τ is the time constant;

[0079]

[0080] The generation of pulses actually selects and emphasizes a feature in the feature map. Only those features with sufficient intensity to exceed the neuron threshold will be transmitted by the neuron in the form of pulses. Through the emission and transmission of pulses, information is further propagated and processed in the network. These pulses can be received by subsequent neurons and used as their input, so as to continue tasks such as feature extraction, classification or recognition.

[0081] The salient trace encodes the temporal relationship of the synapses between presynaptic and postsynaptic neurons within a positive time window. If the presynaptic spike triggers the postsynaptic neuron within the time window, the connection will be strengthened; otherwise, it will be weakened, as determined by a threshold bias x. offset Determine; when the presynaptic pulse arrives, the intensity of the change depends on the time difference between the presynaptic pulse and the postsynaptic pulse, which is calculated as follows:

[0082]

[0083] The output layer is no longer represented by a scalar, but by a vector; the capsule of the digital capsule layer represents not only the confidence of the existence of a certain object category, but also all the attributes extracted from its parts; the length of the instantiation vector represents the confidence of the individual existence of a capsule, and its length is close to 1 if and only if the individual it represents is the only individual present in the image; therefore, in order to allow multiple categories, the following formula is used to calculate each class represented by each capsule of the digital capsule layer:

[0084] L r =T r max(0,m + -||u r ||) 2 +λ(1-T r )max(0,||u r ||-m - ) 2

[0085] Among them, if all capsules in each layer exist, and m + , m - and λ is the hyperparameter to be tuned, then T r is equal to 1.

[0086] When pre-training the capsule network based on spiking neurons, the decoder is used as a reconstruction regularizer. The output of the correct digital capsule layer is used as the input of the decoder and the image is reconstructed. The loss function is the Euclidean distance between the reconstructed image and the input image. The margin loss function is used to convert each individual margin loss L rAdd together to calculate the final score of the training phase. Use the reconstruction regularizer to encourage all output capsules to learn meaningful features and improve the ability to resist interference. The average value of the L2 loss between the input image and the decoder output is calculated by a special factor and added to the margin loss. Only the correct digital capsule layer vectors are used during training, while incorrect vectors are ignored. The decoder forces the capsule to learn features that are useful for reconstructing the original image. The closer the reconstructed image is to the input image, the better.

[0087] An electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of a method for anti-interference target recognition combining a pulse neural network and a capsule network described in the first specific implementation mode when executing the computer program. The computer device of the present invention may be a device including a processor and a memory, such as a single-chip microcomputer including a central processing unit. In addition, the processor is used to implement the steps of the above-mentioned method for anti-interference target recognition combining a pulse neural network and a capsule network when executing the computer program stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0088] The computer-readable storage medium of the present invention may be any form of storage medium that can be read by a processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. A computer program is stored on the computer-readable storage medium. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned anti-interference target recognition method combining a pulse neural network and a capsule network can be implemented. The computer program includes a computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device that can carry the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An anti-interference target recognition method combining a pulse neural network and a capsule network, characterized in that: The following steps are involved: Step 1: Build a capsule network based on spiking neurons and perform pre-training; The capsule network based on pulse neurons includes a convolution layer, a deep convolution layer, a main capsule layer and a digital capsule layer; The convolution layer is used to extract basic features in the input image and map the input image to a higher dimensional space; The deep convolutional layer is used to create a vector representation of the basic features of the image; The main capsule layer is used to generate a combination of basic features of the image to generate impulse capsules; The digital capsule layer uses self-attention routing to select features and update routing weights, and outputs recognition results and confidence levels; Step 2: Input the image to be recognized into the pre-trained capsule network based on spiking neurons, and take the classification result with the highest confidence in the digital capsule layer as the recognition result of the target.

2. The anti-interference target recognition method combining a pulse neural network and a capsule network according to claim 1, characterized in that: The capsule network based on pulse neurons adopts LIF neurons as basic neurons, and LIF neurons continuously receive pulses δ from presynaptic neurons. j (t), and dynamically changes its membrane potential V i (t); when the membrane potential V i (t) is greater than the threshold V th When the membrane potential V i (t) is greater than the threshold V th When the membrane potential V i (t) will be minus the threshold value V th , as shown below: Where C and R are the membrane capacitance and membrane resistance respectively, w ij is the synaptic weight between the i-th LIF neuron and the j-th LIF neuron, N is the number of presynaptic neurons, and dt is V i (t) is the minimum simulation time step; V reset represents the reset potential, δ i (t) represents the pulse emitted by the i-th LIF neuron; Convert the above formula into discrete form as shown below: Here, λ is the attenuation factor of membrane potential. The convolution layer extracts the features of the input image through convolution operations and generates a feature map. Each value in each feature map represents the degree of feature activation of the input image in a specific area, which is the input signal of the LIF neuron. The membrane potential of the LIF neuron will be updated according to the size and weight of the input feature value according to the above formula; at each time step, the LIF neuron will receive multiple values ​​in the feature map as input signals until the membrane potential exceeds the threshold, the neuron will emit a pulse and reset the membrane potential.

3. The anti-interference target recognition method combining a pulse neural network and a capsule network according to claim 2 is characterized in that: The capsule network based on pulse neurons includes multiple convolutional layers. Each convolutional layer performs a conventional convolution operation on the input image, extracts the basic features of the image, and activates the LIF neurons before inputting them to the next layer. The specific process is represented by the following formula: F r+1 (X r )=LIF(Conv kxk (X r )) Among them, X r Represents the input image data matrix of the rth layer; F r+1 (X) represents the output image data matrix of the rth layer, that is, the input image data matrix of the r+1th layer; Conv kxk (X r ) represents the feature map obtained after the input image data matrix is ​​convolved with a convolution kernel of size kxk.

4. The anti-interference target recognition method combining a pulse neural network and a capsule network according to claim 3 is characterized in that: The deep convolution layer of the capsule network based on pulse neurons uses a deep separable convolution with linear activation, and first performs channel-by-channel convolution, where one convolution kernel is responsible for one channel, and one channel is convolved by only one convolution kernel. The number of channels of the feature map generated by this process is exactly the same as the number of channels of the input, and the feature map cannot be expanded. Moreover, this operation performs convolution operations on each channel of the input layer independently, and cannot effectively utilize the feature information of different channels at the same spatial position. Therefore, it is necessary to perform point-by-point convolution again, and perform weighted combination of the feature map of the previous step in the depth direction to generate a new feature map; the deep separable convolution can simplify and reduce the number of parameters required for the capsule creation process, reduce the computational complexity, make it easier to learn the correlation between features, and better capture the information in the input image, and make full use of the filtering function of the image to intelligently extract all capsule features.

5. The anti-interference target recognition method combining a pulse neural network and a capsule network according to claim 4, characterized in that: In the main capsule layer of the capsule network based on spiking neurons, the weights are determined using a self-attention dynamic routing algorithm; let u i and v j Represent the i-th low-level capsule and the j-th high-level capsule, respectively, w ij is the weight matrix of all affine transformations between two adjacent capsule layers, through w ij Each lower-level capsule has a prediction vector of a higher-level capsule c ij is a matrix containing all coupling coefficients generated by the self-attention algorithm, d ij is the logarithmic weight prior matrix of all other weights obtained through discriminant learning at the same time; finally, the prediction results are weighted summed to obtain s j , as shown in the following formula: The coupling coefficient is calculated starting from the self-attention tensor using the following formula: Each capsule in a higher layer contains a symmetric matrix A r ,parameter It helps to stabilize training and helps maintain a balance between coupling coefficients and logarithmic prior probabilities. a is a set parameter and r represents the rth layer. Each self-attention matrix contains the score consistency of each capsule prediction combination in the lower layer. The matrix of all coupling coefficients is calculated by the following formula: The sum of the coupling coefficients between the capsule in the rth layer and all the capsules in the next r+1 layer is 1. The initial logarithmic prior probability is added to the coupling coefficients in turn to obtain the effective routing weights. This process remains the same when multiple capsule layers are stacked on each other to create a deeper level.

6. The anti-interference target recognition method combining a pulse neural network and a capsule network according to claim 5, characterized in that: In the digital capsule layer of the capsule network based on spiking neurons, all information is represented by a vector of binary pulses, and the original self-attention dynamic routing is extended to SNN as shown in the following formula: In order to take into account the activities of pre- and post-synaptic neurons and the relative order of their spike firings and to update the network weights in a shorter time, the synaptic trace x is used. pre To record the peak information of the neuron; the arrival of each presynaptic pulse will leave a trace. When there is a pulse, this trace will be updated to 1. When there is no pulse, it will decay exponentially, as shown in the following formula, where τ is the time constant; The generation of pulses actually selects and emphasizes a feature in the feature map. Only those features with sufficient intensity to exceed the neuron threshold will be transmitted by the neuron in the form of pulses. Through the emission and transmission of pulses, information is further propagated and processed in the network. These pulses will be received by subsequent neurons and used as their inputs to continue feature extraction, classification or recognition tasks. The salient trace encodes the temporal relationship of the synapses between presynaptic and postsynaptic neurons within a positive time window. If the presynaptic spike triggers the postsynaptic neuron within the time window, the connection will be strengthened; otherwise, it will be weakened, as determined by a threshold bias x. offset Determine; when the presynaptic pulse arrives, the intensity of the change depends on the time difference between the presynaptic pulse and the postsynaptic pulse, which is calculated as follows: The output layer is no longer represented by a scalar, but by a vector; the capsule of the digital capsule layer represents not only the confidence of the existence of a certain object category, but also all the attributes extracted from its parts; the length of the instantiation vector represents the confidence of the individual existence of a capsule, and its length is close to 1 if and only if the individual it represents is the only individual present in the image; therefore, in order to allow multiple categories, the following formula is used to calculate each class represented by each capsule of the digital capsule layer: L r =T r max(0,m + -||u r ||) 2 +λ(1-T r )max(0,||u r ||-m - ) 2 Among them, if all capsules in each layer exist, and m + , m - and λ is the hyperparameter to be tuned, then T r is equal to 1.

7. The anti-interference target recognition method combining a pulse neural network and a capsule network according to claim 6, characterized in that: During the pre-training of the capsule network based on spiking neurons, the output of the correct digital capsule layer is used as the input of the decoder, and the image is reconstructed. The loss function is the Euclidean distance between the reconstructed image and the input image. The margin loss function is used to convert each individual margin loss L r Add together to calculate the final score of the training phase; use the reconstruction regularizer to encourage all output capsules to learn meaningful features and improve the ability to resist interference. The average value of the L2 loss between the input image and the decoder output is calculated by a special factor and added to the margin loss; Only correct digit capsule layer vectors are used during training, while incorrect vectors are ignored; the decoder forces the capsule to learn features that are useful for reconstructing the original image, and the reconstructed image is as close to the input image as possible.

8. A computer device / equipment / system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images