Image analysis method based on spiking neural network and related apparatus

Abstract

Embodiments of the present application relate to the technical field of artificial intelligence, and disclose an image analysis method and device based on a pulse neural network, computer equipment and a computer readable storage medium, the method comprising: acquiring an image to be analyzed; inputting the image to be analyzed into a pulse neural network with inhibitory neurons; the neural network comprising a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse self-feedback module and an inference module; the inhibitory pulse self-feedback module comprising at least one inhibitory neuron for dynamically emitting inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons; signal transmission between neurons of the pulse neural network is performed through integer pulses; the neurons comprise excitatory neurons and inhibitory neurons in the convolution-based SNN module and the Transformer-based SNN module; and outputting an analysis result. In the above manner, the embodiments of the present application improve the prediction accuracy of the model and significantly reduce the computing resources.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of artificial intelligence, in particular to an image analysis method and device based on a spiking neural network, computer equipment and a computer readable storage medium. BACKGROUND

[0002] At present, as the third generation of neural networks, the spiking neural network (SNN) has the brain-like information processing characteristics due to the simulation of the pulse discharge mechanism of biological neurons and the event-driven computing characteristics, and exhibits significant potential in efficient computing in the field of low-power intelligence, gradually attracting widespread attention from the academic and industrial circles.

[0003] In the SNN, neurons exchange information through discrete pulse signals, and adopt the "integration-fire" dynamic discharge mechanism similar to biological neurons. Under this mechanism, each neuron accumulates the received pulse signals to its membrane potential, and when the membrane potential reaches a predetermined threshold, the neuron will generate a pulse and reset its potential according to a specific rule. This event-driven mechanism ensures that the SNN performs only a small amount of sparse addition calculation when necessary, thereby significantly reducing power consumption compared to traditional artificial neural network (ANN) models. For example, the Loihi2 neuro-morphic chip has a typical inference power consumption as low as several milliwatts, which can easily meet the demand for lightweight and efficient edge devices such as intelligent sensors and wearable devices. Although the SNN has great potential in low-power computing, the applicant found that there are problems such as insufficient feature expression ability and high bias of redundant pulses in the traditional pulse firing mechanism through pulse visualization of the existing SNN network, which makes the deployment and application of SNN on resource-constrained edge devices still face many challenges. SUMMARY

[0004] In view of the above problems, the embodiment of the present application provides an image analysis method and device based on a spiking neural network, computer equipment and a computer readable storage medium, which are used to solve the problems of poor image prediction accuracy, low efficiency and unsuitability for edge devices of the existing spiking neural network.

[0005] According to an aspect of the embodiment of the present application, an image analysis method based on a spiking neural network is provided, which comprises:

[0006] acquiring an image to be analyzed;

[0007] The image to be analyzed is input into a spiking neural network with inhibitory neurons; wherein the spiking neural network with inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse feedback module, and an inference module; the inhibitory pulse feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons; the neurons of the spiking neural network transmit signals through integer pulses; the neurons include excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module, and the inhibitory neurons;

[0008] Output the analysis results of the image to be analyzed.

[0009] In one alternative approach, each of the inhibitory neurons in the inhibitory impulse self-feedback module is positioned between adjacent excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module.

[0010] The step of inputting the image to be analyzed into a spiking neural network with inhibitory neurons includes:

[0011] The input pulse signal corresponding to the image to be analyzed is input into the convolution-based SNN module for local feature extraction, and then input into the Transformer-based SNN module to output a globally correlated spatiotemporal feature pulse sequence.

[0012] During signal transmission, the inhibition neurons in the inhibition pulse self-feedback module inhibit the integer pulses output by each excitation neuron of the convolution-based SNN module and the Transformer-based SNN module, respectively.

[0013] The inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis result of the image to be analyzed.

[0014] In one optional embodiment, during signal transmission, the inhibition neurons in the inhibition pulse self-feedback module inhibit the integer pulses output by each excitatory neuron of the convolution-based SNN module and the Transformer-based SNN module, respectively, including:

[0015] After receiving the input pulse signal corresponding to the image to be analyzed at the current moment, the target excitatory neuron generates the integer pulse; the target excitatory neuron is any one of a plurality of excitatory neurons;

[0016] The membrane potential of the adjacent excitatory neurons at the next time step is updated based on the membrane potential of the adjacent excitatory neurons at the current time step, the integer pulse output by the target excitatory neuron at the current time step, and the inhibition signal of the target inhibitory neuron; the adjacent excitatory neurons are the excitatory neurons adjacent to the target excitatory neuron; the target inhibitory neuron is the inhibitory neuron between the target neuron and the adjacent excitatory neurons.

[0017] In one alternative approach, after the target excitatory neuron receives the input signal corresponding to the image to be analyzed at the current moment, it generates the integer pulse, including:

[0018] The integer pulse output by the target excitatory neuron n at the current time t is represented as: in, ∈{0,1,2,...,D}, represents the integer pulse value of the target excitatory neuron n at the current time t; This represents the membrane potential of the target excitatory neuron n at time t; Let be the integer firing function of the target neuron n at the current time t; where, the integer firing function is defined. for: in, This means restricting V to the interval [0, D], where D ∈ N. + It is the maximum allowed pulse value. Represents the nearest integer; the membrane potential of the adjacent excitatory neuron at the next time t+1 is updated based on the membrane potential of the adjacent excitatory neuron at the current time, the output signal of the target excitatory neuron at the current time t, and the inhibition signal of the target inhibitory neuron, including:

[0019] The membrane potential of the adjacent excitatory neuron j at time t+1 is determined according to the following formula:

[0020] in, Indicates target inhibitory neuron i For the inhibition signal of the adjacent excitatory neuron j, G(·) is the inhibition function, which uses a linear mapping to characterize the inhibition strength; , where is the positive connection weight of the adjacent neuron j; This is the amplification factor to suppress the signal; This represents the set of neighboring neurons that are connected to the adjacent neuron j.

[0021] In one alternative approach, the inference module is based on a game-theoretic early exit mechanism; the inference module includes multiple nodes; each node includes a discriminant head and an exit point;

[0022] The inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis results of the image to be analyzed, including:

[0023] When the spatiotemporal feature pulse sequence is transmitted to the current node, the current utility corresponding to the current node is calculated; the current node is any one of the plurality of nodes; the current utility is calculated based on the first detection accuracy, the first cost saving, and the first information entropy of the image sample at the current exit point.

[0024] When the current utility is greater than the expected utility, exit at the current exit point and obtain the analysis result of the image to be analyzed;

[0025] When the current utility is less than or equal to the expected utility, continue to pass it to the next exit point;

[0026] The next exit point is taken as the current exit point, and the process of determining the magnitude of the current utility and the expected utility of the next exit point continues until the current utility is greater than the expected utility or the last node is reached, at which point the process exits and the analysis result of the image to be analyzed is obtained.

[0027] In an alternative embodiment, before inputting the image to be analyzed into a spiking neural network with inhibitory neurons, the method further includes:

[0028] Obtain image samples;

[0029] The image samples are input into the spiking neural network with inhibitory neurons for training to obtain sample analysis results;

[0030] Based on the sample analysis results and image sample labels, a preset loss function is used to calculate the predicted total loss;

[0031] During the backpropagation phase, the gradient propagation path of the predicted total loss with respect to all parameters in the spiking neural network is calculated; wherein, for the inhibitory neuron, the gradient of the predicted total loss with respect to the parameters of the inhibitory neuron is obtained by using the chain rule.

[0032] Based on the gradient propagation path of all parameters in the spiking neural network, all parameters of the spiking neural network are updated;

[0033] The image sample is then input into the updated spiking neural network to continue iterative training with parameter updates until a trained spiking neural network with inhibitory neurons is obtained.

[0034] In one alternative approach, after inputting the Transformer-based SNN module, the output is a globally correlated spatiotemporal feature pulse sequence, including:

[0035] Perform a separable convolution operation on the target pulse signal to obtain the first pulse data;

[0036] Selective local pattern attention is applied to the first pulse data to obtain the second pulse data; the selective local pattern attention includes a pulse attention matrix and an SLPA module, the SLPA module being used to control the numerical range of the attention score;

[0037] The second pulse data is processed by a multilayer perceptron at the channel dimension to obtain the output predicted pulse sequence.

[0038] According to another aspect of the present invention, an image analysis apparatus based on a spiking neural network is provided, comprising:

[0039] The acquisition module is used to acquire the image to be analyzed.

[0040] A prediction module is used to input the image to be analyzed into a spiking neural network with inhibitory neurons; wherein the spiking neural network with inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse feedback module, and an inference module; the inhibitory pulse feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons; the neurons of the spiking neural network transmit signals through integer pulses; the neurons include excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module, and the inhibitory neurons;

[0041] The output module is used to output the analysis results of the image to be analyzed.

[0042] According to another aspect of the present invention, a computer device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0043] The memory is used to store at least one executable instruction that causes the processor to perform the operation of the image analysis method based on the spiking neural network.

[0044] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing at least one executable instruction, which, when executed on a computer device, causes the computer device to perform the operation of the image analysis method based on a spiking neural network.

[0045] This invention provides an embodiment of the invention that acquires an image to be analyzed; inputs the image to be analyzed into a spiking neural network (SNN) with inhibitory neurons; wherein the SNN with inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse self-feedback module, and an inference module; the inhibitory pulse self-feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons; the neurons of the spiking neural network transmit signals through integer pulses; the neurons include excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module, and the inhibitory neurons; and outputs the analysis result of the image to be analyzed. In this embodiment, firstly, integer pulses are fired in the network, and inhibitory neurons are added. By firing inhibitory pulses, the membrane potential of neurons is dynamically regulated to optimize the firing accuracy of neurons while significantly reducing pulse density. Secondly, addressing the problem that self-attention mechanisms consume a high proportion of computational resources in resource-constrained environments, this application also proposes a lightweight Spike Lite-Pool Attention (SLPA) module. By fusing attention matrices from asynchronous long-pooling paths, it achieves efficient fusion of cross-scale features, effectively reducing computational costs while maintaining discriminative ability. Finally, to alleviate resource bottlenecks and efficiency issues during the inference phase, this application explores the impact of sample complexity and network depth on detection accuracy and proposes an early termination mechanism based on Nash equilibrium theory in game theory. This mechanism adaptively optimizes the inference path using game theory, significantly reducing the computational resources and time overhead required for inference while ensuring the reasonableness of the prediction.

[0046] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0047] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0048] Figure 1 A flowchart illustrating the image analysis method based on a spiking neural network provided in an embodiment of the present invention is shown.

[0049] Figure 2 A schematic diagram of the structure of the spiking neural network provided in an embodiment of the present invention is shown;

[0050] Figure 3 This diagram illustrates a comparison of signal transmission between a spiking neural network with added inhibitory neurons and a traditional spiking neural network, as provided in an embodiment of the present invention.

[0051] Figure 4 A schematic diagram of the algorithm for the inference module in a spiking neural network provided in an embodiment of the present invention is shown;

[0052] Figure 5 This diagram illustrates a game tree for the inference module in a spiking neural network provided in an embodiment of the present invention.

[0053] Figure 6 A schematic diagram of the structure of the image analysis device based on a spiking neural network provided in an embodiment of the present invention is shown;

[0054] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0055] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0056] This section further elaborates on the problems identified by the applicant regarding existing SNN models. Due to the generally limited computing resources, storage space, and power consumption budgets of edge devices, more stringent requirements are placed on the feature representation capabilities and computational efficiency of the deployed network models. Currently, lightweight and efficient SNN models for edge device deployments still fall short. Through pulse visualization of existing SNNs, the applicant discovered the following main phenomena: 1. Neurons in the target region do not consistently generate effective pulses. 2. A large number of invalid pulses exist in the background region.

[0057] This indicates that SNNs currently face the following main challenges: First, insufficient feature representation capability. The traditional binary spiking mechanism makes it difficult for neurons to finely characterize the spatial details and dynamic changes of the input signal, resulting in the model's inability to effectively extract and express key features, severely limiting the predictive performance of SNNs. Second, severe spiking redundancy. This is because there is still a significant gap between SNNs and traditional ANNs in terms of task prediction accuracy. To improve the performance of SNNs, single-time-step and multi-time-step models optimize accuracy by increasing spiking intensity and time step size, respectively. For example, using multi-spiking thresholds; increasing spiking density; adjusting neuronal synapses to integrate dense spiking at a single time step; and adjusting the time step size according to the task. However, while the above methods improve the model's accuracy to some extent, they lead to the generation of a large number of redundant spiking within the network, increasing computational load and energy consumption costs, severely restricting the deployment of the model on edge platforms with limited computing resources. They also result in higher spiking frequency and computational complexity, greatly limiting their application on edge devices. Therefore, how to effectively improve the feature representation capability of SNNs while reducing spiking redundancy has become a crucial breakthrough for promoting the application of SNNs on edge devices.

[0058] Based on this, this application proposes an image analysis method based on spiking neural networks. This method overcomes the bottlenecks of traditional SNNs in terms of energy efficiency and accuracy, and provides a new paradigm for the efficient deployment of brain-like computing models in resource-constrained scenarios. Figure 1 A flowchart of an image analysis method based on a spiking neural network, provided in an embodiment of the present invention, is shown. This method is executed by a computer device. The computer device can be a computer, a cloud computing device, an edge storage device, a wearable device, etc., and the embodiments of the present invention do not impose specific limitations. Figure 1 As shown, the method includes the following steps:

[0059] Step 110: Obtain the image to be analyzed.

[0060] The image to be analyzed can be an image that requires analysis such as image recognition, image classification, object detection, image semantic segmentation, and image neuromorphic recognition.

[0061] Step 120: Input the image to be analyzed into a spiking neural network with inhibitory neurons.

[0062] In embodiments of the present invention, such as Figure 2As shown, the spiking neural network based on inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse feedback module, and an inference module. The inhibitory pulse feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons. Signal transmission between neurons in the spiking neural network occurs via integer pulses. The neurons include excitatory neurons from the convolution-based SNN module and the Transformer-based SNN module, as well as the inhibitory neurons.

[0063] In this embodiment, each of the inhibitory neurons in the inhibitory impulse self-feedback module is positioned between adjacent excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module.

[0064] In this embodiment of the invention, after obtaining the trained spiking neural network with inhibitory neurons, the following steps are specifically performed when analyzing the image to be analyzed:

[0065] Step 1201: The input pulse signal corresponding to the image to be analyzed is input into the convolution-based SNN module for local feature extraction, and then input into the Transformer-based SNN module to output a globally correlated spatiotemporal feature pulse sequence.

[0066] The convolution-based SNN module described in this embodiment combines four typical variants of the inverted bottleneck structure to enhance expressive power and adapt to the needs of different network depths. The convolution-based SNN module adopts a standard PW-DW-PW architecture combined with the inverted bottleneck structure, which can be any of the four variants, each with unique characteristics. For example, variant a significantly improves the model's capacity by spatially mixing the expanded activation features, but its computational cost also increases accordingly; variant b performs spatial mixing before expansion, achieving large kernel size spatial mixing at a lower cost; variant c can expand network depth and receptive field at low cost, combining the advantages of the ConvNext-Like structure and IBB; variant d consists of two 1×1 point convolutions (PW) stacked together, with an activation layer and a normalization layer embedded in the middle, a structure that is accelerator-friendly. However, optimal performance usually requires use in conjunction with other modules. In this embodiment, a suitable structure can be flexibly selected based on the feature expression requirements of different network depths. In particular, when variant a is used, the computation flow of the Conv-based SNNBlock can be described as follows:

[0067] X' = X + SpikeSepConv(X) ;

[0068] X" = X' + ChannelConv(X') ;

[0069] SpikeSepConv(X) = Conv pw2 (SN(Conv dw2 (SN(Conv pw1 (SN(Conv dw1 (SN(X))))))));

[0070] ChannelConv(X') = Conv(SN(Conv(SN(X')))

[0071] in, SpikeSepConv(·) It is a pulse-driven inverted bottleneck structure. ChannelConv (·) contains two ordinary convolutional modules used to adjust feature channels; Conv pw1 (·), Conv pw2 (·) represents pointwise convolutions. Conv dw1 (·), Conv dw2 (·) stands for depthwise convolution. Conv (·) represents a regular convolutional module. SN (·) represents the spiking neuron layer. The BN layer is omitted in this embodiment for simplification.

[0072] Traditional self-attention modules rely heavily on matrix multiplication, resulting in high energy consumption due to the lack of sparsity in their computation. To address this issue, existing technologies typically incorporate pooling layers into the attention mechanism to reduce matrix dimensionality; however, this leads to a decrease in network accuracy. To resolve this problem, in this embodiment of the invention, the Transformer-based SNN module integrates attention matrices with different pooling steps, thereby reducing computational complexity while maintaining attention expressiveness. Specifically, the Transformer-based SNN module can be represented as follows: X' = X + SpikeSepConv (X) ; X" = X' + SLPA(X') ; X'" = X" + ChannelMLP(X")

[0073] in, X" = X' + SLPA(X') This indicates a separable convolution operation; ChannelMLP(X") It is a multilayer perceptron. X'" = X" + ChannelMLP(X") The selective local pattern attention submodule includes the impulse attention matrix and the SLPA module: SLPA(X') .in, SLPA(X') The specific calculation method is as follows:

[0074]

[0075] ;

[0076] ;

[0077] SLPA(X) ;

[0078] in, It is a pulse attention matrix. It is an average pooling layer. It is a linear layer. It is a concatenation function. The role of scale is to control the numerical range of attention scores, prevent the softmax output from becoming extreme, and thus improve training stability and generalization performance.

[0079] Specifically, the input pulse signal corresponding to the image to be analyzed is input into the convolution-based SNN module for local feature extraction to obtain the target pulse signal. The obtained target pulse signal is then input into the Transformer-based SNN module to output a globally correlated spatiotemporal feature pulse sequence. This includes:

[0080] Perform a separable convolution operation on the target pulse signal to obtain the first pulse data;

[0081] Selective local pattern attention is applied to the first pulse data to obtain the second pulse data; the selective local pattern attention includes a pulse attention matrix and an SLPA module, the SLPA module being used to control the numerical range of the attention score;

[0082] The second pulse data is processed by a multilayer perceptron in the channel dimension to obtain an output predicted pulse sequence. A separable convolution operation is performed on the target pulse signal to obtain first pulse data. Selective local pattern attention is applied to the first pulse data to obtain second pulse data. The selective local pattern attention includes a pulse attention matrix and an SLPA module, wherein the SLPA module is used to control the numerical range of the attention score. The second pulse data is processed by a multilayer perceptron in the channel dimension to obtain a globally correlated spatiotemporal feature pulse sequence, which is the output predicted pulse sequence.

[0083] Step 1202: During signal transmission, the inhibition neurons in the inhibition pulse self-feedback module inhibit the integer pulses output by each excitation neuron of the convolution-based SNN module and the Transformer-based SNN module, respectively.

[0084] In embodiments of the present invention, such as Figure 3As shown, the suppressed impulse self-feedback module, by introducing suppressor neurons, suppresses the accumulation of noise potentials, enabling the network to focus more on the target region. Specifically, it optimizes impulse information transmission through the following key steps: 1. Using integer impulse values ​​as the basic form of signal transmission between neurons; 2. Introducing suppressor neurons between spiking neurons to transmit suppressor signals to adjacent neurons, thereby regulating local information flow; 3. Suppressor neurons adaptively adjust the firing balance of the neuron population in the network by receiving backpropagated gradient information (this part plays a role in the model training process).

[0085] Specifically, after receiving the input pulse signal corresponding to the image to be analyzed at the current moment, the target excitatory neuron generates the integer pulse. The target excitatory neuron can be any one of a plurality of excitatory neurons.

[0086] In this embodiment of the invention, the integer pulse output by the target excitatory neuron n at the current time t is represented as: in, ∈{0,1,2,...,D}, represents the integer pulse value of the target excitatory neuron n at the current time t; This represents the membrane potential of the target excitatory neuron n at time t; Let be the integer firing function of the target neuron n at the current time t.

[0087] Here, the integer distribution function is defined. for: in, This means restricting the membrane potential V to the interval [0, D], where D ∈ N. + It is the maximum allowed pulse value. It represents the nearest integer.

[0088] The membrane potential of adjacent excitatory neurons at the next time step is updated based on the membrane potential of the adjacent excitatory neurons at the current time step, the integer pulse output by the target excitatory neuron at the current time step, and the inhibitory signal of the target inhibitory neuron. Here, the adjacent excitatory neurons are the excitatory neurons adjacent to the target excitatory neuron; the target inhibitory neuron is the inhibitory neuron between the target neuron and the adjacent excitatory neurons.

[0089] Specifically, the adjacent excitatory neurons j The membrane potential at the next time step t+1 is based on the membrane potential of the adjacent excitatory neurons at the current time. The output signal of the target excitatory neuron at the current time t and the inhibitory signals of the target inhibitory neurons Update.

[0090] The membrane potential of the adjacent excitatory neuron j at time t+1 is determined according to the following formula:

[0091] in, Indicates target inhibitory neuron i For the inhibition signal of the adjacent excitatory neuron j, G(·) is the inhibition function, which uses a linear mapping to characterize the inhibition strength; , where is the positive connection weight of the adjacent neuron j; This is the amplification factor to suppress the signal; This represents the set of neighboring neurons that are connected to the adjacent neuron j. Step 1203: The inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis result of the image to be analyzed.

[0092] In the SNN model inference process, the discrimination difficulty of different samples varies significantly: some samples obtain sufficiently accurate representations at shallow network layers, while others require deeper feature representations to achieve high-confidence predictions. This difference essentially stems from the dynamic relationship between the complexity of the information contained in the sample and the feature extraction capability within the network. Based on this key observation, this invention introduces a game theory-based early-retreat mechanism to dynamically optimize the inference path for different samples and accurately determine the most suitable network depth for each sample. By modeling the utility game between samples and network depth, this invention achieves precise allocation of computing resources, effectively alleviating the computational latency and power consumption bottlenecks in edge device deployment while ensuring prediction quality, thus opening up a new paradigm in the fields of intelligent edge computing and dynamic network inference. Figure 4 As shown, the inference module of this embodiment is based on the early exit mechanism of game theory; the inference module includes multiple nodes; each node includes a discriminator and an exit point. In the network structure using this early exit method, each potential exit point is equipped with an independent discriminator. During the inference phase, when the information flow is passed to each exit point, the system will jointly calculate the corresponding utility function based on the detection accuracy of the current node, the model parameter scale, and the information entropy of the sample. Subsequently, this utility will be compared with the expected utility of the next exit point: if the current utility is greater than the expected utility of the next layer, the model chooses to exit early at the current node; if it is less, it continues to pass to the next layer and performs subsequent calculations. In the figure, early exit can be achieved by completing the inference at the third exit point. Assume that there are n exit points in the deep neural network, numbered sequentially as 1, 2, ..., n; at each exit point, the network will generate analysis results, accompanied by a certain amount of computational cost. Let This represents the cumulative computational cost from network input to the i-th exit point, where If the network makes an early exit decision at the i-th exit point, the computational cost saving relative to a full run (up to the final exit point) can be defined as: Obviously The larger the value, the more computing resources are saved by exiting early.

[0093] Specifically, when the spatiotemporal feature pulse sequence is transmitted to the current node, the current utility corresponding to the current node is calculated; the current node is any one of the plurality of nodes; the current utility is calculated based on the first detection accuracy, the first cost saving, and the first information entropy of the image sample at the current exit point.

[0094] When the current utility is greater than the expected utility, exit at the current exit point and obtain the analysis result of the image to be analyzed.

[0095] If the current utility is less than or equal to the expected utility, continue to the next exit point.

[0096] The next exit point is taken as the current exit point, and the process of determining the magnitude of the current utility and the expected utility of the next exit point continues until the current utility is greater than the expected utility or the last node is reached, at which point the process exits, obtaining the analysis result of the image to be analyzed. Specifically, in this embodiment of the invention, a local utility function is pre-constructed. A utility function is constructed at each exit point to characterize the comprehensive benefits of early exit in terms of prediction accuracy and computational cost savings. Let the prediction accuracy of the current exit point be... The utility gained from immediately exiting at the current exit point i can be defined as: ,in, It is a utility function. It is the information entropy of the sample. These are the weighted coefficients for accuracy, computational savings, and sample complexity, respectively, with specific values ​​determined through experimental experience. The expected utility for continuing computation—that is, abandoning the current exit point and utilizing subsequent nodes—is denoted as: in, Indicates the next exit point i+1. Expected utility that can be obtained. At the last exit point, the network must exit, at which point there are no further computational options and no additional computational cost savings. After constructing the local utility function, a sequential game process is executed: since the decision at each exit point has a terminating effect (i.e., once a node makes an early exit decision, the decision-making process of subsequent exit points terminates), this problem can be regarded as a sequential stopping game of perfect information. This embodiment of the invention employs a non-cooperative game model, creating a competitive and cooperative relationship between each exit point. Each exit point, based on the principle of maximizing its own utility, decides whether to exit early or continue with subsequent computations under given conditions. The Nash equilibrium state among the decision-makers ensures the stability of their respective decisions and the efficiency of the overall system. When any decision-maker's adjustment of its decision cannot further improve its own utility, the system reaches a game equilibrium state, thereby achieving optimal allocation of overall computational resources. Its decision-making process can be described as follows: When the network reaches the current exit point... i At that time, the prediction accuracy of the current node has been obtained. and the costs that can be saved by exiting early. As for subsequent exit points, since the actual prediction accuracy is unknown, the expected value can be estimated using historical statistics from a pre-trained model to derive the expected utility. At the current exit point i, compare the expected future utility of exiting immediately versus continuing computation. Based on prior knowledge of the network's accuracy distribution at different exit points during training, we can determine the utility function. The function exhibits an upwardly convex distribution at different exit points. Therefore, if this function has a maximum value, this maximum value must be unique. If, at the current exit point i, early exit is considered to achieve the optimal balance between prediction accuracy and computational cost savings, an exit decision is made; otherwise, computation continues in hopes of obtaining better utility later. Once early exit is chosen at an exit point, the decision-making process immediately terminates, and the network outputs the prediction result at that point. The game tree of the game process is as follows: Figure 5 As shown. If all exit points choose to continue, then a forced exit will eventually occur at exit point n, and the desired effect will be achieved. That is, the boundary conditions are This recursive relationship conforms to the Bellman optimality principle, and the optimal strategy for each level can be determined by backtracking from the endpoint. Finally, the decision rule for the current exit point i is expressed as: This strategy set This constitutes a Nash equilibrium in a non-cooperative sequential game, meaning that, assuming other exit strategies remain unchanged, no layer has an incentive to unilaterally change its exit decision. This equilibrium ensures that the inference path reaches a stable and optimal balance between accuracy and computational cost, significantly improving the overall resource allocation efficiency of the system.

[0097] In this embodiment of the invention, before inputting the image to be analyzed into a spiking neural network with inhibitory neurons, the spiking neural network with inhibitory neurons is first trained in the following manner:

[0098] Step 001: Obtain image samples. These image samples are images that require prediction training for image recognition, image classification, object detection, image semantic segmentation, or image neuromorphic recognition. In this embodiment of the invention, the sample images are pre-labeled to obtain corresponding image sample labels, and the sample images with these labels are used as multiple image samples in the image sample set.

[0099] Step 002: Input the image sample into the spiking neural network with inhibitory neurons for training to obtain the sample prediction result.

[0100] Step 003: Based on the sample prediction results and image sample labels, calculate the total prediction loss using a preset loss function.

[0101] The embodiments of the present invention do not specifically limit the specific loss function, and can be a loss function based on firing rate, a loss function based on pulse time, or a loss function based on membrane potential.

[0102] Step 004: In the backpropagation phase, calculate the gradient propagation path of the total prediction loss L for all parameters in the spiking neural network.

[0103] In this embodiment of the invention, considering that an inhibitory impulse self-feedback module is set up, during the backpropagation stage, for each inhibitory neuron of the inhibitory impulse self-feedback module, the gradient of the predicted total loss L with respect to the parameters of the inhibitory neuron is obtained by using the chain rule, thus obtaining the gradient propagation path of the parameters of the inhibitory neuron.

[0104] Specifically, let inhibitory neurons be... i At any moment t The output suppression signal is The gradient propagation path is then: in, To inhibit neurons i The parameters, Let represent the membrane potential of the adjacent excitatory neuron j (the excitatory neuron adjacent to the target excitatory neuron i) at time t+1. This membrane potential is obtained from the membrane potential update formula: Therefore, we obtain: in, L represents the total loss, where L is the amplification factor for the inhibitory signal of neuron i on its neighboring neuron j. This amplification factor is pre-tuned using different amplification factor values ​​trained on a set of image samples. Therefore, the parameters of the inhibitory neuron... The update can be performed using gradient descent: in, The learning rate is used. The input during training is: training dataset D = {( x k ,y k )} (Includes input samples) x k and corresponding tags y k The parameters are: total number of training rounds E, learning rate η, and loss function L(⋅). The output consists of: network parameters W and inhibitory neuron parameters θ.

[0105] Step 005: Update all parameters in the spiking neural network according to the gradient propagation path of all parameters in the spiking neural network.

[0106] Step 006: Input the image sample into the updated spiking neural network and continue iterative training to update the parameters until a trained spiking neural network with inhibitory neurons is obtained.

[0107] Step 130: Output the analysis results of the image to be analyzed.

[0108] In this embodiment of the invention, firstly, integer pulse firing is employed in the network, and inhibitory neurons are added. By firing inhibitory pulses, the neuronal membrane potential is dynamically adjusted to optimize the firing accuracy of neurons while significantly reducing pulse density. Secondly, addressing the issue of high computational resource consumption in resource-constrained environments, this application proposes a lightweight Spike Lite-Pool Attention (SLPA) module. By fusing attention matrices from asynchronous long-pooling paths, it achieves efficient fusion of cross-scale features, effectively reducing computational costs while maintaining discriminative ability. Finally, to alleviate resource bottlenecks and efficiency issues in the inference stage, this application explores the impact of sample complexity and network depth on detection accuracy and proposes an early-retreat mechanism based on Nash equilibrium theory in game theory. This adaptively optimizes the inference path using game theory, significantly reducing the computational resources and time overhead required for inference while ensuring the reasonableness of predictions.

[0109] Figure 6 A schematic diagram of the structure of an image analysis device based on a spiking neural network provided in an embodiment of the present invention is shown. Figure 6 As shown, the device 200 includes:

[0110] Acquisition module 210 is used to acquire the image to be analyzed;

[0111] The prediction module 220 is used to input the image to be analyzed into a spiking neural network with inhibitory neurons; wherein, the spiking neural network with inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse feedback module, and an inference module; the inhibitory pulse feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons; the neurons of the spiking neural network transmit signals through integer pulses; the neurons include excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module, and the inhibitory neurons;

[0112] The output module 230 is used to output the analysis results of the image to be analyzed.

[0113] In one alternative approach, each of the inhibitory neurons in the inhibitory impulse self-feedback module is positioned between adjacent excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module.

[0114] The step of inputting the image to be analyzed into a spiking neural network with inhibitory neurons includes:

[0115] The input pulse signal corresponding to the image to be analyzed is input into the convolution-based SNN module for local feature extraction, and then input into the Transformer-based SNN module to output a globally correlated spatiotemporal feature pulse sequence.

[0116] During signal transmission, the inhibition neurons in the inhibition pulse self-feedback module inhibit the integer pulses output by each excitatory neuron of the convolution-based SNN module and the Transformer-based SNN module, respectively.

[0117] The inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis result of the image to be analyzed.

[0118] In one optional embodiment, during signal transmission, the inhibition neurons in the inhibition pulse self-feedback module respectively inhibit the integer pulses output by each excitatory neuron of the convolution-based SNN module and the Transformer-based SNN module, including: the target excitatory neuron generates the integer pulse after receiving the input pulse signal corresponding to the image to be analyzed at the current time; the target excitatory neuron is any one of a plurality of excitatory neurons;

[0119] The membrane potential of the adjacent excitatory neurons at the next time step is updated based on the membrane potential of the adjacent excitatory neurons at the current time step, the integer pulse output by the target excitatory neuron at the current time step, and the inhibition signal of the target inhibitory neuron; the adjacent excitatory neurons are the excitatory neurons adjacent to the target excitatory neuron; the target inhibitory neuron is the inhibitory neuron between the target neuron and the adjacent excitatory neurons.

[0120] In one alternative approach, after the target excitatory neuron receives the input signal corresponding to the image to be analyzed at the current moment, it generates the integer pulse, including:

[0121] The integer pulse output by the target excitatory neuron n at the current time t is represented as: in, ∈{0,1,2,...,D}, represents the integer pulse value of the target excitatory neuron n at the current time t; This represents the membrane potential of the target excitatory neuron n at time t; Let be the integer firing function of the target neuron n at the current time t; where, the integer firing function is defined. for: in, This means restricting V to the interval [0, D], where D ∈ N. + It is the maximum allowed pulse value. Represents the nearest integer; the membrane potential of the adjacent excitatory neuron at the next time t+1 is updated based on the membrane potential of the adjacent excitatory neuron at the current time, the output signal of the target excitatory neuron at the current time t, and the inhibition signal of the target inhibitory neuron, including: determining the membrane potential of the adjacent excitatory neuron j at time t+1 according to the following formula: in, Indicates target inhibitory neuron i For the inhibition signal of the adjacent excitatory neuron j, G(·) is the inhibition function, which uses a linear mapping to characterize the inhibition strength; , where is the positive connection weight of the adjacent neuron j; This is the amplification factor to suppress the signal; This represents the set of neighboring neurons that are connected to the adjacent neuron j. In one alternative approach, the inference module is based on a game-theoretic early exit mechanism; the inference module includes multiple nodes; each node includes a discriminant head and an exit point.

[0122] The inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis results of the image to be analyzed, including:

[0123] When the spatiotemporal feature pulse sequence is transmitted to the current node, the current utility corresponding to the current node is calculated; the current node is any one of the plurality of nodes; the current utility is calculated based on the first detection accuracy, the first cost saving, and the first information entropy of the image sample at the current exit point.

[0124] When the current utility is greater than the expected utility, exit at the current exit point and obtain the analysis result of the image to be analyzed;

[0125] When the current utility is less than or equal to the expected utility, continue to pass it to the next exit point;

[0126] The next exit point is taken as the current exit point, and the process of determining the magnitude of the current utility and the expected utility of the next exit point continues until the current utility is greater than the expected utility or the last node is reached, at which point the process exits and the analysis result of the image to be analyzed is obtained.

[0127] In one alternative embodiment, before inputting the image to be analyzed into a spiking neural network with inhibitory neurons, the apparatus further includes:

[0128] The sample module is used to acquire image samples;

[0129] The training module is used to input the image samples into the spiking neural network with inhibitory neurons for training, and to obtain sample analysis results;

[0130] The loss calculation module is used to calculate the predicted total loss based on the sample analysis results and image sample labels using a preset loss function.

[0131] The gradient calculation module is used to calculate the gradient propagation path of the total predicted loss with respect to all parameters in the spiking neural network during the backpropagation phase; wherein, for the inhibitory neuron, the gradient of the total predicted loss with respect to the parameters of the inhibitory neuron is obtained by using the chain rule.

[0132] The parameter update module is used to update all parameters of the spiking neural network according to the gradient propagation path of all parameters in the spiking neural network;

[0133] The iterative training module is used to input the image samples into the updated spiking neural network and continue iterative training with parameter updates until a trained spiking neural network with inhibitory neurons is obtained.

[0134] In one alternative approach, after inputting the Transformer-based SNN module, the output is a globally correlated spatiotemporal feature pulse sequence, including:

[0135] Perform a separable convolution operation on the target pulse signal to obtain the first pulse data;

[0136] Selective local pattern attention is applied to the first pulse data to obtain the second pulse data; the selective local pattern attention includes a pulse attention matrix and an SLPA module, the SLPA module being used to control the numerical range of the attention score;

[0137] The second pulse data is processed by a multilayer perceptron at the channel dimension to obtain the output predicted pulse sequence.

[0138] This invention provides an embodiment of the invention that acquires an image to be analyzed; inputs the image to be analyzed into a spiking neural network (SNN) with inhibitory neurons; wherein the SNN with inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse self-feedback module, and an inference module; the inhibitory pulse self-feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons; the neurons of the spiking neural network transmit signals through integer pulses; the neurons include excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module, and the inhibitory neurons; and outputs the analysis result of the image to be analyzed. In this embodiment, firstly, integer pulses are fired in the network, and inhibitory neurons are added. By firing inhibitory pulses, the membrane potential of neurons is dynamically regulated to optimize the firing accuracy of neurons while significantly reducing pulse density. Secondly, addressing the problem that self-attention mechanisms consume a high proportion of computational resources in resource-constrained environments, this application also proposes a lightweight Spike Lite-Pool Attention (SLPA) module. By fusing attention matrices from asynchronous long-pooling paths, it achieves efficient fusion of cross-scale features, effectively reducing computational costs while maintaining discriminative ability. Finally, to alleviate resource bottlenecks and efficiency issues during the inference phase, this application explores the impact of sample complexity and network depth on detection accuracy and proposes an early termination mechanism based on Nash equilibrium theory in game theory. This mechanism adaptively optimizes the inference path using game theory, significantly reducing the computational resources and time overhead required for inference while ensuring the reasonableness of the prediction.

[0139] Figure 7 The diagram shows a structural schematic of a computer device provided in an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computer device.

[0140] likeFigure 7 As shown, the computer device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0141] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other network elements such as clients or other servers. The processor 402 executes program 410, specifically performing the relevant steps described above in the embodiment of the image analysis method based on a spiking neural network.

[0142] Specifically, program 410 may include program code, which includes computer-executable instructions.

[0143] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0144] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0145] Specifically, program 410 can be called by processor 402 to cause the computer device to perform the following operations:

[0146] Acquire the image to be analyzed;

[0147] The image to be analyzed is input into a spiking neural network with inhibitory neurons; wherein the spiking neural network with inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse feedback module, and an inference module; the inhibitory pulse feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons; the neurons of the spiking neural network transmit signals through integer pulses; the neurons include excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module, and the inhibitory neurons;

[0148] The analysis results of the image to be analyzed are output. This invention provides a computer-readable storage medium storing at least one executable instruction. When this executable instruction is executed on a computer device, it causes the computer device to perform the image analysis method based on a spiking neural network as described in any of the above method embodiments.

[0149] Executable instructions can be used to cause computer devices to perform the following operations:

[0150] Acquire the image to be analyzed;

[0151] The image to be analyzed is input into a spiking neural network with inhibitory neurons; wherein the spiking neural network with inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse feedback module, and an inference module; the inhibitory pulse feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons; the neurons of the spiking neural network transmit signals through integer pulses; the neurons include excitatory neurons in the convolution-based SNN module and the Transformer-based SNN module, and the inhibitory neurons;

[0152] Output the analysis results of the image to be analyzed.

[0153] This invention provides an image analysis device based on a spiking neural network for performing the aforementioned image analysis method based on a spiking neural network.

[0154] This invention provides a computer program that can be called by a processor to cause a computer device to execute the image analysis method based on a spiking neural network in any of the above method embodiments.

[0155] This invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed on a computer, cause the computer to perform the image analysis method based on a spiking neural network as described in any of the above method embodiments.

[0156] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0157] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0158] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0159] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0160] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. An image analysis method based on a spiking neural network, characterized in that, The method includes: Acquire the image to be analyzed; The image to be analyzed is input into a spiking neural network with inhibitory neurons. This spiking neural network includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse feedback module, and an inference module. The inhibitory pulse feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons. Signal transmission between neurons in the spiking neural network occurs via integer pulses. Each neuron includes excitatory neurons in the convolution-based and Transformer-based SNN modules, as well as the inhibitory neurons. Each inhibitory neuron in the inhibitory pulse feedback module is positioned between adjacent excitatory neurons in the convolution-based and Transformer-based SNN modules. The inference module is based on a game-theoretic early termination mechanism. The inference module includes multiple nodes; each node includes a discriminator and an exit point. The step of inputting the image to be analyzed into a spiking neural network with inhibitory neurons includes: the input pulse signal corresponding to the image to be analyzed is input into the convolution-based SNN module for local feature extraction, and then input into the Transformer-based SNN module to output a globally correlated spatiotemporal feature pulse sequence; during signal transmission, the inhibitory neurons in the inhibitory pulse self-feedback module inhibit the integer pulses output by each excitatory neuron of the convolution-based SNN module and the Transformer-based SNN module respectively; the inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis result of the image to be analyzed. During signal transmission, the inhibition neurons in the inhibition pulse self-feedback module inhibit the integer pulses output by each excitatory neuron of the convolution-based SNN module and the Transformer-based SNN module, respectively. This includes: the target excitatory neuron generating the integer pulse after receiving the input pulse signal corresponding to the image to be analyzed at the current time; the target excitatory neuron is any one of multiple excitatory neurons; the membrane potential of adjacent excitatory neurons at the next time step is updated based on the membrane potential of the adjacent excitatory neurons at the current time step, the integer pulse output by the target excitatory neuron at the current time step, and the inhibition signal of the target inhibition neuron; the adjacent excitatory neurons are excitatory neurons adjacent to the target excitatory neuron; the target inhibition neuron is the inhibition neuron between the target excitatory neuron and the adjacent excitatory neurons. The inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis result of the image to be analyzed, including: when the spatiotemporal feature pulse sequence is passed to the current node, calculating the current utility corresponding to the current node; the current node is any node among the plurality of nodes; the current utility is calculated based on the first detection accuracy, the first cost saving, and the first information entropy of the image sample at the current exit point; determining the magnitude of the current utility and the expected utility at the next exit point; the expected utility is calculated based on the second detection accuracy, the second cost saving, and the second information entropy of the image sample at the next exit point; when the current utility is greater than the expected utility, exiting at the current exit point and obtaining the analysis result of the image to be analyzed; when the current utility is less than or equal to the expected utility, continuing to pass to the next exit point; taking the next exit point as the current exit point, continuing to execute the process of determining the magnitude of the current utility and the expected utility at the next exit point until the current utility is greater than the expected utility or the last node is reached, exiting and obtaining the analysis result of the image to be analyzed; and outputting the analysis result of the image to be analyzed.

2. The method according to claim 1, characterized in that, After receiving the input signal corresponding to the image to be analyzed at the current moment, the target excitatory neuron generates the integer pulse, including: The integer pulse output by the target excitatory neuron n at the current time t is represented as: ; in, , represents the integer pulse value of the target excitatory neuron n at the current time t; This represents the membrane potential of the target excitatory neuron n at time t; Let n be the integer firing function of the target neuron n at the current time t; Here, the integer distribution function is defined. for: ; in, This means restricting V to the interval Inside, It is the maximum allowed pulse value. Represents the nearest integer; The membrane potential of the adjacent excitatory neurons at the next time t+1 is updated based on the membrane potential of the adjacent excitatory neurons at the current time, the output signal of the target excitatory neuron at the current time t, and the inhibition signal of the target inhibitory neuron, including: The membrane potential of the adjacent excitatory neuron j at time t+1 is determined according to the following formula: ； in, Indicates target inhibitory neuron i The inhibitory signal on the adjacent excitatory neuron j, The suppression function is a linear mapping used to characterize the suppression strength. , where is the positive connection weight of the adjacent excited neuron j; This is the amplification factor to suppress the signal; This represents the set of neighboring neurons that have a connection with the adjacent excitatory neuron j.

3. The method according to any one of claims 1 or 2, characterized in that, Before inputting the image to be analyzed into a spiking neural network with inhibitory neurons, the method further includes: Obtain image samples; The image samples are input into the spiking neural network with inhibitory neurons for training to obtain sample analysis results; Based on the sample analysis results and image sample labels, a preset loss function is used to calculate the predicted total loss; During the backpropagation phase, the gradient propagation path of the predicted total loss with respect to all parameters in the spiking neural network is calculated; wherein, for the inhibitory neuron, the gradient of the predicted total loss with respect to the parameters of the inhibitory neuron is obtained by using the chain rule. Based on the gradient propagation path of all parameters in the spiking neural network, all parameters of the spiking neural network are updated; The image sample is then input into the updated spiking neural network to continue iterative training with parameter updates until a trained spiking neural network with inhibitory neurons is obtained.

4. The method according to claim 1, characterized in that, After inputting the Transformer-based SNN module, it outputs a globally correlated spatiotemporal feature pulse sequence, including: Perform a separable convolution operation on the target pulse signal to obtain the first pulse data; Selective local pattern attention is applied to the first pulse data to obtain the second pulse data; the selective local pattern attention includes a pulse attention matrix and an SLPA module, the SLPA module being used to control the numerical range of the attention score; The second pulse data is processed by a multilayer perceptron at the channel dimension to obtain the output predicted pulse sequence.

5. An image analysis device based on a spiking neural network, characterized in that, The device includes: The acquisition module is used to acquire the image to be analyzed. A prediction module is used to input the image to be analyzed into a spiking neural network with inhibitory neurons. The spiking neural network with inhibitory neurons includes a convolution-based SNN module, a Transformer-based SNN module, an inhibitory pulse feedback module, and an inference module. The inhibitory pulse feedback module includes at least one inhibitory neuron for dynamically firing inhibitory pulses to regulate the membrane potential state of surrounding excitatory neurons. Signal transmission between neurons in the spiking neural network occurs via integer pulses. Each neuron includes excitatory neurons in the convolution-based and Transformer-based SNN modules and the inhibitory neurons. Each inhibitory neuron in the inhibitory pulse feedback module is positioned between adjacent excitatory neurons in the convolution-based and Transformer-based SNN modules. The inference module is based on a game-theoretic early termination mechanism. The inference module includes multiple nodes; each node includes a discriminator and an exit point. The step of inputting the image to be analyzed into a spiking neural network with inhibitory neurons includes: the input pulse signal corresponding to the image to be analyzed is input into the convolution-based SNN module for local feature extraction, and then input into the Transformer-based SNN module to output a globally correlated spatiotemporal feature pulse sequence; during signal transmission, the inhibitory neurons in the inhibitory pulse self-feedback module inhibit the integer pulses output by each excitatory neuron of the convolution-based SNN module and the Transformer-based SNN module respectively; the inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis result of the image to be analyzed. During signal transmission, the inhibitory neurons in the inhibition pulse self-feedback module inhibit the integer pulses output by each excitatory neuron of the convolution-based SNN module and the Transformer-based SNN module, respectively. This includes: the target excitatory neuron generating the integer pulse after receiving the input pulse signal corresponding to the image to be analyzed at the current time; the target excitatory neuron is any one of a plurality of excitatory neurons; the membrane potential of adjacent excitatory neurons at the next time step is updated based on the membrane potential of the adjacent excitatory neurons at the current time step, the integer pulse output by the target excitatory neuron at the current time step, and the inhibition signal of the target inhibitory neuron; the adjacent excitatory neurons are excitatory neurons adjacent to the target excitatory neuron; the target inhibitory neuron is the inhibitory neuron between the inhibitory neuron and the adjacent excitatory neurons. The inference module receives the globally correlated spatiotemporal feature pulse sequence output by the Transformer-based SNN module, performs inference, and obtains the analysis result of the image to be analyzed, including: when the spatiotemporal feature pulse sequence is passed to the current node, calculating the current utility corresponding to the current node; the current node is any node among the plurality of nodes; the current utility is calculated based on the first detection accuracy, the first cost saving, and the first information entropy of the image sample at the current exit point; determining the magnitude of the current utility and the expected utility at the next exit point; the expected utility is calculated based on the second detection accuracy, the second cost saving, and the second information entropy of the image sample at the next exit point; when the current utility is greater than the expected utility, exiting at the current exit point and obtaining the analysis result of the image to be analyzed; when the current utility is less than or equal to the expected utility, continuing to pass to the next exit point; taking the next exit point as the current exit point, continuing to execute the process of determining the magnitude of the current utility and the expected utility at the next exit point, until the current utility is greater than the expected utility or the last node is reached, exiting and obtaining the analysis result of the image to be analyzed; The output module is used to output the analysis results of the image to be analyzed.

6. A computer device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation of the image analysis method based on a spiking neural network as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on a computer device, causes the computer device to perform the operation of the image analysis method based on a spiking neural network as described in any one of claims 1-4.

Images