Image processing method and device based on pulse neural network, equipment and medium

Abstract

The application discloses an image processing method and device based on a pulse neural network, equipment and a medium. The method comprises the following steps: encoding an initial image input into an initial pulse neural network into a binary pulse image; sequentially performing a minus 1 operation, an AND operation and an XOR operation based on the binary pulse image to obtain a logic operation row; detecting all the least significant bits in the binary pulse image according to the logic operation row, wherein the least significant bits comprise a position index; performing convolution calculation on the pulses in the preset convolution block range according to the position index of the least significant bits by each convolution unit in the PE array to obtain a convolution membrane potential; checking and analyzing the convolution membrane potential based on a potential threshold, and outputting a pulse feature map. The application can solve the technical problem that the existing image processing scheme based on the pulse neural network is time-consuming and labor-consuming, and the actual application efficiency is low.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural networks, and particularly relates to an image processing method and device based on a spiking neural network, equipment and a medium. BACKGROUND

[0002] A spiking neural network (SNN) is an artificial neural network that transmits and processes information based on pulses. In a biological neural system, neurons also transmit internal information through electrical pulses. Unlike a traditional artificial neural network (ANN), a spiking neuron can simulate a biological neuron and has an event-driven characteristic. When a neuron receives a stimulus / pulse, the neuron will be active, otherwise it will be in a dormant state. Although the accuracy of the SNN model cannot reach the effect achieved by the ANN model, the SNN model consumes much less computing resources and power than the ANN model. At present, the SNN has been widely studied and applied to various high-performance computing tasks.

[0003] The spiking neural network can be simulated by a software simulation tool. However, since the transmission and calculation of information in the spiking neural network are simulated by software, a lot of time is required, so the parallelism and efficiency of SNN information processing cannot be well reflected, resulting in low efficiency of image processing based on the spiking neural network. Even if the simulated algorithm is deployed on a hardware device, the running and reasoning efficiency is still low, which is difficult to meet the rapid image processing requirements of the current edge device. SUMMARY

[0004] The present application provides an image processing method and device based on a spiking neural network, equipment and a medium, which solves the technical problem that the existing image processing scheme based on the spiking neural network is time-consuming and labor-intensive, resulting in low efficiency in actual application.

[0005] Therefore, the present application provides an image processing method based on a spiking neural network, comprising:

[0006] encoding an initial image input into an initial spiking neural network into a binary pulse image;

[0007] sequentially performing a minus 1 operation, an AND operation and an XOR operation based on the binary pulse image to obtain a logic operation row;

[0008] detecting all the least significant pulses in the binary pulse image according to the logic operation row, the least significant pulses comprising a position index;

[0009] convolve the pulses within a preset convolution block range according to the position index of the lowest bit pulse through each convolution unit in the PE array to obtain a convolution membrane potential;

[0010] check and analyze the convolution membrane potential based on a potential threshold, and output a pulse feature map.

[0011] Preferably, the binary pulse image is sequentially subjected to minus 1, AND calculation and XOR calculation based on the binary pulse image to obtain a logic operation row, including:

[0012] The binary pulse image is decomposed into multiple binary rows to obtain a binary row;

[0013] After the binary row is subtracted by 1 and AND calculated with the binary row, a de-position binary value is obtained;

[0014] The de-position binary value is XOR calculated with the binary row to obtain a logic operation row.

[0015] Preferably, the convolution membrane potential is obtained by convolving the pulses within a preset convolution block range according to the position index of the lowest bit pulse through each convolution unit in the PE array, and further including:

[0016] The pulse response range of each convolution unit in the PE array is determined according to the core position and the core size to obtain a preset convolution block range.

[0017] Preferably, the convolution membrane potential is checked and analyzed based on a potential threshold, and a pulse feature map is output, including:

[0018] It is judged whether the convolution membrane potential exceeds the potential threshold, if yes, the convolution membrane potential is set to 0, and 1 is output at the same time;

[0019] If not, the membrane potential decay processing is performed on the convolution membrane potential, and 0 is output at the same time;

[0020] The pulse feature map is generated in combination with the output result, and the output result includes 1 and 0.

[0021] The second aspect of the present application provides an image processing device based on a pulse neural network, including:

[0022] An image encoding unit is configured to encode an initial image input into an initial pulse neural network into a binary pulse image;

[0023] A sparse calculation unit is configured to sequentially perform minus 1, AND calculation and XOR calculation based on the binary pulse image to obtain a logic operation row;

[0024] a sparse detection unit configured to detect all the least significant pulses in the binary pulse image according to the logical operation row, the least significant pulses including a position index;

[0025] a convolution calculation unit configured to perform convolution calculation on the pulses within a preset convolution block range according to the position index of the least significant pulse by each convolution unit in the PE array, to obtain a convolution membrane potential;

[0026] an analysis output unit configured to perform inspection analysis on the convolution membrane potential based on a potential threshold, and output a pulse feature map.

[0027] Preferably, the sparse calculation unit is specifically configured to:

[0028] divide the binary pulse image into a plurality of binary rows to obtain a binary row;

[0029] perform AND calculation on the binary row after subtracting 1 from the binary row and the binary row to obtain a de-position binary value;

[0030] perform XOR calculation on the de-position binary value and the binary row to obtain a logical operation row.

[0031] Preferably, the device further comprises:

[0032] a range determination unit configured to determine a pulse response range of each convolution unit in the PE array according to a kernel position and a kernel size, to obtain a preset convolution block range.

[0033] Preferably, the analysis output unit is specifically configured to:

[0034] determine whether the convolution membrane potential exceeds a potential threshold, if yes, set the convolution membrane potential to 0, and simultaneously output 1;

[0035] if not, perform membrane potential decay processing on the convolution membrane potential, and simultaneously output 0;

[0036] combine the output results to generate a pulse feature map, the output results including 1 and 0.

[0037] The third aspect of the present application provides an image processing device based on a spiking neural network, the device comprising a processor and a memory;

[0038] the memory is configured to store program code and transmit the program code to the processor;

[0039] the processor is configured to execute the image processing method based on the spiking neural network according to the instructions in the program code.

[0040] The fourth aspect of the present application provides a computer readable storage medium for storing program code for executing the pulse neural network-based image processing method of the first aspect.

[0041] From the above technical solutions, the embodiments of the present application have the following advantages:

[0042] In the present application, a pulse neural network-based image processing method is provided, which includes: encoding an initial image input into an initial pulse neural network into a binary pulse image; sequentially performing subtraction 1, AND calculation and XOR calculation based on the binary pulse image to obtain a logic operation row; detecting all the least significant pulses in the binary pulse image according to the logic operation row, the least significant pulses including a position index; performing convolution calculation on the pulses in a preset convolution block range according to the position index of the least significant pulse by each convolution unit in the PE array to obtain a convolution membrane potential; and checking and analyzing the convolution membrane potential based on a potential threshold, and outputting a pulse feature map.

[0043] In the pulse neural network-based image processing method provided by the present application, based on the sparse characteristics of pulse convolution, a binary encoding calculation strategy is used to detect the pulse sparse characteristics of each input pulse image, and the least significant pulse that needs to be focused on is marked with a position index; then, convolution calculation is strictly performed according to the position index and the preset convolution block range in the PE array calculation stage, without the need for additional judgment and time-consuming operations, so that the PE calculation is more targeted; and the preset convolution block range can also quickly filter out the pulses in the range for calculation. The entire process can improve the processing efficiency of the image input into the pulse neural network from different angles. Therefore, the present application can solve the technical problem of time-consuming and labor-consuming in the existing pulse neural network-based image processing scheme, resulting in low actual application efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 A flowchart of the pulse neural network-based image processing method provided by the embodiments of the present application is shown;

[0045] Figure 2 A structural diagram of the pulse neural network-based image processing device provided by the embodiments of the present application is shown;

[0046] Figure 3 A whole process example diagram of the pulse neural network-based image processing provided by the embodiments of the present application is shown;

[0047] Figure 4 A hardware platform deployment example diagram of the pulse neural network-based image processing method provided by the application examples of the present application is shown. DETAILED DESCRIPTION

[0048] In order for those skilled in the art to better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0049] For the convenience of understanding, please refer to Figure 1 The embodiments of the image processing method based on the pulse neural network provided by the present application comprise:

[0050] Step 101, encode an initial image input into an initial pulse neural network as a binary pulse image.

[0051] It should be noted that in the SNN, the transmission signal between layers is a binary (0, 1) signal, and at the same time, the pulse neuron does not produce a pulse at every time unit. Therefore, the data processed by the pulse neural network contains a large amount of invalid data, i.e. 0, making the sparsity of internal data very high. According to the sparsity characteristics of the pulse signal, a sparse detection mechanism is designed in the present embodiment to detect the pulse signal from the pulse input image, which can output an event address of an effective pulse in a single cycle and transmit it to the processing unit, so that each calculation of the calculation unit is necessary and targeted, so that the calculation of the network model is more efficient.

[0052] The initial image is a data matrix, and after being encoded into a binary pulse image, it becomes a binary data matrix with values of 0 or 1, which is convenient for subsequent binary calculation and sparse detection analysis. The initial image can be an image required for detection and analysis of an actual scene.

[0053] Step 102, sequentially performing minus 1, AND calculation and XOR calculation based on the binary pulse image to obtain a logic operation row.

[0054] Further, step 102 comprises:

[0055] decomposing the binary pulse image into a plurality of binary rows to obtain a binary row;

[0056] performing AND calculation between the binary row and the binary row after subtracting 1 to obtain a de-position binary value;

[0057] performing XOR calculation between the de-position binary value and the binary row to obtain a logic operation row.

[0058] Step 103, detecting all the least significant pulses in the binary pulse image according to the logic operation row, the least significant pulses including a position index.

[0059] It should be noted that if the size of the binary pulse image is 5x5, 5 rows of data can be obtained by decomposition, and the binary data of each row can be stored in a 5-bit register, which can be regarded as a 5-bit binary row x, assuming 10100. Then, the binary row x is subtracted by 1, so that the lowest bit 1 of x will be shifted to become 0, and the next lowest bit 0 will become 1, so that the binary number y, i.e. 10011, can be obtained. Then, the binary number y is calculated with the binary row x, and the shifted binary value z, i.e. 10000, can be obtained. The number higher than the shifted 1 will not change, and the others will become 0. Finally, the shifted binary value z is calculated with the binary row x by XOR operation, and a lowest bit pulse of the row can be detected.

[0060] It can be understood that since one detection operation can only find a lowest bit pulse of the current row, each binary row obtained by decomposition needs to repeat the above logical calculation multiple times to detect all 1s, i.e. lowest bit pulses, in the binary row x, and obtain all lowest bit pulses, i.e. 1s, in all binary rows. It should be emphasized that the next detection calculation and analysis are repeated on the logical operation row obtained after the XOR calculation, not the original binary row x. These 1s have specific positions in the binary row, so the lowest bit pulses can all generate a corresponding position index, which can be used to determine the position of the unit pulse in the subsequent calculation process and improve the calculation efficiency.

[0061] Step 104, the pulse in the preset convolution block range is calculated by each convolution unit in the PE array according to the position index of the lowest bit pulse, and the convolution membrane potential is obtained.

[0062] Further, step 104 further includes:

[0063] The pulse response range of each convolution unit in the PE array is determined according to the core position and the core size, and the preset convolution block range is obtained.

[0064] It should be noted that since each PE simulates a pulse neuron, it has a determined coordinate information relative to a convolution calculation output channel, which can be called ID or core position. Moreover, each convolution kernel has a size, i.e. core size. Therefore, the present embodiment can determine the pulse response range of each convolution unit, i.e. the preset convolution block range, according to the core position and the core size before the pulse response event occurs.

[0065] For the SCNN network, the range is the size of a convolution kernel, referred to as a block. When the input pulse image is subjected to sparse detection, an output of pulse position indexes is obtained, and the PE response unit determines, according to the block range obtained above, which pulse coordinates are within the block range. The pulses within the range are cached and input to the processing unit for convolution calculation. The convolution calculation is performed on the pulses within the preset convolution block range according to the position index of the lowest bit pulse to obtain the convolution membrane potential. Each convolution unit can quickly locate the pulses that can be convolved, and each unit pulse can be quickly matched to the corresponding convolution unit, thereby improving the processing speed of the network.

[0066] It should be noted that the PE array in the embodiment can perform not only convolution calculation but also checking and analyzing of the convolution membrane potential, pooling operation, and the like, and is not limited to only convolution calculation. The convolution calculation is to multiply all the pulses within the convolution kernel range by the corresponding weight value and accumulate the membrane potential. Since the input in the pulse neural network is 0 or 1, the multiplication of 1 and the weight is still the weight itself. The pulses obtained by the PE through sparse response only need to accumulate the weight value corresponding to the position coordinate index of the pulse to the membrane potential, that is, to perform addition calculation. The pooling operation is implemented based on an or gate calculation, and details are not described herein.

[0067] In step 105, the convolution membrane potential is checked and analyzed based on a potential threshold, and a pulse feature map is output.

[0068] Further, in step 105, the following steps are included.

[0069] It is determined whether the convolution membrane potential exceeds the potential threshold. If yes, the convolution membrane potential is set to 0, and 1 is output.

[0070] If no, the convolution membrane potential is subjected to membrane potential decay processing, and 0 is output.

[0071] The pulse feature map is generated in combination with the output result, and the output result includes 1 and 0.

[0072] It should be noted that the checking and analyzing of the convolution membrane potential is still completed in the PE array, and the potential threshold can be set according to actual conditions and is not limited herein. The checking and analyzing mainly determines whether the convolution membrane potential exceeds the potential threshold. If yes, the convolution membrane potential is set to 0, and the output result is 1, which means that a pulse is fired by the neuron. If no, the membrane potential decay processing is performed, that is, the binary convolution membrane potential stored in the register is right shifted by one bit. The output result is 0, that is, no pulse is generated.

[0073] Please refer to Figure 3The embodiment as a whole is to firstly encode and convert the input image, then detect the pulse, and determine the position coordinates of the pulse event; then set the response pulse input range of the neuron according to the position and size of the neuron; finally, judge whether the detected pulse in all input images is in the response range of the neuron, if so, the pulse will be input into the neuron to start convolution calculation. This calculation data flow can complete the standard convolution.

[0074] In order to make the PE array support different calculation modes, the data flow design needs to be modified and extended. Compared with the calculation process of standard convolution, the calculation of transposed convolution is more complex. First, the input image needs to be expanded, that is, 0 is filled between the rows and columns of pixels, and the expanded image is padded. Then, the expanded image is subjected to standard convolution to obtain the output image, and the output image is padded to obtain the final transposed convolution calculation result.

[0075] That is, the convolution in the PE array in the embodiment can be adaptively selected according to actual conditions, such as standard convolution, transposed convolution, residual convolution and dilated convolution. Taking transposed convolution as an example, since the row and column filling between pixels is needed, the sparse response mechanism needs to be modified compared with standard convolution. After judging that several pulses are input in the preset convolution block range of the PE response unit, the row and column coordinates of the pulse coordinate index in the response range are multiplied by 2, so as to achieve the effect of row and column filling between pixels. The subsequent response process is consistent with that of standard convolution. When selecting other convolutions, the actual calculation process may also need to be slightly adjusted according to the characteristics of each convolution. Here, it will not be described in detail, which is the adjustment that can be expected in combination with the prior art and does not affect the execution of the embodiment.

[0076] Please refer to Figure 4If the image processing method based on the pulse neural network provided in the application is deployed on a hardware architecture platform, the calculation unit includes a processing unit PE array responsible for completing pulse convolution and pooling calculation, a Controller, a PEController, and a sparse detection module. At the same time, multiple Buffer or Memory are designed to complete the buffering and storage of calculation data. The Controller is responsible for judging the input channel level and the control of calculation unit multiplexing, and the PEController is responsible for the calculation control of the output channel level. The Controller state machine is responsible for the control of the input channel and the control of the calculation unit multiplexing. When the Controller receives a calculation start signal, the state will be transferred from idle to acquire the current input channel data state, in which the pulse event detection mechanism is started to record all valid pulse events of the input. When the data detection of the corresponding channel is completed, it will jump to the PE configuration state. Each PE can simulate a pulse neuron of an output layer, that is, it has a specific position information. The PE configuration is to set the specific position of the neuron simulated by the current PE, and generate its pulse event response range according to the position. When the ID configuration of each PE is completed, the data calculation of the corresponding image is started, and the specific calculation process is controlled by the PE Controller.

[0077] It should be noted that generally speaking, the calculation of an image cannot be completed at one time, so the calculation of the image needs to be completed by multiplexing the calculation unit. For example, the output image size is 8x8, but the calculation unit is only 2x2, so the calculation unit needs to be multiplexed 16 times to complete the calculation. Therefore, after receiving the calculation completion signal, it is judged whether the multiplexing is completed. If the calculation unit still needs to be multiplexed, it will jump back to the PE configuration state to complete the configuration of other blocks to complete the calculation, until all multiplexing is completed. Finally, it is judged whether all input channels have completed the calculation. The completion of all input channels represents that the calculation of the current layer has been completed.

[0078] The PE Controller is responsible for completing the calculation of all output channels of a single block image. When the PE Controller receives a start signal, it will acquire the weight of the corresponding input channel and output channel. When the weight acquisition is completed, the membrane potential update calculation of a single output channel is started. After the calculation is completed, it is judged whether all output channels have completed the calculation. As can be seen from here, the calculation data flow of a single convolution calculation layer is also single instruction stream multi-data stream. According to the data of a single input channel, the update calculation of the membrane potential of multiple output channel pulse neurons is completed. If the update of all output channels is not completed, the weight data of the corresponding input channel to the output channel is updated, and the calculation is restarted until all output channels are calculated.

[0079] PE ARRAY is a PE array, each PE can simulate a neuron. PE ARRAY is controlled by PEController, after receiving image data, the input image is distributed to each PE, and the LIF neuron accumulation, leakage and pulse firing calculation are completed.

[0080] Spike Buffer is used to store the pulses fired by neurons, input images. After PE_ARRAY calculation is completed, some neurons simulated by PE will fire pulses, these pulses become stored in Spike buffer, used as the input image of the next layer of network. If it is just starting to calculate, Spike Buffer will read the input image from DDR.

[0081] Weight Buffer is used to store weights. When PE_ARRAY calculation is completed, Weight Buffer will input the corresponding weights into PE according to the position of the neuron simulated by the current PE to complete the convolution calculation.

[0082] Neuron State Buffer is used to store the membrane potential of neurons. When PE_ARRAY starts to calculate, the value of the membrane potential is read from Neuron State Buffer, and the updated membrane potential is written back to Neuron State Buffer after calculation is completed.

[0083] In the image processing method based on the pulse neural network provided by the embodiment of the application, based on the sparse characteristics of the pulse convolution, a binary coding calculation strategy is used to detect the pulse sparse characteristics of each input pulse image, and the least significant pulse that needs to be focused on is marked with a position index. Then, in the PE array calculation stage, the convolution calculation is strictly performed according to the position index and the preset convolution block range, without additional judgment and time-consuming operation, so that the PE calculation is more targeted, and the preset convolution block range can also quickly select the pulses in the range for calculation. The whole process can improve the processing efficiency of the image input pulse neural network from different angles. Therefore, the embodiment of the application can solve the technical problem that the existing image processing scheme based on the pulse neural network is time-consuming and labor-consuming, and leads to low actual application efficiency.

[0084] For ease of understanding, please refer to Figure 2 The embodiment of the application provides an image processing device based on a pulse neural network, which comprises:

[0085] The image encoding unit 201 is configured to encode an initial image input into an initial pulse neural network into a binary pulse image.

[0086] The sparse calculation unit 202 is configured to sequentially perform subtraction 1, AND calculation and XOR calculation based on the binary pulse image to obtain a logic operation row;

[0087] The sparse detection unit 203 is configured to detect all the least significant pulses in the binary pulse image according to the logic operation row, wherein the least significant pulse comprises a position index;

[0088] The convolution calculation unit 204 is configured to perform convolution calculation on the pulses in the preset convolution block range according to the position index of the least significant pulse by each convolution unit in the PE array to obtain a convolution membrane potential;

[0089] The analysis output unit 205 is configured to perform checking analysis on the convolution membrane potential based on a potential threshold and output a pulse feature map.

[0090] Further, the sparse calculation unit 202 is specifically configured to:

[0091] divide the binary pulse image into a plurality of binary rows to obtain a binary row;

[0092] perform AND calculation on the binary row after subtracting 1 from the binary row to obtain a de-position binary value;

[0093] perform XOR calculation on the de-position binary value and the binary row to obtain a logic operation row.

[0094] Further, the device further comprises:

[0095] The range determination unit 206 is configured to determine the pulse response range of each convolution unit in the PE array according to the core position and the core size to obtain a preset convolution block range.

[0096] Further, the analysis output unit 205 is specifically configured to:

[0097] determine whether the convolution membrane potential exceeds the potential threshold, if yes, set the convolution membrane potential to 0 and simultaneously output 1;

[0098] if not, perform membrane potential decay processing on the convolution membrane potential and simultaneously output 0;

[0099] combine the output results to generate the pulse feature map, wherein the output results comprise 1 and 0.

[0100] The application further provides an image processing device based on a pulse neural network, which comprises a processor and a memory;

[0101] The memory is configured to store program code and transmit the program code to the processor;

[0102] The processor is configured to execute the image processing method based on the pulse neural network in the above method embodiments according to instructions in the program code.

[0103] The application further provides a computer-readable storage medium for storing program code for executing the pulse neural network-based image processing method in the method embodiments.

[0104] In several embodiments provided in the application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described apparatus embodiments are merely schematic. The division of the units is merely a logical function division. There can be another division manner for actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0105] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0106] In addition, each functional unit in the various embodiments of the application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0107] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for executing all or part of the steps of the method described in the various embodiments of the application by a computer device (which can be a personal computer, a server, or a network device, etc.). The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), a random access memory (English full name: Random Access Memory, English abbreviation: RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0108] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of image processing based on a spiking neural network, characterized in that, The method comprises the following steps: encoding an initial image input into an initial pulse neural network into a binary pulse image; performing subtraction 1, AND calculation and XOR calculation based on the binary pulse image to obtain a logic operation row, specifically comprising the following steps: decomposing the binary pulse image into multiple rows of binary data to obtain a binary row; performing AND calculation between the binary row and the binary row after subtracting 1 from the binary row to obtain a de-position binary value; performing XOR calculation between the de-position binary value and the binary row to obtain a logic operation row; detecting all the least significant pulses in the binary pulse image according to the logic operation row, wherein the least significant pulses comprise a position index; determining a pulse response range of each convolution unit in a PE array according to a kernel position and a kernel size to obtain a preset convolution block range; performing convolution calculation on the pulse in the preset convolution block range by each convolution unit in the PE array according to the position index of the least significant pulse to obtain a convolution membrane potential; performing checking analysis on the convolution membrane potential based on a potential threshold and outputting a pulse feature map, specifically comprising the following steps: determining whether the convolution membrane potential exceeds the potential threshold, if yes, setting the convolution membrane potential to 0 and simultaneously outputting 1; if not, performing membrane potential decay processing on the convolution membrane potential and simultaneously outputting 0; generating a pulse feature map in combination with the output result, wherein the output result comprises 1 and 0.

2. The image processing apparatus based on the spiking neural network, characterized by, The method comprises the following steps: an image encoding unit is configured to encode an initial image input into an initial pulse neural network into a binary pulse image; a sparse calculation unit is configured to perform subtraction 1, AND calculation and XOR calculation based on the binary pulse image to obtain a logic operation row, and the sparse calculation unit is specifically configured to: decompose the binary pulse image into multiple rows of binary data to obtain a binary row; perform AND calculation between the binary row and the binary row after subtracting 1 from the binary row to obtain a de-position binary value; perform XOR calculation between the de-position binary value and the binary row to obtain a logic operation row; a sparse detection unit is configured to detect all the least significant pulses in the binary pulse image according to the logic operation row, wherein the least significant pulses comprise a position index; a range determination unit is configured to determine a pulse response range of each convolution unit in a PE array according to a kernel position and a kernel size to obtain a preset convolution block range; a convolution calculation unit is configured to perform convolution calculation on the pulse in the preset convolution block range by each convolution unit in the PE array according to the position index of the least significant pulse to obtain a convolution membrane potential; an analysis and output unit is configured to perform checking analysis on the convolution membrane potential based on a potential threshold and output a pulse feature map, and the analysis and output unit is specifically configured to: determine whether the convolution membrane potential exceeds the potential threshold, if yes, set the convolution membrane potential to 0 and simultaneously output 1; if not, perform membrane potential decay processing on the convolution membrane potential and simultaneously output 0; generate a pulse feature map in combination with the output result, wherein the output result comprises 1 and 0.

3. An image processing device based on a spiking neural network, characterized by The device comprises a processor and a memory; the memory is configured to store program code and transmit the program code to the processor; The processor is configured to execute the pulse neural network-based image processing method according to the instructions in the program code.

4. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store program code for executing the pulse neural network-based image processing method according to claim 1.

Images