Image processing method and device based on spiking neural network, equipment and medium

By encoding the initial image into a binary pulse image and performing specific calculations, detecting the lowest pulse position index, and combining the convolutional calculation of the PE array, the problem of low image processing efficiency of pulse neural networks in the prior art is solved, and more efficient Image processing.

CN118968089AActive Publication Date: 2024-11-15GUANGDONG UNIV OF TECH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411210381.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-30
Publication Date
2024-11-15
Estimated Expiration
2044-08-30

AI Technical Summary

Technical Problem

The existing image processing scheme based on pulsed neural networks is inefficient, resulting in low practical application efficiency.

Method used

By encoding the input initial image into a binary pulse image, performing 1 and exclusive OR calculation, detecting the lowest-bit pulse position index, and performing convolutional calculations in the PE array, the output pulse feature map is analyzed based on the potential threshold.

Benefits of technology

The efficiency of image processing is improved, and through sparse characteristics and binary encoding calculation strategies, unnecessary judgment and time-consuming operations are reduced, and processing speed is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968089A_ABST
    Figure CN118968089A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method and device based on a pulse neural network, equipment and a medium. The method comprises the following steps: encoding an initial image input into an initial pulse neural network into a binary pulse image; subtracting 1, AND calculation and XOR calculation in sequence based on the binary pulse image to obtain a logic operation line; detecting all lowest bit pulses in the secondary pulse image according to the logic operation line, wherein the lowest bit pulses comprise position indexes; performing convolution calculation on pulses in a preset convolution block range through each convolution unit in the PE array according to the position index of the lowest bit pulse to obtain a convolution film potential; and checking and analyzing the potential of the convolutional film based on the potential threshold, and outputting a pulse characteristic pattern. The technical problem that the actual application efficiency is low due to the fact that an existing pulse neural network-based image processing scheme wastes time and labor can be solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of neural network technology, and in particular to an image processing method, device, equipment and medium based on a pulse neural network. Background Art

[0002] Spiking Neural Networks (SNNs) are artificial neural networks that process information based on pulses. In biological neural systems, each neuron also transmits internal information through electrical pulses. Unlike traditional artificial neural networks (ANNs), spiking neurons can imitate biological neurons and have event-driven characteristics. When a neuron receives a stimulus / pulse, it will become active, otherwise it will remain dormant. Although the SNN model cannot achieve the same accuracy as the ANN model, its computing resource consumption and power consumption are much smaller than those of the ANN. At present, SNN has been widely studied and applied to various high-performance computing tasks.

[0003] Pulsing neural networks can be simulated based on software simulation tools. However, since software is used to simulate the transmission and computation of information in spiking neural networks, it takes a lot of time, so it cannot well reflect the parallelism and efficiency of SNN information processing, resulting in low efficiency of image processing based on spiking neural networks. Even if the simulated algorithm is deployed on hardware devices, its operation and reasoning efficiency is very low, and it is difficult to adapt to the current fast image processing requirements of edge devices. Summary of the invention

[0004] The present application provides a pulse neural network-based image processing method, apparatus, device and medium, which are used to solve the technical problem that the existing pulse neural network-based image processing solutions are time-consuming and labor-intensive, resulting in low efficiency in actual application.

[0005] In view of this, the first aspect of the present application provides an image processing method based on a pulse neural network, comprising:

[0006] Encoding an initial image input into an initial spike neural network into a binary spike image;

[0007] Subtract 1, perform AND calculation and XOR calculation in sequence based on the binary pulse image to obtain a logic operation row;

[0008] Detecting all the least significant pulses in the binary pulse image according to the logic operation row, wherein the least significant pulse includes a position index;

[0009] By each convolution unit in the PE array performing convolution calculation on the pulse within the preset convolution block range according to the position index of the least significant pulse, a convolution membrane potential is obtained;

[0010] The convolution membrane potential is checked and analyzed based on the potential threshold, and a pulse characteristic diagram is output.

[0011] Preferably, the step of performing subtraction, AND calculation and XOR calculation in sequence based on the binary pulse image to obtain a logic operation row includes:

[0012] Decomposing the binary pulse image into a plurality of lines of binary data to obtain binary lines;

[0013] Subtract 1 from the binary row and perform AND calculation with the binary row to obtain a binary value;

[0014] The carry binary value is XORed with the binary row to obtain a logic operation row.

[0015] Preferably, the convolution calculation is performed on the pulses within the preset convolution block range according to the position index of the least significant pulse by each convolution unit in the PE array to obtain the convolution membrane potential, and the method further includes:

[0016] The impulse response range of each convolution unit in the PE array is determined according to the kernel position and kernel size to obtain the preset convolution block range.

[0017] Preferably, the checking and analyzing the convolution membrane potential based on the potential threshold and outputting a pulse characteristic diagram includes:

[0018] Determine whether the convolution membrane potential exceeds the potential threshold, if so, set the convolution membrane potential to 0 and output 1;

[0019] If not, the convolution membrane potential is subjected to membrane potential attenuation processing and 0 is outputted;

[0020] A pulse characteristic diagram is generated in combination with the output results, wherein the output results include 1 and 0.

[0021] The second aspect of the present application provides an image processing device based on a pulse neural network, comprising:

[0022] An image encoding unit, used for encoding an initial image input into an initial spike neural network into a binary spike image;

[0023] A sparse calculation unit, used for performing subtraction, AND calculation and XOR calculation in sequence based on the binary pulse image to obtain a logic operation row;

[0024] A sparse detection unit, used for detecting all the least significant pulses in the binary pulse image according to the logic operation row, wherein the least significant pulse includes a position index;

[0025] A convolution calculation unit, configured to perform convolution calculation on the pulses within the preset convolution block range according to the position index of the least significant pulse through each convolution unit in the PE array, so as to obtain a convolution membrane potential;

[0026] An analysis output unit is used to check and analyze the convolution membrane potential based on a potential threshold and output a pulse characteristic diagram.

[0027] Preferably, the sparse computing unit is specifically used for:

[0028] Decomposing the binary pulse image into a plurality of lines of binary data to obtain binary lines;

[0029] Subtract 1 from the binary row and perform AND calculation with the binary row to obtain a binary value;

[0030] The carry binary value is XORed with the binary row to obtain a logic operation row.

[0031] Preferably, it also includes:

[0032] The range determination unit is used to determine the impulse response range of each convolution unit in the PE array according to the kernel position and the kernel size, so as to obtain the preset convolution block range.

[0033] Preferably, the analysis output unit is specifically used for:

[0034] Determine whether the convolution membrane potential exceeds the potential threshold, if so, set the convolution membrane potential to 0 and output 1;

[0035] If not, the convolution membrane potential is subjected to membrane potential attenuation processing and 0 is outputted;

[0036] A pulse characteristic diagram is generated in combination with the output results, wherein the output results include 1 and 0.

[0037] A third aspect of the present application provides an image processing device based on a pulse neural network, the device comprising a processor and a memory;

[0038] The memory is used to store program code and transmit the program code to the processor;

[0039] The processor is used to execute the image processing method based on pulse neural network described in the first aspect according to the instructions in the program code.

[0040] The fourth aspect of the present application provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the image processing method based on pulse neural network described in the first aspect.

[0041] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0042] In the present application, an image processing method based on a pulse neural network is provided, including: encoding an initial image input into an initial pulse neural network into a binary pulse image; performing subtraction, AND calculation, and XOR calculation in sequence based on the binary pulse image to obtain a logic operation row; detecting all the lowest-order pulses in the binary pulse image according to the logic operation row, and the lowest-order pulse includes a position index; performing convolution calculation on the pulses within the preset convolution block range according to the position index of the lowest-order pulse by each convolution unit in the PE array to obtain a convolution membrane potential; checking and analyzing the convolution membrane potential based on a potential threshold, and outputting a pulse characteristic diagram.

[0043] In the image processing method based on pulse neural network provided by the present application, based on the sparse characteristics of pulse convolution, a binary coding calculation strategy is adopted to detect the pulse sparse characteristics of each input pulse image, and the lowest-order pulse that needs attention is marked with a position index; then, in the PE array calculation stage, the convolution calculation is strictly performed according to the position index and the preset convolution block range, without the need for additional judgment and time-consuming operations, making the PE calculation more targeted; and the preset convolution block range can also quickly filter out the pulses within the range for calculation. The whole process can improve the processing efficiency of the image after inputting the pulse neural network from different angles. Therefore, the present application can solve the technical problem that the existing image processing scheme based on pulse neural network is time-consuming and labor-intensive, resulting in low efficiency in actual application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of a flow chart of an image processing method based on a spiking neural network provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of an image processing device based on a spiking neural network provided in an embodiment of the present application;

[0046] Figure 3 An example diagram of the overall process of image processing based on a spiking neural network provided in an embodiment of the present application;

[0047] Figure 4 An example diagram of the hardware platform deployment of the pulse neural network-based image processing method provided in the application example of this application. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] For easier understanding, see Figure 1 , an embodiment of the image processing method based on the pulse neural network provided by the present application includes:

[0050] Step 101: Encode the initial image input into the initial spike neural network into a binary spike image.

[0051] It should be noted that in SNN, the transmission signal between layers is a binary (0, 1) signal, and the pulse neuron does not generate a pulse in every time unit; therefore, the data processed by the pulse neural network contains a large amount of invalid data, that is, 0, which makes the sparsity of the internal data very high. According to the sparsity characteristics of the pulse signal, this embodiment designs a sparse detection mechanism to detect the pulse signal from the pulse input image, which can output an event address of a valid pulse in a single cycle and transmit it to the processing unit, so that each calculation of the calculation unit is necessary and targeted, so that the calculation of the network model is more efficient.

[0052] The initial image is a data matrix. After being encoded into a binary pulse image, it becomes a binary data matrix with values ​​of 0 or 1, which is convenient for subsequent binary calculations and sparse detection analysis. The initial image can be the image required for detection and analysis in the actual scene.

[0053] Step 102: Subtract 1, perform AND calculation and XOR calculation in sequence based on the binary pulse image to obtain a logic operation row.

[0054] Further, step 102 includes:

[0055] Decomposing the binary pulse image into multiple lines of binary data to obtain binary lines;

[0056] Subtract 1 from the binary row and perform AND calculation with the binary row to get the binary value of the decimal place.

[0057] The carry binary value is XORed with the binary row to obtain a logical operation row.

[0058] Step 103: Detect all the least significant pulses in the binary pulse image according to the logic operation, wherein the least significant pulse includes a position index.

[0059] It should be noted that if the size of the binary pulse image is 5×5, it can be decomposed into 5 rows of data. Each row of binary data can be stored in a 5-bit register and can be regarded as a 5-bit binary row x, assuming it is 10100; then, subtract 1 from the binary row x, so that the lowest bit 1 of x will be replaced and become 0, and the 0 at the next lower position will become 1, so that the binary number y can be obtained, that is, 10011; then, the binary number y is ANDed with the binary row x to obtain the replaced binary value z, that is, 10000, and the numbers higher than the replaced 1 will not change, and the others will become 0; finally, the replaced binary value z is XORed with the binary row x to detect a lowest-bit pulse of the row.

[0060] It is understandable that since one detection operation can only find a lowest-order pulse in the current row, each binary row obtained by decomposition needs to repeat the above logic calculation multiple times to detect all 1s in binary row x, that is, the lowest-order pulse, and obtain all the lowest-order pulses in all binary rows, that is, 1. It should be emphasized that the next detection calculation analysis is repeated on the calculation of the logic operation row obtained after the XOR calculation, not the original binary row x. These 1s have specific positions in the binary row, so the lowest-order pulse can generate a corresponding position index, which is used to clarify the position of the unit pulse in the subsequent calculation process and improve the calculation efficiency.

[0061] Step 104: Perform convolution calculation on the pulses within the preset convolution block range according to the position index of the lowest-order pulse by each convolution unit in the PE array to obtain the convolution membrane potential.

[0062] Furthermore, step 104, before that, also includes:

[0063] The impulse response range of each convolution unit in the PE array is determined according to the kernel position and kernel size to obtain the preset convolution block range.

[0064] It should be noted that, since each PE simulates a pulse neuron, it has a certain coordinate information relative to a convolution calculation output channel, which can be called ID or kernel position; and each convolution kernel of the convolution calculation has a size, namely, the kernel size. Therefore, this embodiment can determine the impulse response range of each convolution unit according to the kernel position and kernel size before the impulse response event occurs, namely, the preset convolution block range.

[0065] For the SCNN network, this range is the size of a convolution kernel, called a block. When the input pulse image passes through sparse detection and outputs pulse position indexes, the PE response unit will determine which pulses have coordinates within the block range based on the block range obtained above. The pulses within the range will be cached and input into the processing unit for convolution calculation. That is, the pulses within the current preset convolution block range are convolved according to the position index of the lowest-order pulse to obtain the convolution membrane potential. Each convolution unit can quickly locate the pulse that can be convolved by itself, and each unit pulse can also be quickly matched to the corresponding convolution unit, thereby improving the processing speed of the network.

[0066] It should be noted that, in addition to performing convolution calculations, the PE array in this embodiment can also perform inspection and analysis of the convolution membrane potential, as well as pooling operations, etc., not just convolution calculations. The convolution calculation is to multiply all pulse inputs within the convolution kernel by the corresponding weight value and add them to the membrane potential. Since the inputs in the pulse neural network are all 0 or 1, 1 multiplied by the weight is still the weight itself. The pulses obtained by PE through sparse response only need to add the weight values ​​corresponding to the position coordinate indexes of these pulses to the membrane potential, that is, perform addition calculations. The pooling operation is implemented based on the OR gate calculation, and the details are not repeated here.

[0067] Step 105: Check and analyze the convolution membrane potential based on the potential threshold, and output a pulse characteristic diagram.

[0068] Further, step 105 includes:

[0069] Determine whether the convolution membrane potential exceeds the potential threshold. If so, set the convolution membrane potential to 0 and output 1 at the same time;

[0070] If not, the convolution membrane potential is subjected to membrane potential attenuation processing and 0 is output;

[0071] The output results are combined to generate a pulse characteristic diagram, and the output results include 1 and 0.

[0072] It should be noted that the inspection and analysis of the convolution membrane potential is still completed in the PE array, and the potential threshold can be set according to the actual situation and is not limited here. The inspection and analysis is mainly to determine whether the convolution membrane potential exceeds the potential threshold. If it exceeds, the convolution membrane potential is reset to 0, and then the output result is 1, which means that the neuron has emitted a pulse. If it does not exceed the potential threshold, the membrane potential needs to be attenuated, specifically, the binary convolution membrane potential stored in the register is shifted right by one position; then the output result is 0, that is, no pulse is generated.

[0073] See also Figure 3In general, this embodiment converts the input image into code first, then detects the pulse, and determines the position coordinates of the pulse event; then sets the range of the response pulse input of the neuron according to the position and size of the neuron; finally, determines whether the pulses in all input images detected are within the neuron response range. If so, the pulses are input into the neuron to start the convolution calculation. This calculation data flow can complete the standard convolution.

[0074] In order to make the PE array support different computing modes, the data flow design needs to be modified and expanded. Compared with the calculation process of standard convolution, the calculation of transposed convolution is more complicated. First, the input image needs to be expanded, that is, 0 is filled between the rows and columns of pixels, and the output padding is performed on the expanded image. Then, the standard convolution is performed on the expanded image to obtain the output image, and the output image is padded to obtain the final transposed convolution calculation result.

[0075] That is to say, the convolution in the PE array in this embodiment can be adaptively selected according to actual conditions, such as standard convolution, transposed convolution, residual convolution and hole convolution. Taking transposed convolution as an example, since it is necessary to fill the rows and columns between pixels, the sparse response mechanism needs to be modified compared to the standard convolution. After determining that there are several pulse inputs in the preset convolution block range of the PE response unit, the row and column coordinates of the coordinate index of the pulse recorded in the response range are multiplied by 2, so as to achieve the effect of filling the rows and columns between pixels. The subsequent response process is consistent with the standard convolution. When selecting other convolutions, it may also be necessary to make subtle adjustments to the actual calculation process according to the characteristics of each convolution. I will not go into details here. This is an adjustment that can be expected in combination with the prior art and does not affect the execution of this embodiment.

[0076] See also Figure 4, if the image processing method based on pulse neural network provided by the present application is deployed on a hardware architecture platform, the computing unit includes a processing unit PE array responsible for completing pulse convolution and pooling calculations, a controller, a PEController and a sparse detection module. At the same time, the design has multiple buffers or memories to complete the buffering and storage of calculation data. Among them, the Controller is responsible for judging the input channel level and the control of the multiplexing of the computing unit, and the PEController is responsible for the calculation control of the output channel level. The Controller state machine is responsible for the control of the input channel and the control of the multiplexing of the computing unit. When the Controller receives the calculation start signal, the state will be transferred from idle to the state of obtaining the current input channel data. In this state, the pulse event detection mechanism will be started to record all valid pulse events of the input. When the data detection of the corresponding channel is completed, it will jump to the configuration PE state. Each PE can simulate a pulse neuron of the output layer, that is, it has a specific location information. Configuring PE is to set the specific position of the neuron simulated by the current PE, and generate its pulse event response range according to this position. When the ID configuration of each PE is completed, the data calculation of the corresponding image begins, and the specific calculation process is controlled by the PE Controller.

[0077] It should be noted that, generally speaking, an image cannot be calculated all at once, so it is necessary to multiplex the computing units to complete the image calculation. For example, the output image size is 8×8, but the computing units are only 2×2, so it is necessary to reuse the computing units 16 times to complete the calculation. Therefore, after receiving the calculation completion signal, it will determine whether the multiplexing is completed. If it is still necessary to continue to reuse the computing unit, it will jump back to the configuration PE state to re-complete the configuration to complete the calculation of other blocks until all multiplexing is completed. Finally, it is determined whether all input channels have completed the calculation. The completion of all input channels means that the calculation of this layer has been completed.

[0078] PE Controller is responsible for completing the calculation of all output channels of a single image. When PE Controller receives the start signal, it will obtain the weights of the corresponding input channels and output channels. When the weights are obtained, the membrane potential update calculation of a single output channel will be started. When the calculation is completed, it is determined whether all output channels have been calculated. From this, it can be seen that the calculation data flow of a single convolution calculation layer is also a single instruction stream and multiple data streams. The update calculation of the membrane potential of multiple output channel pulse neurons is completed based on the data of a single input channel. If the update of all output channels is not completed, the weight data of the corresponding input channel to the output channel is updated, and the calculation is restarted until all output channels are calculated.

[0079] PE ARRAY is a PE array, each PE can simulate a neuron. PE ARRAY is controlled by PEController, and after receiving the image data, it distributes the input image to each PE to complete the accumulation, leakage and pulse emission calculation of LIF neurons.

[0080] Spike Buffer is used to store pulses emitted by neurons and input images. When PE_ARRAY calculation is completed, some neurons simulated by PE will emit pulses, and these pulses will be stored in Spike Buffer and used as input images for the next layer of the network. If the calculation has just started, Spike Buffer will read the input image from DDR.

[0081] Weight Buffer is used to store weights. When the PE_ARRAY calculation is completed, the Weight Buffer will input the corresponding weights into the PE according to the position of the neuron simulated by the current PE to complete the convolution calculation.

[0082] Neuron State Buffer is used to store the membrane potential of neurons. When PE_ARRAY starts calculating, the value of the membrane potential is read from Neuron State Buffer. After the calculation is completed, the updated membrane potential is written back to Neuron State Buffer.

[0083] In the image processing method based on pulse neural network provided by the embodiment of the present application, based on the sparse characteristics of pulse convolution, a binary coding calculation strategy is adopted to detect the pulse sparse characteristics of each input pulse image, and the lowest-order pulse that needs attention is marked with a position index; then, in the PE array calculation stage, the convolution calculation is strictly performed according to the position index and the preset convolution block range, without the need for additional judgment and time-consuming operations, making the PE calculation more targeted; and the preset convolution block range can also quickly filter out the pulses within the range for calculation. The whole process can improve the processing efficiency of the image after the pulse neural network is input from different angles. Therefore, the embodiment of the present application can solve the technical problem that the existing image processing scheme based on pulse neural network is time-consuming and labor-intensive, resulting in low efficiency in actual application.

[0084] For easier understanding, see Figure 2 The present application provides an embodiment of an image processing device based on a pulse neural network, comprising:

[0085] An image encoding unit 201, used for encoding an initial image input to the initial spike neural network into a binary spike image;

[0086] A sparse calculation unit 202, used for performing subtraction, AND calculation and XOR calculation in sequence based on the binary pulse image to obtain a logic operation row;

[0087] A sparse detection unit 203, configured to detect all the least significant pulses in the binary pulse image according to the logic operation row, wherein the least significant pulse includes a position index;

[0088] A convolution calculation unit 204 is used to perform convolution calculation on the pulses within the preset convolution block range according to the position index of the least significant pulse through each convolution unit in the PE array to obtain a convolution membrane potential;

[0089] The analysis output unit 205 is used to check and analyze the convolution membrane potential based on the potential threshold and output a pulse characteristic diagram.

[0090] Furthermore, the sparse computing unit 202 is specifically configured to:

[0091] Decomposing the binary pulse image into multiple lines of binary data to obtain binary lines;

[0092] Subtract 1 from the binary row and perform AND calculation with the binary row to get the binary value of the decimal place.

[0093] The carry binary value is XORed with the binary row to obtain a logical operation row.

[0094] Furthermore, it also includes:

[0095] The range determination unit 206 is used to determine the impulse response range of each convolution unit in the PE array according to the kernel position and the kernel size to obtain a preset convolution block range.

[0096] Furthermore, the analysis output unit 205 is specifically used for:

[0097] Determine whether the convolution membrane potential exceeds the potential threshold. If so, set the convolution membrane potential to 0 and output 1 at the same time;

[0098] If not, the convolution membrane potential is subjected to membrane potential attenuation processing and 0 is output;

[0099] The output results are combined to generate a pulse characteristic diagram, and the output results include 1 and 0.

[0100] The present application also provides an image processing device based on a pulse neural network, the device comprising a processor and a memory;

[0101] The memory is used to store the program code and transmit the program code to the processor;

[0102] The processor is used to execute the image processing method based on the pulse neural network in the above method embodiment according to the instructions in the program code.

[0103] The present application also provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the image processing method based on pulse neural network in the above method embodiment.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0108] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method based on a pulse neural network, characterized in that: include: Encoding an initial image input into an initial spike neural network into a binary spike image; Subtract 1, perform AND calculation and XOR calculation in sequence based on the binary pulse image to obtain a logic operation row; Detecting all the least significant pulses in the binary pulse image according to the logic operation row, wherein the least significant pulse includes a position index; By each convolution unit in the PE array performing convolution calculation on the pulse within the preset convolution block range according to the position index of the least significant pulse, a convolution membrane potential is obtained; The convolution membrane potential is checked and analyzed based on the potential threshold, and a pulse characteristic diagram is output.

2. The image processing method based on pulse neural network according to claim 1, characterized in that: The step of sequentially performing subtraction, AND calculation, and XOR calculation based on the binary pulse image to obtain a logic operation row includes: Decomposing the binary pulse image into a plurality of lines of binary data to obtain binary lines; Subtract 1 from the binary row and perform AND calculation with the binary row to obtain a binary value; The carry binary value is XORed with the binary row to obtain a logic operation row.

3. The image processing method based on pulse neural network according to claim 1, characterized in that: The convolution calculation is performed on the pulses within the preset convolution block range according to the position index of the least significant pulse by each convolution unit in the PE array to obtain the convolution membrane potential, which also includes: The impulse response range of each convolution unit in the PE array is determined according to the kernel position and kernel size to obtain the preset convolution block range.

4. The image processing method based on pulse neural network according to claim 1, characterized in that: The checking and analyzing of the convolution membrane potential based on the potential threshold and outputting a pulse characteristic diagram include: Determine whether the convolution membrane potential exceeds the potential threshold, if so, set the convolution membrane potential to 0 and output 1; If not, the convolution membrane potential is subjected to membrane potential attenuation processing and 0 is outputted; A pulse characteristic diagram is generated in combination with the output results, wherein the output results include 1 and 0.

5. An image processing device based on a pulse neural network, characterized in that: include: An image encoding unit, used for encoding an initial image input into an initial spike neural network into a binary spike image; A sparse calculation unit, used for performing subtraction, AND calculation and XOR calculation in sequence based on the binary pulse image to obtain a logic operation row; A sparse detection unit, used for detecting all the least significant pulses in the binary pulse image according to the logic operation row, wherein the least significant pulse includes a position index; A convolution calculation unit, configured to perform convolution calculation on the pulses within the preset convolution block range according to the position index of the least significant pulse through each convolution unit in the PE array, so as to obtain a convolution membrane potential; An analysis output unit is used to check and analyze the convolution membrane potential based on a potential threshold and output a pulse characteristic diagram.

6. The image processing device based on pulse neural network according to claim 5, characterized in that: The sparse computing unit is specifically used for: Decomposing the binary pulse image into a plurality of lines of binary data to obtain binary lines; Subtract 1 from the binary row and perform AND calculation with the binary row to obtain a binary value; The carry binary value is XORed with the binary row to obtain a logic operation row.

7. The image processing device based on pulse neural network according to claim 5, characterized in that: Also includes: The range determination unit is used to determine the impulse response range of each convolution unit in the PE array according to the kernel position and the kernel size, so as to obtain the preset convolution block range.

8. The image processing device based on pulse neural network according to claim 5, characterized in that: The analysis output unit is specifically used for: Determine whether the convolution membrane potential exceeds the potential threshold, if so, set the convolution membrane potential to 0 and output 1; If not, the convolution membrane potential is subjected to membrane potential attenuation processing and 0 is outputted; A pulse characteristic diagram is generated in combination with the output results, wherein the output results include 1 and 0.

9. An image processing device based on a pulse neural network, characterized in that The device comprises a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the image processing method based on pulse neural network according to any one of claims 1-4 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the image processing method based on pulse neural network according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Data processing method and device based on spiking neural network

    CN114819122A

  • Pulse convolutional neural network accelerator

    CN115329934A

  • Data classification method for realizing rapid reasoning and effective learning based on spiking neural network

    CN118133121A

  • Monitoring potential of neuron circuits

    US20180211160A1