Convolution Processor, Method and Electronic Device for Spiking Neural Network

Abstract

The present invention provides a convolutional processor, method and electronic device for a spiking neural network, which can be applied to the fields of deep learning and artificial intelligence technologies. The convolutional processor for the spiking neural network includes: a first computing unit, in which 2<supgt;N1< / supgt> spiking convolution results based on N1 first weights are pre-stored, and the first computing unit is configured to receive N1 first features, and find a spiking convolution result matching the N1 first features from the 2<supgt;N1< / supgt> spiking convolution results as a first convolution result; wherein, the spiking convolution result includes K parts, the first computing unit includes K lookup circuits, the k-th lookup circuit stores the k-th part of each spiking convolution result, and the k-th lookup circuit is configured to find the k-th part matching the N1 first features from the k-th part of each stored spiking convolution result as the k-th part of the first convolution result.

Description

Technical Field

[0001] The present invention relates to the technical fields of deep learning and artificial intelligence, and more particularly to a convolutional processor, method, and electronic device for a spiking neural network. Background Art

[0002] The spiking neural network that simulates the transmission of brain neurons has sparse characteristics in the processed data and also has an event-triggering attribute. Not all data at each moment needs to be fully calculated, which reduces the overall computational amount. It is an efficient brain-inspired computing structure. Therefore, the spiking neural network is widely used in fields such as image recognition and robot control.

[0003] In the related art, when performing convolution calculations using the general convolutional calculation structure of the spiking neural network, a large number of adders at multiple levels are required to accumulate and calculate each weight participating in the convolution calculation. The accelerator hardware computing resources are occupied more, the fan-out is larger, and the utilization rate of the accelerator hardware computing resources is lower. Summary of the Invention

[0004] In view of the above problems, the present invention provides a convolutional processor, method, and electronic device for a spiking neural network.

[0005] According to the first aspect of the present invention, there is provided a convolutional processor for a spiking neural network. The convolutional processor includes: a first calculation unit, in which 2 N1 spiking convolution results based on N1 first weights are pre-stored. The first calculation unit is configured to receive N1 first features, and find, from the 2 N1 spiking convolution results, the spiking convolution result that matches the N1 first features as the first convolution result; wherein, the spiking convolution result includes K parts, the first calculation unit includes K lookup circuits, the kth lookup circuit stores the kth part of each spiking convolution result, the kth lookup circuit is any one of the K lookup circuits, the kth part is the part corresponding to the kth lookup circuit among the K parts, and the kth lookup circuit is configured to find, from the stored kth parts of each spiking convolution result, the kth part that matches the N1 first features as the kth part of the first convolution result, where 1 ≤ k ≤ K and k is an integer, and N1 and K are integers greater than 1.

[0006] According to the second aspect of the present invention, there is provided a convolutional processing method for a spiking neural network, which is executed by the above-mentioned convolutional processor. The method includes: the first calculation unit receives N1 first features; the first calculation unit finds from the pre-stored 2 N1Among the pulse convolution results, find the pulse convolution results that match the above N1 first features as the first convolution result; wherein, the above pulse convolution results include K parts, the above first calculation unit includes K search circuits, and the k-th search circuit stores the k-th part of each pulse convolution result. The k-th search circuit is any search circuit among the K search circuits, and the k-th part is the part corresponding to the k-th search circuit among the above K parts. The above 2 from the pre-stored 2 pulse convolution results based on N1 first weights N1 Among the pulse convolution results, finding the pulse convolution results that match the above N1 first features as the first convolution result includes: the k-th search circuit finds the k-th part that matches the above N1 first features from the k-th part of each stored pulse convolution result as the k-th part of the above first convolution result, where 1 ≤ k ≤ K and k is an integer, and N1 and K are integers greater than 1.

[0007] According to the third aspect of the present invention, an electronic device is provided, including the above convolution processor.

[0008] According to the convolution processor for a spiking neural network provided by the embodiments of the present invention, since the pulse convolution results include K parts, the first calculation unit includes K search circuits, and the k-th search circuit stores the k-th part of each pulse convolution result, therefore, the k-th search circuit can be used to find the k-th part that matches the N1 first features from the k-th part of each stored pulse convolution result as the k-th part of the first convolution result. Furthermore, without using adders and multipliers, the K search circuits included in the first calculation unit can be used to quickly find each part of the first convolution result, and quickly obtain the first convolution result found by the first calculation unit from 2 N1 Among the pulse convolution results, the first convolution result that matches the N1 first features reduces the usage amount of adders in the related art. Since the minimum logic unit of the accelerator hardware computing resources includes more search circuits and fewer adders, while reducing the usage amount of adders, the usage amount of the minimum logic unit can be reduced, the occupancy of the accelerator hardware computing resources can be reduced, the fan-out is small, the design area of the related hardware, and the utilization rate of the accelerator hardware computing resources is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Through the following description of the embodiments of the present invention with reference to the drawings, the above content and other objects, features, and advantages of the present invention will become clearer.

[0010] Figure 1A Shows the schematic diagram of the convolution calculation of the spiking neural network.

[0011] Figure 1B Shows the schematic diagram of using multiple adders in the related art to accelerate the convolution calculation of the spiking neural network.

[0012] Figure 2 Shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the first embodiment of the present invention.

[0013] Figure 3 Shows a schematic diagram of the spiking convolution result according to an embodiment of the present invention.

[0014] Figure 4 Shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the second embodiment of the present invention.

[0015] Figure 5 Shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the third embodiment of the present invention.

[0016] Figure 6 Shows a schematic diagram of a lookup table logic circuit according to an embodiment of the present invention.

[0017] Figure 7 Shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the fourth embodiment of the present invention.

[0018] Figure 8 Shows a schematic diagram of the spiking convolution result according to another embodiment of the present invention.

[0019] Figure 9 Shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the fifth embodiment of the present invention.

[0020] Figure 10 Shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the sixth embodiment of the present invention.

[0021] Figure 11 Shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the seventh embodiment of the present invention.

[0022] Figure 12 Shows a flowchart of a convolutional processing method for a spiking neural network according to an embodiment of the present invention.

[0023] Figure 13 Shows a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed implementation manners

[0024] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In the following detailed description, for the sake of explanation, numerous specific details are set forth in order to provide a comprehensive understanding of the embodiments of the present invention. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present invention.

[0025] The terms used herein are merely for describing specific embodiments and are not intended to limit the present invention. The terms "comprising", "including" and the like used herein indicate the presence of the described features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] In the case of using expressions such as "at least one of A, B, and C", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0028] Figure 1A The schematic diagram of the convolutional calculation performed by the spiking neural network is shown.

[0029] As Figure 1A shown, the feature matrix 110 input into the spiking neural network includes a plurality of feature data. Since the calculation feature of the spiking neural network is to use single-bit pulse data instead of multi-bit data to participate in the convolutional calculation, the feature data included in the feature matrix 110 is 0 or 1. When the spiking neural network performs convolutional calculation, the feature matrix 110 is multiplied by the weight matrix 120 to obtain the convolutional matrix 130. Among them, the weight matrix 120 includes a plurality of weights corresponding to the plurality of feature data. The values in the convolutional matrix 130 are accumulated to obtain the convolutional result "w2 + w3 + w4 + w5 + w7 + w9".

[0030] From Figure 1A it can be seen that when Figure 1AWhen the feature matrix 110 and the weight matrix 120 in [it] perform a 3×3 convolution calculation, the feature data serves as the control signal for data selection, controlling whether the data participating in the addition calculation is weight data or 0 data. Therefore, when the spiking neural network performs a convolution calculation, the multiplier can be omitted, and only a multi-stage adder unit is used to accumulate each weight participating in the convolution calculation to obtain the convolution result.

[0031] Figure 1B FIG. shows a schematic diagram of accelerating the convolution calculation of a spiking neural network by using multiple adders in the related art.

[0032] As Figure 1B shown, when the convolution processor of the spiking neural network in the related art performs a 3×3 convolution calculation on multiple feature data 111 included in the feature matrix and multiple weights 121 included in the weight matrix, the multiplier is omitted, and only the multi-stage adder unit 140 is used to accumulate each weight participating in the convolution calculation to obtain the convolution result. Among them, the preset value can be 0.

[0033] According to Figure 1B it can be known that although the convolution processor of the spiking neural network in the related art omits the multiplier and only uses the adder unit when performing a 3×3 convolution calculation, 4 levels of a total of 9 adders are used, and the levels of the adder tree used are relatively many. Among them, the adder unit can be a multi-stage addition calculation structure implemented by using logic gates and flip-flops in the accelerator hardware resources. And the accelerator hardware resources usually include resources such as look-up tables, logic gates, and flip-flops. Therefore, the convolution processor of the spiking neural network in the related art uses a large number of logic gates and flip-flop resources, wasting a lot of resources such as look-up tables (Look-Up-Table, LUT) in the accelerator hardware resources.

[0034] For example, when the logic unit includes 4 6-input 2-output look-up tables and resources such as logic gates and flip-flops, and each logic gate included in the logic unit constitutes an adder, when the convolution processor of the spiking neural network in the related art performs a 3×3 convolution calculation, using 9 adders is equivalent to using 9 logic units, and the fan-out will become 9, occupying more hardware computing resources and having a larger fan-out. In the case of large-scale, multi-layer convolution calculations, the fan-out amount will become unacceptable, increasing the design area of the related hardware. And the related art wastes multiple look-up table resources in 9 look-up tables, and the utilization rate of the accelerator hardware computing resources is low. Moreover, the larger the fan-out, the lower the calculation peak frequency. In the case where the related art occupies more hardware computing resources and has a larger fan-out, the possible calculation main frequency is reduced. Among them, the 6-input 2-output look-up table represents a look-up table including 6 input terminals and 2 output terminals.

[0035] In view of this, embodiments of the present invention provide a convolutional processor, method, and electronic device for a spiking neural network, which can be applied to the fields of deep learning and artificial intelligence technology.

[0036] Figure 2 FIG. 4 shows a schematic structural diagram of a convolutional processor for a spiking neural network according to a first embodiment of the present invention.

[0037] As Figure 2 shown, the convolutional processor for a spiking neural network may include a first calculation unit 210.

[0038] In the first calculation unit 210, 2 N1 pulse convolution results based on N1 first weights may be pre-stored. The first calculation unit 210 may be configured to receive N1 first features F1, F2, …, FN1, and search for a pulse convolution result that matches the N1 first features F1, F2, …, FN1 from the 2 N1 pulse convolution results as the first convolution result. Here, N1 is an integer greater than 1.

[0039] According to an embodiment of the present invention, N1 can be selected according to actual situations and is not limited herein.

[0040] For example, when N1 is 4, 16 pulse convolution results based on 4 first weights may be pre-stored in the first calculation unit 210. The first calculation unit 210 may receive 4 first features, and use the 4 first features as addresses to search for a pulse convolution result that matches the 4 first features from the 16 pulse convolution results as the first convolution result.

[0041] For example, when N1 is 5, 32 pulse convolution results based on 5 first weights may be pre-stored in the first calculation unit 210. The first calculation unit 210 may receive 5 first features, and use the 5 first features as addresses to search for a pulse convolution result that matches the 5 first features from the 32 pulse convolution results as the first convolution result.

[0042] For example, when N1 is 9, 512 pulse convolution results based on 9 first weights may be pre-stored in the first calculation unit 210. The first calculation unit 210 may receive 9 first features, and use the 9 first features as addresses to search for a pulse convolution result that matches the 9 first features from the 512 pulse convolution results as the first convolution result.

[0043] The pulse convolution result may include K parts. The first calculation unit 210 may include K lookup circuits, for example, the first lookup circuit LUT1, the second lookup circuit LUT2, …, the Kth lookup circuit LUTK. The kth lookup circuit stores the kth part of each pulse convolution result. The kth lookup circuit is any one of the K lookup circuits, and the kth part is the part corresponding to the kth lookup circuit among the K parts. For example, assuming the pulse convolution result is 10-bit data, the 10-bit pulse convolution result can be divided into 5 parts, each part being 2 bits, and the 5 parts of the pulse convolution result can be stored in 5 lookup circuits respectively.

[0044] The kth lookup circuit is used to look up the kth part that matches N1 first features F1, F2, …, FN1 from the stored kth part of each pulse convolution result as the kth part of the first convolution result, where 1 ≤ k ≤ K and k is an integer, and K is an integer greater than 1. For example, the kth lookup circuit can store the kth part of each pulse convolution result by address. When looking up, N1 first features F1, F2, …, FN1 can be used as the target query addresses to look up the kth part that matches the target query addresses from the stored kth part of each pulse convolution result.

[0045] As Figure 2 shown, the first lookup circuit LUT1 can store the first part of each pulse convolution result. The first lookup circuit LUT1 can look up the first part that matches N1 first features F1, F2, …, FN1 from the stored first part of each pulse convolution result as the first part S1 of the first convolution result.

[0046] The second lookup circuit LUT2 can store the second part of each pulse convolution result. The second lookup circuit LUT2 can look up the second part that matches N1 first features F1, F2, …, FN1 from the stored second part of each pulse convolution result as the second part S2 of the first convolution result.

[0047] And so on, the kth lookup circuit can store the kth part of each pulse convolution result. The Kth lookup circuit LUTK can look up the Kth part that matches N1 first features F1, F2, …, FN1 from the stored Kth part S K of each pulse convolution result as the Kth part S K of the first convolution result.

[0048] In this way, the first calculation unit 210 generates the convolution result of N1 first features F1, F2, …, FN1 and N1 first weights, that is, the first convolution result, where the K lookup circuits in the first calculation unit 210 respectively output K parts S1, S2, …, S of the first convolution resultK 。

[0049] According to the convolutional processor for spiking neural networks provided by an embodiment of the present invention, since the spiking convolution result includes K parts, the first calculation unit includes K lookup circuits, and the k-th lookup circuit stores the k-th part of each spiking convolution result. Therefore, the k-th lookup circuit can be used to find the k-th part that matches N1 first features from the stored k-th parts of each spiking convolution result as the k-th part of the first convolution result. Furthermore, without using adders and multipliers, the K lookup circuits included in the first calculation unit can be used to quickly find each part of the first convolution result, and quickly obtain the first convolution result found by the first calculation unit from 2 N1 spiking convolution results that match N1 first features, reducing the usage of adders in the related art. Since the minimum logic unit of the accelerator hardware computing resources includes more lookup circuits and fewer adders, while reducing the usage of adders, the usage of the minimum logic unit can be reduced, the occupancy of the accelerator hardware computing resources can be reduced, the fan-out is small, the design area of the related hardware, and the utilization rate of the accelerator hardware computing resources can be improved.

[0050] According to the embodiment of the present invention, since the convolutional processor for spiking neural networks provided by the embodiment of the present invention has a smaller fan-out compared to the related art, it can balance the reduction in the computing main frequency caused by low resource utilization and increased fan-out in the related art.

[0051] Next, in conjunction with Figure 3 and Figure 4 , another embodiment of the convolutional processor according to the embodiment of the present invention will be described. Figure 3 FIG. shows a schematic diagram of spiking convolution according to an embodiment of the present invention. Figure 4 FIG. shows a schematic structural diagram of a convolutional processor for spiking neural networks according to a second embodiment of the present invention.

[0052] As Figure 3 shown, both the feature matrix 110 and the weight matrix 120 are 3×3 matrices. Here, taking the convolution of 4 first features at positions 00, 01, 02, and 10 and 4 first weights w1, w2, w3, and w4 as an example, the structure and working principle of the convolutional processor in the case of N1 = 4 will be described. 00 represents the first row and the first column, 01 represents the first row and the second column, 02 represents the first row and the third column, and 10 represents the second row and the first column. The total number of spiking convolution results that can be generated by 4 first weights w1, w2, w3, and w4 is 16. The number of bits M of the spiking convolution result is 10. The 10-bit data of 16 spiking convolution results can be divided into 5 parts, each part being 2 bits. In this case, N1 = 4, M = 10, and K = 5.

[0053] As shown Figure 4 in the figure, the convolutional processor for the spiking neural network may include a first type of first computing unit 210A. The first type of first computing unit 210A may include 5 lookup circuits LUT1, LUT2, …, LUT5, and the 5 parts of each pulse convolution result are respectively stored in the 5 lookup circuits LUT1, LUT2, …, LUT5. For example, if the pulse convolution result is 10-bit data, the first part of the pulse convolution result may be the 1st and 2nd bits of the 10-bit data, the second part is the 3rd and 4th bits of the 10-bit data, and so on. The 1st and 2nd bits of each of the 16 pulse convolution results may be stored in the first lookup circuit LUT1, the 3rd and 4th bits are stored in the second lookup circuit LUT2, and so on.

[0054] The lookup circuit may have N1 input terminals and m output terminals, where M = mK, M is the number of bits of the first convolution result, and m is the number of bits of each part of the first convolution result. The kth lookup circuit may be used to respectively receive N1 first features from the N1 input terminals, search for the kth part that matches the N1 first features from the kth part of each stored pulse convolution result as the kth part of the first convolution result, and output the m bits of the kth part of the first convolution result through the m output terminals respectively. In Figure 4 the example, N = 4, M = 10, K = 5, so m = 2. That is to say, each lookup circuit has 4 input terminals and 2 output terminals. The 4 input terminals are respectively used to receive 4 first features located at positions 00, 01, 02, and 10 in the feature matrix 110, and the 2 output terminals are used to output the corresponding 2-bit part of the first convolution result. For example, the first lookup circuit LUT1 outputs the 1st and 2nd bits of the first convolution result, the second lookup circuit LUT2 outputs the 3rd and 4th bits of the first convolution result, the third lookup circuit LUT3 outputs the 5th and 6th bits of the first convolution result, and so on.

[0055] According to the embodiments of the present invention, the kth lookup circuit is used to respectively receive N1 first features from the N1 input terminals, search for the kth part that matches the N1 first features from the kth part of each stored pulse convolution result as the kth part of the first convolution result, and output the m bits of the kth part of the first convolution result through the m output terminals respectively. Furthermore, without using adders and multipliers, each lookup circuit can quickly search for the m bits of the first convolution result, and K lookup circuits can quickly search for the M-bit first convolution result, reducing the usage amount of adders in the related art, reducing the occupancy of accelerator hardware computing resources, and improving the utilization rate of accelerator hardware computing resources.

[0056] It can be utilized Figure 4The convolution processor for a spiking neural network shown is configured to perform convolution calculations on the first features at positions 00, 01, 02, and 10 in the feature matrix 110 and the four first weights w1, w2, w3, and w4 at the corresponding positions in the weight matrix 120.

[0057] The k-th lookup circuit can be used to receive four first features respectively through four input terminals of the k-th lookup circuit, look up the k-th part that matches the four first features from the k-th parts of the stored respective pulse convolution results as the k-th part of the first convolution result, and output 2 bits of the k-th part of the first convolution result respectively through two output terminals of the k-th lookup circuit.

[0058] According to an embodiment of the present invention, when the lookup circuit has four input terminals and two output terminals, N1 = 4, M = 10, and K = 5, by using the four input terminals of the k-th lookup circuit to receive four first features respectively, looking up the k-th part that matches the four first features from the k-th parts of the stored respective pulse convolution results as the k-th part of the first convolution result, and outputting 2 bits of the k-th part of the first convolution result respectively through two output terminals of the k-th lookup circuit, it is possible to quickly look up 2-bit data in the first convolution result by using each lookup circuit without using adders and multipliers, quickly look up a 10-bit first convolution result by using five lookup circuits, reduce the usage amount of adders in the related art, reduce the occupancy of accelerator hardware computing resources, and improve the utilization rate of accelerator hardware computing resources.

[0059] Similarly, taking the following Figure 3 3×3 convolution as an example, referring to Figure 5 to describe the structure and working principle of the convolution processor for a spiking neural network according to another embodiment of the present invention.

[0060] Figure 5 Fig. shows a schematic structural diagram of a convolution processor for a spiking neural network according to the third embodiment of the present invention.

[0061] In this embodiment, the convolution processor for a spiking neural network may include a plurality of first computing units and a first adder. The first adder can sum the first convolution results output by the plurality of first computing units to obtain a first output. For example, as Figure 5 shown, the convolution processor includes two first-type first computing units 210A and a first adder 220. The first adder 220 can sum the first convolution results output by the two first-type first computing units 210A to obtain a first output.

[0062] In some embodiments, the convolutional processor may further include a second computing unit 230 and a second adder 240. The second computing unit 230 may be configured to receive N2 second features, perform a convolution calculation on the received N2 second features and N2 second weights to obtain a second convolution result, where N2 is an integer greater than or equal to 1. The second adder 240 may be configured to sum the first output and the second convolution result provided by the second computing unit 230 to obtain a second output.

[0063] Similarly, with reference to Figure 3 , taking a 3×3 convolution as an example, the Figure 5 convolutional processor will be described.

[0064] N1 = 4, M = 10, K = 5. Each of the two first-type first computing units 210A includes five lookup circuits LUT1 to LUT5. The lookup circuit may have four input terminals and two output terminals. Figure 5 The first-type first computing unit 210A of Figure 4 has a similar structure and function to the first-type first computing unit 210A in

[0065] Combined with Figure 3 and Figure 5 , one of the two first-type first computing units 210A ( Figure 5 the left first-type first computing unit 210A in Figure 5 ) may perform a convolution calculation on the four first features at positions 00, 01, 02, 10 in the feature matrix 110 and the four first weights w1, w2, w3, and w4 at the corresponding positions in the weight matrix 120 to obtain a 10-bit first convolution result. The other first-type first computing unit 210A (

[0066] the right first-type first computing unit 210A in Figure 5 ) may perform a convolution calculation on the four first features at positions 11, 12, 20, 21 in the feature matrix 110 and the first weights w5, w6, w7, and w8 at the corresponding positions in the weight matrix 120 to obtain another 10-bit first convolution result.

[0066] According to the embodiments of the present invention, by using the first adder to sum the first convolution results output by multiple first computing units, a first output is obtained. Compared with the related art, the number of adders used is greatly reduced, the resource utilization rate is improved, the number of adder stages is reduced, and the inference calculation efficiency is improved.

[0067] In some embodiments, the second computing unit 230 may include a selector. The selector may be used to select between N2 second weights and a preset reference value according to N2 second features to obtain a second convolution result. According to an embodiment of the present invention, the preset reference value may be selected according to actual circumstances and is not limited herein. For example, the preset reference value may be 0. For example, in combination with Figure 3 and Figure 5 , when N2 = 1, the second computing unit 230 may use a single selector to perform convolution calculation on the feature at position 22 in the feature matrix 110 (i.e., the second feature) and the weight w9 at position 22 in the weight matrix 120 (i.e., the second weight). Specifically, one input terminal of the selector receives the weight w9 at position 22 in the weight matrix, another input terminal receives the preset reference value 0, the control terminal of the selector receives the feature at position 22 in the feature matrix (i.e., the second feature), and the output terminal of the selector outputs the second convolution result. When the feature received by the control terminal of the selector is 0, the selector outputs the preset reference value 0, and when the feature received by the control terminal of the selector is 1, the selector outputs the weight w9. Although Figure 5 describes an example with a single selector and a single second adder 240, however, the embodiments of the present invention are not limited thereto, and the number of selectors and second adders may be set according to needs. For example, when N2 = 2, the second computing unit 230 may include two selectors, which are respectively used for convolution calculation of 2 features and 2 weights.

[0068] According to an embodiment of the present invention, by using a first adder to sum the first convolution results output by multiple first computing units to obtain a first output, using a second computing unit to receive N2 second features, performing convolution calculation on the received N2 second features and N2 second weights to obtain a second convolution result, and using a second adder to sum the first output and the second convolution result provided by the second computing unit to obtain a second output, the technical means can quickly perform convolution calculation on N1 first features and N2 second features by using 2 adders and multiple lookup circuits, quickly obtain the second output. Compared with the related art, fewer adders are used, the resource utilization rate is improved, the number of stages of the adder is reduced, and the inference calculation efficiency is improved.

[0069] According to an embodiment of the present invention, in a convolutional processor for a spiking neural network, which includes 2 first computing units, a first adder, a second computing unit, and a second adder, when N1 = 4, N2 = 1, M = 10, K = 5, and the lookup circuit has 4 input terminals and 2 output terminals, a 3×3 convolutional calculation is implemented by using the convolutional processor of the spiking neural network, and a 3×3 convolutional calculation is quickly performed by using 10 lookup circuits, 2 adders, and a selector. Compared with the related art which requires 9 adders, that is, 9 logic units are used, the adders used in the present invention are greatly reduced, and only 3 logic units are required, and the logic units used are greatly reduced. A large number of lookup circuit resources, logic gates, and flip-flop resources in the accelerator hardware computing resources are fully utilized. The influence of resource utilization rate and fan-out is balanced. Compared with the related art, the resource utilization rate is improved, the number of stages of the adder is reduced, and the inference calculation efficiency is improved.

[0070] The lookup circuit in the above embodiment may be a lookup table logic circuit. The following will refer to Figure 6 to describe the lookup table logic circuit.

[0071] Figure 6 FIG. shows a schematic diagram of a lookup table logic circuit according to an embodiment of the present invention.

[0072] As Figure 6 shown, the lookup table logic circuit may have 6 input terminals A1 to A6 and 2 output terminals O5 and O6. The lookup table logic circuit is a basic implementation unit of a field programmable gate array (FPGA, Field Programmable Gate Array‌), which is evenly distributed in large numbers at all positions of the FPGA and is the least scarce hardware resource on FPGA-based accelerators. The lookup circuit in the convolutional processor according to an embodiment of the present invention may be implemented by a lookup table logic circuit. For example, if a 4-input 2-output lookup circuit is required in the convolutional processor, then 4 of the 6 input terminals (such as A1 to A4) of the lookup table logic circuit can be used as the input terminals of the lookup circuit, and the 2 output terminals O5 and O6 of the lookup table logic circuit can be used as the output terminals of the lookup circuit.

[0073] According to an embodiment of the present invention, in a convolutional processor for a spiking neural network, it includes 2 first computing units, a first adder, a second computing unit, and a second adder. When N1 = 4, N2 = 1, M = 10, K = 5, and the lookup circuit is a lookup table logic circuit with 4 input terminals and 2 output terminals, a 3×3 convolution calculation is implemented using the convolutional processor of the spiking neural network, and 3×3 convolution calculation is quickly performed using 10 lookup table logic circuits, 2 adders, and a selector. Compared with the related art that requires 9 adders, the present invention reduces the number of adders to 2, 1, or even no adder is needed. It makes full use of a large number of lookup table logic circuit resources, logic gates, and flip-flop resources in the FPGA. It balances the influence of resource utilization rate and fan-out. Compared with the related art, it improves the resource utilization rate, reduces the number of adder stages, and improves the inference calculation efficiency.

[0074] Figure 7 FIG. 4 shows a schematic structural diagram of a convolutional processor for a spiking neural network according to a fourth embodiment of the present invention.

[0075] As Figure 7 shown, the convolutional processor for a spiking neural network may include at least one acceleration module, and each acceleration module may have a structure as Figure 5 shown, that is, it includes two first-type first computing units 210A, a first adder 220, a second computing unit 230, and a second adder 240. The above Figure 5 descriptions of the first-type first computing unit 210A, the first adder 220, the second computing unit 230, and the second adder 240 also apply to this embodiment and will not be elaborated here.

[0076] Multiple Figure 7 shown acceleration modules can be used to perform 3×3 convolution calculations on different feature matrices 110 and corresponding weight matrices 120 in parallel, further improving the convolution calculation efficiency.

[0077] As Figure 7 shown, the convolutional processor for a spiking neural network may further include a comparator 250. The comparator 250 can be used to compare the second output provided by the second adder 240 with a preset threshold, and output a pulse signal when the second output is greater than or equal to the preset threshold. For example, the comparator 250 can also compare the sum of multiple second outputs with the preset threshold, and output a pulse signal when the sum result is greater than or equal to the preset threshold.

[0078] In some embodiments, Figure 7The convolutional processor for spiking neural networks therein may further include a cache unit 260. The cache unit 260 can be used to accumulate a plurality of consecutive second outputs and cache the accumulated result. Among them, the plurality of consecutive second outputs can be the second outputs continuously output by the same acceleration module for multiple times, or the second outputs sequentially output by different acceleration modules.

[0079] The comparator 250 can add the currently output second output and the cached accumulated result, compare the added result with a preset threshold, and output a pulse signal when the added result is greater than or equal to the preset threshold, and set the cache unit 260 to 0.

[0080] According to an embodiment of the present invention, the preset threshold can be selected according to the actual situation and is not limited herein.

[0081] According to an embodiment of the present invention, by using a comparator to compare the second output with a preset threshold and outputting a pulse signal when the second output is greater than or equal to the preset threshold, a pulse signal is generated only for the second output that meets the requirements.

[0082] Next, refer to Figure 8 and Figure 9 to elaborate in detail on the structure and principle of the convolutional processor according to another embodiment of the present invention.

[0083] Figure 8 FIG. shows a schematic diagram of pulse convolution according to another embodiment of the present invention. Figure 9 FIG. shows a schematic structural diagram of a convolutional processor for spiking neural networks according to the fifth embodiment of the present invention.

[0084] As Figure 8 shown, the number of pulse convolution results corresponding to the 5 first features at positions 11, 12, 20, 21, and 22 in the feature matrix 110 and the 5 first weights w5, w6, w7, w8, and w9 at the corresponding positions in the weight matrix 120 can be 32. The number of bits of the maximum value among the 32 pulse convolution results can be 11 bits. The 11-bit data of the 32 pulse convolution results can be divided into 6 parts respectively. The first part to the fifth part can each be 2 bits, and the sixth part can be 1 bit. The input signal of the lookup circuit can be a pulse signal.

[0085] It is possible to utilize Figure 9 the convolutional processor for spiking neural networks shown to implement convolutional processing on the 5 first features at positions 11, 12, 20, 21, and 22 in the feature matrix 110 and the 5 first weights w5, w6, w7, w8, and w9 at the corresponding positions in the weight matrix 120.

[0086] AsFigure 9 As shown, the convolutional processor for the spiking neural network may include a first computing unit 210B of the second type. Different from Figure 4 the first type of computing unit 210A, Figure 9 the number of lookup circuits and the number of input and output terminals of the lookup circuits in the first computing unit 210B of the second type are different.

[0087] As Figure 9 shown, the first computing unit 210B of the second type may include six lookup circuits LUT1 to LUT6. Each of the lookup circuits LUT1 to LUT6 may have five input terminals and two output terminals. N1 = 5, M = 11, K = 6, and M is the number of bits of the first convolution result.

[0088] Thirty-two pulse convolution results based on five first weights may be pre-stored in the first computing unit 210B of the second type. The first computing unit 210B of the second type may be configured to receive five first features, and find a pulse convolution result matching the five first features from the 32 pulse convolution results as the first convolution result. The pulse convolution result may include six parts, which are respectively stored in the six lookup circuits LUT1 to LUT6.

[0089] The first lookup circuit LUT1 may receive five first features located at positions 11, 12, 20, 21, and 22 in the feature matrix, find the first part matching the five first features from the first part of each stored pulse convolution result as the first part of the first convolution result, and output the two bits of the first part of the first convolution result through the two output terminals of the first lookup circuit respectively.

[0090] The second lookup circuit LUT2 may receive five first features located at positions 11, 12, 20, 21, and 22 in the feature matrix, find the second part matching the five first features from the second part of each stored pulse convolution result as the second part of the first convolution result, and output the two bits of the second part of the first convolution result through the two output terminals of the second lookup circuit respectively, and so on.

[0091] The sixth lookup circuit LUT6 may receive five first features located at positions 11, 12, 20, 21, and 22 in the feature matrix, find the sixth part matching the five first features from the sixth part of each stored pulse convolution result as the sixth part of the first convolution result, and output one bit of the sixth part of the first convolution result through one output terminal of the sixth lookup circuit. The other output terminal of the sixth lookup circuit may be set to invalid, that is, no output is provided. Figure 9 The lookup circuits LUT1 to LUT6 in

[0092] According to an embodiment of the present invention, the lookup circuit in the convolutional processor for a spiking neural network has 5 input terminals and 2 output terminals. When N1 = 5, M = 11, and K = 6, where M is the number of bits of the first convolution result, without using an adder, the first 5 lookup circuits can be used to quickly look up 2 bits of the first convolution result respectively, and the last lookup circuit can be used to quickly look up 1 bit of the first convolution result, so as to quickly look up the 11-bit first convolution result by using 6 lookup circuits, reduce the usage amount of adders in the related art, reduce the occupation amount of accelerator hardware computing resources, and improve the utilization rate of accelerator hardware computing resources.

[0093] It can utilize Figure 10 The convolutional processor for a spiking neural network shown can implement 3×3 convolution processing on the feature matrix 110 and the weight matrix 120.

[0094] Figure 10 FIG. shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the sixth embodiment of the present invention.

[0095] As Figure 10 shown, the convolutional processor for a spiking neural network may include a first type of first computing unit 210A, a second type of first computing unit 210B, and a first adder 220. The first type of first computing unit 210A can generate a 10-bit first convolution result according to 4 first features. The second type of first computing unit 210B can generate an 11-bit first convolution result according to 5 first features. The first type of first computing unit 210A may have the same structure as the first type of first computing unit in any of the above embodiments, and the second type of first computing unit 210B may have the same structure as the second type of first computing unit in any of the above embodiments, which will not be elaborated here. Taking Figure 8 the 3×3 convolution shown as an example, the first type of first computing unit 210A can perform lookup convolution on 4 features at positions 00, 01, 02, and 10 in the feature matrix 110 and 4 weights w1, w2, w3, and w4 at the corresponding positions in the weight matrix 120 to obtain a 10-bit first convolution result. The second type of first computing unit 210B can perform lookup convolution on 5 features at positions 11, 12, 20, 21, and 22 in the feature matrix 110 to obtain an 11-bit first convolution result.

[0096] The first adder 220 can sum the first convolution results output by the first type of first computing unit 210A and the second type of first computing unit 210B to obtain a first output.

[0097] According to an embodiment of the present invention, a comparator can also be used to compare the first output with a preset threshold, and output a pulse signal when the first output is greater than or equal to the preset threshold, so as to generate a pulse signal only for the second output that meets the requirements.

[0098] The first type of first computing unit 210A includes 5 lookup circuits each having 4 input terminals and 2 output terminals. The second type of first computing unit 210B includes 6 lookup circuits each having 5 input terminals and 2 output terminals.

[0099] According to an embodiment of the present invention, in a convolutional processor for a spiking neural network including 2 first computing units and a first adder, one of the two first computing units generates a 10-bit first convolution result based on 4 first features, and the other of the two first computing units generates an 11-bit first convolution result based on 5 first features. One first computing unit includes 5 lookup circuits each having 4 input terminals and 2 output terminals, and the other first computing unit includes 6 lookup circuits each having 5 input terminals and 2 output terminals. In this case, a 3×3 convolution calculation is implemented using the convolutional processor of the spiking neural network, and a 3×3 convolution calculation is quickly performed using 11 lookup circuits and 1 adder. Compared with the related art that requires 9 adders, that is, 9 logic units, the amount of adder data used in the present invention is greatly reduced, and only 3 logic units are required, so fewer logic units are used. The large number of lookup circuit resources, logic gates, and flip-flop resources in the accelerator hardware computing resources are fully utilized. The influence of resource utilization rate and fan-out is balanced. Compared with the related art, the resource utilization rate is improved, the number of adder stages is reduced, and the inference calculation efficiency is improved.

[0100] According to an embodiment of the present invention, the lookup circuit included in one first computing unit can be a lookup table logic circuit, and the lookup circuit included in the other first computing unit can be a lookup table logic circuit.

[0101] According to an embodiment of the present invention, a convolutional processor for a spiking neural network includes 2 first computing units and a first adder. One of the two first computing units generates a 10-bit first convolution result based on 4 first features, and the other first computing unit generates an 11-bit first convolution result based on 5 first features. One first computing unit includes 5 lookup table logic circuits each having 4 input terminals and 2 output terminals, and the other first computing unit includes 6 lookup table logic circuits each having 5 input terminals and 2 output terminals. In this case, a 3×3 convolution calculation is implemented using the convolutional processor of the spiking neural network, and a 3×3 convolution calculation is quickly performed using 11 lookup table logic circuits and 1 adder. Compared with the related art which requires 9 adders, that is, 9 logic units, the amount of adder data used in the present invention is greatly reduced, and only 3 logic units are required, so fewer logic units are used. The large number of lookup circuit resources, logic gates, flip-flop resources, etc. in the accelerator hardware computing resources are fully utilized. The influence of resource utilization rate and fan-out is balanced. Compared with the related art, the resource utilization rate is improved, the number of adder stages is reduced, and the inference calculation efficiency is improved.

[0102] The convolutional processor for a spiking neural network provided by the embodiment of the present invention can be used for 3×3 convolution. When performing 3×3 convolution calculation, the first computing unit can be 1.

[0103] When implementing 3×3 convolution using 1 first computing unit, the number of pulse convolution results corresponding to 9 first features and 9 first weights can be 512. The number of bits of the maximum value among the 512 pulse convolution results is 12 bit.

[0104] When performing 3×3 convolution calculation, if no adder is used and only lookup circuits are used, since the input terminals of some lookup circuits, such as the lookup table logic circuits in FPGA, are at most 6, which does not meet the input width of 9 pulse signals, a carry problem will occur. Therefore, a design as shown in Figure 11 is required to implement the convolution calculation.

[0105] Figure 11 FIG. shows a schematic structural diagram of a convolutional processor for a spiking neural network according to the seventh embodiment of the present invention.

[0106] As Figure 11As shown, the convolutional processor for the spiking neural network may include a first computing unit 210C of the third type. The first computing unit 210C of the third type includes K lookup circuits. The lookup circuits in the first computing unit 210C of the third type have a structure different from that of the lookup circuits in the foregoing embodiments, which is here referred to as the second type of lookup circuit and is represented by 211 to distinguish it from the lookup circuits in the foregoing embodiments (also referred to as the first type of lookup circuit). For the sake of simplicity, Figure 11 only the first lookup circuit 211 is marked in Figure 11 . The remaining lookup circuits have similar structures and will not be elaborated here.

[0107] In the first computing unit 210C of the third type, 2 N1 pulse convolution results based on N1 first weights may be pre-stored. The first computing unit 210 may be used to receive N1 first features and look up the pulse convolution result matching the N1 first features from the 2 N1 pulse convolution results as the first convolution result. Wherein, N1 is an integer greater than 1.

[0108] The N1 first features are divided into a first group and a second group. For example, the first group of 9 first features may be Figure 8 the 4 first features at 00, 01, 02, and 10 in the first feature matrix 110 in Figure 8 . The second group of 9 first features may be Figure 8 the 5 first features at 11, 12, 20, 21, and 22 in the first feature matrix 110 in Figure 8 . The k-th part of the first convolution result includes a first sub-part and a second sub-part.

[0109] The second type of lookup circuit 211 may include a first sub-circuit 2111 and a second sub-circuit 2112.

[0110] In the k-th second type of lookup circuit, the first sub-circuit 2111 may store the first sub-part in the k-th part of each pulse convolution result and the carry flag corresponding to the first sub-part in the k-th part of each pulse convolution result. In Figure 11 Figure 11 , "jin" represents the carry flag. The second sub-circuit 2112 may store the second sub-part in the k-th part of each pulse convolution result and the carry flag corresponding to the second sub-part in the k-th part of each pulse convolution result. The first sub-circuit 2111 may determine the first sub-part and the carry flag in the k-th part of the first convolution result according to the first group of first features. The second sub-circuit 2112 may be used to determine the second sub-part in the k-th part of the first convolution result according to the carry flag output by the first sub-circuit 2111 and the second group of first features.

[0111] According to an embodiment of the present invention, the first sub - circuit determining the first sub - part of the first convolution result and the carry flag according to the first group of first features may include: determining the first group of first features as the first target query address; according to the first target query address, searching for the first sub - part in the k - th part of each stored pulse convolution result that matches the first group of first features as the first sub - part in the k - th part of the first convolution result, and searching for the carry flag that matches the first sub - part in the k - th part of the first convolution result from the carry flags corresponding to the first sub - parts in the k - th part of each pulse convolution result.

[0112] According to an embodiment of the present invention, the second sub - circuit determining the second sub - part of the first convolution result according to the carry flag output by the first sub - circuit and the second group of first features may include: determining the second group of first features and the carry flag output by the first sub - circuit as the second target query address; according to the second target query address, searching for the second sub - part in the k - th part of each stored pulse convolution result that matches the second group of first features as the second sub - part in the k - th part of the first convolution result.

[0113] According to an embodiment of the present invention, a comparator may also be used to compare the first convolution result with a preset threshold, and when the first convolution result is greater than or equal to the preset threshold, output a pulse signal to generate a pulse signal only for the second output that meets the requirements.

[0114] According to an embodiment of the present invention, since the N1 first features are divided into a first group and a second group, the k - th part of the first convolution result includes a first sub - part and a second sub - part, and the k - th search circuit includes a first sub - circuit and a second sub - circuit. Therefore, the first sub - circuit can be used to determine the first sub - part in the k - th part of the first convolution result and the carry flag according to the first group of first features, and the second sub - circuit can be used to determine the second sub - part in the k - th part of the first convolution result according to the carry flag output by the first sub - circuit and the second group of first features. Furthermore, when N1 is greater than the number of input terminals of the smallest lookup - table logic circuit in the accelerator hardware computing resources, without using adders and multipliers, two smallest lookup - table logic circuits can be used to quickly search for the k - th part of the first convolution result, and 2K smallest lookup - table logic circuits can be used to quickly search for the first convolution result, reducing the usage amount of adders in the related art. And since the smallest logic unit of the accelerator hardware computing resources includes more lookup - table logic circuits and fewer adders, while reducing the usage amount of adders, the usage amount of the smallest logic unit can be reduced, the occupancy of the accelerator hardware computing resources can be reduced, the fan - out is small, the design area of the related hardware, and the utilization rate of the accelerator hardware computing resources can be improved.

[0115] For example, Figure 10When the convolutional processor of the spiking neural network is used for 3×3 convolution, N1 = 9, M = 12, K = 4, the first sub-circuit 2111 has 4 input terminals and 2 output terminals, and the second sub-circuit 2112 has 6 input terminals and 2 output terminals, where M is the number of bits of the first convolution result.

[0116] According to an embodiment of the present invention, the first sub-circuit may have 4 input terminals and 2 output terminals. One output terminal of the first sub-circuit outputs 1 bit of the first convolution result, and the other output terminal of the first sub-circuit outputs a carry flag. The second sub-circuit may have 6 input terminals and 2 output terminals. The 2 output terminals of the second sub-circuit output 2 bits of data in the first convolution result. Therefore, the first sub-circuit and the second sub-circuit can output 3 bits of data in the first convolution result, that is, the k-th lookup circuit can output 3 bits of data in the first convolution result.

[0117] According to an embodiment of the present invention, the first sub-circuit may also have 5 input terminals and 2 output terminals. Alternatively, the first sub-circuit may also have 6 input terminals and 2 output terminals. Among them, when the first sub-circuit has 5 input terminals or 6 input terminals, only 4 input terminals in the first sub-circuit may be used.

[0118] According to an embodiment of the present invention, in the case where the convolutional processor of the spiking neural network includes 1 first computing unit, N1 = 9, M = 12, K = 4, the first sub-circuit has 4 input terminals and 2 output terminals, and the second sub-circuit has 6 input terminals and 2 output terminals, and M is the number of bits of the first convolution result, 3×3 convolution calculation is implemented by using the convolutional processor of the spiking neural network, and 3×3 convolution calculation is quickly performed by using 4 lookup circuits. Compared with the related technology that requires 9 adders, that is, 9 logic units, the present invention can not use adders and only requires 2 logic units, and uses fewer logic units. The large number of lookup circuit resources in the accelerator hardware computing resources are fully utilized. Compared with the related technology, the resource utilization rate is improved, the number of adder stages is reduced, and the inference calculation efficiency is improved.

[0119] For example, the first sub-circuit 2111 and the second sub-circuit 2112 may both be lookup table logic circuits. The first sub-circuit 2111 and the second sub-circuit 2112 may both be Figure 6 the lookup table logic circuits shown.

[0120] According to an embodiment of the present invention, in the convolutional processor of the spiking neural network, there is 1 first computing unit, N1 = 9, M = 12, K = 4. The first sub-circuit has 4 input terminals and 2 output terminals, and the second sub-circuit has 6 input terminals and 2 output terminals. When M is the number of bits of the first convolution result and both the first sub-circuit and the second sub-circuit are lookup table logic circuits, the convolutional processor of the spiking neural network is used to implement 3×3 convolution calculation, and 3×3 convolution calculation is quickly implemented using 4 lookup table logic circuits. Compared with the related art that requires 9 adders, that is, 9 logic units are used, the present invention can not use adders and only requires 2 logic units, using fewer logic units. It makes full use of a large number of lookup circuit resources in the accelerator hardware computing resources. Compared with the related art, it improves the resource utilization rate, reduces the number of adder stages, and improves the inference calculation efficiency.

[0121] Based on the above convolutional processor for the spiking neural network, an embodiment of the present invention further provides a convolutional processing method for the spiking neural network.

[0122] Figure 12 The flowchart of the convolutional processing method for the spiking neural network according to an embodiment of the present invention is shown. Figure 12 The shown convolutional processing method for the spiking neural network can be executed by the above convolutional processor for the spiking neural network.

[0123] As Figure 12 shown, the convolutional processing method for the spiking neural network may include operations S1210 to S1220.

[0124] In operation S1210, the first computing unit receives N1 first features.

[0125] In operation S1220, the first computing unit looks up, from the 2 N1 pre-stored pulse convolution results based on N1 first weights, the pulse convolution result that matches the N1 first features as the first convolution result. Among them, the pulse convolution result includes K parts, the first computing unit includes K lookup circuits, the kth lookup circuit stores the kth part of each pulse convolution result, the kth lookup circuit is any lookup circuit among the K lookup circuits, and the kth part is the part corresponding to the kth lookup circuit among the K parts

[0126] For operation S1220, from the 2 N1Among the pulse convolution results, finding the pulse convolution result that matches N1 first features as the first convolution result may include: the k-th search circuit searches for the k-th part that matches the N1 first features from the k-th part of each stored pulse convolution result as the k-th part of the first convolution result, where 1 ≤ k ≤ K and k is an integer, and N1 and K are integers greater than 1.

[0127] According to an embodiment of the present invention, the search circuit has N1 input terminals and m output terminals, where M = mK, M is the number of bits of the first convolution result, and m is the number of bits of each part of the first convolution result.

[0128] From the pre-stored 2 N1 pulse convolution results based on N1 first weights, finding the pulse convolution result that matches N1 first features as the first convolution result includes: the k-th search circuit respectively receives N1 first features from the N1 input terminals, searches for the k-th part that matches the N1 first features from the k-th part of each stored pulse convolution result as the k-th part of the first convolution result, and outputs the m bits of the k-th part of the first convolution result through the m output terminals respectively.

[0129] According to an embodiment of the present invention, the search circuit is a lookup table logic circuit.

[0130] According to an embodiment of the present invention, there are multiple first calculation units, and the convolution processor further includes a first adder. The convolution processing method for the spiking neural network further includes: the first adder sums the first convolution results output by the multiple first calculation units to obtain a first output.

[0131] According to an embodiment of the present invention, the convolution processor further includes a second calculation unit and a second adder. The convolution processing method for the spiking neural network further includes: the second calculation unit receives N2 second features, performs a convolution calculation on the received N2 second features and N2 second weights to obtain a second convolution result, where N2 is an integer greater than or equal to 1; the second adder sums the first output and the second convolution result provided by the second calculation unit to obtain a second output.

[0132] According to an embodiment of the present invention, the convolution processor is used for 3×3 convolution, the number of first calculation units is 2, N1 = 4, N2 = 1, M = 10, K = 5, and M is the number of bits of the first convolution result. The search circuit has 4 input terminals and 2 output terminals.

[0133] The k-th lookup circuit looks up the k-th part that matches N1 first features from the k-th parts of the stored respective pulse convolution results. The k-th part of the first convolution result includes: the k-th lookup circuit receives 4 first features respectively through 4 input terminals of the k-th lookup circuit, looks up the k-th part that matches the 4 first features from the k-th parts of the stored respective pulse convolution results as the k-th part of the first convolution result, and outputs 2 bits of the k-th part of the first convolution result respectively through 2 output terminals of the k-th lookup circuit.

[0134] According to an embodiment of the present invention, N2 = 1, and the second calculation unit includes a selector. The second calculation unit receives N2 second features, performs convolution calculation on the received N2 second features and N2 second weights to obtain a second convolution result, which includes: the selector selects between N2 second weights and a preset reference value according to the N2 second features to obtain the second convolution result.

[0135] According to an embodiment of the present invention, N1 first features are divided into a first group and a second group. The k-th part of the first convolution result includes a first sub-part and a second sub-part, and the k-th lookup circuit includes a first sub-circuit and a second sub-circuit. The k-th lookup circuit looks up the k-th part that matches N1 first features from the k-th parts of the stored respective pulse convolution results. The k-th part of the first convolution result includes: the first sub-circuit determines the first sub-part and a carry flag in the k-th part of the first convolution result according to the first group of first features, and the second sub-circuit determines the second sub-part in the k-th part of the first convolution result according to the carry flag output by the first sub-circuit and the second group of first features.

[0136] According to an embodiment of the present invention, both the first sub-circuit and the second sub-circuit are lookup table logic circuits.

[0137] According to an embodiment of the present invention, the convolution processor is used for 3×3 convolution, N1 = 9, M = 12, K = 4. The first sub-circuit has 4 input terminals and 2 output terminals, and the second sub-circuit has 6 input terminals and 2 output terminals, where M is the number of bits of the first convolution result.

[0138] According to an embodiment of the present invention, the convolution processor is used for 3×3 convolution, and the number of first calculation units is 2. One of the two first calculation units is used to generate a 10-bit first convolution result according to 4 first features, and the other of the two first calculation units is used to generate an 11-bit first convolution result according to 5 first features. One first calculation unit includes 5 lookup circuits each having 4 input terminals and 2 output terminals, and the other first calculation unit includes 6 lookup circuits each having 5 input terminals and 2 output terminals.

[0139] According to an embodiment of the present invention, the convolutional processor further includes: a comparator. The method for convolutional processing of a spiking neural network further includes: the comparator compares the second output with a preset threshold, and outputs a pulse signal when the second output is greater than or equal to the preset threshold.

[0140] Figure 13 The structural block diagram of an electronic device according to an embodiment of the present invention is shown.

[0141] As Figure 13 shown, the electronic device 1300 may include a convolutional processor 1310 for a spiking neural network. The electronic device 1300 may be a server for computing convolution or a server cluster for computing convolution.

[0142] The electronic device 1300 may include multiple convolutional processors 1310 for a spiking neural network, which are used to perform parallel calculations on multiple first features and multiple first weights, further improving the calculation efficiency.

[0143] Those skilled in the art can understand that the features described in various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0144] The embodiments of the present invention have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present invention, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present invention.

Claims

1. A convolutional processor for a spiking neural network, characterized in that The convolutional processor includes: The first computing unit, in which 2 pulse convolution results based on N1 first weights are pre-stored, and the first computing unit is configured to receive N1 first features and find, from the 2 pulse convolution results, the pulse convolution result that matches the N1 first features as the first convolution result; N1 2 N1 pulse convolution results, and find the pulse convolution result that matches the N1 first features as the first convolution result; Among them, the pulse convolution result includes K parts. The first calculation unit includes K lookup circuits. The k-th lookup circuit stores the k-th part of each pulse convolution result. The k-th lookup circuit is any one of the K lookup circuits. The k-th part is the part corresponding to the k-th lookup circuit among the K parts. The k-th lookup circuit is used to find the k-th part that matches the N1 first features from the stored k-th parts of each pulse convolution result as the k-th part of the first convolution result, where 1 ≤ k ≤ K and k is an integer, and N1 and K are integers greater than 1. Among them, the lookup circuit has N1 input terminals and m output terminals, M = mK, M is the number of bits of the first convolution result, m is the number of bits of each part of the first convolution result, and the k-th lookup circuit outputs the m bits of the k-th part of the first convolution result through m output terminals respectively. Alternatively, the N1 first features are divided into a first group and a second group. The k-th part of the first convolution result includes a first sub-part and a second sub-part. The k-th lookup circuit includes a first sub-circuit and a second sub-circuit. The first sub-circuit is used to determine the first sub-part and the carry flag in the k-th part of the first convolution result according to the first group of first features. The second sub-circuit is used to determine the second sub-part in the k-th part of the first convolution result according to the carry flag output by the first sub-circuit and the second group of first features.

2. The convolutional processor according to claim 1, wherein In the case where the lookup circuit has N1 input terminals and m output terminals, where M = mK, M is the number of bits of the first convolution result, and m is the number of bits of each part of the first convolution result; The k-th lookup circuit is used to receive the N1 first features from the N1 input terminals respectively, and find the k-th part that matches the N1 first features from the stored k-th parts of each pulse convolution result as the k-th part of the first convolution result.

3. The convolutional processor according to claim 2, characterized in that, The lookup circuit is a lookup table logic circuit.

4. The convolutional processor according to claim 2, wherein There are multiple first calculation units. The convolutional processor further includes: a first adder, which is used to sum the first convolution results output by the multiple first calculation units to obtain a first output.

5. The convolutional processor according to claim 4, wherein, The convolutional processor further includes: A second calculation unit, which is used to receive N2 second features, perform a convolution calculation on the received N2 second features and N2 second weights to obtain a second convolution result, where N2 is an integer greater than or equal to 1; A second adder, which is used to sum the first output and the second convolution result provided by the second calculation unit to obtain a second output.

6. The convolutional processor according to claim 5, characterized in that, The convolutional processor is used for 3×3 convolution. The number of first calculation units is 2, N1 = 4, N2 = 1, M = 10, K = 5, and M is the number of bits of the first convolution result; The lookup circuit has 4 input terminals and 2 output terminals. Among them, the k-th lookup circuit is used to receive 4 first features respectively through the 4 input terminals of the k-th lookup circuit, look up the k-th part that matches the 4 first features from the k-th part of each pulse convolution result stored, as the k-th part of the first convolution result, and output 2 bits of the k-th part of the first convolution result respectively through the 2 output terminals of the k-th lookup circuit.

7. The convolutional processor according to claim 5, wherein N2 = 1. The second calculation unit includes: a selector, which is used to select between N2 second weights and a preset reference value according to N2 second features to obtain a second convolution result.

8. The convolutional processor according to claim 1, wherein Both the first sub-circuit and the second sub-circuit are lookup table logic circuits.

9. The convolutional processor according to claim 1, characterized in that, When the N1 first features are divided into a first group and a second group, the k-th part of the first convolution result includes a first sub-part and a second sub-part, and the k-th lookup circuit includes a first sub-circuit and a second sub-circuit, the convolution processor is used for 3×3 convolution, N1 = 9, M = 12, K = 4. The first sub-circuit has 4 input terminals and 2 output terminals, and the second sub-circuit has 6 input terminals and 2 output terminals. Among them, M is the number of bits of the first convolution result.

10. The convolutional processor according to claim 1, characterized in that The convolution processor is used for 3×3 convolution, and the number of the first calculation units is 2. The convolution processor includes a first-type first calculation unit and a second-type first calculation unit. When the lookup circuit has N1 input terminals and m output terminals, M = mK, M is the number of bits of the first convolution result, and m is the number of bits of each part of the first convolution result, the first calculation unit is the first-type first calculation unit. The first-type first calculation unit is used to generate a 10-bit first convolution result according to 4 first features. When the first calculation unit is the second-type first calculation unit, the second-type first calculation unit is used to generate an 11-bit first convolution result according to 5 first features. The first-type first calculation unit includes 5 lookup circuits with 4 input terminals and 2 output terminals, and the second-type first calculation unit includes 6 lookup circuits with 5 input terminals and 2 output terminals; The convolution processor further includes: a first adder, which is used to sum the first convolution results output by the first-type first calculation unit and the second-type first calculation unit to obtain a first output.

11. The convolutional processor according to claim 5, characterized in that, The convolution processor further includes: a comparator, which is used to compare the second output with a preset threshold, and output a pulse signal when the second output is greater than or equal to the preset threshold.

12. A convolutional processing method for a spiking neural network, which is executed by the convolutional processor according to claim 1, characterized in that, The method includes: The first calculation unit receives N1 first features; The first calculation unit searches for the pulse convolution result that matches the N1 first features from the pre-stored 2 pulse convolution results based on N1 first weights as the first convolution result; N1 among the pulse convolution results, the pulse convolution result that matches the N1 first features is found as the first convolution result; Among them, the pulse convolution result includes K parts, the first calculation unit includes K lookup circuits, the k-th lookup circuit stores the k-th part of each pulse convolution result, the k-th lookup circuit is any one of the K lookup circuits, the k-th part is the part corresponding to the k-th lookup circuit among the K parts, and from the pre-stored 2 N1 pulse convolution results based on N1 first weights, looking up the pulse convolution result that matches the N1 first features as the first convolution result includes: the k-th lookup circuit looks up the k-th part that matches the N1 first features from the k-th part of each stored pulse convolution result as the k-th part of the first convolution result, where 1 ≤ k ≤ K and k is an integer, and N1 and K are integers greater than 1; Among them, the lookup circuit has N1 input terminals and m output terminals, M = mK, M is the number of bits of the first convolution result, m is the number of bits of each part of the first convolution result, and the k-th lookup circuit outputs m bits of the k-th part of the first convolution result respectively through m output terminals; Alternatively, the N1 first features are divided into a first group and a second group. The k-th part of the first convolution result includes a first sub-part and a second sub-part. The k-th lookup circuit includes a first sub-circuit and a second sub-circuit. The k-th lookup circuit looks up the k-th part that matches the N1 first features from the k-th parts of the stored respective pulse convolution results as the k-th part of the first convolution result, which includes: the first sub-circuit determines the first sub-part and a carry flag in the k-th part of the first convolution result according to the first group of first features, and the second sub-circuit determines the second sub-part in the k-th part of the first convolution result according to the carry flag output by the first sub-circuit and the second group of first features.

13. The method according to claim 12, wherein When the lookup circuit has N1 input terminals and m output terminals, where M = mK, M is the number of bits of the first convolution result, and m is the number of bits of each part of the first convolution result, the process of looking up, from the pre-stored 2 N1 pulse convolution results based on N1 first weights, for a pulse convolution result that matches the N1 first features as the first convolution result includes: The k-th lookup circuit receives the N1 first features from N1 input terminals respectively, and looks up the k-th part that matches the N1 first features from the k-th parts of the stored respective pulse convolution results as the k-th part of the first convolution result.

14. The method according to claim 13, wherein There are multiple first calculation units, and the convolution processor further includes a first adder. The method further includes: the first adder sums the first convolution results output by the multiple first calculation units to obtain a first output.

15. The method according to claim 14, characterized in that, The convolution processor further includes a second calculation unit and a second adder. The method further includes: The second calculation unit receives N2 second features, performs convolution calculation on the received N2 second features and N2 second weights to obtain a second convolution result, where N2 is an integer greater than or equal to 1. The second adder sums the first output and the second convolution result provided by the second calculation unit to obtain a second output.

16. The method according to claim 15, characterized in that, The convolution processor is used for 3×3 convolution, the number of the first calculation units is 2, N1 = 4, N2 = 1, M = 10, K = 5, and M is the number of bits of the first convolution result. The lookup circuit has 4 input terminals and 2 output terminals. Among them, the k-th lookup circuit looks up the k-th part that matches the N1 first features from the k-th parts of the stored respective pulse convolution results as the k-th part of the first convolution result, which includes: the k-th lookup circuit receives 4 first features through the 4 input terminals of the k-th lookup circuit respectively, looks up the k-th part that matches the 4 first features from the k-th parts of the stored respective pulse convolution results as the k-th part of the first convolution result, and outputs the 2 bits of the k-th part of the first convolution result through the 2 output terminals of the k-th lookup circuit respectively.

17. The method according to claim 15, characterized in that, N2 = 1, the second calculation unit includes a selector. The second calculation unit receives N2 second features, and the convolution calculation of the received N2 second features and N2 second weights to obtain a second convolution result includes: The selector selects between the N2 second weights and a preset reference value according to the N2 second features to obtain a second convolution result.

18. The convolution processing method according to claim 15, wherein The convolution processor further includes: a comparator. The method further includes: The comparator compares the second output with a preset threshold, and outputs a pulse signal when the second output is greater than or equal to the preset threshold.

19. An electronic device, comprising the convolutional processor according to any one of claims 1 to 11.

Images