Convolution calculation method and related device

Abstract

The embodiment of the present application discloses a convolution calculation method, which includes: a data processing device obtains convolution kernel data and generates a target space matrix, and the data processing device generates a convolution result according to the sub-unit combination relationship of the intermediate matrix, the intermediate matrix is ​​calculated by the target space matrix and the reference matrix, the reference matrix indicates the pixel information of the picture, and the sub-unit combination relationship of the intermediate matrix is ​​the relationship between the sub-units of the intermediate matrix in the process of calculating the convolution result. Generating the convolution result according to the sub-unit combination relationship of the intermediate matrix greatly reduces the occupation of the internal storage space.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular to a convolution calculation method and related devices. Background Art

[0002] In recent years, artificial intelligence (AI) has achieved explosive growth. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. Since the birth of artificial intelligence, the theory and technology of artificial intelligence have become increasingly mature, and the application field has continued to expand. It can be imagined that the technological products brought by artificial intelligence in the future will be the "containers" of human wisdom.

[0003] Along with the third wave of artificial intelligence development, Winograd FFT (WINO algorithm) has been widely used in the accelerated calculation of convolutional neural network CNN. The WINO algorithm maps the input calculation data to another space and replaces some multiplication operations with addition and subtraction operations. Since the speed of addition and subtraction operations is much higher than that of multiplication operations in the hardware implementation process, a significant acceleration effect is achieved.

[0004] However, when implementing the WINO algorithm, it is necessary to use the three-dimensional matrix CUBE module to complete the WINO general matrix multiplication (GEMM) operation and the vector (VEC) module to complete the WINO inverse calculation. During the operation, the WINO GEMM operation will be completed first, and its result will be saved and used for the WINO Inverse calculation. Saving the result of the WINO GEMM operation requires a large amount of storage space. Summary of the invention

[0005] A first aspect of an embodiment of the present application provides a convolution calculation method, including:

[0006] The data processing device obtains convolution kernel data, which is a third-order matrix, and generates a target space matrix, which is mapped from the convolution kernel data, wherein the target space matrix is ​​a fourth-order matrix. The data processing device generates a convolution result according to the sub-unit combination relationship of the intermediate matrix, which includes four result units. The intermediate matrix includes 16 sub-units. The intermediate matrix is ​​calculated by the target space matrix and the reference matrix. The reference matrix indicates the pixel information of the image. The sub-unit combination relationship of the intermediate matrix is ​​the relationship between the sub-units of the intermediate matrix in the process of calculating the convolution result.

[0007] The present application scheme generates convolution results according to the sub-unit combination relationship of the intermediate matrix, which greatly reduces the occupancy of the internal storage space, and the WINO GEMM calculation results are used synchronously, reducing the data interaction between VEC and the internal storage space, and reducing the data reading and writing of the internal storage space.

[0008] Based on the first aspect of the embodiments of the present application, in a first implementation manner of the first aspect of the embodiments of the present application, the 16 sub-units of the intermediate matrix are the first sub-unit to the sixteenth sub-unit, and the sorting order of the first sub-unit to the sixteenth sub-unit is according to their positions in the intermediate matrix, from the first row and the first column to the fourth row and the fourth column, such as the first sub-unit is the first sub-unit of the first row of the intermediate matrix, the second sub-unit is the second sub-unit of the first row, and so on.

[0009] An embodiment of the present application provides a sub-unit sorting order of an intermediate matrix.

[0010] Based on the first implementation manner of the first aspect of the embodiments of the present application, in the second implementation manner of the first aspect of the embodiments of the present application, the sub-unit combination relationship of the intermediate matrix includes: the fifth sub-unit and the ninth sub-unit are a group, the sixth sub-unit and the tenth sub-unit are a group, the eighth sub-unit and the twelfth sub-unit are a group, and the seventh sub-unit and the eleventh sub-unit are a group.

[0011] Based on any one of the first aspect of the embodiments of the present application to the second aspect of the first aspect of the embodiments of the present application, in the third aspect of the first aspect of the embodiments of the present application, the specific process of the data processing device generating a convolution result according to the sub-unit combination relationship of the intermediate matrix can be that the data processing device calculates the target sub-unit, the target sub-unit belongs to the sub-unit of the intermediate matrix, the data processing device determines whether the target sub-unit needs to be directly added or subtracted with the current data, the current data is the data obtained in the previous unit time before calculating the target sub-unit, if so, the data processing device performs addition or subtraction operations on the target sub-unit and the current data; if not, the data processing device saves the target sub-unit.

[0012] An embodiment of the present application provides a specific process in which a data processing device generates a convolution result according to a sub-unit combination relationship of an intermediate matrix.

[0013] A second aspect of an embodiment of the present application provides a data processing device, which executes the method of the aforementioned first aspect and any implementation method of the first aspect.

[0014] A third aspect of an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method of the first aspect and any implementation method of the first aspect.

[0015] A fourth aspect of the embodiments of the present application provides a computer software product. When the computer program product is executed on a computer, it enables the computer to execute the method of the first aspect and any one of the implementation methods of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of a traditional convolution calculation process in an embodiment of the present application;

[0017] Figure 2 This is a schematic diagram of the convolution calculation process corresponding to the WINO algorithm in the embodiment of the present application;

[0018] Figure 3 This is a structural diagram of a data processing device in an embodiment of the present application;

[0019] Figure 4 A schematic diagram of the calculation sequence of the sub-units of the intermediate matrix in the embodiment of the present application;

[0020] Figure 5 A schematic diagram of a convolution calculation method in an embodiment of the present application;

[0021] Figure 6 Another schematic diagram of the calculation order of the sub-units of the intermediate matrix in the embodiment of the present application;

[0022] Figure 7 A schematic diagram of a process of the WINO algorithm in an embodiment of the present application;

[0023] Figure 8 This is another structural schematic diagram of the data processing device in the embodiment of the present application;

[0024] Fig. 9 This is another structural schematic diagram of the data processing device in the embodiment of the present application;

[0025] Fig.10 This is another structural schematic diagram of the data processing device in the embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the present application, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there may be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or its similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b and c can be single or multiple. It is worth noting that "at least one" can also be interpreted as "one or more items".

[0027] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0028] The first, second, etc. descriptions appearing in the embodiments of the present application are only used for illustration and distinction of the description objects. There is no order, nor do they indicate any special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0029] The embodiment of the present application provides a convolution calculation method, which is suitable for WINO calculation with a convolution kernel size of 3*3, and reduces the occupation of data storage space during the calculation process.

[0030] The data processing device in the embodiment of the present application may be a terminal device, also known as user equipment (UE), mobile station (MS), mobile terminal (MT), etc., which refers to a device that provides voice and / or data connectivity to users. For example, a handheld device with wireless connection function, a vehicle-mounted device, etc. At present, some examples of terminals are: mobile phones, tablet computers, laptops, PDAs, mobile Internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc.

[0031] With the third wave of artificial intelligence development, the WINO algorithm has been widely used in the accelerated calculation of convolutional neural networks (CNNs). The WINO algorithm maps the input calculation data to another space and replaces some multiplication operations with addition and subtraction operations. Since the speed of addition and subtraction operations is much higher than that of multiplication operations in the hardware implementation process, a significant acceleration effect is achieved. For example, in the field of video surveillance processing, the WINO algorithm is used many times in the data processing process of portrait recognition after the smart camera collects portrait photos. In smart terminal devices, such as the data processing process of using the face recognition unlocking function of smart phones, the WINO algorithm is also used many times.

[0032] Taking image processing as an example, for the calculation of convolution kernel size of 3*3, Figure 1Figure 1 is a schematic diagram of the traditional calculation process. In order to obtain the four convolution calculation results r0-r3, the data processing device first performs a dot multiplication operation on the pixel value (k0, k1, k2, k4, k5, k6, k8, k9, k10) of the image input during the current WINO calculation and the convolution kernel data (Weight: w0, w1, w2, w3, w4, w5, w6, w7, w8) to obtain r0; the second time, the (k1, k2, k3, k5, k6, k7, k9, k10, k11) in the Input image and the Weight (w0, w1, w2, w3, w4, w5, w6, w7, w8) are performed a dot multiplication operation to obtain r1; the third time, the Input The dot multiplication of (k4, k5, k6, k8, k9, k10, k12, k13, k14) in the image and Weight (w0, w1, w2, w3, w4, w5, w6, w7, w8) is performed to obtain r2; the fourth time, the dot multiplication of (k5, k6, k7, k9, k10, k11, k13, k14, k15) in the Input image and (w0, w1, w2, w3, w4, w5, w6, w7, w8) is performed to obtain r3. Therefore, the data processing device needs to perform 4*9=36 multiplication operations in total.

[0033] Figure 2 The following is a calculation diagram of the WINO algorithm. For the WINO calculation with a convolution kernel size of 3*3, the calculation process includes:

[0034] 1. Data transformation operation. The data processing device maps the 3*3 convolution kernel data (Weight) to the 4*4 target space matrix ( Figure 2 The U space matrix in the image is used to map the 4*4 Input image data to the 4*4 V space matrix through the BT / B matrix operation.

[0035] 2. WINO GEMM operation. The data processing device multiplies the u data and v data at corresponding positions of the 4*4 U space matrix and the 4*4 V space matrix to obtain a 4*4 intermediate matrix (U*V matrix). The intermediate matrix includes 16 sub-units s0 to s15, where each sub-unit is obtained by multiplying the u data and the v data, for example, u0*v0=s0, u1*v1=s1, etc.

[0036] 3. WINO Inverse calculation: The 4*4 intermediate matrix is ​​operated with the AT / A matrix to obtain a 2*2 calculation result Q matrix, which includes four result units q0 to q3.

[0037] The above data transform operation can be calculated by offline preprocessing, so the actual number of effective multiplication operations is only 4*4=16 times in the GEMM operation. Figure 1 The traditional 3*3 convolution calculation process shown in the figure can reduce the multiplication calculation complexity by 36 / 16=2.25 times. In particular, the GEMM operation and the Inverse calculation process can be expanded to obtain the following calculation expression:

[0038] q0=s0+s4+s8+s1+s5+s9+s2+s6+s10;

[0039] q1=s1+s5+s9-s2-s6-s10-s3-s7-s11;

[0040] q2=s4-s8-s12+s5-s9-s13+s6-s10-s14;

[0041] q3=s5-s9-s13-s6+s10+s14-s7+s11+s15.

[0042] Combination Figure 3 As shown in the structural diagram, the technical solution of this embodiment is designed based on the AI ​​computing acceleration engine. The CUBE module includes M*N groups of multiplication and accumulation calculation units composed of MUL / ACC, wherein the multiplication and accumulation calculation unit MUL is used to complete the WINO GEMM calculation.

[0043] When implementing the WINO algorithm, the CUBE module is needed to complete the GEMM operation, and the VEC module is needed to complete the WINO Inverse calculation. During the operation, the WINO GEMM operation will be completed first, and its result will be saved and used for the WINO Inverse calculation. Saving the result of the WINO GEMM operation requires a lot of storage space (memory), such as Figure 4 shown.

[0044] See also Figure 5 In an embodiment of the present application, a convolution calculation method includes:

[0045] 501. The data processing device obtains convolution kernel data;

[0046] Data data obtains the convolution kernel data, which is a third-order matrix. Figure 2 In this embodiment, the convolution kernel matrix is ​​represented by a Weight matrix, including: w0, w1, w2, w3, w4, w5, w6, w7, w8.

[0047] 502. The data processing device generates a target space matrix;

[0048] The target space matrix is ​​obtained by mapping the convolution kernel data. The target space matrix is ​​a fourth-order matrix. Figure 2 In this embodiment, the target space matrix is ​​represented by the U matrix, including u0 to u15.

[0049] 503. The data processing device determines the sub-unit combination relationship of the intermediate matrix;

[0050] The intermediate matrix includes the first sub-unit to the sixteenth sub-unit, a total of 16 sub-units, see Figure 2 The U*V matrix in the embodiment is represented by s0 to s15, and s0 to s15 are arranged in sequence from the first row and the first column to the fourth row and the fourth column in the intermediate matrix.

[0051] The sub-unit combination relationship of the intermediate matrix is ​​the relationship between the sub-units of the intermediate matrix in the process of calculating the convolution result, such as s5 and s9 are a group, s6 and s10 are a group, s4 and s8 are a group, and s7 and s11 are a group.

[0052] 504. The data processing device adjusts the calculation order of the sub-units of the intermediate matrix according to the combination relationship of the sub-units of the intermediate matrix;

[0053] The specific order adjustment rules are: the front and back distribution and internal order of the four groups (s5, s9), (s6, s10), (s4, s8), and (s7, s11) are interchangeable, that is, the calculation order is arranged in the front or back, and the order of the two calculations in the brackets can be interchanged; s13, s1 are distributed before and after (s5, s9); s14, s2 are distributed before and after (s6, s10); s0, s12 are distributed after (s4, s8); s3, s15 are distributed after (s7, s11); ((s4, s8), s0, s12) are distributed after (s13, (s5, s9), s1); ((s7, s11), s3, s5) are distributed after (s14, (s6, s10), s2).

[0054] Since the multiplication and addition result value of the current cycle needs to be used for accumulation / subtraction after another cycle, refer to Figure 6 In the embodiment of the present application, the calculation order of the sub-units of the intermediate matrix can be: s13->s14->s5->s6s9->s10->s1->s2->s4->s7->s8->s11->s0->s3->s12->s15.

[0055] 505. The data processing device generates a convolution result.

[0056] Figure 7It is a schematic diagram of the calculation process of the convolution result in the embodiment of the present application, wherein S_num is the s serial number corresponding to the WINO GEMM calculation at the current moment; ACC0_op represents the specific operation calculation content of the ACC0 accumulation or subtraction calculation unit at the current moment; ACC0_src0 and ACC0_src1 represent the sources of the two input operation data of the ACC0 accumulation or subtraction calculation unit at the current moment; P.SUM represents the calculation result from the WINO GEMM at the current moment, that is, it is used to save the data corresponding to S_num; Accum0 represents the calculation result from ACC0 at the previous moment; bufc represents a temporary cache space; ACC0_dest represents whether the calculation result of the ACC0 accumulation or subtraction calculation unit at the current moment needs to be written to bufc for temporary cache processing; NA represents that after the calculation is completed, the calculation result is saved in ACC0 or ACC1 and does not need to be moved to the cache; ACC1_op, ACC1_src0, ACC1_src1, and ACC1_dest have similar meanings.

[0057] During the calculation period from cycle 0 to cycle 3 of the first round (cycle0-cycle3), WINO GEMM calculations corresponding to s13 / s14 / s5 / s6 are performed respectively. During the calculation period from cycle0 to cycle3 of other rounds, WINO GEMM calculations corresponding to s13 / s14 / s5 / s6 and WINO Inverse calculations corresponding to ACC1 are performed simultaneously, and ACC0 is not calculated. During the calculation period from cycle4 to cycle15, WINO GEMM calculations corresponding to s9 / s10 / s1 / s2 / s4 / s7 / s8 / s11 / s0 / s3 / s12 / s15 and WINO Inverse calculations corresponding to ACC0 / ACC1 are performed respectively.

[0058] Since the WINO GEMM calculation results s13 and s14 corresponding to the zeroth cycle cycle0 and the first cycle cycle1 are used by ACC1 for accumulation or subtraction operations in the sixth cycle cycle6 and the seventh cycle cycle7, respectively, the WINO GEMM calculation results of s13 and s14 are first cached in the cache space bufc[0] and bufc[1] of the data processing device, respectively. Similarly, the calculation result (s1+s5+s9) of ACC0 at cycle6 is first cached in bufc[1] for calculation by ACC1 in the 13th cycle cycle13. The calculation result (s5-s9-s13) of ACC1 at the sixth cycle cycle6 is first cached in bufc[4] for calculation by ACC1 in the next cycle cycle3. The calculation result (s2+s6+s10) of ACC0 at cycle7 is first cached in bufc[0] for calculation by ACC1 in cycle12. In cycle 7, the ACC1 calculation results (s6-s10-s14) are first cached to bufc[5] for use in the next cycle 2 ACC1 calculation. In cycle 10, the ACC1 calculation results (s5-s9-s13+s4-s8) are first cached to bufc[0] for use in the next cycle 0 ACC1 calculation. In cycle 11, the ACC1 calculation results (s6-s10-s14+s7-s11) are first cached to bufc[1] for use in the next cycle 1 ACC1 calculation.

[0059] In the embodiment of the present application, only the internal storage space bufc[0] to bufc[5] is required, which greatly reduces the occupation of the internal storage space (in the prior art, s0 to s15 need to be stored in the internal storage space of the data processing device, that is, occupying bufc[0] to bufc

[15] ), and the WINO GEMM calculation results, such as the calculation result s5 of cycle2, are used in cycle4. VEC does not need to interact with the internal storage space for data, reducing the data reading and writing of the internal storage space.

[0060] Specifically, the calculation steps involved in q0=s0+s4+s8+s1+s5+s9+s2+s6+s10 are as follows:

[0061] Step 1: cycle 2, P.SUM, s5

[0062] Step 2: cycle 4, ACC0, s5+s9

[0063] Step3: cycle6, ACC0, s5+s9+s1

[0064] Step4: cycle8, ACC0, s5+s9+s1+s4

[0065] Step5: cycle10, ACC0, s5+s9+s1+s4+s8

[0066] Step6: cycle12, ACC1, s5+s9+s1+s4+s8+s2+s6+s10

[0067] Step7: cycle14, ACC1, s5+s9+s1+s4+s8+s2+s6+s10+s0

[0068] Where Step represents a step, such as Step1 represents the first step, Step2 represents the second step, and so on. Cycle represents a cycle, cycle2 represents the second cycle, cycle4 represents the fourth cycle, and so on. The multiplication and addition result value of the current cycle needs to be used for addition / subtraction operations every other cycle. For example, the result of the second cycle can be used for addition / subtraction operations in the fourth cycle, that is, if the result of the second cycle is needed in the fourth cycle, the result of the second cycle does not need to occupy the internal storage space of the data processing device for storage. (P.SUM, s5) means saving the calculation result of s5 to the product sum result (product sum, P.SUM), and (ACC0, s5+s9) means calculating s5+s9 by ACC0.

[0069] The calculation steps involved in q1=s1+s5+s9-s2-s6-s10-s3-s7-s11 are as follows:

[0070] Step 1: cycle 3, P.SUM, s6

[0071] Step2: cycle5, ACC0, s6+s10

[0072] Step3: cycle7, ACC0, s6+s10+s2

[0073] Step4: cycle9, ACC0, s6+s10+s2+s7

[0074] Step5: cycle11, ACC0, s6+s10+s2+s7+s11

[0075] Step6: cycle13, ACC1, s1+s5+s9-s6-s10-s2-s7-s11

[0076] Step7: cycle15, ACC1, s1+s5+s9-s6-s10-s2-s7-s11-s3

[0077] The calculation steps involved in q2=s4-s8-s12+s5-s9-s13+s6-s10-s14 are as follows:

[0078] Step 1: cycle 2, P.SUM, s5

[0079] Step 2: cycle 4, ACC1, s5-s9

[0080] Step3: cycle6, ACC1, s5-s9-s13

[0081] Step4: cycle8, ACC1, s5-s9-s13+s4

[0082] Step5: cycle10, ACC1, s5-s9-s13+s4-s8

[0083] Step 6: Next cycle 0, ACC1, s5-s9-s13+s4-s8-s12

[0084] Step7: Next cycle 2, ACC1, s5-s9-s13+s4-s8-s12+s6-s10-s14

[0085] The calculation steps involved in q3=s5-s9-s13-s6+s10+s14-s7+s11+s15 are as follows:

[0086] Step 1: cycle 3, P.SUM, s6

[0087] Step2: cycle5, ACC1, s6-s10

[0088] Step3: cycle7, ACC1, s6-s10-s14

[0089] Step4: cycle9, ACC1, s6-s10-s14+s7

[0090] Step5: cycle11, ACC1, s6-s10-s14+s7-s11

[0091] Step 6: Next cycle 1, ACC1, s6-s10-s14+s7-s11-s15

[0092] Step7: Next cycle3, ACC1, s5-s9-s13-s6+s10+s14-s7+s11+s15

[0093] According to the above calculation steps, the convolution results (q0, q1, q2, q3) can be obtained in cycle14 and cycle15 of the current round and cycle2 and cycle3 of the next round respectively.

[0094] The convolution calculation method in the embodiment of the present application is described above. The device in the embodiment of the present application is described below. Figure 8 In the embodiments of the present application, one embodiment of the data processing device includes:

[0095] The acquisition unit 801 is used to acquire convolution kernel data.

[0096] The generating unit 802 is used to generate a target space matrix and to generate a convolution result according to a sub-unit combination relationship of the intermediate matrix.

[0097] The generating unit 802 may include:

[0098] A computation unit is used to compute the target subunit.

[0099] The judgment unit is used to judge whether the target subunit needs to perform addition or subtraction operations directly with the current data.

[0100] The operation unit is used to perform addition and subtraction operations on the target subunit and the current data.

[0101] A save unit is used to save the target subunit.

[0102] In this embodiment, the operations performed by each unit are the same as those in the previous embodiment. Figure 5 In the illustrated embodiment, the operations performed by the data processing device are similar to those described above and will not be described in detail here.

[0103] Fig. 9 It is a structural diagram of a data processing device provided in an embodiment of the present application. The data processing device 900 may include one or more processors 901 and a memory 905. The memory 905 stores one or more application programs or data.

[0104] The memory 905 may be a volatile storage or a persistent storage. The program stored in the memory 905 may include one or more modules, each of which may include a series of instruction operations in the data processing device 900. Furthermore, the processor 901 may be configured to communicate with the memory 905 and execute a series of instruction operations in the memory 905 on the data processing device 900.

[0105] The data processing device 900 may also include one or more power supplies 902, one or more wired or wireless network interfaces 903, one or more input and output interfaces 904, and / or one or more operating systems, such as any one of Microsoft Windows, Android, Mac OS, Unix, and Linux.

[0106] The processor 901 can execute the operations performed by the data processing device in any of the aforementioned embodiments, and the details are not repeated here.

[0107] See also Fig.10 , the data and processing equipment includes a printed circuit board (PCB), on which a central processing unit (CPU), memory, an accelerated computing engine and other chips required for computing are installed. The memory module is used to cache the collected WINO computing data and the processing results of the accelerated computing engine. The CPU controls the processing process of the accelerated computing engine by issuing configuration parameters. The accelerated computing engine is an independent chip, which can be a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), or it can be integrated into a system on chip (SoC) of a data processing device; the CPU and the accelerated computing engine are interconnected using a high-speed serial computer expansion bus standard (PCIe) interface (for independent chips) or an advanced extensible interface (AXI) interface (for SoC).

[0108] See also Fig.10, the CPU in the data and processing device sends the input image and weight data required for WINO calculation to the zero-level (Level 0) input cache space A (L0A) and the zero-level (Level 0) input cache space B (L0B) cache (memory) module through the AXI / PCIe interface. The CUBE module completes the WINO GEMM and WINO Inverse calculations according to the input data content and outputs the calculation results to the zero-level (Level 0) output cache space C (L0C) cache module. The WINO calculation results are returned from the L0C cache module to the memory space of the terminal device / data center server through the AXI / PCIe interface.

[0109] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0110] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A convolution calculation method, applied to the WINO algorithm scenario, characterized in that: include: The data processing device obtains convolution kernel data, where the convolution kernel data is a third-order matrix; The data processing device generates a target space matrix, wherein the target space matrix is ​​obtained by mapping the convolution kernel data, and the target space matrix is ​​a fourth-order matrix; The data processing device generates a convolution result according to a sub-unit combination relationship of an intermediate matrix, wherein the convolution result includes four result units, the intermediate matrix includes 16 sub-units, the intermediate matrix is ​​calculated by the target space matrix and the reference matrix, the reference matrix indicates pixel information of the image, and the sub-unit combination relationship of the intermediate matrix is ​​the relationship between the sub-units of the intermediate matrix during the process of calculating the convolution result.

2. The method according to claim 1, characterized in that The 16 subunits of the intermediate matrix are the first subunit to the sixteenth subunit, and the first subunit to the sixteenth subunit are arranged in sequence from the first row and the first column to the fourth row and the fourth column according to their positions in the intermediate matrix.

3. The method according to claim 2, characterized in that The sub-unit combination relationship of the intermediate matrix includes: the fifth sub-unit and the ninth sub-unit are a group, the sixth sub-unit and the tenth sub-unit are a group, the eighth sub-unit and the twelfth sub-unit are a group, and the seventh sub-unit and the eleventh sub-unit are a group.

4. The method according to any one of claims 1 to 3, characterized in that The data processing device generates a convolution result according to the sub-unit combination relationship of the intermediate matrix, including: The data processing device calculates a target sub-unit, the target sub-unit belonging to a sub-unit of the intermediate matrix; The data processing device determines whether the target subunit needs to be directly added or subtracted with current data, where the current data is the data obtained in the previous unit time before the target subunit is calculated; If yes, the data processing device performs addition and subtraction operations on the target subunit and the current data; If not, the data processing device saves the target subunit.

5. A data processing device, applied to the WINO algorithm scenario, characterized in that: include: An acquisition unit, used to acquire convolution kernel data, wherein the convolution kernel data is a third-order matrix; A generating unit, used for generating a target space matrix, wherein the target space matrix is ​​obtained by mapping the convolution kernel data, and the target space matrix is ​​a fourth-order matrix; The generation unit is also used to generate a convolution result according to the sub-unit combination relationship of the intermediate matrix, the convolution result includes four result units, the intermediate matrix includes 16 sub-units, the intermediate matrix is ​​calculated by the target space matrix and the reference matrix, the reference matrix indicates the pixel information of the picture, and the sub-unit combination relationship of the intermediate matrix is ​​the relationship between the sub-units of the intermediate matrix in the process of calculating the convolution result.

6. The device according to claim 5, characterized in that The 16 subunits of the intermediate matrix are the first subunit to the sixteenth subunit, and the first subunit to the sixteenth subunit are arranged in sequence from the first row and the first column to the fourth row and the fourth column according to their positions in the intermediate matrix.

7. The device according to claim 6, characterized in that The sub-unit combination relationship of the intermediate matrix includes: the fifth sub-unit and the ninth sub-unit are a group, the sixth sub-unit and the tenth sub-unit are a group, the eighth sub-unit and the twelfth sub-unit are a group, and the seventh sub-unit and the eleventh sub-unit are a group.

8. The device according to any one of claims 5 to 7, characterized in that The generating unit comprises: A calculation unit, used for calculating a target sub-unit, wherein the target sub-unit belongs to a sub-unit of the intermediate matrix; A judging unit, used to judge whether the target subunit needs to be directly added or subtracted with current data, where the current data is the data obtained in the previous unit time before the target subunit is calculated; If yes, the operation unit performs addition and subtraction operations on the target subunit and the current data; If not, the storage unit stores the target subunit.

9. A computer storage medium, characterized in that: The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 4.

10. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 4.

Images