Hardware acceleration method, device, system and readable storage medium for pooling algorithm

Abstract

The present invention discloses a hardware acceleration method, device, system and readable storage medium of a pooling algorithm. The method comprises: when a calculation instruction is detected, non-filled data in a calculation window is obtained through a data sorting module, and the non-filled data is sent to a first calculation module; first filling data is input into the first calculation module through a control module, and a first calculation result is obtained through the first calculation module according to the non-filled data and the first filling data; the first calculation result is input into a second calculation module through the first calculation module, and second filling data is input into the second calculation module through the control module; a second calculation result is obtained through the second calculation module according to the first calculation result and the second filling data; the present invention inputs the first filling data into the first calculation module and the second filling data into the second calculation module through the control module, thereby improving the performance of the hardware accelerator and thus improving the calculation speed of the pooling algorithm.

Description

Technical Field

[0001] The present invention relates to the field of hardware design technology, and in particular to a hardware acceleration method, device, system and readable storage medium for a pooling algorithm. Background Art

[0002] The pooling algorithm is a very commonly used algorithm in neural network calculations. It reduces the number of output feature vectors by downsampling. Most existing hardware accelerators in neural networks are implemented by continuously reading padding data from the memory. Storing the padding data in the memory takes up storage space, and reading the padding data from the memory and sending it to the calculation module increases the transmission time. In addition, the padding data needs to be calculated separately when the pooling algorithm is calculated, which also requires a lot of calculation time, greatly reducing the performance of the hardware accelerator, and thus reducing the calculation speed of the pooling algorithm.

[0003] Therefore, how to improve the performance of hardware accelerators to increase the calculation speed of pooling algorithms is an urgent problem to be solved. Summary of the invention

[0004] The main purpose of the present invention is to propose a hardware acceleration method, device, system and readable storage medium for a pooling algorithm, aiming to solve the problem of how to improve the performance of a hardware accelerator to increase the calculation speed of a pooling algorithm.

[0005] To achieve the above object, the present invention provides a hardware acceleration method for a pooling algorithm, and the hardware acceleration method for the pooling algorithm comprises the following steps:

[0006] When a calculation instruction is detected, non-filled data in the calculation window is acquired through a data sorting module, and the non-filled data is sent to the first calculation module;

[0007] Inputting first filling data to the first calculation module through a control module, and obtaining a first calculation result according to the non-filling data and the first filling data through the first calculation module;

[0008] Inputting the first calculation result into a second calculation module through the first calculation module, and inputting second filling data into the second calculation module through the control module;

[0009] A second calculation result is obtained by the second calculation module according to the first calculation result and the second filling data.

[0010] Preferably, when a calculation instruction is detected, the step of obtaining non-filled data in the calculation window by the data sorting module includes:

[0011] When a calculation instruction is detected, a target image corresponding to the calculation instruction is obtained, and the target image is selected through a calculation window according to a preset rule;

[0012] The non-filled data of the portion framed by the calculation window in the target image is identified and acquired through the data sorting module, and the non-filled data located in the same column in the calculation window are sent to the first calculation module respectively.

[0013] Preferably, the step of inputting the first filling data to the first calculation module through the control module includes:

[0014] Determine a first serial port to be enabled corresponding to the first computing module according to the number of padding data rows above or below the target image, the height of the target image, the accumulated value of the column step length corresponding to the computing window, and the window height corresponding to the computing window;

[0015] The first serial port to be enabled is enabled through the control module to input first filling data into the first calculation module.

[0016] Preferably, the step of obtaining a first calculation result according to the non-filled data and the first filled data by the first calculation module includes:

[0017] The non-filled data and the first filled data in each column of the calculation window are input into an adder or a comparator through the first calculation module to obtain a first calculation result.

[0018] Preferably, the step of inputting second filling data to the second calculation module through the control module comprises:

[0019] Determine the second serial port to be enabled corresponding to the second calculation module according to the number of padded data columns on the left or right side of the target image, the width of the target image, the accumulated value of the row stride corresponding to the calculation window, and the window width corresponding to the calculation window;

[0020] According to a first preset clock cycle, the second serial port to be enabled is enabled by the control module to input second filling data into the second calculation module.

[0021] Preferably, the step of obtaining a second calculation result according to the first calculation result and the second filling data by the second calculation module includes:

[0022] The first calculation result and the second filling data are stored in a register through the second calculation module, and the first calculation result and the second filling data are input into an adder and a divider according to a second preset clock cycle, or the first calculation result and the second filling data are input into a comparator to obtain a second calculation result.

[0023] Preferably, after the step of obtaining the second calculation result according to the first calculation result and the second filling data by the second calculation module, the hardware acceleration method of the pooling algorithm further includes:

[0024] According to the row step or column step corresponding to the calculation window, the calculation window is controlled to move according to a preset movement rule, and the steps are performed: non-filled data in the calculation window is obtained through a data sorting module to obtain a target feature map.

[0025] In addition, to achieve the above purpose, the present invention also provides a hardware acceleration device for a pooling algorithm, the hardware acceleration device for the pooling algorithm comprising:

[0026] An acquisition module, configured to acquire non-filled data in the calculation window through the data sorting module when a calculation instruction is detected, and send the non-filled data to the first calculation module;

[0027] a first input module, configured to input first filling data to the first calculation module through a control module, and obtain a first calculation result according to the non-filling data and the first filling data through the first calculation module;

[0028] a second input module, configured to input the first calculation result into a second calculation module through the first calculation module, and to input second filling data into the second calculation module through the control module;

[0029] A calculation module is used to obtain a second calculation result according to the first calculation result and the second filling data through the second calculation module.

[0030] Preferably, the acquisition module is also used for:

[0031] When a calculation instruction is detected, a target image corresponding to the calculation instruction is obtained, and the target image is selected through a calculation window according to a preset rule;

[0032] The non-filled data of the portion framed by the calculation window in the target image is identified and acquired through the data sorting module, and the non-filled data located in the same column in the calculation window are sent to the first calculation module respectively.

[0033] Preferably, the first input module is further used for:

[0034] Determine a first serial port to be enabled corresponding to the first computing module according to the number of padding data rows above or below the target image, the height of the target image, the accumulated value of the column step length corresponding to the computing window, and the window height corresponding to the computing window;

[0035] The first serial port to be enabled is enabled through the control module to input first filling data into the first calculation module.

[0036] Preferably, the first input module further includes a calculation module, and the calculation module is used to:

[0037] The non-filled data and the first filled data in each column of the calculation window are input into an adder or a comparator through the first calculation module to obtain a first calculation result.

[0038] Preferably, the second input module is further used for:

[0039] Determine the second serial port to be enabled corresponding to the second calculation module according to the number of padded data columns on the left or right side of the target image, the width of the target image, the accumulated value of the row stride corresponding to the calculation window, and the window width corresponding to the calculation window;

[0040] According to a first preset clock cycle, the second serial port to be enabled is enabled by the control module to input second filling data into the second calculation module.

[0041] Preferably, the calculation module is also used for:

[0042] The first calculation result and the second filling data are stored in a register through the second calculation module, and the first calculation result and the second filling data are input into an adder and a divider according to a second preset clock cycle, or the first calculation result and the second filling data are input into a comparator to obtain a second calculation result.

[0043] Preferably, the calculation module is also used for:

[0044] According to the row step or column step corresponding to the calculation window, the calculation window is controlled to move according to a preset movement rule, and the steps are performed: non-filled data in the calculation window is obtained through a data sorting module to obtain a target feature map.

[0045] In addition, to achieve the above-mentioned purpose, the present invention also provides a hardware acceleration system for a pooling algorithm, the hardware acceleration system for the pooling algorithm comprising: a memory, a processor, and a hardware acceleration program for the pooling algorithm stored in the memory and runnable on the processor, and when the hardware acceleration program for the pooling algorithm is executed by the processor, the steps of the hardware acceleration method for the pooling algorithm as described above are implemented.

[0046] In addition, to achieve the above-mentioned purpose, the present invention also provides a readable storage medium, which is a computer-readable storage medium, and a hardware acceleration program of the pooling algorithm is stored on the computer-readable storage medium. When the hardware acceleration program of the pooling algorithm is executed by the processor, the steps of the hardware acceleration method of the pooling algorithm as described above are implemented.

[0047] The hardware acceleration method of the pooling algorithm proposed in the present invention obtains non-filled data in the calculation window through a data sorting module when a calculation instruction is detected, and sends the non-filled data to a first calculation module; inputs first filling data to the first calculation module through a control module, and obtains a first calculation result through the first calculation module according to the non-filled data and the first filling data; inputs the first calculation result to the second calculation module through the first calculation module, and inputs second filling data to the second calculation module through the control module; obtains a second calculation result through the second calculation module according to the first calculation result and the second filling data; the present invention inputs the first filling data to the first calculation module and the second filling data to the second calculation module through the control module, thereby improving the performance of the hardware accelerator and thus improving the calculation speed of the pooling algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the system structure of the hardware operating environment involved in the embodiment of the present invention;

[0049] Figure 2 It is a flowchart of the first embodiment of the hardware acceleration method of the pooling algorithm of the present invention;

[0050] Figure 3 It is a schematic diagram of the target picture of the present invention;

[0051] Figure 4 It is a connection diagram of each module in the hardware accelerator of the present invention;

[0052] Figure 5 It is a schematic diagram of the connection between the first calculation module and the control module of the present invention;

[0053] Figure 6 It is a connection diagram of the second calculation module and the control module of the present invention.

[0054] The realization of the purpose, functional features and advantages of the present invention will be further explained with reference to the accompanying drawings in combination with the embodiments. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0056] like Figure 1 As shown, Figure 1It is a schematic diagram of the system structure of the hardware operating environment involved in the embodiment of the present invention.

[0057] The system of the embodiment of the present invention may be a PC or a server system.

[0058] like Figure 1 As shown, the system may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0059] Those skilled in the art will understand that Figure 1 The system structure shown in the figure does not constitute a limitation of the system, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0060] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a hardware acceleration program of a pooling algorithm.

[0061] Among them, the operating system is a program that manages and controls the hardware acceleration tools and software resources of the portable pooling algorithm, and supports the operation of the network communication module, the user interface module, the hardware acceleration program of the pooling algorithm and other programs or software; the network communication module is used to manage and control the network interface 1002; the user interface module is used to manage and control the user interface 1003.

[0062] exist Figure 1 In the hardware acceleration tool of the pooling algorithm shown, the hardware acceleration tool of the pooling algorithm calls the hardware acceleration program of the pooling algorithm stored in the memory 1005 through the processor 1001, and performs the operations in each embodiment of the hardware acceleration method of the pooling algorithm described below.

[0063] Based on the above hardware structure, an embodiment of a hardware acceleration method of the pooling algorithm of the present invention is proposed.

[0064] Reference Figure 2 , Figure 2The flowchart of the first embodiment of the hardware acceleration method of the pooling algorithm of the present invention is as follows:

[0065] Step S10, when a calculation instruction is detected, obtaining non-filled data in the calculation window through the data sorting module, and sending the non-filled data to the first calculation module;

[0066] Step S20, inputting first filling data into the first calculation module through the control module, and obtaining a first calculation result according to the non-filling data and the first filling data through the first calculation module;

[0067] Step S30, inputting the first calculation result into the second calculation module through the first calculation module, and inputting second filling data into the second calculation module through the control module;

[0068] Step S40: obtaining a second calculation result according to the first calculation result and the second filling data by the second calculation module.

[0069] The hardware acceleration method of the pooling algorithm of this embodiment is applied to the hardware accelerator of the pooling algorithm of the image processing mechanism. The hardware accelerator of the pooling algorithm can be applied to a terminal or a PC device to improve the calculation speed of the terminal or PC device running the pooling algorithm. For the convenience of description, the hardware accelerator is used as an example for description. The hardware accelerator includes but is not limited to a control module, a data sorting module, a first calculation module and a second calculation module. When the hardware accelerator detects a calculation instruction, it obtains the target image corresponding to the calculation instruction, and selects the target image through the calculation window according to the preset rule; the non-filled data of the selected part of the calculation window in the target image is identified and obtained through the data sorting module, and the non-filled data is sent to the first calculation module; the hardware accelerator inputs the first filling data to the first calculation module through the control module, and obtains the first calculation result according to the non-filled data and the first filling data through the first calculation module; the hardware accelerator inputs the first calculation result to the second calculation module through the first calculation module, and inputs the second filling data to the second calculation module through the control module; the hardware accelerator obtains the second calculation result according to the first calculation result and the second filling data through the second calculation module.

[0070] It should be noted that the target image obtained by the hardware accelerator is Figure 3 As shown, "P" represents the padding data (padding value) in the target image, and the other numbers represent the non-padding data in the target image. The non-padding data is the data carried by the target image itself. It can be understood that Figure 3 This is only used as an example and is not intended to limit the target image. Depending on the actual situation, the target image may also insert padding data or more padding data at other locations; Figure 4 As shown, Figure 4 It is a connection diagram of each module in the hardware accelerator, wherein the control module is responsible for controlling the data sorting module, the first computing module and the second computing module. The data sorting module inputs the corresponding numbers into the first computing module, and the first computing module inputs the corresponding data into the second computing module to obtain the calculation result of the pooling algorithm for the target image.

[0071] The hardware acceleration method of the pooling algorithm of this embodiment, when a calculation instruction is detected, obtains non-filled data in the calculation window through the data sorting module, and sends the non-filled data to the first calculation module; inputs first filling data to the first calculation module through the control module, and obtains a first calculation result through the first calculation module according to the non-filled data and the first filling data; inputs the first calculation result to the second calculation module through the first calculation module, and inputs second filling data to the second calculation module through the control module; obtains a second calculation result through the second calculation module according to the first calculation result and the second filling data; the present invention improves the performance of the hardware accelerator by inputting the first filling data to the first calculation module and the second filling data to the second calculation module through the control module, thereby improving the calculation speed of the pooling algorithm.

[0072] The following is a detailed description of each step:

[0073] Step S10, when a calculation instruction is detected, obtaining non-filled data in the calculation window through the data sorting module, and sending the non-filled data to the first calculation module;

[0074] In this embodiment, when the hardware accelerator detects a calculation instruction, it obtains the target image corresponding to the calculation instruction, selects the target image through the calculation window according to the preset rule, and then controls the data sorting module through the control module to identify and obtain the non-filled data of the selected part of the calculation window in the target image, and sends the non-filled data to the first calculation module. It can be understood that if Figure 3 As shown, Figure 3 The black box in the upper left corner refers to the calculation window, which includes filled data "P" and non-filled data "0, 1, 8, 9". The data sorting module identifies and obtains the non-filled data "0, 1, 8, 9" in the calculation window, and inputs the non-filled data into the first calculation module.

[0075] Specifically, when a calculation instruction is detected, the step of obtaining non-filled data in the calculation window through the data sorting module includes:

[0076] Step a, when a calculation instruction is detected, obtaining a target image corresponding to the calculation instruction, and selecting the target image through a calculation window according to a preset rule;

[0077] In this step, when the hardware accelerator detects a calculation instruction, it obtains the target image corresponding to the calculation instruction, and selects the target image through the calculation window according to the preset rules; for example, the relevant R&D personnel set the height and width of the calculation window to 3, and the preset rule is to select the target image from the upper left corner of the target image through the calculation window, and then use 1 as the row step and column step. The calculation window moves in the row direction of the target image according to the row step, and the calculation window moves in the column direction of the target image according to the column step. The hardware accelerator selects the target image through the calculation window according to the preset rules. It should be noted that the row step refers to the step length of the calculation window in the row direction of the target image, that is, the width direction, and the column step refers to the step length of the calculation window in the column direction of the target image, that is, the height direction.

[0078] Step b: identifying and acquiring the non-filled data of the portion of the target image framed by the calculation window through a data sorting module, and sending the non-filled data located in the same column in the calculation window to the first calculation module respectively.

[0079] In this step, the hardware accelerator controls the data sorting module through the control module to identify and obtain the non-filled data of the selected part of the calculation window in the target image, and packages the non-filled data in the same column in the calculation window and sends them to the first calculation module respectively, such as: Figure 3 , when the calculation window is located at Figure 3 When the black box in the upper left corner of the image is located, the filling data "P" and the non-filling data "0, 1, 8, 9" in the target image are selected in the calculation window, where the non-filling data "0 and 8" are in the same column, and the non-filling data "1 and 9" are in another column. The control module controls the data sorting module to pack the non-filling data "0 and 8" into a set [8,0], and pack the non-filling data "1 and 9" into [9,1], and then send [8,0] and [9,1] to the first calculation module respectively; further, when the calculation window moves to Figure 3 , at this time, the first row of the calculation box is all filled with data "P", and the second and third rows are all non-filled data "0, 1, 2, 8, 9, 10", among which, the non-filled data "0 and 8" are in the same column, the non-filled data "1 and 9" are in the same column, and the non-filled data "2 and 10" are in the same column. Since the non-filled data "0 and 8" and "1 and 9" have been stored in the register of the second calculation module in the last calculation process, the control module controls the data sorting module to pack the non-filled data "2 and 10" into [2, 10], and then sends [2, 10] to the first calculation module; the situation when the calculation window is located at other positions on the target image is similar to the above situation, which will not be repeated here.

[0080] Step S20, inputting first filling data into the first calculation module through the control module, and obtaining a first calculation result according to the non-filling data and the first filling data through the first calculation module;

[0081] In this embodiment, the hardware accelerator controls the first calculation module through the control module to input the first filling data into the first calculation module and obtains the first calculation result according to the non-filling data and the first filling data through the first calculation module; Figure 5 , Figure 5 : is a connection diagram of the first computing module and the control module, wherein PH represents the first filling data, pub_kh0_en, pub_kh1_en and pub_kh2_en represent three serial ports to be enabled in the first computing module, the three serial ports to be enabled are connected to the control module, and "+ / M" represents an adder or a comparator. In the initial stage, pub_kh0_en=pub_kh1_en=pub_kh2_en=0, that is, the three serial ports to be enabled are all at a low level. When the hardware accelerator recognizes that the first computing module needs to use the first filling data for calculation, the hardware accelerator controls the corresponding serial ports to be enabled through the control module to enable, so that one or more of pub_kh0_en, pub_kh1_en or pub_kh2_en is equal to 1, that is, at a high level, and "PH", that is, the first filling data, is input to the first computing module. The first computing module obtains a first calculation result according to the non-filling data and the first filling data. It should be noted that PH corresponds to the filling data located above and below the target image, that is, for example Figure 3 The "P" values ​​of the first and last rows shown; the number of serial ports to be enabled in the first calculation module can be modified accordingly according to the width of the calculation window. For example, when the size of the calculation window is 5 rows and 5 columns, the number of serial ports to be enabled in the first calculation module should be 5. It should be noted that pub_kh0_en, pub_kh1_en and pub_kh2_en in the first calculation module represent the first row, the second row and the third row in the calculation window respectively. When a serial port is enabled, the first padding data is input to the corresponding row.

[0082] Specifically, the step of inputting the first filling data to the first calculation module through the control module includes:

[0083] Step c, determining the first serial port to be enabled corresponding to the first calculation module according to the number of padding data rows above or below the target image, the height of the target image, the accumulated column step length corresponding to the calculation window, and the window height corresponding to the calculation window;

[0084] In this step, the hardware accelerator obtains the number of rows of filled data above the target image, the height of the target image, the accumulated value of the column step corresponding to the calculation window, and the window height corresponding to the calculation window, and determines the first serial port to be enabled corresponding to the first computing module based on the number of rows of filled data above the target image, the height of the target image, the accumulated value of the column step corresponding to the calculation window, and the window height corresponding to the calculation window, where the first serial port to be enabled may include one or more of the serial ports to be enabled in the first computing module.

[0085] When judging whether the calculation window contains the padding data above the target image, it is assumed that PU represents the number of padding data rows above the target image, H represents the height of the target image, H_CNT represents the column step accumulation value corresponding to the calculation window, and KH represents the window height corresponding to the calculation window; when it is determined that PU>H_CNT is not true, it is determined that the calculation window does not contain the padding data above the target image, and it is not necessary to determine the first serial port to be enabled. For example, the calculation window is located at Figure 3 When the black frame in the figure moves toward the height direction of the target image by a row step of 1, the values ​​in the calculation window are respectively "P, 0, 1" in the first row, "P, 8, 9" in the second row, and "P, 16, 17" in the third column. At this time, the calculation window does not contain the padding data above the target image, so there is no need to determine the first serial port to be enabled; if it is determined that PU>H_CNT is established, it is determined that the calculation window contains the padding data above the target image, and the difference between PU and H_CNT is further calculated. This difference is the number of rows of the padding data above the calculation window. For example, the calculation window is located at Figure 3 When the black box is located at the position shown, H_CNT=0, PU=1, PU>H_CNT is established, the difference between PU and H_CNT is 1, and the corresponding 1 is greater than 0, so pub_kh0_en is used as the first serial port to be enabled. For another example, the calculation window is located at Figure 3 When the black box in the middle moves one column step 1 position in the width direction of the target image, the values ​​in the calculation window are "P, P, P" in the first row, "0, 1, 2" in the second row, and "8, 9, 10" in the third column. At this time, the calculation window contains the padding data above the target image, and the difference between PU and H_CNT is 1, and the corresponding 1 is greater than 0, so pub_kh0_en is used as the first serial port to be enabled. For another example, assuming that the first and second rows of the calculation window are all padding data, the difference between PU and H_CNT is 2, and the corresponding 2 is greater than 0 and 1, so pub_kh0_en and pub_kh1_en are used as the first serial ports to be enabled. It should be noted that the above method of determining the first serial port to be enabled is limited to use when the calculation window moves in the column direction of the target image.

[0086] When judging whether the calculation window contains the fill data below the target image, if the calculation window box is selected as follows Figure 3 When the lower left corner area of ​​the target image is shown, the first line of the calculation window is "P, 48, 49", the second line is "P, 56, 57", and the third line is "P, P, P". At this time, H_CNT+KH>PU+H is established, and it is determined that the calculation window contains the padding data below the target image. Then, the difference between H_CNT+KH and PU+H is calculated. This difference is the number of rows of padding data below the calculation window. At this time, the difference is 1, and the window height KH is 3. Correspondingly, according to the difference + x>=KH, x=2 is obtained. It is determined that pub_kh2_en is the first serial port to be enabled; when the data selected in the calculation window is "P, 40, 41" in the first row, "P, 48, 49" in the second row, and "P, 56, 57" in the third row, the calculation window does not contain the padding data below the target image, then H_CNT+KH>PU+H does not hold, and there is no need to determine the first serial port to be enabled. It should be noted that the above method of determining the first serial port to be enabled is only limited to use when the calculation window moves in the column direction of the target image, that is, the height direction.

[0087] Step d: enabling the first serial port to be enabled by the control module to input first filling data into the first calculation module.

[0088] In this step, after determining the first serial port to be enabled, the hardware accelerator enables the first serial port to be enabled through the control module to input the first filling data to the first computing module; Figure 3For the target image shown in the figure, when the calculation window contains the filling data above the target image, the hardware accelerator determines that pub_kh0_en is the first serial port to be enabled, and the hardware accelerator enables pub_kh0_en through the control module, so that pub_kh0_en=1, that is, it is at a high level, and the first filling data is input to the first calculation module; when the calculation window contains the filling data below the target image, the hardware accelerator determines that pub_kh2_en is the first serial port to be enabled, and the hardware accelerator enables pub_kh2_en through the control module, so that pub_kh2_en=1, that is, it is at a high level, and the first filling data is input to the first calculation module. It should be noted that the calculation window cannot include the filling data above and below the target image at the same time; generally speaking, the values ​​corresponding to the filling data above and below the target image are the same, so the first filling data is recorded in the hardware accelerator when the hardware accelerator obtains the target image, and when the corresponding serial port is enabled, the first filling data is input into the first calculation module, and there is no need to store the first filling data in the storage separately, which saves the time of reading the first filling data from the storage and improves the performance of the hardware accelerator.

[0089] Specifically, the step of obtaining a first calculation result according to the non-filled data and the first filled data by the first calculation module includes:

[0090] Step e: inputting the non-filled data and the first filled data in each column of the calculation window into an adder or a comparator through the first calculation module to obtain a first calculation result.

[0091] In this step, the hardware accelerator controls the first calculation module through the control module to input the non-filled data and the first filled data in each column of the calculation window into the adder or comparator to obtain the first calculation result; such as: Figure 3 The target image is assumed to be Figure 3When the calculation window is located at the position of the black frame, the data sorting module packs the non-filled data "0, 8" in the second column and the non-filled data "1, 9" in the third column to obtain [8,0] and [9,1], and inputs [8,0] and [9,1] into the first calculation module. The control module controls pub_kh0_en=1 of the first calculation window, and inputs the first filling data PH=P=1 into the first calculation module. At this time, the first calculation module obtains [8,0,1] and [9,1,1]. When the pooling algorithm is MAX POOL (the maximum value of the calculation window), the first calculation module inputs [8,0,1] and [9,1,1] into the comparator, and the comparator counts the maximum values ​​of [8,0,1] and [9,1,1] respectively, and obtains the first calculation result of [8,9]. When the pooling algorithm is Average When POOL (calculating the average value of the window), the first calculation module inputs [8, 0, 1] and [9, 1, 1] into the adder, and the adder adds the values ​​in [8, 0, 1] and [9, 1, 1] respectively, and the calculation result is [9, 11]. It should be noted that the first filling data is the filling data above and below the target image, that is, the data above and below the target image represented by "P", and the first filling data PH in the first calculation module is P.

[0092] Step S30, inputting the first calculation result into the second calculation module through the first calculation module, and inputting second filling data into the second calculation module through the control module;

[0093] In this step, the hardware accelerator inputs the first calculation result into the second calculation module through the first calculation module. The hardware accelerator judges whether the calculation window contains the padding data on the left or right side of the target image. If the calculation window contains the padding data on the left or right side of the target image, the hardware accelerator inputs the second padding data into the second calculation module through the control module; for example, referring to Figure 6 , Figure 6: is a connection diagram of the second computing module and the control module, wherein PW represents the second filling data, pub_kw0_en, pub_kw1_en and pub_kw2_en represent three serial ports to be enabled in the second computing module, the three serial ports to be enabled are connected to the control module, "Reg" represents a register, "+ / M" represents an adder or a comparator, and "DIV" represents a divider. In the initial stage, pub_kw0_en=pub_kw1_en=pub_kw2_en=0, that is, the three serial ports to be enabled are all at a low level. When the hardware accelerator recognizes that the second computing module needs to use the second filling data for calculation, the hardware accelerator controls the corresponding serial ports to be enabled through the control module to enable, so that one or more of pub_kw0_en, pub_kw1_en or pub_kw2_en is equal to 1, that is, at a high level, and "PW", i.e., the second filling data, is input to the second computing module; it should be noted that, if Figure 3 The "P" values ​​of the first and last columns shown correspond to the padding data located on the left and right of the target image, respectively. When the pooling algorithm is MAX POOL, the second padding data PW=P, and when the pooling algorithm is Average POOL, the second padding data PW=3*P; the number of serial ports to be enabled in the second calculation module can be modified accordingly according to the width of the calculation window. For example, when the size of the calculation window is 5 rows and 5 columns, the number of serial ports to be enabled in the second calculation module should be 5. It should be noted that pub_kw0_en, pub_kw1_en, and pub_kw2_en in the second calculation module represent the first, second, and third columns in the calculation window, respectively. When a serial port is enabled, the second padding data is input to the corresponding row.

[0094] Specifically, the step of inputting the second filling data to the second calculation module through the control module includes:

[0095] Step f, determining the second serial port to be enabled corresponding to the second calculation module according to the number of padded data columns on the left or right side of the target image, the width of the target image, the accumulated value of the row stride corresponding to the calculation window, and the window width corresponding to the calculation window;

[0096] In this step, the hardware accelerator determines the second serial port to be enabled corresponding to the second computing module based on the number of padding data columns on the left side of the target image, the width of the target image, the accumulated value of the row step corresponding to the calculation window, and the window width corresponding to the calculation window; wherein the second serial port to be enabled may include one or more of the serial ports to be enabled in the second computing module.

[0097] When judging whether the calculation window contains the padding data on the left side of the target image, it is assumed that PL represents the number of padding data rows on the left side of the target image, W represents the width of the target image, W_CNT represents the accumulated row step length corresponding to the calculation window, and KW represents the window width corresponding to the calculation window. For example, the calculation window is located at Figure 3 When the black frame in the middle moves one column step 1 in the width direction of the target image, the values ​​in the calculation window are "P, P, P" in the first row, "0, 1, 2" in the second row, and "8, 9, 10" in the third column. At this time, it is determined that PL>W_CNT is not true, and it can be determined that the calculation window does not contain the padding data on the left side of the target image, so there is no need to determine the first serial port to be enabled; for example, the calculation window is located at Figure 3 When the black box in the middle moves a row step of 1 in the height direction of the target image, the values ​​in the calculation window are respectively "P, 0, 1" in the first row, "P, 8, 9" in the second row, and "P, 16, 17" in the third column. At this time, it is determined that PU>H_CNT holds, and it is determined that the calculation window contains the padding data on the left side of the target image. The difference between PL and W_CNT is further calculated. This difference is the number of columns of padding data on the left side of the calculation window. At this time, the number of columns containing padding data on the left side of the target image in the calculation window is 1, and the difference between PL and W_CNT is 1, and the corresponding 1 is greater than 0. Therefore, pub_kw0_en is used as the first serial port to be enabled. Assuming that the first and second columns of the calculation window are all padding data, the difference between PL and W_CNT is 2, and the corresponding 2 is greater than 0 and 1. Therefore, pub_kh0_en and pub_kh1_en are used as the second serial ports to be enabled. It should be noted that the above method of determining the second serial port to be enabled is limited to when the calculation window moves in the row direction of the target image, that is, the width direction of the target image.

[0098] When judging whether the calculation window contains the fill data on the right side of the target image; when the calculation window box is selected as follows Figure 3When the upper right corner area of ​​the target image is shown, the first line of the calculation window is "P, P, P", the second line is "6, 7, P", and the third line is "14, 15, P". At this time, W_CNT+KW>PL+W is established. It is determined that the number of columns containing padding data on the right side of the target image in the calculation window is 1. Then, the difference between W_CNT+KW and PL+W is calculated. This difference is the number of rows of padding data on the right side of the calculation window. At this time, the difference is 1, and the window width KW is 3. Correspondingly, according to the difference + x>=KW, x= 2, then it is determined that pub_kh2_en is the first serial port to be enabled; when, for example, the data selected in the calculation window is "P, P, P" in the first line, "5, 6, 7" in the second line, and "13, 14, 15" in the third line, the calculation window does not contain the padding data on the right side of the target image, then W_CNT+KW>PL+W does not hold, and there is no need to determine the first serial port to be enabled. It should be noted that the above method of determining the first serial port to be enabled is limited to use when the calculation window moves in the row direction, that is, the width direction, of the target image.

[0099] Step g: according to a first preset clock cycle, enabling the second serial port to be enabled by the control module to input second filling data into the second calculation module.

[0100] In this step, the hardware accelerator enables the second serial port to be enabled through the control module according to the first preset clock cycle, so as to input the second filling data to the second computing module, such as: Figure 3 For the target image shown in the figure, when the calculation window contains the padding data on the left side of the target image, the hardware accelerator determines that pub_kw0_en is the second serial port to be enabled, and the hardware accelerator enables pub_kw0_en through the control module according to the first preset clock cycle, so that pub_kw0_en=1, that is, it is at a high level, and the second padding data is input to the second calculation module; when the calculation window contains the padding data on the right side of the target image, the hardware accelerator determines that pub_kw2_en is the second serial port to be enabled, and the hardware accelerator enables pub_kw2_en through the control module, so that pub_kw2_en=1, that is, it is at a high level, and the second padding data is input to the second calculation module. It should be noted that the calculation window cannot include the padding data on the left and the right of the target image at the same time; generally speaking, the values ​​corresponding to the padding data on the left and the right of the target image are the same, so the second padding data is recorded in the hardware accelerator when the hardware accelerator obtains the target image, and when the corresponding serial port is enabled, the second padding data is input into the second calculation module, and there is no need to store the second padding data in the storage separately, which saves the time of reading the second padding data from the storage and improves the performance of the hardware accelerator.

[0101] Step S40: obtaining a second calculation result according to the first calculation result and the second filling data by the second calculation module.

[0102] In this embodiment, the hardware accelerator controls the second calculation module through the control module to obtain the second calculation result according to the first calculation result and the second filling data. Figure 6 , Figure 6 : is a connection diagram of the second computing module and the control module, where PW represents the second filling data, pub_kw0_en, pub_kw1_en and pub_kw2_en represent three serial ports to be enabled in the second computing module, and the three serial ports to be enabled are connected to the control module, "Reg" represents a register, "+ / M" represents an adder or a comparator, and "DIV" represents a divider. In the initial stage, pub_kw0_en=pub_kw1_en=pub_kw2_en=0, that is, the three serial ports to be enabled are all at a low level. When the hardware accelerator recognizes that the second computing module When the module needs to use the second filling data for calculation, the hardware accelerator controls the corresponding serial port to be enabled through the control module, so that one or more of pub_kw0_en, pub_kw1_en or pub_kw2_en is equal to 1, that is, it is at a high level, and "PW", that is, the second filling data, is input to the second calculation module. The second calculation module inputs the first calculation result stored in the register and the second filling data input by the control module into the adder and divider according to the specific pooling algorithm, or inputs the first calculation result and the second filling data into the comparator to obtain the second calculation result.

[0103] Specifically, step S40 includes:

[0104] The first calculation result and the second filling data are stored in a register through the second calculation module, and the first calculation result and the second filling data are input into an adder and a divider according to a second preset clock cycle, or the first calculation result and the second filling data are input into a comparator to obtain a second calculation result.

[0105] In this step, the hardware accelerator controls the second calculation module through the control module to store the first calculation result and the second filling data in the register, and inputs the first calculation result and the second filling data into the adder and the divider according to the second preset clock cycle, or inputs the first calculation result and the second filling data into the comparator to obtain the second calculation result; such as: Figure 3 The target image is assumed to be Figure 3In the calculation window, P=1, when the calculation window is located at the position of the black frame, it is determined that the calculation window contains the padding data on the left side of the target image, so pub_kw0_en needs to be enabled. When the pooling algorithm is MAXPOOL (the maximum value of the calculation window), the first calculation result is [8,9]. The first calculation module inputs the first calculation result [8,9] into the second calculation module. The second calculation module stores "8" in the first calculation result in register Reg1 and "9" in the first calculation result in register Reg2. The second padding data PW=P=1 enabled by the control module for inputting pub_kw0_en is stored in register Reg0. Then, the first calculation result and the second padding data are input into the comparator according to the second preset clock cycle, that is, [1,8,9] is given to the comparator. The comparator obtains the second calculation result 9; when the pooling algorithm is AveragePOOL (calculating the average value of the window), the first calculation module obtains the calculation result [9,11], the first calculation module inputs the first calculation result [9,11] into the second calculation module, the second calculation module stores "9" in the first calculation result in register Reg1, stores "11" in the first calculation result in register Reg2, stores the second filling data PW=3*P=3 enabled by the control module for pub_kw0_en in register Reg0, and then inputs the first calculation result and the second filling data into the adder and divider according to the second preset clock cycle, that is, [3,9,11] is input into the adder and divider, and the second calculation result is obtained: (11+9+3) / 3=7.66. It should be noted that the second filling data is the filling data on the left and right sides of the target image, that is, the data represented by "P" on the left and right sides of the target image, and is represented by PW in the second calculation module.

[0106] When the hardware accelerator of the pooling algorithm of this embodiment detects a calculation instruction, it obtains the target image corresponding to the calculation instruction, and selects the target image through the calculation window according to the preset rules; identifies and obtains the non-filled data of the selected part of the calculation window in the target image through the data sorting module, and sends the non-filled data to the first calculation module; the hardware accelerator inputs the first filling data to the first calculation module through the control module, and obtains the first calculation result according to the non-filled data and the first filling data through the first calculation module; the hardware accelerator inputs the first calculation result to the second calculation module through the first calculation module, and inputs the second filling data to the second calculation module through the control module; the hardware accelerator obtains the second calculation result according to the first calculation result and the second filling data through the second calculation module, thereby improving the performance of the hardware accelerator and thus improving the calculation speed of the pooling algorithm.

[0107] Furthermore, based on the first embodiment of the hardware acceleration method for the pooling algorithm of the present invention, a second embodiment of the hardware acceleration method for the pooling algorithm of the present invention is proposed.

[0108] The difference between the second embodiment of the hardware acceleration method of the pooling algorithm and the first embodiment of the hardware acceleration method of the pooling algorithm is that after step S40, the following steps are further included:

[0109] Step h, according to the row step or column step corresponding to the calculation window, control the calculation window to move according to the preset movement rule, and execute the step: obtain the non-filled data in the calculation window through the data sorting module to obtain the target feature map.

[0110] In this embodiment, the hardware accelerator controls the calculation window from the following: Figure 3 , the non-filled data in the calculation window is obtained through the data sorting module and the subsequent steps are executed, and then the calculation window is controlled to move in the row direction of the target image, that is, the width direction. Each time it moves, the non-filled data in the calculation window is obtained through the data sorting module and the subsequent steps are executed. When the calculation window moves to the rightmost side of the target image, that is, the calculation window contains: the first row "P, P, P", the second row "6, 7, P", the third row "14, 15, P", and the non-filled data in the calculation window is obtained through the data sorting module and the subsequent steps are completed. After the calculation is completed, the calculation window is controlled to return to the following Figure 3 The position of the black box shown in the figure is determined, and according to the column step size corresponding to the calculation window, the calculation module is controlled to move downward so that the calculation window contains: the first row "P, 0, 1", the second row "P, 8, 9", the third row "P, 16, 17", and so on. The calculation window is controlled to move until the calculation of the entire image is completed and the target feature map corresponding to the target image is obtained.

[0111] The hardware accelerator of the pooling algorithm of this embodiment controls the calculation window to move according to a preset movement rule based on the row step or column step corresponding to the calculation window, and executes the steps of: obtaining non-filled data in the calculation window through a data sorting module to obtain a target feature map, which helps to improve the performance of the hardware accelerator and further helps to improve the calculation speed of the pooling algorithm.

[0112] The present invention also provides a hardware acceleration device for a pooling algorithm. The hardware acceleration device for a pooling algorithm of the present invention comprises:

[0113] A receiving module, configured to generate a corresponding configuration file set according to the IP file set when receiving the IP file set;

[0114] An acquisition module, configured to acquire non-filled data in the calculation window through the data sorting module when a calculation instruction is detected, and send the non-filled data to the first calculation module;

[0115] a first input module, configured to input first filling data to the first calculation module through a control module, and obtain a first calculation result according to the non-filling data and the first filling data through the first calculation module;

[0116] a second input module, configured to input the first calculation result into a second calculation module through the first calculation module, and to input second filling data into the second calculation module through the control module;

[0117] A calculation module is used to obtain a second calculation result according to the first calculation result and the second filling data through the second calculation module.

[0118] Preferably, the acquisition module is also used for:

[0119] When a calculation instruction is detected, a target image corresponding to the calculation instruction is obtained, and the target image is selected through a calculation window according to a preset rule;

[0120] The non-filled data of the portion framed by the calculation window in the target image is identified and acquired through the data sorting module, and the non-filled data located in the same column in the calculation window are sent to the first calculation module respectively.

[0121] Preferably, the first input module is further used for:

[0122] Determine a first serial port to be enabled corresponding to the first computing module according to the number of padding data rows above or below the target image, the height of the target image, the accumulated value of the column step length corresponding to the computing window, and the window height corresponding to the computing window;

[0123] The first serial port to be enabled is enabled through the control module to input first filling data into the first calculation module.

[0124] Preferably, the first input module further includes a calculation module, and the calculation module is used to:

[0125] The non-filled data and the first filled data in each column of the calculation window are input into an adder or a comparator through the first calculation module to obtain a first calculation result.

[0126] Preferably, the second input module is further used for:

[0127] Determine the second serial port to be enabled corresponding to the second calculation module according to the number of padded data columns on the left or right side of the target image, the width of the target image, the accumulated value of the row stride corresponding to the calculation window, and the window width corresponding to the calculation window;

[0128] According to a first preset clock cycle, the second serial port to be enabled is enabled by the control module to input second filling data into the second calculation module.

[0129] Preferably, the calculation module is also used for:

[0130] The first calculation result and the second filling data are stored in a register through the second calculation module, and the first calculation result and the second filling data are input into an adder and a divider according to a second preset clock cycle, or the first calculation result and the second filling data are input into a comparator to obtain a second calculation result.

[0131] Preferably, the calculation module is also used for:

[0132] According to the row step or column step corresponding to the calculation window, the calculation window is controlled to move according to a preset movement rule, and the steps are performed: non-filled data in the calculation window is obtained through a data sorting module to obtain a target feature map.

[0133] The invention also provides a hardware acceleration system for a pooling algorithm.

[0134] The hardware acceleration system of the pooling algorithm of the present invention comprises: a hardware accelerator, a memory, a processor, and a hardware acceleration program of the pooling algorithm stored in the memory and executable on the processor. When the hardware acceleration program of the pooling algorithm is executed by the processor, the steps of the hardware acceleration method of the pooling algorithm as described above are implemented.

[0135] Among them, the method implemented when the hardware acceleration program of the pooling algorithm running on the processor is executed can refer to the various embodiments of the hardware acceleration method of the pooling algorithm of the present invention, which will not be repeated here.

[0136] The invention also provides a readable storage medium.

[0137] The readable storage medium is a computer-readable storage medium. The computer-readable storage medium of the present invention stores a hardware acceleration program of the pooling algorithm. When the hardware acceleration program of the pooling algorithm is executed by the processor, the steps of the hardware acceleration method of the pooling algorithm as described above are implemented.

[0138] Among them, the method implemented when the hardware acceleration program of the pooling algorithm running on the processor is executed can refer to the various embodiments of the hardware acceleration method of the pooling algorithm of the present invention, which will not be repeated here.

[0139] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0140] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal system (which can be a mobile phone, computer, server, or network system, etc.) to execute the methods described in each embodiment of the present invention.

[0142] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A hardware acceleration method for a pooling algorithm, characterized in that: The hardware acceleration method of the pooling algorithm comprises the following steps: When a calculation instruction is detected, non-filled data in the calculation window is acquired through a data sorting module, and the non-filled data is sent to the first calculation module; Inputting first filling data to the first calculation module through a control module, and obtaining a first calculation result according to the non-filling data and the first filling data through the first calculation module; Inputting the first calculation result into a second calculation module through the first calculation module, and inputting second filling data into the second calculation module through the control module; Obtaining a second calculation result according to the first calculation result and the second filling data by the second calculation module; The step of inputting the first filling data to the first calculation module through the control module includes: Determining a first serial port to be enabled corresponding to the first computing module according to the number of padding data rows above or below the target image, the height of the target image, the accumulated column step length corresponding to the computing window, and the window height corresponding to the computing window, wherein the target image is obtained according to the computing instruction when the computing instruction is detected; Performing an enabling operation on the first serial port to be enabled through the control module, so as to input first filling data into the first calculation module; The step of inputting the second filling data to the second calculation module through the control module includes: Determine the second serial port to be enabled corresponding to the second calculation module according to the number of padded data columns on the left or right side of the target image, the width of the target image, the accumulated value of the row stride corresponding to the calculation window, and the window width corresponding to the calculation window; According to a first preset clock cycle, the second serial port to be enabled is enabled by the control module to input second filling data into the second calculation module.

2. The hardware acceleration method of the pooling algorithm according to claim 1, characterized in that: The step of acquiring the non-filled data in the calculation window by the data sorting module when the calculation instruction is detected, and sending the non-filled data to the first calculation module comprises: When a calculation instruction is detected, the target image is selected through the calculation window according to a preset rule; The non-filled data of the portion framed by the calculation window in the target image is identified and acquired through the data sorting module, and the non-filled data located in the same column in the calculation window are sent to the first calculation module respectively.

3. The hardware acceleration method of the pooling algorithm according to claim 1, characterized in that: The step of obtaining a first calculation result according to the non-filled data and the first filled data by the first calculation module includes: The non-filled data and the first filled data in each column of the calculation window are input into an adder or a comparator through the first calculation module to obtain a first calculation result.

4. The hardware acceleration method of the pooling algorithm according to claim 1, characterized in that: The step of obtaining a second calculation result according to the first calculation result and the second filling data by the second calculation module includes: The first calculation result and the second filling data are stored in a register through the second calculation module, and the first calculation result and the second filling data are input into an adder and a divider according to a second preset clock cycle, or the first calculation result and the second filling data are input into a comparator to obtain a second calculation result.

5. The hardware acceleration method of the pooling algorithm according to claim 1, characterized in that: After the step of obtaining the second calculation result according to the first calculation result and the second filling data by the second calculation module, the hardware acceleration method of the pooling algorithm further includes: According to the row step or column step corresponding to the calculation window, the calculation window is controlled to move according to a preset movement rule, and the steps are performed: non-filled data in the calculation window is obtained through a data sorting module to obtain a target feature map.

6. A hardware acceleration device for a pooling algorithm, characterized in that: The hardware acceleration device of the pooling algorithm includes: An acquisition module, configured to acquire non-filled data in the calculation window through the data sorting module when a calculation instruction is detected, and send the non-filled data to the first calculation module; a first input module, configured to input first filling data to the first calculation module through a control module, and obtain a first calculation result according to the non-filling data and the first filling data through the first calculation module; a second input module, configured to input the first calculation result into a second calculation module through the first calculation module, and to input second filling data into the second calculation module through the control module; a calculation module, configured to obtain a second calculation result according to the first calculation result and the second filling data through the second calculation module; The first input module is further used to determine the first serial port to be enabled corresponding to the first calculation module according to the number of rows of padding data above or below the target image, the height of the target image, the accumulated value of the column step corresponding to the calculation window, and the window height corresponding to the calculation window, wherein the target image is obtained according to the calculation instruction when the calculation instruction is detected; and the first serial port to be enabled is enabled by the control module to input the first padding data to the first calculation module; The second input module is also used to determine the second serial port to be enabled corresponding to the second calculation module according to the number of padding data columns on the left or right side of the target image, the width of the target image, the accumulated value of the row step corresponding to the calculation window, and the window width corresponding to the calculation window; according to the first preset clock cycle, the second serial port to be enabled is enabled through the control module to input the second padding data into the second calculation module.

7. A hardware acceleration system for a pooling algorithm, characterized in that: The hardware acceleration system of the pooling algorithm includes: a hardware accelerator, a memory, a processor, and a hardware acceleration program of the pooling algorithm stored in the memory and executable on the processor. When the hardware acceleration program of the pooling algorithm is executed by the processor, the steps of the hardware acceleration method of the pooling algorithm as described in any one of claims 1 to 5 are implemented.

8. A readable storage medium, characterized in that: The readable storage medium is a computer-readable storage medium, on which a hardware acceleration program of a pooling algorithm is stored. When the hardware acceleration program of the pooling algorithm is executed by a processor, the steps of the hardware acceleration method of the pooling algorithm as described in any one of claims 1 to 5 are implemented.

Images