Block computing method, system, device and storage medium based on data flow

Abstract

The embodiment of the present invention discloses a block calculation method, system, device and storage medium based on data flow. The method includes: determining whether the buffer in the neural network calculation engine can store the input feature map to be calculated; if the buffer cannot store the input feature map to be calculated, dividing the input feature map to be calculated into blocks; and driving the neural network calculation engine to calculate the input feature map to be calculated after the block calculation to obtain an output feature map. The embodiment of the present invention improves the calculation efficiency of the neural network based on the data flow architecture when the input feature map is too large.

Description

Technical Field

[0001] The embodiments of the present invention relate to neural network technology, and in particular to a block computing method, system, device and storage medium based on data flow. Background Art

[0002] With the rapid development of computers, the calculation of neural network data is becoming more and more important. The calculation of neural networks requires a large amount of data. The traditional instruction set architecture has to go through several stages to complete a complete operation, including the instruction fetch stage, instruction decoding stage, instruction execution stage, memory access stage, and result write-back stage. The entire process is extremely inefficient, while the data flow architecture completes the entire calculation process based on the continuous flow of data without the participation of the instruction set. Compared with the instruction set architecture, the data flow architecture can maximize the performance and efficiency of the chip, but because the data flow architecture requires data to flow at all times, it places strict requirements on storage, bus, and bandwidth.

[0003] The data flow architecture relies on the fact that before the neural network computing engine can start calculating, it needs to use the direct memory access device (DMA) to place the input feature map of a certain layer in the data buffer on the chip, and then let the neural network computing engine start calculating.

[0004] However, due to the limitation of hardware resources, for the computing layer with too large input feature map, the data buffer in the neural network computing engine cannot load it all, which greatly affects the computing efficiency of the data flow architecture. Summary of the invention

[0005] Embodiments of the present invention provide a data stream-based block calculation method, system, device and storage medium to improve the calculation efficiency of a neural network based on a data stream architecture when the input feature map is too large.

[0006] To achieve this purpose, an embodiment of the present invention provides a block calculation method based on data stream, the method comprising:

[0007] Determine whether the buffer in the neural network computing engine can store the input feature map to be calculated;

[0008] If the buffer cannot store the input feature map to be calculated, dividing the input feature map to be calculated into blocks;

[0009] The neural network computing engine is driven to calculate the input feature map to be calculated after the blocks are divided to obtain the output feature map.

[0010] Furthermore, before dividing the input feature map to be calculated into blocks, the method includes:

[0011] The input feature map to be calculated is converted into a three-dimensional matrix, where the three-dimensional matrix includes a plurality of original blocks.

[0012] Furthermore, dividing the input feature map to be calculated into blocks includes:

[0013] The original blocks of the three-dimensional matrix are divided into blocks along the height direction or the length direction of the three-dimensional matrix.

[0014] Furthermore, dividing the original blocks of the three-dimensional matrix along the height direction or the length direction of the three-dimensional matrix includes:

[0015] Obtaining the size of each of the original blocks;

[0016] Calculate the height or length of each original block according to the size of each original block;

[0017] determining an overlap between each of the original blocks;

[0018] Determine the number of divided original blocks after the original block is divided according to the height or length of each of the original blocks in the overlapping part;

[0019] The original block is divided into blocks according to the number of the original blocks after the division.

[0020] Furthermore, after dividing the original blocks of the three-dimensional matrix along the height direction or the length direction of the three-dimensional matrix, the method further comprises:

[0021] The parameter information of the original block after the block division is updated.

[0022] Further, the updating of the parameter information of the original block after the block division includes:

[0023] Calculating parameter information of the original block after the block division according to the number of the original blocks after the block division;

[0024] The parameter information of the original block after segmentation is converted into register information to be configured to each of the original block after segmentation and the direct memory access device in the neural network computing engine.

[0025] Furthermore, the calculation process of the neural network computing engine includes convolution, pooling, convolution superposition pooling, deconvolution, dilated convolution, data replication and bilinear upsampling.

[0026] On the one hand, an embodiment of the present invention further provides a block computing system based on data flow, the system comprising:

[0027] A judgment module, used to judge whether the buffer in the neural network computing engine can store the input feature map to be calculated;

[0028] A block division module, used for dividing the input feature map to be calculated into blocks if the buffer cannot store the input feature map to be calculated;

[0029] A computing module is used to drive the neural network computing engine to calculate the input feature map to be calculated after the block is divided to obtain an output feature map.

[0030] On the other hand, an embodiment of the present invention further provides a computer device, which includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement a method provided in any embodiment of the present invention.

[0031] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method provided in any embodiment of the present invention.

[0032] The embodiment of the present invention determines whether the buffer in the neural network computing engine can store the input feature map to be calculated; if the buffer cannot store the input feature map to be calculated, the input feature map to be calculated is divided into blocks; and the neural network computing engine is driven to calculate the divided input feature map to be calculated to obtain an output feature map. This solves the problem that the data buffer in the neural network computing engine cannot load all the calculation layers of the input feature map that is too large, which greatly affects the computing efficiency of the data flow architecture. This achieves the effect of improving the computing efficiency of the neural network based on the data flow architecture when the input feature map is too large. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flowchart of a block calculation method based on data stream provided in the first embodiment of the present invention;

[0034] Figure 2 1 is a flow chart of a block calculation method based on data stream provided in Embodiment 2 of the present invention;

[0035] Figure 3 It is a schematic diagram of the structure of a three-dimensional matrix of an input feature map provided in Embodiment 2 of the present invention;

[0036] Figure 4 is a flowchart of step S230 provided in the second embodiment of the present invention;

[0037] Figure 5 It is a schematic diagram of the structure of the original block when the convolution or pooling algorithm is adopted according to the second embodiment of the present invention;

[0038] Figure 6 It is a schematic diagram of the structure of the original block when the convolution or pooling algorithm is adopted according to the second embodiment of the present invention;

[0039] Figure 7a It is a schematic diagram of the structure of the original block in the convolution process when the convolution stacking pooling algorithm is adopted according to the second embodiment of the present invention;

[0040] Figure 7b It is a schematic diagram of the structure of the original block in the pooling process when the convolution stacking pooling algorithm is adopted according to the second embodiment of the present invention;

[0041] Figure 7c It is a schematic diagram of the structure of the original block after block division output when the convolution stacking pooling algorithm is adopted according to the second embodiment of the present invention;

[0042] Figure 8a is a schematic diagram of the structure of the original block when the deconvolution algorithm is adopted according to the second embodiment of the present invention;

[0043] Figure 8b It is a schematic diagram of the structure of the original block after the block division output when the deconvolution algorithm is adopted according to the second embodiment of the present invention;

[0044] Figure 9a is a schematic diagram of the structure of the original block when the dilated convolution algorithm is adopted according to the second embodiment of the present invention;

[0045] Figure 9b It is a schematic diagram of the structure of the original block after block division output when the dilated convolution algorithm is adopted according to the second embodiment of the present invention;

[0046] Fig.10a It is a schematic diagram of the structure of the original block without setting the offset value when the bilinear upsampling algorithm is adopted according to the second embodiment of the present invention;

[0047] Fig.10b It is a structural schematic diagram of an original block of an offset value when a bilinear upsampling algorithm is used according to the second embodiment of the present invention;

[0048] Fig.11a It is a schematic diagram of the structure of the original block without setting the offset value when the data replication algorithm is adopted according to the second embodiment of the present invention;

[0049] Fig.11b It is a schematic diagram of the structure of the original block of the offset value when the data duplication algorithm is adopted according to the second embodiment of the present invention;

[0050] Fig.12 It is a structural diagram of a block computing system based on data flow provided in Embodiment 3 of the present invention;

[0051] Fig.13 A schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0053] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0054] In addition, the terms "first", "second", etc. can be used in this article to describe various directions, actions, steps or elements, but these directions, actions, steps or elements are not limited by these terms. These terms are only used to distinguish a first direction, action, step or element from another direction, action, step or element. For example, without departing from the scope of the present application, the first module can be referred to as the second module, and similarly, the second module can be referred to as the first module. Both the first module and the second module are modules, but they are not the same module. The terms "first", "second", etc. cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can expressly or implicitly include one or more of the features. In the description of the embodiment of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0055] Embodiment 1

[0056] like Figure 1 As shown, the first embodiment of the present invention provides a block calculation method, system, device and storage medium based on data stream, and the method includes:

[0057] S110, determining whether the buffer in the neural network computing engine can store the input feature map to be calculated.

[0058] S120: If the buffer cannot store the input feature map to be calculated, divide the input feature map to be calculated into blocks.

[0059] In this embodiment, before the calculation of the neural network based on the data flow architecture begins, it is necessary to determine whether the buffer in the neural network calculation engine can store the input feature map to be calculated. Among them, the input feature map to be calculated includes multiple layers of input feature maps, and the judgment is made layer by layer. If the input feature map buffer of the current layer can store it, then there is no need to perform block processing. If the input feature map of the next layer cannot be stored in the cache, it means that the input feature map of that layer needs to be processed in blocks until the input feature maps of all layers of the input feature map to be calculated are judged.

[0060] S130, driving the neural network computing engine to calculate the input feature map to be calculated after the blocks are divided to obtain an output feature map.

[0061] In this embodiment, after the input feature map of one layer is judged, if it does not need to be processed in blocks, the neural network calculation engine is directly driven to calculate the input feature map of the layer to obtain the output feature map. After the input feature map of the next layer is judged, if it needs to be processed in blocks, the neural network calculation engine is driven to calculate the input feature map of the layer after the block processing to obtain the output feature map. Because the neural network calculation in the data flow architecture is continuous, after the input feature map of each layer is judged, if it needs to be processed in blocks, it is directly calculated after the block processing. If it does not need to be processed in blocks, it is directly calculated. While the input feature map of this layer is calculated, it is judged whether the input feature map of the next layer needs to be processed in blocks, so that the entire calculation process is continuous. Even if there is an input feature map that cannot be stored in the buffer, the data flow architecture can be realized. It should be noted that the input feature map after blocking is equivalent to increasing the number of layers of the input feature map, and the neural network calculation engine or other hardware does not perceive it, and will not increase hardware consumption.

[0062] The embodiment of the present invention determines whether the buffer in the neural network computing engine can store the input feature map to be calculated; if the buffer cannot store the input feature map to be calculated, the input feature map to be calculated is divided into blocks; and the neural network computing engine is driven to calculate the divided input feature map to be calculated to obtain an output feature map. This solves the problem that the data buffer in the neural network computing engine cannot load all the calculation layers of the input feature map that is too large, which greatly affects the computing efficiency of the data flow architecture. This achieves the effect of improving the computing efficiency of the neural network based on the data flow architecture when the input feature map is too large.

[0063] Embodiment 2

[0064] like Figure 2 As shown, the second embodiment of the present invention provides a block calculation method, system, device and storage medium based on data stream. The second embodiment of the present invention is further explained on the basis of the first embodiment of the present invention. The method includes:

[0065] S210, determining whether the buffer in the neural network computing engine can store the input feature map to be calculated.

[0066] S220: If the buffer cannot store the input feature map to be calculated, convert the input feature map to be calculated into a three-dimensional matrix, where the three-dimensional matrix includes a plurality of original blocks.

[0067] S230 , dividing the original blocks of the three-dimensional matrix into blocks along the height direction or the length direction of the three-dimensional matrix.

[0068] In this embodiment, Figure 3 As shown, when the input feature map to be calculated is divided into blocks, it is first necessary to convert the input feature map to be calculated into a three-dimensional matrix, wherein the three-dimensional matrix includes multiple original blocks, and the three-dimensional matrix includes three directions, namely, a length (Channel) direction 10, a height (Height) direction 20, and a width (Width) direction 30. Since the data of the input feature map to be calculated are arranged in the order of Height-Width-Channel in the buffer, that is, arranged in the order of height direction 20-width direction 30-length direction 10, in actual situations, most of the input feature maps to be calculated are mainly cut and divided into blocks in the height direction 20, and in some cases, the length direction 10 is cut and divided into blocks.

[0069] Specifically, after obtaining the feature matrix of the current layer input feature map in the input feature map to be calculated, the original blocks of the three-dimensional matrix are divided into blocks along the height direction 20 or the length direction 10 of the three-dimensional matrix to obtain the original blocks after division, that is, the division of the input feature map of the current layer is completed, and the calculation stage is entered. Then, it is judged whether the input feature map of the next layer needs to be divided into blocks. If necessary, the input feature map of the next layer is converted into a three-dimensional matrix and divided into blocks to obtain the original blocks after division, until all the input feature maps that need to be divided into blocks in the input feature map to be calculated are divided into blocks, and the input feature maps of each layer are immediately input into the buffer after the block processing for the neural network calculation engine to obtain and calculate.

[0070] Further, such as Figure 4 As shown, step S230 may specifically include:

[0071] S231. Obtain the size of each original block.

[0072] S232. Calculate the height or length of each original block according to the size of each original block.

[0073] S233: Determine the overlapping portion between each original block.

[0074] S234: Determine the number of divided original blocks after the original blocks are divided according to the height or length of each original block in the overlapping part.

[0075] S235 , dividing the original block into blocks according to the number of the original blocks after the block division.

[0076] In this embodiment, in the process of dividing the original blocks of the three-dimensional matrix along the height direction or the length direction of the three-dimensional matrix, specifically, firstly, the size of each original block of the three-dimensional matrix is ​​obtained, then the height or length of each original block is calculated according to the size of each original block, and the overlapping part between each original block is determined, then the number of divided original blocks after the original block is divided is determined according to the height or length of each original block in the overlapping part, and finally the original block is divided according to the number of divided original blocks. Among them, the calculation process of the neural network calculation engine includes convolution, pooling, convolution plus pooling, deconvolution, dilated convolution, data copy and bilinear upsampling.

[0077] In one embodiment, if Figure 5 As shown, Figure 5 The asterisk in represents a two-dimensional data point (not indicating the length direction of the three-dimensional matrix), and the dotted box represents the convolution kernel of the original block. When the original block needs to use the convolution or pooling algorithm, it can be found that there is an overlap between the two original blocks, where the number of rows in the overlapping part can be obtained by the following formula:

[0078] o=kh-s

[0079] Among them, o (overlap) is the number of rows of the overlapping part, kh (kernel height) is the height of the convolution kernel, and s (stride) is the stride of the convolution kernel. After determining the size of the overlapping part, the number of original blocks after the block division in the height direction can be calculated according to the following formula:

[0080] cnt=(ih-ih f -ih l +ih m -1) / (ih m -o)+2

[0081] Among them, Figure 6 As shown, cnt is the number of original blocks after segmentation, ih is the input height, ih f and ih l are the input heights of the first and last original blocks, ih mis the input height of the middle raw block. The reason they are different is that the first raw block and the last raw block contain padding. In order to maximize the use of hardware resources, the amount of data that can be stored in the first raw block and the last raw block will be different from that of ordinary raw blocks.

[0082] If the blocks are divided according to the length information, there is no need to consider padding. The number of original blocks after the blocks are divided is:

[0083] cnt=(c-1) / c m +1

[0084] Among them, c is the number of original blocks in the original length direction, c m is the number of original blocks in the length direction after segmentation.

[0085] In one embodiment, if Figure 7a , Figure 7b and Figure 7c As shown in the figure, when the original block needs to use the convolution superposition pooling algorithm, compared with the algorithm using convolution or pooling alone, the overlapping part of the pooling operator needs to be considered. The calculation formula of the overlapping part is:

[0086] o=kh+s*(kh p -s p -1)

[0087] Among them, kh p Represents the height of the pooled convolution kernel, s p Indicates the stride of the pooled convolution kernel.

[0088] In one embodiment, if Figure 8a and Figure 8b As shown in the figure, when the original block needs to use the deconvolution algorithm, because the deconvolution algorithm inserts padding in the middle of the data before performing the convolution operation, the original block in the middle will have padding division. Here we assume that the original block in the middle has upper padding but no lower padding. The calculation formula for the overlapping part is:

[0089] o=kh / s

[0090] The formula result is an integer, and pad is filled on top. up The calculation formula is:

[0091] pad up =kh%s

[0092] In one embodiment, if Figure 9a and Figure 9b As shown in the figure, when the original block needs to use the dilated convolution algorithm, it can be regarded as the convolution kernel is enlarged and then the convolution is performed normally. The convolution kernel is recalculated as:

[0093] kh=kh*dr-dr+1

[0094] Where dr is the dilated rate. The hollow asterisks in the figure represent the two-dimensional data points of a valid row of an original block, that is, a convolution kernel does not need all the two-dimensional data points to perform calculations, so in order to save hardware resources, only the two-dimensional data points of the valid row are transmitted to the buffer.

[0095] In one embodiment, if Fig.10a As shown in the figure, when the original block needs to use the bilinear upsampling algorithm, the endpoints in multiple matrices connected by dotted lines in the figure are the input two-dimensional data points, and the remaining points are the two-dimensional data output points. Since the operator has no convolution kernel, each two-dimensional data output point requires the corresponding four two-dimensional data input points to calculate, so the overlap between each original block is fixed to 1. In order to ensure the accuracy of the calculation, as shown in Fig.10b As shown, except for the first original block, the subsequent original blocks require an offset value to record the position of the two-dimensional data input point in the height direction (because the value of each two-dimensional data output point is related to its mapping location on the two-dimensional data input point), which makes the output parameters of each original block need to be recalculated when the original block parameters are calculated in a loop.

[0096] In one embodiment, if Fig.11a As shown, when the original block needs to use a data replication algorithm, it is similar to the block method of bilinear upsampling, such as Fig.11b As shown, in addition to the overlapping part being 0, the current original block is also required to calculate an offset value to indicate the starting position of the first row of two-dimensional data output points of the next original block.

[0097] It should be noted that the operators of the above algorithms can be combined, that is, the corresponding block algorithms can also be combined accordingly to achieve block division for a variety of different algorithms.

[0098] S240: Calculate parameter information of the original blocks after segmentation according to the number of the original blocks after segmentation.

[0099] S250, converting the parameter information of the original block after segmentation into register information to configure the direct memory access device in each original block after segmentation and the neural network computing engine.

[0100] In this embodiment, after the block division is completed, the original block after the block division is obtained. Since the block division will cause the parameter change, if the original block of the input feature map of the current layer is processed by block division, the parameter information of the original block after the block division needs to be updated. Specifically, the parameter information of the original block after the block division is first calculated according to the number of the original blocks after the block division, and then the parameter information of the original block after the block division is converted into register information to be configured for each original block after the block division, and updated to the direct memory access device in the neural network calculation engine, and finally the address information of the next original block after the block division is calculated according to the parameter information of the original block after the previous block division, so that the parameter information and address information of each original block after the block division are updated, and the neural network calculation engine can calculate the original block after the block division.

[0101] S260, driving the neural network computing engine to calculate the input feature map to be calculated after the blocks are divided to obtain an output feature map.

[0102] The embodiment of the present invention obtains the size of each original block. The height or length of each original block is calculated according to the size of each original block. The overlapping part between each original block is determined. The number of original blocks after the original block is divided is determined according to the height or length of each original block in the overlapping part. The original block is divided according to the number of original blocks after the division. The problem that the data buffer in the neural network computing engine cannot load all the computing layers of the input feature map that is too large, which greatly affects the computing efficiency of the data flow architecture, is solved, and the effect of improving the computing efficiency of the neural network based on the data flow architecture is achieved when the input feature map is too large.

[0103] Embodiment 3

[0104] like Fig.12 As shown, the third embodiment of the present invention provides a data flow-based block computing system 100. The data flow-based block computing system 100 provided in the third embodiment of the present invention can execute the data flow-based block computing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. The data flow-based block computing system 100 includes a judgment module 200, a block module 300 and a computing module 400.

[0105] Specifically, the judgment module 200 is used to judge whether the buffer in the neural network computing engine can store the input feature map to be calculated; the blocking module 300 is used to block the input feature map to be calculated if the buffer cannot store the input feature map to be calculated; the calculation module 400 is used to drive the neural network computing engine to calculate the input feature map to be calculated after blocking to obtain the output feature map.

[0106] In this embodiment, the blocking module 300 is also used to convert the input feature map to be calculated into a three-dimensional matrix, and the three-dimensional matrix includes multiple original blocks. The blocking module 300 is specifically used to block the original blocks of the three-dimensional matrix along the height direction or the length direction of the three-dimensional matrix. The blocking module 300 is also specifically used to obtain the size of each original block; calculate the height or length of each original block according to the size of each original block; determine the overlapping part between each original block; determine the number of original blocks after the original block is blocked according to the height or length of each original block in the overlapping part; and block the original block according to the number of original blocks after blocking.

[0107] Furthermore, the block module 300 is also used to update the parameter information of the original block after the block. The block module 300 is specifically used to calculate the parameter information of the original block after the block according to the number of the original blocks after the block; convert the parameter information of the original block after the block into register information to configure each original block after the block and the direct memory access device in the neural network computing engine.

[0108] In this embodiment, the calculation process of the neural network calculation engine includes convolution, pooling, convolution superposition pooling, deconvolution, dilated convolution, data replication and bilinear upsampling.

[0109] Embodiment 4

[0110] Fig.13 This is a schematic diagram of the structure of a computer device 12 provided in Embodiment 4 of the present invention. Fig.13 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Fig.13 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0111] like Fig.13 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0112] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0113] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0114] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Fig.13 not shown, usually called a "hard drive"). Although Fig.13 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0115] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0116] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the computer device 12, and / or may communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the computer device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the computer device 12, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0117] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, for example, implementing the method provided in the embodiment of the present invention:

[0118] Determine whether the buffer in the neural network computing engine can store the input feature map to be calculated;

[0119] If the buffer cannot store the input feature map to be calculated, dividing the input feature map to be calculated into blocks;

[0120] The neural network computing engine is driven to calculate the input feature map to be calculated after the blocks are divided to obtain the output feature map.

[0121] Embodiment 5

[0122] Embodiment 5 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method provided in all the embodiments of the present invention is implemented:

[0123] Determine whether the buffer in the neural network computing engine can store the input feature map to be calculated;

[0124] If the buffer cannot store the input feature map to be calculated, dividing the input feature map to be calculated into blocks;

[0125] The neural network computing engine is driven to calculate the input feature map to be calculated after the blocks are divided to obtain the output feature map.

[0126] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0127] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0128] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0129] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A block calculation method based on data flow, characterized in that: include: Determine whether the buffer in the neural network computing engine can store the input feature map to be calculated; If the buffer cannot store the input feature map to be calculated, dividing the input feature map to be calculated into blocks; Driving the neural network computing engine to calculate the input feature map to be calculated after the blocks are divided to obtain an output feature map; The step of dividing the input feature map to be calculated into blocks includes: Converting the input feature map to be calculated into a three-dimensional matrix, wherein the three-dimensional matrix includes a plurality of original blocks; The dividing the input feature map to be calculated into blocks comprises: Dividing the original blocks of the three-dimensional matrix into blocks along the height direction or the length direction of the three-dimensional matrix; The dividing the original blocks of the three-dimensional matrix along the height direction or the length direction of the three-dimensional matrix comprises: Obtaining the size of each of the original blocks; Calculate the height or length of each original block according to the size of each original block; determining an overlap between each of the original blocks; Determine the number of original blocks after the original block is divided according to the height or length of each original block in the overlapping part; The original block is divided into blocks according to the number of the original blocks after the division.

2. The method according to claim 1, characterized in that After dividing the original blocks of the three-dimensional matrix along the height direction or the length direction of the three-dimensional matrix, the method further comprises: The parameter information of the original block after the block division is updated.

3. The method according to claim 2, characterized in that The parameter information of the original block after updating the block includes: Calculating parameter information of the original block after the block division according to the number of the original blocks after the block division; The parameter information of the original block after segmentation is converted into register information to be configured to each of the original block after segmentation and the direct memory access device in the neural network computing engine.

4. The method according to claim 1, characterized in that: The computing process of the neural network computing engine includes convolution, pooling, convolution superposition pooling, deconvolution, dilated convolution, data replication and bilinear upsampling.

5. A block computing system based on data flow, characterized in that: include: A judgment module, used to judge whether the buffer in the neural network computing engine can store the input feature map to be calculated; A block division module, used for dividing the input feature map to be calculated into blocks if the buffer cannot store the input feature map to be calculated; A calculation module, used to drive the neural network calculation engine to calculate the input feature map to be calculated after the blocks are divided to obtain an output feature map; The block module is also used to convert the input feature map to be calculated into a three-dimensional matrix, where the three-dimensional matrix includes a plurality of original blocks; The block division module is specifically used to divide the original blocks of the three-dimensional matrix into blocks along the height direction or the length direction of the three-dimensional matrix; The block segmentation module is further specifically used to obtain the size of each of the original blocks; Calculate the height or length of each original block according to the size of each original block; determining an overlap between each of the original blocks; Determine the number of original blocks after the original block is divided according to the height or length of each original block in the overlapping part; The original block is divided into blocks according to the number of the original blocks after the division.

6. A computer device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images