Data processing method and device of neural network, computer device and storage medium

Abstract

Embodiments of the present application disclose a data processing method and device of a neural network, computer equipment and a storage medium. The method comprises: obtaining an input feature map of a layer of a neural network to be processed and a cache size of a neural network engine; dividing the input feature map according to the cache size, so that the cache of the neural network engine can accommodate each block result; and reassembling each block result to obtain an output feature map. The technical solution provided by the embodiments of the present application divides the feature map of the neural network, so that each block result can be cached and stored, the number of DMA calls is reduced, and the computing power and real-time performance of the neural network are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of deep learning, and particularly relate to a data processing method and device of a neural network, a computer device and a storage medium. BACKGROUND

[0002] With the rapid development of deep learning, in order to better simulate real network scenarios, the scale of operators in a single network model of deep learning is constantly increasing. Based on existing hardware and software architectures, the difficulty of data adaptation is increasing. With the increasing scale of AI networks, the neural network needs to move data between DDR and cache more frequently during calculation, which affects the computing power of neural network calculation. Meanwhile, the efficiency of moving data between DDR and cache also affects the real-time performance of neural network. SUMMARY

[0003] Embodiments of the present application provide a data processing method and device of a neural network, a computer device and a storage medium, to reduce the number of DMA (a hardware technology responsible for moving data between DDR and cache) calls, thereby improving the computing power and real-time performance of neural network calculation.

[0004] In a first aspect, embodiments of the present application provide a data processing method of a neural network, which comprises:

[0005] obtaining an input feature map of a layer of a neural network to be processed and a cache size of a neural network engine;

[0006] dividing the input feature map into blocks according to the cache size, so that the cache of the neural network engine can accommodate each block result;

[0007] re-assembling each block result to obtain an output feature map.

[0008] Optionally, the dividing the input feature map into blocks according to the cache size comprises:

[0009] determining a feature map block direction according to the type of the neural network to be processed, and a height of an overlapping area between each two adjacent block results along the feature map block direction;

[0010] determining a data block size of each block result along the feature map block direction according to the cache size and the height of the overlapping area.

[0011] Optionally, the height of the overlapping area is fixed, and the determining the height of the overlapping area between each two adjacent block results along the feature map block direction according to the type of the neural network to be processed comprises:

[0012] The overlap region height is determined according to a convolution kernel size of a current layer of the neural network to be processed and a convolution kernel moving step.

[0013] Optionally, the overlap region height is floating, and the overlap region height between every two adjacent block results in the feature map block direction is determined according to the type of the neural network to be processed, including:

[0014] The overlap region height is determined according to a start row index of a current block result and an end row index of a previous block result.

[0015] Optionally, the data block size of each block result in the feature map block direction is determined according to the cache size and the overlap region height, including:

[0016] The number of blocks in the feature map block direction is determined according to the cache size and the overlap region height.

[0017]

[0018] Wherein, block_num represents the number of blocks, ceil() represents rounding up, float() represents converting to a floating point number, i represents the length of the input feature map in the feature map block direction, block_size represents the length of the cache size in the feature map block direction, and overlap represents the overlap region height.

[0019] The data block size is determined according to the number of blocks.

[0020] Optionally, after the data block size of each block result in the feature map block direction is determined according to the cache size and the overlap region height, the method further includes:

[0021] Convolution kernel output adaptation and DMA address adaptation are performed according to the data block size.

[0022] Optionally, before the data block size of each block result in the feature map block direction is determined according to the cache size and the overlap region height, the method further includes:

[0023] The block mode is matched according to the type of the neural network to be processed, so as to determine the calculation method of the data block size according to the block mode.

[0024] The minimum parameter set required by the block process is adapted according to the block mode, the feature map block direction and the overlap region height.

[0025] In a second aspect, an embodiment of the present application further provides a data processing device of a neural network, which comprises:

[0026] An input feature map acquisition module is configured to acquire an input feature map of a layer of a neural network to be processed and a cache size of a neural network engine.

[0027] A feature map partitioning module is configured to partition the input feature map according to the cache size, so that each partitioned result can be accommodated in the cache of the neural network engine.

[0028] An output feature map assembling module is configured to reassemble each partitioned result to obtain an output feature map.

[0029] In a third aspect, an embodiment of the present application further provides a computer device, which comprises:

[0030] one or more processors;

[0031] a memory configured to store one or more programs;

[0032] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method of the neural network provided by any embodiment of the present application.

[0033] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the data processing method of the neural network provided by any embodiment of the present application.

[0034] The data processing method of the neural network provided by the embodiment of the present application firstly acquires an input feature map of a layer of a neural network to be processed and a cache size of a neural network engine, then partitions the input feature map according to the cache size, so that each partitioned result can be accommodated in the cache of the neural network engine, and reassembles each partitioned result to obtain a processed output feature map. The data processing method of the neural network provided by the embodiment of the present application partitions the feature map of the neural network, so that each partitioned result can be cached, the data can be retained in the cache, the number of DMA calls is reduced, and the computing power and real-time performance of the neural network are improved. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 a flowchart of the data processing method of the neural network provided by the first embodiment of the present application;

[0036] Figure 2 a structural schematic diagram of the data processing device of the neural network provided by the second embodiment of the present application;

[0037] Figure 3 a structural schematic diagram of the computer device provided by the third embodiment of the present application. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0039] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0040] Example 1

[0041] Figure 1 This is a flowchart of a neural network data processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where neural networks are used for related calculations. This method can be executed by the neural network data processing device provided in this embodiment of the invention. This device can be implemented in hardware and / or software, and is generally integrated into a computer device. Figure 1 As shown, the specific steps include the following:

[0042] S11. Obtain the input feature map of the first layer of the neural network to be processed and the cache size of the neural network engine.

[0043] Specifically, the input and output of a neural network are mainly represented using feature maps. The input data of each layer of the neural network, i.e., the input feature map, can be processed separately. The method provided in this embodiment is applicable to feature maps of each layer. The processing of all feature maps can be completed in batches according to a certain order, and the currently processed feature map can be used as the input feature map. The input feature map can be multi-dimensional data, specifically a three-dimensional data block. Since the feature map is to be divided into blocks for complete storage in a cache, the cache size of the neural network engine can be obtained first, so that the subsequent block division process can be completed based on this cache size.

[0044] S12. Divide the input feature map into blocks according to the cache size so that the cache of the neural network engine can accommodate the results of each block.

[0045] Specifically, after obtaining the cache size of the neural network engine, the input data can be cut according to the cache size, that is, the feature map is divided into blocks, so that the cache size is referred to so that each block result can be completely accommodated by the cache, so that the cut data can effectively utilize the cache.

[0046] Optionally, the dividing the input feature map according to the cache size comprises: determining a feature map block direction according to the type of the neural network to be processed, and determining a height of an overlapping region between every two adjacent block results in the feature map block direction; determining a data block size of each block result in the feature map block direction according to the cache size and the height of the overlapping region.

[0047] Specifically, a block order can be preset to divide the input feature map in each direction according to the block order until the cache can accommodate the data, and when a direction itself satisfies the condition of cache accommodation, the block process in the direction can be not performed. For example, for a three-dimensional data block, theoretically, the block can be performed in the height (h), width (w) and channel (c) directions, and assuming that the preset block order is hwc, it is determined whether the h direction needs to be divided (that is, whether the length of the input feature map in the h direction is greater than the length of the cache in the h direction), if yes, the input feature map is divided in the h direction, and if not, the h direction is fixed as the size of the convolution kernel, and then the w direction and the c direction are sequentially determined whether to be divided, and then the feature map block direction that needs to be divided is determined from each direction of the input feature map. In particular, some neural networks have specific direction requirements, such as GAP which can only be divided in the c direction and does not need to be dynamically divided, and the like, so the required feature map block direction can be determined according to the type of the neural network to be processed on the basis of the preset block order and in combination with the requirements of each neural network.

[0048] After the required feature map block direction is determined, each feature map block direction can be processed respectively, and if multiple directions need to be divided, the block principles in the multiple directions are the same, and one of the directions (such as the h direction) is described below. In the process of dividing, the overlapping part between two adjacent block results needs to be considered, so that the data of the two block results can be completely calculated, and therefore, the height of the overlapping region between each two adjacent block results needs to be determined in each feature map block direction, so that the data block size of each block result in the corresponding feature map block direction can be determined according to the cache size and the height of the overlapping region.

[0049] Further optionally, the determining the data block size of each of the block results in the feature map block direction according to the cache size and the overlap height comprises: determining the number of blocks in the feature map block direction according to the cache size and the overlap height.

[0050]

[0051] wherein block_num represents the number of blocks, ceil() represents rounding up, float() represents converting to a floating-point number, i represents the length of the input feature map in the feature map block direction, block_size represents the length of the cache size in the feature map block direction, and overlap represents the overlap height; the data block size is determined according to the number of blocks. Specifically, the input feature map can be first divided into several blocks in the feature map block direction according to the above formula, and then the input feature map can be divided according to the determined number of blocks and the cache size, so as to determine the data block size of each block result. The pad compensation is mainly to solve the problem that some neural network operators (such as deconvolution) need to add redundant data in front and back of the data, and the pad information is contained in i. In addition, block_size can be modified according to different neural network operators, such as the convolutional neural network needs block_size to be an integer multiple of the height of the convolution kernel.

[0052] On the basis of the above technical solutions, optionally, the overlap height is fixed, and the determining the overlap height between every two adjacent block results in the feature map block direction according to the type of the neural network to be processed comprises: determining the overlap height according to the convolution kernel size and the convolution kernel moving step of the current layer of the neural network to be processed. Alternatively, optionally, the overlap height is floating, and the determining the overlap height between every two adjacent block results in the feature map block direction according to the type of the neural network to be processed comprises: determining the overlap height according to the starting row index of the current block result and the ending row index of the previous block result.

[0053] Specifically, for different neural networks, the type of the overlap region can also be different. For example, for neural network types such as convolution, deconvolution, atrous convolution, pooling, and elementwise, by analyzing the relationship between the input and the output, it can be found that the overlap region height between two adjacent block results is fixed, and specifically can be overlap = kh-s, where overlap represents the overlap region height, kh represents the convolution kernel height (the length of the convolution kernel size in the corresponding feature map block direction), and s represents the convolution kernel moving step. At the same time, since the overlap region height is fixed, the required block number can be calculated in advance by the above calculation formula of the block number, and the data block size of each block result can be directly obtained.

[0054] For neural network types such as bilinear and upsample, the data will be expanded and reduced according to a non-integer scale, and the input data and the output data need to be able to be accommodated by the cache. If the input and the output can be accommodated by the cache after being blocked, it is equivalent to the above fixed overlap region height mode, but if the output cannot be accommodated by the cache, the size of the input needs to be reduced in reverse until the output can be accommodated by the cache. Therefore, for this type of neural network, the overlap region height can not be calculated in advance to a fixed value, that is, the overlap region height is floating. In this case, the data block size of each block result can be determined step by step, and adjustments can be made in real time according to each output. The overlap region between the upper and lower two adjacent block results is essentially the overlap between the first row of input of the current block result and the last row of input of the previous block result. Therefore, for the current block result, the corresponding overlap region height can be determined according to the starting row index of the current block result and the ending row index of the previous block result. Specifically, the following formula can be used:

[0055]

[0056] wherein, overlap represents the height of the overlap region, floor() represents the down rounding, oh_start_line represents the starting line index of the current block result, oh_last_line represents the ending line index of the last block result, and scale represents the expansion multiple of the neural network operator. The output line index of the current block result in the input can be calculated by oh_start_line / scale, but the result is not necessarily an integer, so floor() is used to obtain the corresponding integer, and when it cannot be divided, the calculated input line will be offset by one line (for the h direction block), so 1 is used for compensation when calculating. After the new overlap height is calculated each time, the data block size of the current block result can be determined according to the cache size and the overlap height by referring to the above process. At the same time, since the floating overlap height needs to be constantly traced back to find the appropriate value, the execution efficiency of the neural network operator is affected, so the fixed algorithm is not used to unify the floating algorithm in this embodiment.

[0057] On the basis of the above technical solutions, after the data block size of each block result in the feature map block direction is determined according to the cache size and the overlap height, the method further comprises: performing convolution kernel output adaptation and DMA address adaptation according to the data block size. Specifically, after the data block size of each block result is determined, all data block sizes and input feature maps can be transmitted to an output adapter to perform convolution kernel output adaptation and DMA address adaptation by the output adapter according to the data block size, and some other special adaptations and the like can also be performed, so as to obtain the related information of the output feature map.

[0058] Based on the above technical solution, optionally, before determining the data block size of each block result in the feature map block direction according to the cache size and the overlap region height, the method further includes: matching the block mode according to the type of the neural network to be processed, so as to determine the calculation method of the data block size according to the block mode; and adapting the minimum set of parameters required for the block process according to the block mode, the feature map block direction, and the overlap region height. Specifically, the method provided in this embodiment can be divided into two parts, namely the feature map block algorithm and the feature map adaptation algorithm. The two parts work together to achieve low coupling, ensuring that the feature map block algorithm does not depend on the specific neural network type. At the same time, it does not require the user to be very familiar with the neural network algorithm and feature map, reducing the user's learning cost. Moreover, for different neural networks, there are some common block principles. The feature map adaptation algorithm can also avoid a lot of redundant operations, thereby saving the computing power required for neural network execution and reducing the impact on the real-time performance of the calculation. Specifically, the feature map adaptation algorithm can be implemented using an input adapter. First, after obtaining the input feature map of one layer of the neural network to be processed, the input feature map is transmitted to the input adapter. The block partitioning mode is matched according to the type of the neural network, i.e., which sub-algorithm and output adaptation method are used, such as fixed mode or floating mode. This allows the calculation method for the data block size to be determined based on the partitioning mode. Then, an overlap region height can be determined. For the fixed mode, this yields the global overlap region height; for the floating mode, it yields the initial overlap region height of the first partitioning result. Subsequent iterations can backtrack to determine new overlap region heights, and padding compensation (for the specific neural network) can be determined. Next, it can be determined whether the input feature map needs to be partitioned, i.e., whether the data size of the input feature map exceeds the buffer size. If not, the algorithm can terminate, and the input feature map can be directly used as the output feature map. If so, the subsequent algorithms continue. Finally, the required feature map partitioning direction is determined; the specific process can be found above and will not be elaborated here. Finally, the minimum set of parameters required for the block segmentation process can be adapted based on the obtained parameters, thereby integrating the parameters required by the general block segmentation algorithm and facilitating unified maintenance of the parameters later. After obtaining the minimum set of parameters, it can be transmitted to the feature map block segmentation algorithm part for calculation. The feature map block segmentation algorithm part can calculate the data block size of each block result in each feature map block direction in a certain order according to the required feature map block direction, and then output it to the output adapter mentioned above.

[0059] S13. Reassemble each of the segmented results to obtain the output feature map.

[0060] Specifically, after the blocking process is completed, each block result can be reassembled according to the obtained blocking information to obtain the required output feature map. Subsequently, when the neural network is used for related calculation, part or all of the block results can be stored in the cache corresponding to the neural network engine according to the blocking information carried therein, so as to reduce the carrying frequency of DMA and ensure that the calculation result of each layer of the neural network can reach the expectation without loss of calculation accuracy.

[0061] The technical scheme provided by the embodiment of the application first acquires the input feature map of a layer of a neural network to be processed and the cache size of a neural network engine, then blocks the input feature map according to the cache size, so that the cache of the neural network engine can accommodate each block result, and then reassembles each block result to obtain the processed output feature map. By blocking the feature map of the neural network, each block result can be cached, so that the data can be retained in the cache, the calling frequency of DMA is reduced, and the computing power and real-time performance of the neural network are improved.

[0062] Embodiment Two

[0063] Figure 2 The structure diagram of the data processing device of the neural network provided by the embodiment two of the application is shown in the figure. The device can be realized by hardware and / or software, and can be integrated in a computer device, and is used for executing the data processing method of the neural network provided by any embodiment of the application. As shown in the figure, the device comprises: Figure 2

[0064] An input feature map acquisition module 21 is configured to acquire the input feature map of a layer of a neural network to be processed and the cache size of a neural network engine.

[0065] A feature map blocking module 22 is configured to block the input feature map according to the cache size, so that the cache of the neural network engine can accommodate each block result.

[0066] An output feature map assembly module 23 is configured to reassemble each block result to obtain an output feature map.

[0067] ​The technical scheme provided by the embodiment of the application first acquires an input feature map of a layer of a neural network to be processed and a cache size of a neural network engine, then blocks the input feature map according to the cache size, so that the cache of the neural network engine can accommodate each block result, and then reassembles each block result, thereby obtaining a processed output feature map. By blocking the feature map of the neural network, each block result can be cached, data can be retained in the cache, the number of DMA calls is reduced, and therefore the computing power of the neural network and the real-time performance of the calculation are improved.

[0068] On the basis of the above technical scheme, optionally, the feature map blocking module 22 comprises:

[0069] The overlap region height determination unit is configured to determine a feature map blocking direction according to the type of the neural network to be processed, and determine an overlap region height between each two adjacent block results along the feature map blocking direction.

[0070] The data block size determination unit is configured to determine a data block size of each block result along the feature map blocking direction according to the cache size and the overlap region height.

[0071] On the basis of the above technical scheme, optionally, the overlap region height is fixed, and the overlap region height determination unit is specifically configured to:

[0072] determine the overlap region height according to a convolution kernel size and a convolution kernel moving step of a current layer of the neural network to be processed.

[0073] On the basis of the above technical scheme, optionally, the overlap region height is floating, and the overlap region height determination unit is specifically configured to:

[0074] determine the overlap region height according to a starting row index of a current block result and an ending row index of a previous block result.

[0075] On the basis of the above technical scheme, optionally, the data block size determination unit comprises:

[0076] The block number determination subunit is configured to determine a block number along the feature map blocking direction according to the cache size and the overlap region height.

[0077]

[0078] wherein block_num represents the number of blocks, ceil() represents rounding up, float() represents converting to a floating-point number, i represents the length of the input feature map in the feature map blocking direction, block_size represents the length of the cache size in the feature map blocking direction, and overlap represents the height of the overlap region.

[0079] The data block size determination subunit is configured to determine the data block size according to the number of blocks.

[0080] On the basis of the above technical solution, the feature map blocking module 22 further comprises:

[0081] The output adaptation unit is configured to, after determining the data block size of each block result in the feature map blocking direction according to the cache size and the height of the overlap region, perform convolution kernel output adaptation and DMA address adaptation according to the data block size.

[0082] On the basis of the above technical solution, the feature map blocking module 22 further comprises:

[0083] The blocking mode determination unit is configured to, before determining the data block size of each block result in the feature map blocking direction according to the cache size and the height of the overlap region, match a blocking mode according to the type of the neural network to be processed, so as to determine the calculation manner of the data block size according to the blocking mode.

[0084] The minimum parameter set adaptation unit is configured to adapt a minimum parameter set required by a blocking process according to the blocking mode, the feature map blocking direction, and the height of the overlap region.

[0085] The neural network data processing device provided by the embodiment of the present application can execute the neural network data processing method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0086] It should be noted that, in the above embodiment of the neural network data processing device, each unit and module included is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be realized; in addition, the specific name of each functional unit is only for easy mutual differentiation, and does not limit the protection scope of the present application.

[0087] Embodiment three

[0088] Figure 3 The structural schematic diagram of the computer device provided by the embodiment three of the present application shows a block diagram of an exemplary computer device suitable for implementing the embodiments of the present application. Figure 3The computer device shown is merely an example and should not impose any limitation on the functions and usage range of the embodiments of the present application. As Figure 3 As shown in the figure, the computer device includes a processor 31, a memory 32, an input device 33 and an output device 34; the number of processors 31 in the computer device can be one or more, Figure 3 In the figure, the processor 31 in the computer device, the memory 32, the input device 33 and the output device 34 can be connected through a bus or other means, Figure 3 In the figure, the connection through the bus is taken as an example.

[0089] The memory 32, as a computer readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the data processing method of the neural network in the embodiments of the present application (for example, the input feature map acquisition module 21, the feature map blocking module 22 and the output feature map assembly module 23 in the data processing device of the neural network). The processor 31 executes the software programs, instructions and modules stored in the memory 32, thereby performing various function applications and data processing of the computer device, that is, implementing the data processing method of the neural network described above.

[0090] The memory 32 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 32 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device or other non-volatile solid-state storage device. In some examples, the memory 32 can further include a memory remotely arranged with respect to the processor 31, and these remote memories can be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0091] The input device 33 can be used to acquire the input feature map of a layer of the neural network to be processed, and to generate key signal inputs related to the user settings and function control of the computer device, etc. The output device 34 can be used to externally transmit the output feature map, etc.

[0092] Embodiment four

[0093] The embodiment four of the present application further provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute a data processing method of a neural network, the method comprising:

[0094] acquiring an input feature map of a layer of the neural network to be processed and a cache size of a neural network engine;

[0095] The input feature map is divided into blocks according to the cache size, so that the cache of the neural network engine can accommodate each block result;

[0096] Each block result is reassembled to obtain an output feature map.

[0097] A storage medium can be any of various types of memory devices or storage devices. The term "storage medium" is intended to include an installation medium, e.g., a CD-ROM, floppy disks, or tape device; a computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; a nonvolatile memory such as a magnetic medium (e.g., a hard drive or optical storage); registers or other similar types of memory elements upon which a computer can store data. The memory can also include other types of storage media such as a flash drive, Zip Drive, tape, etc. The storage medium can further include a second computer system that is connected to the computer system via a network (e.g., the Internet) that allows transfer of data from the second computer system to the computer system. The term "storage medium" can include one or both of both a computer-readable storage medium or a computer-readable transmission medium. A computer-readable storage medium can also store program instructions which are executable by one or more processors. The term "computer-readable transmission medium" refers to a computer-based signal, which can be in an electronic, optical, or other form. The term "computer program product" includes one or both of a computer-readable storage medium or a computer-readable transmission medium.

[0098] Of course, the storage medium provided by the embodiments of the present application includes computer executable instructions, and the computer executable instructions are not limited to the method operations described above, but can also perform related operations in the data processing method of the neural network provided by any embodiment of the present application.

[0099] The computer readable signal medium can include a data signal traveling in baseband or traveling as a part of a carrier wave. Such a traveling data signal can take a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a storage medium and that can communicate, propagate or transport programming for use by or in connection with an instruction execution system, apparatus, or device.

[0100] The program code contained on the computer readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0101] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary universal hardware, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH memory, a hard disk, or an optical disc, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0102] It is noted that the above are only the preferred embodiments of the present application and the technical principles applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A data processing method of a neural network, characterized by, The method comprises the following steps: obtaining an input feature map of a layer of a neural network to be processed and a cache size of a neural network engine; blocking the input feature map according to the cache size, so that the cache of the neural network engine can accommodate each block result; re-assembling each block result to obtain an output feature map; the step of blocking the input feature map according to the cache size comprises: determining a feature map blocking direction according to the type of the neural network to be processed, and a height of an overlapping area between each two adjacent block results along the feature map blocking direction; determining a data block size of each block result along the feature map blocking direction according to the cache size and the height of the overlapping area; the height of the overlapping area is fixed, and the step of determining the height of the overlapping area along the feature map blocking direction according to the type of the neural network to be processed comprises: determining the height of the overlapping area according to a convolution kernel size of a current layer of the neural network to be processed and a convolution kernel moving step, wherein the height of the overlapping area is overlap = kh-s, wherein kh represents the length of the convolution kernel size in the corresponding feature map blocking direction, and s represents the convolution kernel moving step; the height of the overlapping area is floating, and the step of determining the height of the overlapping area along the feature map blocking direction according to the type of the neural network to be processed comprises: determining the height of the overlapping area according to a starting row index of a current block result and an ending row index of a previous block result, wherein the height of the overlapping area is: ； wherein floor() represents rounding down, oh_start_line represents the starting row index of the current block result, oh_last_line represents the ending row index of the previous block result, and scale represents a scaling multiple of a neural network operator.

2. The data processing method of a neural network according to claim 1, wherein, the step of determining the data block size of each block result along the feature map blocking direction according to the cache size and the height of the overlapping area comprises: determining a number of blocks along the feature map blocking direction according to the cache size and the height of the overlapping area: ； wherein, denotes the number of patches, denotes rounding up, denotes converting to a floating point number, denotes the length of the input feature map in the feature map patch direction, denotes the length of the cache size in the feature map patch direction, denotes the overlap region height; determining the data block size according to the number of blocks.

3. The data processing method of a neural network according to claim 1, wherein, after the step of determining the data block size of each block result along the feature map blocking direction according to the cache size and the height of the overlapping area, the method further comprises: performing convolution kernel output adaptation and DMA address adaptation according to the data block size.

4. The data processing method of a neural network according to claim 1, wherein, before the step of determining the data block size of each block result along the feature map blocking direction according to the cache size and the height of the overlapping area, the method further comprises: matching a blocking mode according to the type of the neural network to be processed, so as to determine a calculation manner of the data block size according to the blocking mode; adapting a minimum parameter set required by a blocking process according to the blocking mode, the feature map blocking direction and the height of the overlapping area.

5. A data processing apparatus of a neural network, characterized by, The method comprises the following steps: an input feature map obtaining module is configured to obtain an input feature map of a layer of a neural network to be processed and a cache size of a neural network engine; The feature map partitioning module is configured to partition the input feature map according to the cache size, so that the cache of the neural network engine can accommodate each partitioned result. The output feature map assembling module is configured to reassemble each of the partitioned results to obtain an output feature map. The feature map partitioning module comprises: The overlap height determining unit is configured to determine a feature map partitioning direction according to the type of the neural network to be processed, and determine an overlap height between each two adjacent partitioned results along the feature map partitioning direction. The data block size determining unit is configured to determine a data block size of each of the partitioned results along the feature map partitioning direction according to the cache size and the overlap height. The overlap height is fixed, and the overlap height determining unit is specifically configured to: determine the overlap height according to a convolution kernel size of a current layer of the neural network to be processed and a convolution kernel moving step, the overlap height being overlap = kh-s, where kh represents a length of the convolution kernel size in a corresponding feature map partitioning direction, and s represents the convolution kernel moving step. The overlap height is floating, and the overlap height determining unit is specifically configured to: determine the overlap height according to a starting row index of a current partitioned result and an ending row index of a previous partitioned result, the overlap height being: ； where floor() represents a floor function, oh_start_line represents the starting row index of the current partitioned result, oh_last_line represents the ending row index of the previous partitioned result, and scale represents a scale of a neural network operator.

6. A computer device, comprising: comprise: one or more processors; a memory configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the data processing method of the neural network according to any one of claims 1-4.

7. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the data processing method of the neural network according to any one of claims 1-4.

Images