Neural network compiling method and device, electronic equipment and medium
By optimizing the neural network compilation method, obtaining and merging input data, storing calculation results, and switching operators, the problem of poor compilation performance of large operator networks on the chip is solved, achieving more efficient storage space utilization and reducing import and export overhead.
Patent Information
- Application Number
- CN202510469171.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-15
- Publication Date
- 2025-09-16
AI Technical Summary
In the existing technology, neural networks with large operators have poor performance when compiled on the chip, insufficient storage space utilization, and common grouping schemes lead to high import and export overhead and redundant calculations.
By obtaining the input data of the current operator, determining whether it meets the calculation requirements, merging and supplementing the input data, storing the calculation results, and switching operators until all input data are read, the compilation instruction mapping is finally performed to optimize the compilation process of the network model.
The compilation performance of large operator networks has been optimized, import and export overhead has been reduced, storage space utilization has been improved, and redundant calculations have been avoided.
Smart Images
Figure CN120654760A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of neural network compilation technology, and in particular to a neural network compilation method, device, electronic device, and medium. Background Art
[0002] After quantization, neural networks need to be compiled and ultimately deployed on the chip. Due to the complexity of neural networks, they often contain large operators. These operators cannot be computed all at once within existing chip memory and must be calculated in blocks. This requires importing and exporting data between double-data-rate synchronous dynamic random access memory (SPM) and on-chip memory (DDR).
[0003] On such a network basis, whether the large operator is grouped separately or grouped with other operators, the performance of the operator cannot be improved.
[0004] If the model is treated as a single group, it must be divided into at least three groups, which will result in large import and export overhead.
[0005] If it is grouped with other operators, the other operators will be divided into many blocks, resulting in redundant calculations. Moreover, if not handled properly, there will still be imports and exports.
[0006] Due to these issues, compilation currently still follows a common grouping scheme, which groups operators according to simple rules based on the order of operators in the network model. This approach does not yield good performance for networks with large operators. Furthermore, chip storage space is not fully utilized, and common grouping schemes waste space. Summary of the Invention
[0007] In view of this, the embodiments of the present disclosure provide a neural network compilation method, apparatus, electronic device, and medium to address the existing problems of networks with large operators not achieving good performance. Furthermore, chip storage space cannot be well utilized, and conventional grouping schemes also waste some space.
[0008] A first aspect of the embodiments of the present disclosure provides a neural network compilation method, wherein the above-mentioned God General Network has n operators; the above-mentioned operators are used to compile data in a memory, and the above-mentioned method includes: step 1, obtaining current input data of the current operator; wherein the above-mentioned current input data comes from a second memory; wherein the second memory is used to store the calculation result of the operator or to store data from the first memory; the storage space of the first memory is larger than that of the second memory, and the first memory is used to store the input data of the neural network; the current input data from the first memory is one of several data blocks obtained after block processing of the input data of the above-mentioned neural network; step 2, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 3, if it meets the requirements, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 4, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 5, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 6, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 7, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 8, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 9, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 10, determining whether the current input data of the current operator meets the calculation requirements of the current operator; step 11, determining whether the current input data of the current operator meets the calculation requirements of the current operator If yes, jump to step 5; otherwise, jump to step 4; step 4, obtain supplementary input data, merge the supplementary input data with the current input data to obtain updated current input data, if the above-mentioned current operator is the first operator of the neural network, the above-mentioned supplementary input data is another data block among the several data blocks obtained in sequence, if the above-mentioned current operator is not the first operator of the neural network, the above-mentioned supplementary input data is the calculation result of the above-mentioned other data block after the predecessor operators of the current operator and output by the previous operator; and jump to step 2; step 5, store the current calculation result in the second memory, and switch the current operator; until the input data of the neural network is read and the calculation of the last operator is completed; step 6, map the compilation instructions for n operators to complete the compilation of the neural network.
[0009] According to a second aspect of the embodiments of the present disclosure, a neural network compilation device is provided. The neural network has n operators, and the device includes: a first memory and a second memory; wherein the neural network compilation device performs the following steps: Step 1: obtaining current input data of a current operator; wherein the current input data comes from the second memory; wherein the second memory is used to store calculation results of the operator or to store data from the first memory; the storage space of the first memory is larger than that of the second memory, and the first memory is used to store input data of the neural network; the current input data from the first memory is one of a plurality of data blocks obtained after block processing of the input data of the neural network; Step 2: determining whether the current input data of the current operator meets the calculation requirements of the current operator; Step 3: If it meets the requirements, jump to step 5; otherwise, jump to step 4; Step 4: Get supplementary input data, merge the supplementary input data with the current input data to obtain updated current input data. If the current operator is the first operator of the neural network, the supplementary input data is another data block among several data blocks obtained in sequence. If the current operator is not the first operator of the neural network, the supplementary input data is the calculation result of the other data block after the previous operators of the current operator and output by the previous operator; and jump to step 2; Step 5: Store the current calculation result in the second memory and switch the current operator; until the input data of the neural network is read and the calculation of the last operator is completed; Step 6: Map the compilation instructions for n operators to complete the compilation of the neural network.
[0010] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.
[0011] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.
[0012] Compared with the prior art, the beneficial effects of the embodiment of the present disclosure are as follows: first, step 1, obtain the current input data of the current operator; second, step 2, determine whether the current input data of the current operator meets the calculation requirements of the current operator; then, step 3, if it meets, jump to step 5; otherwise, jump to step 4; thereafter, step 4, obtain supplementary input data, merge the supplementary input data and the current input data to obtain updated current input data, if the above-mentioned current operator is the first operator of the neural network, the above-mentioned supplementary input data is another data block among the several data blocks obtained in sequence, if the above-mentioned current operator is not the first operator of the neural network, the above-mentioned supplementary input data is the calculation result of the above-mentioned other data block after the previous operators of the current operator and output by the previous operator; and jump to step 2; thereafter, step 5, store the current calculation result in the second memory, and switch the current operator; until the input data of the neural network is read and the calculation of the last operator is completed; finally, step 6, map the compilation instructions for n operators to complete the compilation of the neural network. The present disclosure compiles a network model with large operators without generating imports and exports, thereby optimizing the performance of the network model. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0014] Figure 1 is a schematic diagram of an application scenario of a neural network compilation method according to some embodiments of the present disclosure;
[0015] Figure 2 is a flowchart of some embodiments of the neural network compilation method according to the present disclosure;
[0016] Figure 3 is a schematic structural diagram of some embodiments of a neural network compilation device according to the present disclosure;
[0017] Figure 4 It is a structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.
[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.
[0020] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0021] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".
[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0024] Figure 1 2 is a schematic diagram of an application scenario of a neural network compilation method according to some embodiments of the present disclosure.
[0025] exist Figure 1In the application scenario, first, the computing device 101 can obtain the current input data 102 of the current operator. Secondly, the computing device 101 can determine whether the current input data 102 of the current operator meets the calculation requirements 103 of the current operator. Afterwards, if it does, the computing device 101 can store the current calculation result in the second memory and switch the current operator; until the input data of the neural network is read and the last operator calculation is completed, as shown in the figure mark 104; otherwise, the computing device 101 can obtain supplementary input data, merge the supplementary input data with the current input data, and obtain the updated current input data, as shown in the figure mark 105; and jump to "Determine whether the current input data 102 of the current operator meets the calculation requirements 103 of the current operator". Afterwards, the computing device 101 can store the current calculation result in the second memory and switch the current operator; until the input data of the neural network is read and the last operator calculation is completed, as shown in the figure mark 106. Finally, the computing device 101 can map the compilation instructions for n operators to complete the compilation of the neural network, as shown in the figure mark 107.
[0026] It should be noted that the computing device 101 described above can be either hardware or software. When the computing device 101 is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device 101 is software, it can be installed in the hardware devices listed above. It can be implemented as multiple software programs or software modules, for example, to provide distributed services, or as a single software program or software module. No specific limitations are imposed herein.
[0027] It should be understood that Figure 1 The number of computing devices in the embodiment is merely illustrative. Any number of computing devices may be provided according to implementation requirements.
[0028] Figure 2 is a flowchart of some embodiments of the neural network compilation method according to the present disclosure. Figure 2 The neural network compilation method can be obtained by Figure 1 The computing device 101 executes. Figure 2 As shown, the neural network compilation method includes:
[0029] Step 1. Obtain the current input data of the current operator; wherein the current input data comes from the second memory; wherein the second memory is used to store the calculation results of the operator or to store data from the first memory; the storage space of the first memory is larger than that of the second memory, and the first memory is used to store the input data of the neural network; the current input data from the first memory is one of the several data blocks obtained after the input data of the neural network is processed by block processing.
[0030] Specifically, in deep learning, model operators are units of operation that perform specific computational functions, such as convolution and pooling. The data processed by each operator has a specific shape, which is used to describe the size of the data in various dimensions.
[0031] The first memory is an on-chip memory (DDR), and the second memory is a double data rate synchronous dynamic random access memory (SPM). The DDR stores all input data for the model. Due to the limited storage capacity of the chip's SPM, it cannot accommodate all the data required for the model's operators to be calculated simultaneously on the SPM. Therefore, this data is stored in the larger DDR, meaning that the first memory has more storage space than the second memory. During the block-by-block computation process, data is gradually imported from the DDR to the SPM as needed. Due to the SPM's high-speed access, it can quickly provide data support for operator computations. The data stored in the SPM is arranged in the format (n, h, w, c), where (n, h, w, c) represent the sample number, height, width, and channel number dimensions, respectively. The sample number dimension typically represents the batch size, that is, the number of samples processed at a time. For example, in an image classification task, if 32 images are processed simultaneously, then n is equal to 32. For image data, the height dimension represents the number of vertical pixels in the image. The width dimension represents the number of horizontal pixels in the image. For color images, there are usually three channels: red (R), green (G), and blue (B). In this case, c is equal to 3. If it is a feature map after some convolution operations, the number of channels may vary depending on the number of convolution kernels.
[0032] It should be noted that the shape type of each operator in the above neural network is 4-dimensional.
[0033] It's understandable that the aforementioned operators are used to compile data in memory, and that operator calculations are performed sequentially. According to a pre-set order, operator A is calculated first. If the operator's calculation requirements are met, operator B is then calculated, and this sequence continues until all operators have been calculated. This indicates that there is a certain dependency or logical order between operators, and the result of a previous operator's calculation may serve as an input condition or prerequisite for the calculation of a subsequent operator.
[0034] Step 2: Determine whether the current input data of the current operator meets the calculation requirements of the current operator.
[0035] Step 3: If yes, go to step 5; otherwise, go to step 4.
[0036] Step 4: Obtain supplementary input data, merge the supplementary input data with the current input data to obtain updated current input data. If the current operator is the first operator of the neural network, the supplementary input data is another data block among several data blocks obtained in sequence. If the current operator is not the first operator of the neural network, the supplementary input data is the calculation result of the other data block after the predecessor operators of the current operator and output by the previous operator; and jump to step 2.
[0037] Specifically, during the calculation process, the block data imported from DDR is stored, as well as the data merged with the re-imported data when it does not meet the operator calculation requirements. This data is currently participating in the calculation or is prepared for the next step of the calculation.
[0038] Step 5: Store the current calculation result in the second memory and switch the current operator; until the input data of the neural network is read and the last operator calculation is completed.
[0039] In summary, the specific calculation process is as follows: read a row of data (1,1,1,c) in the neural network input data from DDR to SPM; use this row of data as the input data of the current operator and execute step 2; step 2, determine whether the input data of the current operator meets the calculation requirements of the current operator; if it meets the requirements, determine whether the current operator is the last operator. If it is the last operator, determine whether there is any unread neural network input data in DDR. If so, return to step 1. If not, complete the judgment of all operators; if it is not the last operator, use the calculation result of the current operator on this row of data as the input data of the next operator (wherein, the calculation result of the current operator on this row of data is also stored in SPM), switch the operator and return to step 2;
[0040] If the input data of the current operator does not meet the calculation requirements of the current operator (the current operator requires more rows of data for calculation), read the next row of data in the neural network input data from DDR to SPM, and return to step 2 (assuming that the neural network has three operators and the third operator requires 2 rows of data to meet the calculation requirements, then read another row of data from DDR, pass through the first two operators, and merge the two calculation results of the second operator (that is, 2 rows of data) into the input to the third operator).
[0041] It should be noted that the size of each data block is 1, 1, 1, c. The block calculation here can be performed in the dimensions n, h, w, and c, which means that the data can be split in terms of batch size, height, width, and number of channels separately or simultaneously.
[0042] Here we choose to set the data dimension size for a single calculation to 1,1,1,c. The specific meaning is as follows:
[0043] The first 1 corresponds to the n dimension, which means that only one sample is processed at a time, that is, the batch size is 1.
[0044] The second 1 corresponds to the h dimension, indicating that only one pixel row in the height direction of the image is processed each time.
[0045] The third 1 corresponds to the w dimension, indicating that only one pixel column in the width direction of the image is processed at a time.
[0046] c remains unchanged, that is, the data of all channels at the pixel position are processed.
[0047] With this setting, the final data shape is a row of data, which contains the values of the pixel in all channels.
[0048] After obtaining a row of data in the shape of 1,1,1,c, it can be loaded into the chip's memory cells, allowing the operator to perform calculations on this row of data. After the calculation is completed, the data of the next pixel is processed in the same way until the entire operator has processed all the data. This block-based calculation method effectively alleviates chip memory pressure while also ensuring the accuracy of the calculation.
[0049] For example, for a convolution operator, it was originally necessary to perform convolution calculations on the entire batch of images, but now only one pixel point is taken at a time for convolution operation on all channels, gradually completing the convolution process of the entire image.
[0050] The present disclosure minimizes the segmentation of operator data, effectively occupies SPM space, and reduces intermediate imports and exports, thereby achieving compilation optimization.
[0051] Step 6: Map the compilation instructions to the n operators to complete the compilation of the neural network.
[0052] In some embodiments, step 6 includes:
[0053] In response to the completion of calculation of the n operators and the first memory being empty, mapping compilation instructions for the n operators to complete compilation of the neural network; or
[0054] In response to each switching of the current operator, mapping of the compilation instruction is performed on the current operator; until calculations of n operators are completed and the first memory is empty.
[0055] Here, after all judgments are completed, the compilation instructions of each operator of the neural network can be mapped to complete the compilation of the neural network, or the compilation instructions of the operators of the neural network can be mapped to complete the compilation of the neural network during the judgment process.
[0056] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.
[0057] The following are embodiments of the apparatus disclosed herein, which can be used to implement the method embodiments disclosed herein. For details not disclosed in the apparatus embodiments disclosed herein, please refer to the method embodiments disclosed herein.
[0058] Figure 3 Schematic diagram of the structure of some embodiments of the neural network compilation device according to the present disclosure. Figure 3 As shown, the neural network compilation device includes: a first memory 301 and a second memory 302; wherein the neural network compilation device performs the following steps:
[0059] Step 1: Obtain current input data of the current operator; wherein the current input data comes from a second memory; wherein the second memory is used to store calculation results of the operator or to store data from the first memory; the storage space of the first memory is larger than that of the second memory, and the first memory is used to store input data of the neural network; the current input data from the first memory is one of the multiple data blocks obtained after block processing of the input data of the neural network;
[0060] Step 2: Determine whether the current input data of the current operator meets the calculation requirements of the current operator;
[0061] Step 3: If it meets the requirements, go to step 5; otherwise, go to step 4;
[0062] Step 4: Obtain supplementary input data, merge the supplementary input data with the current input data to obtain updated current input data. If the current operator is the first operator of the neural network, the supplementary input data is another data block among the sequentially obtained data blocks. If the current operator is not the first operator of the neural network, the supplementary input data is the calculation result of the other data block after passing through the previous operators of the current operator and output by the previous operator. Then jump to step 2.
[0063] Step 5: Store the current calculation result in the second memory and switch the current operator until the input data of the neural network is read and the last operator calculation is completed;
[0064] Step 6: Map the compilation instructions to the n operators to complete the compilation of the neural network.
[0065] In some optional implementations of some embodiments, the first memory is a double data rate synchronous dynamic random access memory, and the second memory is an on-chip memory.
[0066] In some optional implementations of some embodiments, the shape type of each operator in the above neural network is 4-dimensional.
[0067] In some optional implementations of some embodiments, the data format of each data block stored in the second memory is a preset format; the above preset format includes a sample quantity dimension, a height dimension, a width dimension, and a channel quantity dimension.
[0068] In some optional implementations of some embodiments, the size of each of the above data blocks is 1, 1, 1, c; where c is the number of channels.
[0069] In some optional implementations of some embodiments, step 6 includes:
[0070] In response to the completion of calculation of the n operators and the first memory being empty, mapping compilation instructions for the n operators to complete compilation of the neural network; or
[0071] In response to each switching of the current operator, mapping of the compilation instruction is performed on the current operator; until calculations of n operators are completed and the first memory is empty.
[0072] In some optional implementations of some embodiments, the data blocks stored in the second memory are ordered; and are output in sequence each time.
[0073] Reference below Figure 4 , which shows an electronic device (eg, Figure 1 Schematic diagram of the structure of the computing device 101)400. Figure 4 The server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0074] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. Various programs and data required for the operation of the electronic device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 405. An input / output (I / O) interface 405 is also connected to the bus 405.
[0075] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as needed.
[0076] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.
[0077] It should be noted that in some embodiments of the present disclosure, the computer-readable medium mentioned above may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
[0078] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0079] The computer-readable medium may be included in the device, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains the current input data of the current operator; determines whether the current input data of the current operator meets the calculation requirements of the current operator; if so, jumps to "store the current calculation result in the second memory and switch the current operator; until the input data of the neural network is read and the calculation of the last operator is completed"; otherwise, jumps to "obtain supplementary input data, merge the supplementary input data with the current input data to obtain updated current input data. If the current operator is the first operator of the neural network, the supplementary input data is another data block among several data blocks obtained in sequence. If the current operator is not the first operator of the neural network, the supplementary input data is the calculation result of the other data block after the previous operators of the current operator and output by the previous operator; and jumps to "determine whether the current input data of the current operator meets the calculation requirements of the current operator"; "map the compilation instructions for n operators to complete the compilation of the neural network.
[0080] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0082] The units described in some embodiments of the present disclosure may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as follows: a processor includes an import unit, a determination unit, and a switching unit. The names of these units do not, in some cases, limit the units themselves. For example, the import unit may also be described as a "unit for importing data from a first memory to a second memory."
[0083] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0084] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.
Claims
1. A neural network compilation method, characterized in that: The neural network has n operators, and the method includes: Step 1: Obtain current input data of a current operator; wherein the current input data comes from a second memory; wherein the second memory is used to store calculation results of the operator or to store data from a first memory; the storage space of the first memory is larger than that of the second memory, and the first memory is used to store input data of a neural network; the current input data from the first memory is one of a plurality of data blocks obtained after block processing of the input data of the neural network; Step 2: Determine whether the current input data of the current operator meets the calculation requirements of the current operator; Step 3: If it meets the requirements, go to step 5; otherwise, go to step 4; Step 4: Obtain supplementary input data, merge the supplementary input data with the current input data to obtain updated current input data. If the current operator is the first operator of the neural network, the supplementary input data is another data block among the sequentially obtained data blocks. If the current operator is not the first operator of the neural network, the supplementary input data is the calculation result of the other data block after passing through the previous operators of the current operator and output by the previous operator. Then jump to step 2. Step 5: Store the current calculation result in the second memory and switch the current operator until the input data of the neural network is read and the last operator calculation is completed; Step 6: Map the compilation instructions to the n operators to complete the compilation of the neural network.
2. The neural network compilation method according to claim 1, characterized in that: The first memory is an on-chip memory, and the second memory is a double data rate synchronous dynamic random access memory.
3. The neural network compilation method according to claim 1, characterized in that: The shape type of each operator in the neural network is 4-dimensional.
4. The neural network compilation method according to claim 3, characterized in that: The data format of each data block stored in the second memory is a preset format; the preset format includes a sample quantity dimension, a height dimension, a width dimension, and a channel quantity dimension.
5. The neural network compilation method according to claim 4, characterized in that: The size of each data block is 1, 1, 1, c, where c is the number of channels.
6. The neural network compilation method according to claim 1, characterized in that: The step 6 comprises: In response to the completion of calculation of the n operators and the first memory being empty, mapping compilation instructions for the n operators to complete compilation of the neural network; or In response to each switching of the current operator, mapping of the compilation instruction is performed on the current operator; until calculations of n operators are completed and the first memory is empty.
7. The neural network compilation method according to claim 1, characterized in that: The data blocks stored in the second memory are ordered; and are output in order each time.
8. A neural network compilation device, characterized in that: The neural network has n operators, and the device includes: a first memory and a second memory; wherein the neural network compilation device performs the following steps: Step 1: Obtain current input data of a current operator; wherein the current input data comes from a second memory; wherein the second memory is used to store calculation results of the operator or to store data from a first memory; the storage space of the first memory is larger than that of the second memory, and the first memory is used to store input data of a neural network; the current input data from the first memory is one of a plurality of data blocks obtained after block processing of the input data of the neural network; Step 2: Determine whether the current input data of the current operator meets the calculation requirements of the current operator; Step 3: If it meets the requirements, go to step 5; otherwise, go to step 4; Step 4: Obtain supplementary input data, merge the supplementary input data with the current input data to obtain updated current input data. If the current operator is the first operator of the neural network, the supplementary input data is another data block among the sequentially obtained data blocks. If the current operator is not the first operator of the neural network, the supplementary input data is the calculation result of the other data block after passing through the previous operators of the current operator and output by the previous operator. Then jump to step 2. Step 5: Store the current calculation result in the second memory and switch the current operator until the input data of the neural network is read and the last operator calculation is completed; Step 6: Map the compilation instructions to the n operators to complete the compilation of the neural network.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.