Computation method, device, terminal and storage medium of neural network model

Abstract

The embodiments of the present application disclose a method, device, terminal, and storage medium for calculating a neural network model, and belong to the field of computer technology. The method includes: obtaining a neural network model; determining a first type of progress segment and a second type of progress segment during the operation of the neural network model, wherein the bandwidth required for network operation on the first type of progress segment is greater than a bandwidth threshold, and the bandwidth required for network operation on the second type of progress segment is less than or equal to the bandwidth threshold; and using a neural network processor to access neural network data using a first memory on the first type of progress segment and a second memory on the second type of progress segment to calculate the neural network model, wherein the access speed of the first memory is greater than that of the second memory. This method enables the memory to meet the data throughput requirements of the neural network processor at each operation progress of the neural network model, and fully utilizes the operation performance of the neural network processor.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a calculation method, device, terminal, and storage medium for a neural network model. Background Art

[0002] The Neural Processing Unit (NPU) is an artificial intelligence chip used for computing neural network models.

[0003] During the NPU's neural network model calculations, dynamic random access memory (DRAM) provides storage for input and output data. To meet the NPU's high-speed read and output requirements, increasing DRAM bandwidth or throughput can be used to ensure that data required for neural network model calculations can be transferred quickly to and from the NPU. Summary of the Invention

[0004] The present application provides a method, device, terminal, and storage medium for computing a neural network model. The technical solution is as follows:

[0005] According to one aspect of the present application, a method for calculating a neural network model is provided, the method comprising:

[0006] Get the neural network model;

[0007] Determining a first type of progress segment and a second type of progress segment during the operation of the neural network model, wherein a bandwidth required for network operation in the first type of progress segment is greater than a bandwidth threshold, and a bandwidth required for network operation in the second type of progress segment is less than or equal to the bandwidth threshold; and

[0008] The neural network model is operated by using a neural network processor to access the neural network data using a first memory on the first type of progress segment and to access the neural network data using a second memory on the second type of progress segment, wherein the access speed of the first memory is greater than the access speed of the second memory.

[0009] According to another aspect of the present application, a computing device for a neural network model is provided, the device comprising:

[0010] Acquisition module, used to obtain the neural network model;

[0011] a determination module, configured to determine a first type of progress segment and a second type of progress segment during the operation of the neural network model, wherein the bandwidth required for network operation in the first type of progress segment is greater than a bandwidth threshold, and the bandwidth required for network operation in the second type of progress segment is less than or equal to the bandwidth threshold;

[0012] An operation module is used to use a neural network processor to access neural network data using a first memory on the first type of progress segment and to access the neural network data using a second memory on the second type of progress segment to perform operations on the neural network model, wherein the access speed of the first memory is greater than the access speed of the second memory.

[0013] According to another aspect of the present application, a terminal is provided, comprising:

[0014] processor; and

[0015] Memory; wherein, the memory stores program instructions, and when the processor executes the program instructions, it implements the calculation method of the neural network model as described in the above aspect.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, in which program instructions are stored. When the program instructions are executed by a processor, the calculation method of the neural network model as described in the above aspect is implemented.

[0017] According to one aspect of the present application, a computer program product is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the calculation method of the neural network model as described in the above aspect.

[0018] The beneficial effects of the technical solutions provided in the embodiments of the present application may include:

[0019] The method provided in the present application is to first determine the first type of progress segment and the second type of progress segment of the neural network model in the operation process when calculating the neural network model, the bandwidth required for network operation on the first type of progress segment is greater than the bandwidth threshold, and the bandwidth required for network operation on the second type of progress segment is less than or equal to the bandwidth threshold. Then, in the process of calculating the neural network model, the neural network processor uses a first memory with a relatively faster access speed to access the neural network data on the first type of progress segment, and uses a second memory with a relatively slower access speed on the second type of progress segment, so that the memory can meet the data throughput requirements of the neural network processor at each operation progress of the neural network model, fully utilize the operation performance of the neural network processor, make the neural network processor work nearly saturated or even saturated, and solve the problem of performance degradation of the neural network processor caused by insufficient memory throughput rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly introduce the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 is a structural block diagram of a terminal provided by an exemplary embodiment of the present application;

[0022] Figure 2 is a flow chart of a calculation method of a neural network model provided by an exemplary embodiment of the present application;

[0023] Figure 3 is a flowchart of a calculation method of a neural network model provided by another exemplary embodiment of the present application;

[0024] Figure 4 This is a data access diagram of a neural network processor provided by an exemplary embodiment of the present application;

[0025] Figure 5 is a schematic diagram of calculating the estimated bandwidth of neural network data transmission provided by an exemplary embodiment of the present application;

[0026] Figure 6 This is a schematic diagram of bandwidth allocation for neural network data provided by an exemplary embodiment of the present application;

[0027] Figure 7 is a schematic diagram of progress segmentation provided by an exemplary embodiment of the present application;

[0028] Figure 8It is a structural block diagram of a computing device for a neural network model provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0030] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with certain aspects of the present application as detailed in the appended claims.

[0031] In order to facilitate understanding of the solutions shown in the embodiments of the present application, several nouns appearing in the embodiments of the present application are introduced below.

[0032] Progress segmentation refers to the segmentation of computational progress. Computational progress refers to the completion progress of calculating a neural network model. For example, a percentage can be used to represent the completion progress of calculating a neural network model. The computational progress of the neural network model ranges from 0% to 100%, and when the computational progress reaches 100%, it indicates that the computation of the neural network model is complete. For another example, time can be used to represent the completion progress of calculating a neural network model. The computational progress of the neural network model ranges from the 0th second to the Tth second. When the computational progress reaches the Tth second, it indicates that the computation of the neural network model is complete, where T is a positive integer.

[0033] Exemplarily, the calculation method of the neural network model shown in the embodiment of the present application can be applied in a terminal that has the computing function of the neural network model. The terminal may include a mobile phone, a tablet computer, a laptop computer, a desktop computer, an all-in-one computer, a server, a workstation, a television, a set-top box, smart glasses, a smart watch, a digital camera, a Moving Picture Experts Group Audio Layer IV (MP4) playback terminal, a Moving Picture Experts Group Audio Layer IV (MP5) playback terminal, a learning machine, a point-reading machine, an electronic paper book, an electronic dictionary, a car terminal, a virtual reality (VR) playback terminal, or an augmented reality (AR) playback terminal, etc.

[0034] Figure 1 This is a block diagram of a terminal provided by an exemplary embodiment of the present application. Figure 1As shown, the terminal includes a processor 120 and a memory 140, and the memory 140 stores at least one instruction. The at least one instruction is loaded and executed by the processor 120 to implement the calculation method of the neural network model as described in the various method embodiments of the present application.

[0035] In the present application, terminal 100 is an electronic device with a neural network model computing function. When terminal 100 runs a neural network model, before officially running the neural network model, terminal 100 can simulate the neural network model and determine the estimated bandwidth required for accessing neural network data at each computing progress. Then, based on the relationship between the estimated bandwidth and the bandwidth threshold, terminal 100 divides the entire computing progress of the neural network model into a first type of progress segment and a second type of progress segment. Terminal 100 further determines whether to use a first memory to access neural network data in the first type of progress segment where the estimated bandwidth is greater than the bandwidth threshold, and to use a second memory to access neural network data in the second type of progress segment where the estimated bandwidth is less than or equal to the bandwidth threshold, wherein the access speed of the first memory is greater than the second access speed. Then, the neural network processor in terminal 100 accesses neural network data according to the allocated memory usage of the first type of progress segment and the second type of progress segment to perform neural network model calculation.

[0036] The processor 120 may include one or more processing cores. The processor 120 utilizes various interfaces and circuits to connect various components within the terminal 100. It executes various functions and processes data for the terminal 100 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 140, as well as accessing data stored in the memory 140. Optionally, the processor 120 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 120 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processing unit (NPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content displayed on the display; the NPU is used for computing neural network models, and is particularly adept at processing multimedia data such as video and images; and the modem is used to handle wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 120, but may be implemented by a separate chip.

[0037] The memory 140 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 140 may also include a dynamic random access memory (DRAM) and a static random access memory (SRAM). Optionally, the memory 140 includes a non-transitory computer-readable storage medium. The memory 140 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 140 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data involved in the following various method embodiments, etc.

[0038] Figure 2 1 is a flow chart of a method for calculating a neural network model provided by an exemplary embodiment of the present application. The method for calculating a neural network model can be applied in the terminal shown above, and the method includes:

[0039] Step 210: Obtain a neural network model.

[0040] Exemplarily, the terminal stores a neural network model in a local memory, and the terminal obtains the neural network model from the local memory. Alternatively, the terminal does not store a neural network model in the local memory, and the terminal downloads the neural network model from a server to the terminal via a wired or wireless network.

[0041] Step 220, determining the first type of progress segment and the second type of progress segment during the operation of the neural network model, the bandwidth required for network operation on the first type of progress segment is greater than the bandwidth threshold, and the bandwidth required for network operation on the second type of progress segment is less than or equal to the bandwidth threshold.

[0042] The terminal determines, based on the neural network model, a first type of progress segment and a second type of progress segment of the neural network model during its operation. For example, if the progress segment of the neural network model already exists in the terminal, the first type of progress segment and the second type of progress segment of the neural network model are directly obtained; alternatively, the terminal calculates the progress segment of the neural network model.

[0043] Optionally, the bandwidth threshold is set based on the data transmission bandwidth of the second memory. Optionally, the bandwidth threshold is set to the data transmission bandwidth of the second memory. The above bandwidth threshold is used to determine whether the bandwidth required for the neural network model operation exceeds the data transmission bandwidth provided by the second memory.

[0044] In step 230, the neural network processor uses a first memory to access the neural network data in the first type of progress segment and uses a second memory to access the neural network data in the second type of progress segment to perform neural network model operations, and the access speed of the first memory is greater than the access speed of the second memory.

[0045] After determining the first type of progress segment and the second type of progress segment, the terminal calculates a neural network model via a neural network processor. During the calculation of the neural network model, a first memory is used to access neural network data for the first type of progress segment, and a second memory is used to access neural network data for the second type of progress segment. Exemplarily, the neural network model calculated by the terminal is compiled by a compiler to obtain executable instructions for the neural network processor. The neural network processor then executes the executable instructions to calculate the neural network model.

[0046] Optionally, the first memory includes at least one of a system cache (System Cache, Sys$) and a system buffer memory (System Buffer, SysBuf); the second memory includes DRAM. Exemplarily, the Sys$ and SysBuf are composed of SRAM.

[0047] Exemplarily, in the process of calculating the neural network model, the terminal uses Sys$ and / or SysBuf to access the neural network data in the first type of progress segment, and uses DRAM to access the neural network data in the second type of progress segment.

[0048] It should be noted that the present embodiment does not limit the order in which step 220 and step 230 are executed. The terminal may execute step 220 first and then step 230, or may execute step 220 and step 230 simultaneously.

[0049] In summary, the neural network model calculation method provided in this embodiment first determines the first and second progress segments of the neural network model during the calculation process when calculating the neural network model. The bandwidth required for network calculations in the first progress segment is greater than a bandwidth threshold, and the bandwidth required for network calculations in the second progress segment is less than or equal to the bandwidth threshold. Subsequently, during the process of calculating the neural network model, the neural network processor uses a first memory with a relatively faster access speed to access neural network data in the first progress segment, and uses a second memory with a relatively slower access speed in the second progress segment. This ensures that the memory can meet the data throughput requirements of the neural network processor at each calculation progress of the neural network model, fully utilizing the computing performance of the neural network processor, allowing the neural network processor to operate at near or even full capacity, thereby solving the problem of performance degradation of the neural network processor caused by insufficient memory throughput. Secondly, when the neural network data is stored in Sys$ and / or SysBuf, due to the power-saving performance of Sys$ and / or SysBuf, it is also possible to save power and prevent system overheating during the neural network model calculation process.

[0050] Based on the solution disclosed in the previous embodiment, the method of executing step 220 by the terminal is described in detail. Figure 3 , step 220 may include steps 221 to 224, which are as follows:

[0051] Step 221: query the corresponding relationship table for the progress segment information corresponding to the model identifier of the neural network model.

[0052] The terminal is provided with a correspondence table between progress segment information and model identifiers. The progress segment information includes first-type and second-type progress segments during the operation of the neural network model. After obtaining the neural network model, the terminal first searches the correspondence table for the progress segment information corresponding to the model identifier of the neural network model. If the progress segment information exists in the correspondence table, the terminal executes step 224. If the progress segment information does not exist in the correspondence table, the terminal executes steps 222 and 223.

[0053] Step 222: When there is no progress segmentation information in the correspondence table, simulate the neural network model to obtain the estimated bandwidth required for neural network data access at each operation progress in the simulated operation process.

[0054] The terminal simulates the operation of the neural network model and estimates the bandwidth required for accessing the neural network data at each operation progress when operating the neural network model; the above-mentioned operation progress refers to the completion progress of calculating a neural network model, and the neural network data includes the data read and written when the neural network model is operated.

[0055] During the neural network model's computation, there are input nodes for raw data and output nodes for result data. During the neural network model's intermediate steps, there are also intermediate access nodes for writing and reading intermediate feature data. Accordingly, the neural network data described above includes raw data, intermediate feature data, and result data. Raw data refers to raw data that has not been computed by the neural network model, and result data refers to the result data calculated by the neural network model.

[0056] In one possible implementation, the terminal performs simulation operations on the neural network model and directly calculates the estimated bandwidth required for accessing the neural network data at each operation progress.

[0057] In another possible implementation, the terminal performs simulated operations on the neural network model, and calculates the first estimated bandwidth required for reading the original data, the second estimated bandwidth required for accessing the intermediate feature data, and the third estimated bandwidth required for writing the result data at each operation progress; the first estimated bandwidth, the second estimated bandwidth, and the third estimated bandwidth are superimposed according to the operation progress to obtain the estimated bandwidth.

[0058] Optionally, the neural network model includes n intermediate access nodes for intermediate feature data, where n is a positive integer greater than 1; for the calculation of the second estimated bandwidth, the terminal performs simulation operations on the neural network model, calculates the second estimated bandwidth required for accessing the intermediate feature data on each intermediate access node, and finally obtains n second estimated bandwidths corresponding to the n intermediate access nodes.

[0059] For example, Figure 4 When a neural network model is run on the NPU 11, there are four blocks of data entering and leaving the NPU, namely, one block of raw data input, two blocks of intermediate feature images written and read, and one block of result data output. That is, there is one input node, two intermediate access nodes, and one output node on the above neural network model. Therefore, an input cache 12 for raw data, an intermediate access cache 13, an intermediate access cache 14, and an output cache 15 for result data are required.

[0060] like Figure 5 , the bandwidth required for data access on the above-mentioned four reading nodes is predicted, and an estimated bandwidth curve 21 of the first estimated bandwidth corresponding to the original data on the computing progress, an estimated bandwidth curve 22 of the second estimated bandwidth corresponding to the intermediate feature image 1 on the computing progress, an estimated bandwidth curve 23 of the second estimated bandwidth corresponding to the intermediate feature image 2 on the computing progress, and an estimated bandwidth curve 24 of the third estimated bandwidth corresponding to the result data on the computing progress are obtained. The estimated bandwidth curve 21, the estimated bandwidth curve 22, the estimated bandwidth curve 23, and the estimated bandwidth curve 24 are added according to the corresponding relationship of the computing progress to obtain an estimated bandwidth curve 25 of the overall estimated bandwidth corresponding to the neural network data on the computing progress.

[0061] Step 223 : Determine a first type of progress segment whose estimated bandwidth is greater than a bandwidth threshold, and a second type of progress segment whose estimated bandwidth is less than or equal to the bandwidth threshold.

[0062] The terminal compares the estimated bandwidth with the bandwidth threshold, and divides the progress segments into a first type where the estimated bandwidth is greater than the bandwidth threshold, and a second type where the estimated bandwidth is less than or equal to the bandwidth threshold.

[0063] For example, if the estimated bandwidth of the first L1 consecutive computing progress of a certain computing progress is greater than the bandwidth threshold, and the estimated bandwidth of the last L2 consecutive computing progress of the computing progress is also greater than the bandwidth threshold, the computing progress is divided into the first type of progress segment; L1 and L2 are positive integers.

[0064] For example, after determining the first type of progress segment and the second type of progress segment, the terminal determines the reading progress segment of the original data, the access progress segment of the intermediate feature data, and the writing progress segment of the result data according to the first type of progress segment and the second type of progress segment. Figure 6According to the first type of progress segmentation and the second type of progress segmentation, the location division of the raw data cache 31, the intermediate feature image 1 cache 32, the intermediate feature image 2 cache 33, and the result data cache 34 are determined. In the raw data cache 31, the dot pattern marked on the left oblique line pattern is the portion of the raw data cached in the first memory, and the remaining portion is the portion of the raw data cached in the second memory; in the intermediate feature image 1 cache 32, the dot pattern marked on the grid pattern is the portion of the intermediate feature image 1 cached in the first memory, and the remaining portion is the portion of the intermediate feature image 1 cached in the second memory; in the intermediate feature image 2 cache 33, the dot pattern marked on the oblique grid pattern is the portion of the intermediate feature image 2 cached in the first memory, and the remaining portion is the portion of the intermediate feature image 2 cached in the second memory; in the result data cache 34, the dot pattern marked on the right oblique line pattern is the portion of the result data cached in the first memory, and the remaining portion is the portion of the result data cached in the second memory.

[0065] Exemplarily, after determining the first type of progress segment and the second type of progress segment, the terminal may also determine the first bandwidth of the second memory to be allocated for use on the first type of progress segment, wherein the first bandwidth is less than or equal to the bandwidth threshold; calculate the difference between the estimated bandwidth and the first bandwidth on the first type of progress segment to obtain the second bandwidth; determine the second bandwidth of the first memory to be allocated for use on the first type of progress segment; and further determine the use of the second memory to access the neural network data on the first type of progress segment. Exemplarily, the terminal may randomly determine the first bandwidth that is less than or equal to the bandwidth threshold. Figure 7 , a full DRAM data access solution and a DRAM combined with Sys$ / SysBuf data access solution are given. The terminal determines 4 first-type progress segments and 4 second-type progress segments based on the bandwidth threshold, and determines to use Sys$ or SysBuf to access neural network data on the first-type progress segments, and determines to use DRAM to access neural network data on the second-type progress segments.

[0066] It should be noted that this embodiment does not limit the order of calculating the estimated bandwidth and determining the progress segments. The terminal can first calculate the estimated bandwidth and then determine the progress segments, or it can calculate the estimated bandwidth and determine the progress segments at the same time.

[0067] Step 224: When the corresponding relationship table contains progress segment information, the progress segment information is directly obtained from the corresponding relationship table.

[0068] When the corresponding relationship table contains progress segment information, the terminal directly obtains the progress segment information from the corresponding relationship table based on the model identifier of the neural network model.

[0069] To sum up, the calculation method of the neural network model provided in this embodiment can be directly obtained and used on the premise that the progress segmentation information is stored in the correspondence table. There is no need to calculate the progress segmentation information for each calculated neural network model, thereby improving the calculation efficiency of the neural network model.

[0070] Based on the solution disclosed in the previous embodiment, the terminal can also perform the step of storing the progress segmentation information and the model identifier of the neural network model in the correspondence table after step 223, so that the terminal only needs to calculate the progress segmentation information of the neural network model once, and can obtain it without repeatedly calculating the progress segmentation information in the subsequent application process, thereby improving the computing efficiency of the neural network model and saving the computing space of the terminal.

[0071] Optionally, when the cache duration of the correspondence between the progress segmentation information and the model identifier in the above correspondence table is greater than or exceeds a duration threshold, the correspondence in the correspondence table is cleared; or, the correspondence between the progress segmentation information and the model identifier cached in the correspondence table is cleared periodically, for example, the correspondence between the progress segmentation information and the model identifier cached in the correspondence table is cleared at 24:00 every day.

[0072] The progress segment information is used to instruct the neural network processor to use the first memory to access neural network data in a first type of progress segment, and to use the second memory to access neural network data in a second type of progress segment, when operating the neural network model. Optionally, the progress segment information is used to instruct the neural network processor to use the second bandwidth of the first memory and the first bandwidth of the second memory to access neural network data in the first type of progress segment, and to use the second memory to access neural network data in the second type of progress segment, when operating the neural network model.

[0073] Clearing the correspondence between the progress segment information and the model identifier can save storage space.

[0074] Based on the solution disclosed in the previous embodiment, the terminal can realize the simulation operation of the neural network model through the compiler. Exemplarily, the terminal obtains the model packaging information of the neural network model; compiles the model packaging information through the compiler to obtain the executable instructions of the neural network processor; and simulates the neural network processor to execute the executable instructions through the compiler, so that the progress segmentation information of the neural network model can be calculated. Exemplarily, the compiler also transmits the compiled executable instructions to the neural network processor, so that the neural network processor executes the above executable instructions and then calculates the neural network model. Exemplarily, the compiler can simulate the neural network processor to execute the executable instructions after completing the compilation of the entire model packaging information; or it can simulate the neural network processor to execute the compiled executable instructions while compiling the model packaging information. The above process is not limited in this embodiment.

[0075] Exemplarily, the model encapsulation information of the neural network model may be stored in a local memory of the terminal, or may be downloaded by the terminal from a server.

[0076] In this method, the compiler is applied to determine the segmented storage information of the neural network model, which can be widely applied to various types of terminals and realize the functional expansion of the compiler.

[0077] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0078] Please refer to Figure 8 , which shows a block diagram of a computing device for a neural network model provided by an exemplary embodiment of the present application. The device can be implemented as all or part of a terminal through software, hardware, or a combination of both. The device includes:

[0079] An acquisition module 310 is used to acquire a neural network model;

[0080] a determination module 320 configured to determine a first type of progress segment and a second type of progress segment during the operation of the neural network model, wherein the bandwidth required for network operation in the first type of progress segment is greater than a bandwidth threshold, and the bandwidth required for network operation in the second type of progress segment is less than or equal to the bandwidth threshold; and

[0081] The operation module 330 is used to use a neural network processor to use a first memory to access neural network data on the first type of progress segment, and use a second memory to access the neural network data on the second type of progress segment to perform operations on the neural network model, wherein the access speed of the first memory is greater than the access speed of the second memory.

[0082] In an optional embodiment, the determining module 320 is configured to:

[0083] Performing a simulation operation on the neural network model to obtain an estimated bandwidth required for accessing the neural network data at each operation progress during the simulation operation, wherein the operation progress refers to the completion progress of calculating the neural network model; and

[0084] The first type progress segment whose estimated bandwidth is greater than the bandwidth threshold and the second type progress segment whose estimated bandwidth is less than or equal to the bandwidth threshold are determined.

[0085] In an optional embodiment, the neural network data includes original data, intermediate feature data, and result data; the determination module 320 is configured to:

[0086] Performing simulation operations on the neural network model to calculate a first estimated bandwidth required for reading the original data, a second estimated bandwidth required for accessing the intermediate feature data, and a third estimated bandwidth required for writing the result data at each operation progress; and

[0087] The first estimated bandwidth, the second estimated bandwidth, and the third estimated bandwidth are correspondingly superimposed according to the calculation progress to obtain the estimated bandwidth.

[0088] In an optional embodiment, the determining module 320 is configured to:

[0089] Compiling the model encapsulation information of the neural network model through a compiler to obtain executable instructions for the neural network processor; and

[0090] The compiler simulates the neural network processor to execute the executable instructions.

[0091] In an optional embodiment, the apparatus is provided with a correspondence table between progress segment information and model identifiers; the determination module 320 is configured to:

[0092] Querying the progress segment information corresponding to the model identifier of the neural network model from the corresponding relationship table;

[0093] When the progress segment information does not exist in the correspondence table, determining to perform the simulation operation on the neural network model to obtain the estimated bandwidth required for the neural network data access at each operation progress in the simulation operation process; determining the first type of progress segment with the estimated bandwidth greater than the bandwidth threshold, and the second type of progress segment with the estimated bandwidth less than or equal to the bandwidth threshold, to obtain the progress segment information;

[0094] The progress segment information includes the first type of progress segment and the second type of progress segment.

[0095] In an optional embodiment, the determining module 320 is configured to:

[0096] When the progress segment information exists in the correspondence table, the progress segment information is directly obtained from the correspondence table.

[0097] In an optional embodiment, the determining module 320 is configured to:

[0098] The progress segment information and the model identifier of the neural network model are stored in the corresponding relationship table in correspondence.

[0099] In an optional embodiment, the first memory includes at least one of a system cache and a system buffer memory; and the second memory includes a dynamic random access memory.

[0100] In summary, the neural network model calculation device provided in this embodiment, when calculating a neural network model, first determines the first type of progress segment and the second type of progress segment during the neural network model calculation process. The bandwidth required for network calculation in the first type of progress segment is greater than the bandwidth threshold, and the bandwidth required for network calculation in the second type of progress segment is less than or equal to the bandwidth threshold. Subsequently, during the process of calculating the neural network model, the neural network processor uses a first memory with a relatively faster access speed to access neural network data in the first type of progress segment, and uses a second memory with a relatively slower access speed in the second type of progress segment. This ensures that the memory can meet the data throughput requirements of the neural network processor at each calculation progress of the neural network model, fully utilizing the computing performance of the neural network processor, allowing the neural network processor to operate at near or even full capacity, thereby solving the problem of performance degradation of the neural network processor caused by insufficient memory throughput. Secondly, when the neural network data is stored in Sys$ and / or SysBuf, due to the power-saving performance of Sys$ and / or SysBuf, it is also possible to save power and prevent system overheating during the neural network model calculation process.

[0101] An embodiment of the present application also provides a computer-readable medium, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the neural network model calculation method described in the above embodiments.

[0102] It should be noted that the neural network model calculation device provided in the above embodiment only uses the division of the above functional modules as an example when executing the neural network model calculation method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the neural network model calculation device provided in the above embodiment and the neural network model calculation method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0103] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0104] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0105] The above description is merely an exemplary embodiment that can be implemented in the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for calculating a neural network model, characterized in that: The method comprises: Get the neural network model; determining a first type of progress segment and a second type of progress segment during the operation of the neural network model, wherein a bandwidth required for network operation in the first type of progress segment is greater than a bandwidth threshold, and a bandwidth required for network operation in the second type of progress segment is less than or equal to the bandwidth threshold, wherein the bandwidth threshold is set based on a data transmission bandwidth of a second memory, and the bandwidth threshold is used to determine whether the bandwidth required for the operation of the neural network model exceeds a data transmission bandwidth provided by the second memory; and Using a neural network processor to access neural network data using a first memory in the first type of progress segment and using a second memory in the second type of progress segment to perform operations on the neural network model; The access speed of the first memory is greater than the access speed of the second memory, the first memory includes a static random access memory, and the second memory includes a dynamic random access memory.

2. The method according to claim 1, characterized in that The determining of the first type of progress segment and the second type of progress segment during the operation of the neural network model includes: Performing a simulation operation on the neural network model to obtain an estimated bandwidth required for accessing the neural network data at each operation progress during the simulation operation, wherein the operation progress refers to the completion progress of calculating the neural network model; and The first type progress segment whose estimated bandwidth is greater than the bandwidth threshold and the second type progress segment whose estimated bandwidth is less than or equal to the bandwidth threshold are determined.

3. The method according to claim 2, characterized in that The neural network data includes original data, intermediate feature data and result data; The performing of simulation operation on the neural network model to obtain an estimated bandwidth required for accessing the neural network data at each operation progress during the simulation operation includes: Performing simulation operations on the neural network model to calculate a first estimated bandwidth required for reading the original data, a second estimated bandwidth required for accessing the intermediate feature data, and a third estimated bandwidth required for writing the result data at each operation progress; and The first estimated bandwidth, the second estimated bandwidth, and the third estimated bandwidth are correspondingly superimposed according to the calculation progress to obtain the estimated bandwidth.

4. The method according to claim 2, characterized in that The simulation operation includes: Compiling the model encapsulation information of the neural network model through a compiler to obtain executable instructions for the neural network processor; and The compiler simulates the neural network processor to execute the executable instructions.

5. The method according to any one of claims 2 to 4, characterized in that: Applied in a terminal, the terminal is provided with a correspondence table between progress segment information and model identifiers; Before performing simulation operation on the neural network model to obtain the estimated bandwidth required for accessing the neural network data at each operation progress in the simulation operation process, the method includes: Querying the progress segment information corresponding to the model identifier of the neural network model from the corresponding relationship table; When the progress segment information does not exist in the correspondence table, determining to perform the simulation operation on the neural network model to obtain the estimated bandwidth required for the neural network data access at each operation progress in the simulation operation process; determining the first type of progress segment with the estimated bandwidth greater than the bandwidth threshold, and the second type of progress segment with the estimated bandwidth less than or equal to the bandwidth threshold, to obtain the progress segment information; The progress segment information includes the first type of progress segment and the second type of progress segment.

6. The method according to claim 5, characterized in that The method further comprises: When the progress segment information exists in the correspondence table, the progress segment information is directly obtained from the correspondence table.

7. The method according to claim 5, characterized in that After determining the first type of progress segment whose estimated bandwidth is greater than the bandwidth threshold and the second type of progress segment whose estimated bandwidth is less than or equal to the bandwidth threshold, the method further includes: The progress segment information and the model identifier of the neural network model are stored in the corresponding relationship table in correspondence.

8. The method according to any one of claims 1 to 4, characterized in that: The first memory includes at least one of a system cache and a system buffer memory.

9. A computing device for a neural network model, characterized in that: The device comprises: Acquisition module, used to obtain the neural network model; a determination module, configured to determine a first type of progress segment and a second type of progress segment during the operation of the neural network model, wherein the bandwidth required for network operation in the first type of progress segment is greater than a bandwidth threshold, and the bandwidth required for network operation in the second type of progress segment is less than or equal to the bandwidth threshold, wherein the bandwidth threshold is set based on the data transmission bandwidth of the second memory, and the bandwidth threshold is used to determine whether the bandwidth required for the operation of the neural network model exceeds the data transmission bandwidth provided by the second memory; a computing module configured to, through a neural network processor, access neural network data using a first memory in the first type of progress segment and access neural network data using a second memory in the second type of progress segment, to perform computing of the neural network model; The access speed of the first memory is greater than the access speed of the second memory, the first memory includes a static random access memory, and the second memory includes a dynamic random access memory.

10. A terminal, characterized in that: The terminal includes: processor; and A memory, wherein program instructions are stored on the memory, and when the processor executes the program instructions, the calculation method of the neural network model according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program instructions, and when the program instructions are executed by the processor, the calculation method of the neural network model according to any one of claims 1 to 8 is implemented.

Images