Interface calling device, method, equipment and medium for neural network processing unit

Abstract

The present invention relates to an interface calling device, method, equipment and medium of a neural network processing unit. Through standardized function calling interfaces and encapsulated driver details, users only need to use simple APIs to operate NPUs without having to understand the complex underlying driver and hardware details, which greatly reduces the difficulty of development. The encapsulated API uses a standard programming language, so that users do not need to understand custom structures and data types. The interface design is intuitive and easy to use, which facilitates users to quickly deploy and evaluate performance. A complete application processing flow from device initialization, network loading, pre-processing data, inference execution to resource release is provided, ensuring that the system can run efficiently and orderly at all stages. The encapsulated API includes the functions of initializing and destroying network contexts, ensuring that resources are effectively managed during allocation and release, avoiding resource leakage and insufficient memory problems, efficiently calling the NPU and improving the reliability of the system.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and embedded systems, and relates to an interface calling device, method, equipment and medium of a neural network processing unit. Background Art

[0002] In the field of modern artificial intelligence and embedded systems, neural network processing units (NPUs) are dedicated hardware accelerators that can significantly improve the efficiency and performance of neural network reasoning. NPUs usually work in conjunction with other processors (such as digital signal processors, DSPs) to achieve complex computing tasks through high-speed data transmission and processing.

[0003] In the existing NPU calling technology in actual application, for example, some embedded systems use specific APIs (Application Programming Interface) to communicate with NPU. These APIs usually require developers to understand the specific custom structure, data type, parameter settings and some driver function calling sequences, which are relatively complicated to use. Therefore, in order to solve the technical problems of complex NPU calling and lack of complete processing flow, an efficient and simple solution is needed. Summary of the invention

[0004] In response to the problems existing in the above-mentioned traditional methods, the present invention proposes an interface calling device for a neural network processing unit, an interface calling method for a neural network processing unit, a computer device and a computer-readable storage medium, which can efficiently call NPU and provide a complete processing flow.

[0005] In order to achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] On the one hand, an interface calling device for a neural network processing unit is provided, including a network file context list, an open device application program interface, a close device application program interface, an initialize network application program interface, a destroy network application program interface, and a network reasoning application program interface;

[0007] The network file context list is used to encapsulate the configuration information and parameters required to call the neural network processing unit, and the configuration information and parameters include the neural network processing unit configuration parameters, memory allocation information, model parameters, and input and output data buffer addresses;

[0008] The open device application program interface encapsulates the interaction logic with the underlying driver of the neural network processing unit device, which is used to trigger the underlying driver of the neural network processing unit device to initialize the driver and ensure that the neural network processing unit is in an available state;

[0009] The initialization network API is used to create a network object, allocate computing resources and initialize the network context according to the network inference data structure input by the user;

[0010] The network inference application program interface is used to copy all input data required for the network inference data structure to the memory space planned by the neural network processing unit device, connect all input buffers and output buffers of the network to the effective buffers of the neural network processing unit respectively, refresh the input data cache and notify the neural network processing unit to run the network and wait for the inference operation to be completed, refresh the output data cache and copy the inference result data to the memory space planned by the user; the network inference application program interface supports multiple calls;

[0011] The destroy network API is used to destroy network objects and release allocated computing resources according to the network file context list;

[0012] The shutdown device application program interface is used to release all resources occupied by the neural network processing unit and control the power off of the neural network processing unit.

[0013] On the other hand, a method for calling an interface of a neural network processing unit is also provided, which is applied to the interface calling device of the neural network processing unit. The method for calling an interface of a neural network processing unit comprises the steps of:

[0014] Call to open the device application program interface to initialize the driver, configure the software environment of the neural network processing unit and power on the neural network processing unit device;

[0015] Call the initialization network application program interface to create a network object, allocate computing resources and initialize the network context according to the network inference data structure input by the user;

[0016] Enter the loop part to obtain the inference result data; the loop part includes obtaining input data, pre-processing data, calling the network inference application program interface, post-processing data and judging whether to exit the loop;

[0017] When judging to exit the loop part, call the destroy network application program interface to destroy the network object and release the allocated computing resources;

[0018] Calling the shutdown device application program interface releases all resources occupied by the neural network processing unit and controls the power off of the neural network processing unit.

[0019] On the other hand, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the interface calling method of the above-mentioned neural network processing unit when executing the computer program.

[0020] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the interface calling method of the above-mentioned neural network processing unit are implemented.

[0021] One of the above technical solutions has the following advantages and beneficial effects:

[0022] The interface calling device, method, equipment and medium of the above-mentioned neural network processing unit, through standardized function call interface and encapsulated driver details, users only need to use simple API to operate NPU, without understanding its underlying complex driver and hardware details, which greatly reduces the difficulty of development. The encapsulated API uses a standard programming language, so that users do not need to understand custom structures and data types. The interface design is intuitive and easy to use, which is convenient for users to quickly deploy and evaluate performance. It provides a complete application processing flow from device initialization, network loading, pre-processing data, inference execution to resource release, ensuring that the system can run efficiently and orderly at all stages. The encapsulated API includes the functions of initializing and destroying network contexts, ensuring that resources are effectively managed during allocation and release, avoiding resource leakage and insufficient memory problems, and improving the reliability of the system.

[0023] Compared with traditional technologies, users can call NPU conveniently and efficiently, and achieve efficient data processing and resource management when DSP and NPU work together. In addition, it simplifies the process of NPU application development, improves the usability and reusability of the interface, thereby improving the user development experience and reducing development complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 A schematic diagram of the module structure of an interface calling device of a neural network processing unit in one embodiment;

[0026] Figure 2 A schematic diagram of a process for opening a device API in one embodiment;

[0027] Figure 3 A schematic diagram of a process for closing a device API in one embodiment;

[0028] Figure 4 A schematic diagram of a process flow implemented by a network reasoning API in one embodiment;

[0029] Figure 5 is a schematic diagram of a neural network reasoning process in one embodiment;

[0030] Figure 6 A flowchart of an interface calling method of a neural network processing unit in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0032] It should be noted that the reference to "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The presentation of this phrase at various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It will be appreciated by those skilled in the art that the embodiments described herein may be combined with other embodiments. The term "and / or" used in the specification of the present invention and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0033] The following will describe the implementation of the present invention in detail with reference to the accompanying drawings in the embodiment diagram of the present invention.

[0034] In one embodiment, Figure 1 As shown, an interface calling device 100 for a neural network processing unit is provided, which may include a network file context list 11, an open device application program interface 13, a close device application program interface 15, an initialize network application program interface 17, a destroy network application program interface 19, and a network reasoning application program interface 21. The network file context list 11 is used to encapsulate the configuration information and parameters required to call the neural network processing unit, and the configuration information and parameters include the neural network processing unit configuration parameters, memory allocation information, model parameters, and input and output data buffer addresses. The open device application program interface 13 encapsulates the interaction logic with the underlying driver of the neural network processing unit device, and is used to trigger the underlying driver of the neural network processing unit device to initialize the drive to ensure that the neural network processing unit is in an available state.

[0035] The network initialization application program interface 17 is used to create a network object according to the network reasoning data structure input by the user, allocate computing resources and initialize the network context. The network reasoning application program interface 21 is used to copy all input data required for the network reasoning data structure to the memory space planned by the neural network processing unit device, connect all input buffers and output buffers of the network to the effective buffers of the neural network processing unit respectively, refresh the input data cache and notify the neural network processing unit to run the network and wait for the reasoning operation to be completed, refresh the output data cache and copy the reasoning result data to the memory space planned by the user; the network reasoning application program interface 21 supports multiple calls. The network destruction application program interface 19 is used to destroy the network object and release the allocated computing resources according to the network file context list 11. The shutdown device application program interface 15 is used to release all resources occupied by the neural network processing unit and control the power off of the neural network processing unit.

[0036] It is understandable that Figure 1 As shown, the interface calling device 100 of the neural network processing unit includes a network file context list 11 and five application programming interfaces (APIs), and the five application programming interfaces are: an open device application programming interface 13 (hereinafter referred to as the open device API), a close device application programming interface 15 (hereinafter referred to as the close device API), an initialize network application programming interface 17 (hereinafter referred to as the initialize network API), a destroy network application programming interface 19 (hereinafter referred to as the destroy network API) and a network reasoning application programming interface 21 (hereinafter referred to as the network reasoning API).

[0037] The interface calling device 100 of the above-mentioned neural network processing unit, through standardized function call interface and encapsulated driver details, users only need to use simple API to operate NPU, without understanding its underlying complex driver and hardware details, which greatly reduces the difficulty of development. The encapsulated API uses a standard programming language, so that users do not need to understand custom structures and data types. The interface design is intuitive and easy to use, which is convenient for users to quickly deploy and evaluate performance. It provides a complete application processing flow from device initialization, network loading, pre-processing data, inference execution to resource release, ensuring that the system can run efficiently and orderly at all stages. The encapsulated API includes the functions of initializing and destroying network contexts, ensuring that resources are effectively managed during allocation and release, avoiding resource leakage and insufficient memory problems, and improving the reliability of the system.

[0038] Compared with traditional technologies, users can call NPU conveniently and efficiently, and achieve efficient data processing and resource management when DSP and NPU work together. In addition, it simplifies the process of NPU application development, improves the usability and reusability of the interface, thereby improving the user development experience and reducing development complexity.

[0039] The network file context list 11 includes all configuration information and parameters required for calling the NPU, which may include, for example, NPU configuration parameters, memory allocation information, model parameters, and input and output data buffer addresses.

[0040] By centralizing these configuration information and parameters in one list, parameter management when calling NPU is simplified, so that users only need to process one list when calling NPU-related APIs, without having to provide each parameter one by one. Moreover, since the network file context list 11 encapsulates all necessary configuration information and parameters, different APIs can reuse the same network file context list 11, thereby avoiding the appearance of duplicate code; this design improves the reusability and maintainability of the code and reduces code redundancy. Furthermore, by defining and using a unified network file context list 11, the consistency of all NPU-related API calls is ensured; all APIs follow the same parameter passing method, reducing system crashes or errors caused by parameter passing errors. This consistency design improves the reliability and stability of the system. In addition, the network file context list 11 not only contains the parameters required for initialization and runtime, but also includes information on resource allocation and release. Users can easily initialize and destroy the network context to ensure efficient management and utilization of resources.

[0041] Device Open API: This API encapsulates the interaction logic with the underlying driver of the NPU device. By calling this device open API, the user can trigger the underlying driver to perform necessary initialization operations. Through this encapsulation, the user does not need to understand the implementation details of the underlying driver of the NPU device, and only needs to call the standardized API to complete the device open operation.

[0042] In one embodiment, when the device status is queried by opening the device API, if the device status has been called once, it will be displayed that the neural network processing unit device has been opened and the operation of resetting the device will be entered; resetting the device is used to reset the status register inside the neural network processing unit.

[0043] It is understandable that Figure 2 As shown, the main operations implemented by calling the device opening API include device status query and device reset. When querying the device status, if the device opening API has been called once, it will show that the NPU device has been opened, and directly enter the next step of resetting the device; wherein, resetting the device is used to reset the status register inside the NPU so that the NPU can quickly enter the latest ready state. If the display device is not opened, the following setting operations need to be performed, which include powering on the NPU, setting the NPU clock frequency, and initializing the NPU software environment.

[0044] Specifically, when the NPU device is running in energy-saving mode, the NPU is usually in a power-off state. In order for the NPU to work normally, the NPU device must be powered on first: for example, read and write the NPU power control register to power the NPU device and put it into working state. Then set the NPU clock frequency, that is, read and write the NPU clock control register to configure the NPU clock frequency.

[0045] In one embodiment, when the device API is opened to configure the clock frequency of the neural network processing unit by reading and writing the clock control register of the neural network processing unit, if there is a user-defined clock frequency, the user-defined clock frequency is used to configure the clock frequency value of the neural network processing unit, otherwise the default clock frequency is used to configure the clock frequency value of the neural network processing unit.

[0046] It can be understood that if the user sets a custom clock frequency, the user-defined clock frequency value is used; if the user does not set the clock frequency, the default clock frequency value is used. Then initialize the NPU software environment. Opening the device API includes the steps of setting the NPU software environment. For example, before the NPU starts working, it is necessary to allocate and initialize the multi-segment memory area required by the NPU. These configuration operations are implemented internally by the opening device API to ensure that the NPU runs in the correct environment, thereby improving the stability and reliability of the system.

[0047] In one embodiment, when the device API is shutting down and querying the device status, if the neural network processing unit device has entered a low power consumption state, no operation needs to be performed.

[0048] Disable device API: Figure 3 As shown, firstly, the device status is queried. If the device has entered a low-power state (i.e., energy-saving mode), no operation is required. If the device has not entered a low-power state, two operations are performed: releasing NPU resources and controlling NPU power-off. Among them, releasing NPU resources: before shutting down the NPU device, the shutdown device API is responsible for releasing all resources occupied by the NPU, including memory, cache, and other hardware resources, to ensure that resources are properly managed and released when the NPU device is shut down to prevent resource leakage and system instability. These release operations are implemented internally by the shutdown device API to ensure the integrity and effectiveness of resource management. Control NPU power-off: The shutdown device API calls the underlying driver of the NPU device, reads and writes the power control register of the NPU, and powers off the NPU device, putting it into a low-power or sleep state. Through the power-off operation, the overall power consumption of the system is reduced and the service life of the device is extended.

[0049] Initialize network API: used to create a network object based on the network inference data structure (NBG) input by the user. The initialize network API first receives the NBG input by the user, creates a network object based on the data, and associates the network object to the network file context list 11. The network object contains the structure and parameter information of the network model and is the basis for network inference. The initialize network API then starts to allocate resources and initialize the network stack space. According to the network file context list 11, after creating the network object, the initialize network API needs to allocate the necessary computing resources, including memory, cache, and other hardware resources. Specifically, the initialize network API will initialize the network stack space and deploy all operating resources into the internal memory pool to ensure that the resources required for the network inference process are sufficient and well managed.

[0050] Then, the network initialization API applies for memory space for the parameters that require memory space in the network file context list 11. During the initialization process, the network initialization API checks the parameter list in the network to determine which parameters need to allocate memory space, and applies for appropriate memory space for these parameters through the internal memory management mechanism to ensure that these parameters can be correctly accessed and used during the network inference process. Subsequently, the network initialization API allocates internal memory resources for NBG, deploys all operating resources to the internal memory pool, configures / generates command buffers for the NBG, and patch command buffers for resources in the internal memory allocation.

[0051] In one embodiment, when the network API is initialized to verify the network inference data structure input by the user, if it is checked that the network data of the network inference data structure is inconsistent with the current network model, or the network inference data structure does not meet the established requirements of the system, an error message is returned; the network data of the network inference data structure includes data format and version.

[0052] It can be understood that, finally, the network API is initialized to verify NBG (user input). To ensure the correctness and consistency of network reasoning, the network API will verify the network reasoning data structure input by the user. Specifically, it can include checking whether the network reasoning data structure meets the established requirements of the system, and whether the data format and version are consistent with the current network model; if inconsistency is found or it does not meet the established requirements, the network API will return a corresponding error message to prompt the user to make corrections.

[0053] Destroy network API: used to release network objects. Destroy network API will first destroy the network objects created during the initialization process according to the network file context list 11 to ensure that all data structures related to the network model are correctly released. By destroying network objects, memory and other resources related to the network model can be effectively reclaimed. Next, the Destroy network API releases the memory pool and network stack space; after destroying the network object, the Destroy network API needs to release the memory pool and network stack space allocated during the initialization process. Specifically, the Destroy network API will call the memory management module according to the network file context list 11 to release all memory used in the network inference process to ensure that memory resources are correctly recycled. Finally, the Destroy network API releases the parameter memory space. During the initialization process, the memory space allocated for the network parameters also needs to be released during the destruction process. Therefore, the Destroy network API will traverse the network file context list 11 and release the memory space allocated for these parameters one by one to ensure that there is no memory leak.

[0054] Network Inference API: used to assist NPU in network inference and manage inference results. Figure 4 As shown, the main steps implemented by the network inference API may include:

[0055] Copy all input data required by NBG to the memory space planned by the NPU device;

[0056] Connect input buffers and output buffers; connect all inputs and outputs of the network to the NPU valid buffers respectively. When connecting, the driver will patch the network command buffer to fill the buffer address;

[0057] Refresh the input data cache; operate the NPU to refresh the input data cache to ensure that the latest input data is available for the network;

[0058] Notify the NPU to run the network and wait for the inference operation to be completed; notify the NPU to perform the network inference task, and the NPU starts to perform the inference operation according to the predetermined network model and parameters;

[0059] Refresh the output data cache: Operate the NPU to refresh the output data cache to ensure that the output inference result data is ready for users to read;

[0060] The inference result data is copied to the memory space planned by the user; the inference result data after the inference is completed is copied from the memory of the NPU device to the memory area specified by the user.

[0061] It can be understood that the network reasoning API can be called multiple times, and each time it is called, the network reasoning API will use the currently attached input buffer and output buffer for reasoning. This design allows users to perform multiple reasoning operations in the same network context, which is highly flexible and adapts to different reasoning needs.

[0062] It can be understood that the interface calling device 100 of the neural network processing unit provides a complete NPU calling process for managing the initialization, execution and resource release of the NPU. Figure 5 As shown in the figure, the process includes opening and closing the device, initializing and destroying the network, and pre-processing, reasoning and post-processing of the data. The network reasoning data structure (NBG) is represented by the .nbg file.

[0063] First, call the open device API, initialize the driver, configure the NPU software environment, and power on the NPU device; this step ensures that the NPU device is in a usable state and ready for subsequent operations. Then, according to the binary network inference data structure (NBG) input by the user, call the initialize network API to create a network object, allocate computing resources and initialize the network context; this step ensures that the network model is loaded and ready for inference operations.

[0064] Next, enter the loop part, including obtaining input data, pre-processing data, calling the network inference API, post-processing data, and judging whether to exit. Among them, obtaining input data means obtaining input data required for inference from an external source (such as a sensor or a file). Pre-processing data is to pre-process the input data, including normalization and quantization operations to meet the input requirements of the network model. Call the network inference API to pass the pre-processed input data to the NPU for inference, and store the inference results in the output buffer. Post-processing data is to post-process the inference results, including dequantization, classification, and recognition operations to generate the final output inference result data. Judging whether to exit is to judge whether to exit the loop part according to specific conditions (such as processing all input data or receiving an exit signal). If it is judged to continue the loop, then return to the step of obtaining input data. If it is judged to exit the loop, then call the destruction network API so that after exiting the loop part, the network object is destroyed by destroying the network API, and the resources allocated during the initialization process are released to ensure that all resources are released correctly and prevent resource leakage.

[0065] Finally, the shutdown device API is called to release NPU resources, shut down the driver, and power off the NPU. This step ensures that the NPU device is in the shutdown state and system resources are managed correctly.

[0066] Compared with traditional technologies, the above design standardizes the function call interface, such as providing a user-friendly function call interface, which simplifies the user's call to the NPU without having to understand its underlying driver and data structure. The function call interface is decoupled from the underlying driver of the NPU device. Users do not need to understand the custom structure, data type, parameter setting, and driver function call sequence. They can directly use the standard C language interface to call, which is convenient for users. It also provides a complete set of network inference processing flows, such as defining a network file context list 11, which simplifies the call of NPU-related APIs, ensures the uniformity and convenience of configuration, memory, and parameter management, and covers the complete application process of opening the device, initializing the network, pre-processing data, executing inference, post-processing data, and releasing resources.

[0067] In this way, through standardized function call interfaces and encapsulated driver details, users only need to use simple APIs to operate NPU without having to understand the underlying complex driver and hardware details, which greatly reduces the difficulty of development. The encapsulated API uses standard C language, so that users do not need to understand custom structures and data types. The interface design is intuitive and easy to use, which facilitates users to quickly deploy and evaluate performance. It provides a complete application process from device initialization, network loading, pre-processing data, inference execution to resource release, ensuring that the system can run efficiently and orderly at all stages. The encapsulated API includes the functions of initializing and destroying network contexts, ensuring that resources are effectively managed during allocation and release, avoiding resource leakage and insufficient memory, and improving system reliability.

[0068] Each module in the interface calling device 100 of the neural network processing unit can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a device with data processing function in the form of hardware, or can be stored in the memory of the aforementioned device in the form of software, so that the processor can call and execute the operations corresponding to each module. The aforementioned device can be, but is not limited to, various types of computer devices already available in the art.

[0069] In one embodiment, Figure 6 As shown, a method for calling an interface of a neural network processing unit is provided, which is applied to the above-mentioned device 100 for calling an interface of a neural network processing unit. The method for calling an interface of a neural network processing unit may include the following processing steps S10 to S18:

[0070] S10, calling and opening the device application program interface to initialize the driver, configuring the software environment of the neural network processing unit and powering on the neural network processing unit device;

[0071] S12, calling the initialization network application program interface to create a network object, allocate computing resources and initialize the network context according to the network inference data structure input by the user;

[0072] S14, entering the loop part to obtain the inference result data; the loop part includes obtaining input data, pre-processing data, calling the network inference application program interface, post-processing data and determining whether to exit the loop;

[0073] S16, when it is determined to exit the loop part, calling the destroy network application program interface to destroy the network object and release the allocated computing resources;

[0074] S18, calling the shutdown device application program interface to release all resources occupied by the neural network processing unit and control the neural network processing unit to power off.

[0075] The interface calling method of the above-mentioned neural network processing unit, through standardized function calling interface and encapsulated driver details, users only need to use simple API to operate NPU, without having to understand its underlying complex driver and hardware details, which greatly reduces the difficulty of development. The encapsulated API uses a standard programming language, so that users do not need to understand custom structures and data types. The interface design is intuitive and easy to use, which is convenient for users to quickly deploy and evaluate performance. It provides a complete application processing flow from device initialization, network loading, pre-processing data, inference execution to resource release, ensuring that the system can run efficiently and orderly at all stages. The encapsulated API includes the functions of initializing and destroying network contexts, ensuring that resources are effectively managed during allocation and release, avoiding resource leakage and insufficient memory problems, and improving the reliability of the system.

[0076] Compared with traditional technologies, users can call NPU conveniently and efficiently, and achieve efficient data processing and resource management when DSP and NPU work together. In addition, it simplifies the process of NPU application development, improves the usability and reusability of the interface, thereby improving the user development experience and reducing development complexity.

[0077] In one embodiment, the interface calling method of the neural network processing unit may further include the steps of:

[0078] When determining to continue the loop, return to the step of obtaining input data.

[0079] For the specific limitations and explanations of the interface calling method of the neural network processing unit, please refer to the corresponding limitations and explanations of the interface calling device 100 of the neural network processing unit mentioned above, which will not be repeated here.

[0080] It should be understood that although Figure 6The steps in the method are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Figure 6 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequentially, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0081] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the following processing steps when executing the computer program: calling an open device application program interface to initialize the driver, configuring the software environment of the neural network processing unit and powering on the neural network processing unit device; calling an initialize network application program interface to create a network object according to the network reasoning data structure input by the user, allocating computing resources and initializing the network context; entering a loop part to obtain reasoning result data; the loop part includes obtaining input data, pre-processing data, calling a network reasoning application program interface, post-processing data and judging whether to exit the loop; when judging to exit the loop part, calling a destroy network application program interface to destroy the network object and release the allocated computing resources; calling a close device application program interface to release all resources occupied by the neural network processing unit and control the neural network processing unit to power off.

[0082] In one embodiment, when the processor executes the computer program, it can also implement the steps or sub-steps added in each embodiment of the interface calling method of the neural network processing unit mentioned above.

[0083] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following processing steps are implemented: calling an open device application program interface to initialize the driver, configuring the software environment of the neural network processing unit and powering on the neural network processing unit device; calling an initialize network application program interface to create a network object according to the network reasoning data structure input by the user, allocating computing resources and initializing the network context; entering a loop part to obtain reasoning result data; the loop part includes obtaining input data, pre-processing data, calling a network reasoning application program interface, post-processing data and judging whether to exit the loop; when judging to exit the loop part, calling a destroy network application program interface to destroy the network object and release the allocated computing resources; calling a close device application program interface to release all resources occupied by the neural network processing unit and control the neural network processing unit to power off.

[0084] In one embodiment, when the computer program is executed by a processor, it can also implement the steps or sub-steps added in each embodiment of the interface calling method of the neural network processing unit described above.

[0085] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus dynamic random access memory (RambusDRAM, referred to as RDRAM) and interface dynamic random access memory (DRDRAM).

[0086] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of protection of the invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. An interface calling device for a neural network processing unit, characterized in that: Including network file context list, open device API, close device API, initialize network API, destroy network API and network reasoning API; The network file context list is used to encapsulate the configuration information and parameters required for calling the neural network processing unit, and the configuration information and parameters include the neural network processing unit configuration parameters, memory allocation information, model parameters, and input and output data buffer addresses; The device opening application program interface encapsulates the interaction logic with the underlying driver of the neural network processing unit device, and is used to trigger the underlying driver of the neural network processing unit device to initialize the drive, thereby ensuring that the neural network processing unit is in an available state; The network initialization application program interface is used to create a network object according to the network reasoning data structure input by the user, allocate computing resources and initialize the network context; The network reasoning application program interface is used to copy all input data required for the network reasoning data structure to the memory space planned by the neural network processing unit device, connect all input buffers and output buffers of the network to the effective buffers of the neural network processing unit respectively, refresh the input data cache, notify the neural network processing unit to run the network and wait for the reasoning operation to be completed, refresh the output data cache and copy the reasoning result data to the memory space planned by the user; the network reasoning application program interface supports multiple calls; The destroy network application program interface is used to destroy network objects and release allocated computing resources according to the network file context list; The shutdown device application program interface is used to release all resources occupied by the neural network processing unit and control the neural network processing unit to power off.

2. The interface calling device of the neural network processing unit according to claim 1, characterized in that: When the device status is queried through the device opening application program interface, if the device opening application program interface has been called once, it will be displayed that the neural network processing unit device has been opened and the operation of resetting the device will be entered; the reset device is used to reset the status register inside the neural network processing unit.

3. The interface calling device of the neural network processing unit according to claim 2, characterized in that: When the device application interface is opened to configure the clock frequency of the neural network processing unit by reading and writing the clock control register of the neural network processing unit, if there is a user-defined clock frequency, the user-defined clock frequency is used to configure the clock frequency value of the neural network processing unit, otherwise the default clock frequency is used to configure the clock frequency value of the neural network processing unit.

4. The interface calling device of the neural network processing unit according to any one of claims 1 to 3, characterized in that: When the shutdown device application program interface queries the device status, if the neural network processing unit device has entered a low power consumption state, no operation needs to be performed.

5. The interface calling device of the neural network processing unit according to claim 4, characterized in that: When the initialized network application program interface verifies the network inference data structure input by the user, if it is detected that the network data of the network inference data structure is inconsistent with the current network model, or the network inference data structure does not meet the established requirements of the system, an error message is returned; the network data of the network inference data structure includes data format and version.

6. A method for calling an interface of a neural network processing unit, characterized in that: The interface calling device for the neural network processing unit according to any one of claims 1 to 5, wherein the interface calling method for the neural network processing unit comprises the steps of: Call to open the device application program interface to initialize the driver, configure the software environment of the neural network processing unit and power on the neural network processing unit device to ensure that the neural network processing unit is in an available state; Call the initialization network application program interface to create a network object, allocate computing resources and initialize the network context according to the network inference data structure input by the user; Entering the loop part to obtain the inference result data; the loop part includes obtaining input data, pre-processing data, calling the network inference application program interface, post-processing data and judging whether to exit the loop; When it is determined to exit the loop portion, calling a destroy network application program interface to destroy the network object and release the allocated computing resources; Calling the shutdown device application program interface releases all resources occupied by the neural network processing unit and controls the power off of the neural network processing unit.

7. The interface calling method of the neural network processing unit according to claim 6, characterized in that: Also includes the steps: When it is determined to continue the loop portion, return to the step of obtaining input data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the interface calling method of the neural network processing unit described in claim 6 or 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the interface calling method of the neural network processing unit described in claim 6 or 7 are implemented.

Images