Simulation device, simulation system and simulation method thereof, storage medium

Abstract

Embodiments of the present disclosure provide a simulation device, a simulation system and its simulation method, and a non-transitory computer-readable storage medium. The simulation device is used to simulate a neural network processor, communicates with an object model that simulates the neural network processor, and is configured to be an agent of the object model in the simulation device, including an agent module, a management module set to manage the simulation device, and an interconnection module configured to communicatively connect the agent module and the management module. The simulation device receives a task sent from a host, sends task-related work information to the object model by the agent module, receives feedback information returned after the object model processes the work information by the agent module, and provides the feedback information to the host.

Description

Technical Field

[0001] This application claims the priority of Chinese Patent Application No. 202210613143.4 filed on May 31, 2022, and hereby incorporates by reference the content disclosed in the above Chinese patent application as part of this application.

[0002] Embodiments of the present disclosure relate to a simulation device, a simulation system and its simulation method, and a non-temporary computer-readable storage medium.

Background Art

[0003] With the development of Artificial Intelligence, the amount of parameters of algorithm models has increased rapidly, and the demand for hash rate has been increasing. For conventional hardware architectures (e.g., CPU (Central Processing Unit) / GPU (Graphics Processing Unit)), due to considering the balance between different business needs at the architecture design stage, the hash rate provided for AI applications is limited. As a result, Domain-Specific Accelerator (DSA) emerged. The core idea of DSA is to do specialized things using dedicated hardware as well. Since DSA satisfies applications within one domain rather than one fixed application, DSA can meet the trade-off between flexibility and specialization.

Summary of the Invention

[0004] To introduce the concept in a simplified form, this summary part is provided, and these concepts will be described in detail in the detailed implementation part of the following invention. This summary part is not intended to identify the key features or essential features of the claimed technical solution, nor is it used to limit the scope of the claimed technical solution.

[0005] At least one embodiment of the present disclosure provides a simulation apparatus for simulating a neural network processor, comprising: an agent module configured to communicate with an object model that simulates the neural network processor and to be an agent of the object model in the simulation apparatus; a management module configured to manage the simulation apparatus; and an interconnection module configured to communicate between the agent module and the management module, wherein the simulation apparatus receives a task transmitted from a host, the agent module transmits work information relating to the task to the object model, the agent module receives feedback information returned by the object model after processing the work information, and provides the feedback information to the host.

[0006] At least one embodiment of the present disclosure further provides a simulation system comprising a simulation apparatus described in any one embodiment of the present disclosure, an object model, and a host, wherein the host is configured to acquire the task and transmit the task to the simulation apparatus, and the object model is configured to process the work information and obtain the feedback information.

[0007] At least one embodiment of the present disclosure further provides a simulation method applicable to a simulation system described in any one embodiment of the present disclosure, the simulation method comprising: transmitting a task by a host; receiving and analyzing the task by a simulation device and transmitting work information relating to the task to an object model; processing the work information by the object model and obtaining feedback information; and providing the feedback information to the host by the simulation device.

[0008] At least one embodiment of the present disclosure further provides a non-temporary computer-readable storage medium that stores computer-executable instructions which, when executed by a processor, realize the simulation method described in any one embodiment above. [Brief explanation of the drawing]

[0009] The above and other features, advantages and aspects of each embodiment of this disclosure will become more apparent by referring to the embodiments for carrying out the invention as shown in the drawings below. In all drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and elements are not necessarily drawn to scale. [Figure 1A] This is a schematic diagram of the hardware architecture of a simulation device provided by at least one embodiment of the present disclosure. [Figure 1B] This is a schematic diagram of the hardware architecture of another simulation device provided by at least one embodiment of the present disclosure. [Figure 2] This is a schematic diagram of the hardware architecture of a simulation system provided by at least one embodiment of the present disclosure. [Figure 3] This is a schematic flowchart of the simulation method provided by at least one embodiment of the present disclosure. [Figure 4] This is a schematic diagram of a non-temporary computer-readable storage medium provided by at least one embodiment of the present disclosure. [Figure 5] This is a schematic diagram of the hardware structure of an electronic device provided by at least one embodiment of the present disclosure. [Modes for carrying out the invention]

[0010] The embodiments of this disclosure will be described in more detail below with reference to the drawings. While the drawings show several embodiments of this disclosure, it should be understood that this disclosure is achievable in various ways and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to allow for a more thorough and complete understanding of this disclosure. It should be understood that the drawings and embodiments of this disclosure are for illustrative purposes only and do not limit the scope of protection of this disclosure.

[0011] It should be understood that the steps described in the embodiments of the method of this disclosure may be performed in a different order and / or in parallel. Furthermore, embodiments of the method may include additional steps and / or the described steps may be omitted. The scope of this disclosure is not limited in this respect.

[0012] As used herein, the term “including” and its variations are non-restrictive, meaning “including, but not limited to, …”. The term “based on” means “based at least partially on”. The term “one embodiment” means “at least one embodiment”. The term “another embodiment” means “at least one other embodiment”. The term “several embodiments” means “at least several embodiments”. Relevant definitions of other terms are given below.

[0013] Furthermore, the concepts of "first," "second," etc., as used in this disclosure are merely for the purpose of distinguishing different devices, modules, or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules, or units.

[0014] Furthermore, the modifications “one” and “multiple” as used in this disclosure are illustrative and not restrictive, and should be understood as “one or more” unless otherwise explicitly stated in the context, as can be understood by those skilled in the art.

[0015] The names of messages or information that interact between multiple devices in embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0016] Recent advancements have made the architectural design of AI hardware accelerators increasingly complex. From the initial single-core shared memory to the current homogeneous many-core distributed memory, the core design in AI hardware accelerators has evolved from simple Very Long Instruction Word (VLIW) structures to a coexistence of two implementation architectures: Turing's complete Instruction Set Architecture (ISA) and instruction-driven data access (Master-Slave and Data-Streaming). For early architectural design exploration and later verification of AI hardware accelerators, a complete simulation system for simulating them has become essential; in other words, current AI hardware accelerators lack a simple and efficient simulation framework for architectural design and functional verification.

[0017] Optimization policies for AI compilers primarily include operator fusion, partitioning, and I / O / computation parallel scheduling. These optimization policies constitute a vast optimization search space. The final execution efficiency of each optimization policy differs in the underlying accelerator, and a cost model is needed to evaluate each optimization policy and reduce the optimization search space. The cost model needs to be sufficiently accurate without introducing excessive overhead at the compilation stage. Currently, open-source all-system simulation platforms include the gem5 simulator. The drawbacks of the gem5 simulator include its enormous and complex system, its lack of design for modeling AI accelerators, and its low execution efficiency. Furthermore, the gem5 simulator has too much overhead as a cost model. Therefore, a high-performance cost model is currently needed to evaluate AI compilers.

[0018] At least one embodiment of the present disclosure provides a simulation apparatus used to simulate a neural network processor and includes an agent module, a management module, and an interconnection module. The agent module communicates with an object model that simulates a neural network processor and is configured to be an agent of the object model in the simulation apparatus; the management module is configured to manage the simulation apparatus; and the interconnection module is configured to communicate between the agent module and the management module. The simulation apparatus receives tasks transmitted from a host, the agent module transmits work information about the tasks to the object model, the agent module receives feedback information returned by the object model after processing the work information, and provides the feedback information to the host.

[0019] The simulation device provided by an embodiment of the present disclosure can effectively implement the architecture design, exploration, and functional verification of a neural network processor by modeling the entire system simulation platform of the neural network processor, and can accelerate the structural and mechanism design process of the neural network processor, thereby avoiding the development of the neural network processor being restricted by hardware. Also, this simulation device has a simple structure and is easy to implement.

[0020] At least one embodiment of the present disclosure further provides a simulation system, its simulation method, and a non - transient computer - readable storage medium.

[0021] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings, but the present disclosure is not limited to these specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, details of some known functions and known components are omitted.

[0022] FIG. ۱A is a schematic diagram of the hardware architecture of a simulation device provided by at least one embodiment of the present disclosure, FIG. ۱B is a schematic diagram of the hardware architecture of another simulation device provided by at least one embodiment of the present disclosure, and FIG. ۲ is a schematic diagram of the hardware architecture of a simulation system provided by at least one embodiment of the present disclosure.

[0023] The simulation device provided by an embodiment of the present disclosure can be used to simulate a neural network processor, and the neural network processor can be realized in hardware form, for example, it can be realized as a chip based on a programming language. For example, the neural network processor can be used to implement convolution operations, matrix operations, etc.

[0024] As shown in FIGS. 1A and 1B, in some embodiments of the present disclosure, the simulation device 100 may include an agent module 110, a management module 120, and an interconnection module 130. As shown in FIG. 2, the agent module 110 communicates with an object model 300 that simulates a neural network processor, and is configured to be an agent of the object model in the simulation device 100. That is, the object model 300 is used to simulate the functions of a neural network processor. The management module 120 is configured to manage the simulation device 100, and the interconnection module 130 is configured to communicatively connect the agent module 110 and the management module 120. Note that the simulation device 100 shown in FIG. 2 is the simulation device shown in FIG. 1A, but the simulation device 100 in the simulation system 1000 may be the simulation device shown in FIG. 1B.

[0025] For example, the management module 120 can manage the agent module 110 in order to control the agent module 110 to execute the corresponding functions.

[0026] For example, as shown in FIG. 2, the simulation device 100 receives a task sent from the host 200, sends work information related to the task to the object model by the agent module 110, receives feedback information returned after the object model processes the work information by the agent module 110, and provides the feedback information to the host 200.

[0027] For example, the simulation device 100 is implemented by a virtual simulation platform. For example, the simulation device 100 can be implemented as a QEMU (Quick EMUlator) virtual platform (virt platform) or the like.

[0028] For example, the management module 120 is implemented by a virtual central processing unit, and in some embodiments, the virtual central processing unit may be a RISC (Reduced Instruction Set Computer RISC)-V (where V represents fifth-generation RISC) core.

[0029] For example, the task may be any task that needs to be performed by a neural network processor, such as goal identification or matrix operations (e.g., matrix multiplication).

[0030] For example, communication between the agent module 110 and the host 200 can be performed using at least one method, which may include, for example, a socket method, a shared storage method, and / or a message queue method. As shown in Figure 2, in some embodiments, the agent module 110 and the host 200 communicate using a socket method, a shared storage method, and a message queue method. The socket method may be a Unix Domain Socket (UDS) method. The shared storage method indicates communication using a shared storage file method, for example, the shared storage file may be a / dev / shm / ivshmem file on the host 200. The ivshmem file can enable sharing of memory areas created by the host 200 between different QEMU processes. The message queue method MQ1 can transmit messages using a First Input First Output (FIFO) method.

[0031] For example, communication between the agent module 110 and the host 200 is achieved using a shared storage method for large amounts of data, and communication between small amounts of data is achieved using a socket method and / or a message queue method.

[0032] For example, communication between the agent module 110 and the object model 300 can be conducted in at least one way. For instance, communication between the agent module 110 and the object model 300 can be conducted using a message queue method. As shown in Figure 2, in some embodiments, the agent module 110 and the object model 300 communicate using a message queue method. For example, the agent module 110 sends task-related work information to the object model 300 using message queue MQ2, and the object model 300 sends feedback information to the agent module 110 using message queue MQ3. Similarly, message queue MQ2 and message queue MQ3 can transmit messages using a FIFO (First-In, First-Out) method.

[0033] For example, the object model 300 can be connected to the simulation device 100 in a pluggable manner, meaning that the simulation device 100 can simulate different object models, and for example, different object models can simulate different neural network processors.

[0034] For example, object model 300 can accelerate the computation speed of a neural network processor, save computation time, and improve computation efficiency.

[0035] For example, as shown in Figure 2, the host 200 can acquire tasks from external devices and transmit those tasks to the simulation device 100 using a socket method.

[0036] For example, information such as the parameters of the neural network processor and the inputs and / or outputs related to the task are accessed by the host 200 and the simulation device 100 via shared storage for operations such as reading and writing. For example, the inputs and / or outputs related to the task can be determined based on the type of task. For example, in some embodiments, the task may be to identify a target object in an image and feed back the image with the target object marked to the host 200. In this case, the input image may be the input to the task, and the image identified by the object model 300 and with the target object marked may be the output to the task.

[0037] For example, the simulation device 100 can send feedback information to the host 200 using a message queue method (e.g., synchronize).

[0038] For example, in some embodiments, as shown in Figure 1B, the simulation device 100 may further include an input / output module 140 that can interact with the host 200, for example, by communicating with the host 200 via a socket to receive tasks. For example, the input / output module 140 may be configured to send tasks to an agent module 110 or a management module 120. For example, the input / output module 140 may include a buffer or the like to store tasks.

[0039] For example, as shown in Figures 1A and 1B, the address space of the agent module 110 can include the agent register space Re and the model space Mem1.

[0040] For example, the agent register space Re is used to define the registers of the neural network processor. All of the neural network processor's registers are memory-mapped. Memory mapping means that the device's registers and memory are uniformly addressed. In other words, by positioning the device's registers using memory addresses, the neural network processor's registers are defined within the memory space and can be read and written by being positioned using the memory address method.

[0041] For example, the agent register space Re can communicate with host 200 and object model 300 using a message queue method.

[0042] For example, the model space Mem1 is used to store parameters of a neural network processor and inputs and / or outputs related to the task.

[0043] For example, in some embodiments, the model space Mem1 is shared by the agent module 110 and the host 200, in which case, for example, the model space Mem1 communicates with the host 200 using a shared storage scheme.

[0044] For example, in some embodiments, as shown in Figure 2, the model space Mem1 may be mapped as the memory space Mem2 in the object model. As can be seen from this, the model space Mem1 and the memory space Mem2 share the same address space, and the actual memory is mapped to a shared memory file on the host 200. In other words, the model space Mem1 can actually be accessed by the simulation device 100, the host 200, and the object model 300. For example, by the host 200 writing information such as the parameters of the neural network processor and the inputs and / or outputs related to the task to the model space Mem1, the object model 300 can directly read the information such as the inputs and / or outputs related to the task written to the model space Mem1 and perform the relevant processing.

[0045] For example, in some embodiments, as shown in Figure 2, the address space of the agent module 110 may further include a configuration instruction space Mem3. The configuration instruction space Mem3 is used to store configuration instructions for the neural network processor's registers and control instructions related to tasks.

[0046] For example, in some embodiments, the agent module 110, the management module 120, and the interconnection module 130 can be mounted on a PCIe (peripheral component interconnect express) system. A PCIe system can include several types of devices, such as a root complex (RC), a bridge, a switch, and an endpoint. The root complex is the interface between the CPU and the PCIe bus, and the bridge provides an interface with other buses (e.g., PCI or PCI-x, and yet another PCIe bus), sometimes referred to as bridge forwarding. A switch provides expansion or aggregation capabilities, allowing more devices to be connected to PCIe ports. A switch can function as a packet router, identifying which path a given packet should take based on its address or other routing information, and is a PCIe-to-PCIe bridge. The endpoint is located at the very end of the PCIe bus system topology and is generally considered to be either an initiator (similar to a host in a PCI bus) or a completer (similar to a slave in a PCI bus) for bus operations.

[0047] For example, in some embodiments, the interconnection module 130 may be the root complex in the PCIe system.

[0048] For example, in some embodiments, the agent module 100 may be an endpoint in a PCIe system. For example, the agent module 100 may include a plurality of base address register (BAR) spaces, and the plurality of base address register spaces may include a first base address register space and a second base address register space. In some embodiments, the plurality of base address register spaces may include BAR0 to BAR5 spaces, the first base address register space may be the BAR0 space, and the second base address register space may be the BAR3 space.

[0049] For example, the agent register space Re corresponds to the first base address register space, e.g., the BAR0 space, and the configuration instruction space Mem3 corresponds to the second base address register space, e.g., the BAR3 space. The base address register space refers to the PCIe BAR space, and the simulation device, as a host device, is connected to the HOST via a PCIe interface. In order for the HOST to access the simulation device, it needs to map the space on the simulation device to the PCIe BAR space. If the simulation device has multiple independent accessible spaces, PCIe provides multiple BAR spaces for mapping.

[0050] For example, the configuration instruction space Mem3 is mapped to a shared storage file ivshmem, which is shared by the agent module 110 and the host 200, and the shared storage file ivshmem is located on the host 200. In other words, in the simulation device 100, all access to the BAR3 space is forwarded to the shared storage file ivshmem on the host 200, and the host 200 enables the transmission of configuration instructions to the agent module 110 in the simulation device 100 by writing to the shared storage file ivshmem.

[0051] For example, in some embodiments of this disclosure, as shown in Figure 2, the actual storage of the model space Mem1, storage space Mem2, and configuration instruction space Mem3 is mapped to a shared storage file in the host 200. The model space Mem1 and storage space Mem2 can be mapped to the same location in the shared storage file, and the location where the configuration instruction space Mem3 is mapped to the shared storage file is different from the location where the model space Mem1 / storage space Mem2 are mapped to the shared storage file.

[0052] For example, the agent module 110 is configured to provide the agent register space Re and the content in the model space Mem1 to the object model 300 based on the content in the setting instruction space Mem3, thereby setting up the object model 300, scheduling it, and performing operation simulation. If the model space Mem1 can be mapped as the memory space Mem2 in the object model, the object model 300 can directly access the model space Mem1 to obtain necessary data such as input data related to the task.

[0053] For example, host 200 can interrupt agent module 110 using a socket method, meaning that tasks performed by object model 300, transmitted from host 200, can be executed using an interrupt method.

[0054] For example, in some embodiments, the agent module 110 may include an interrupt register, and the host writes task notification information to the interrupt register and notifies the management module 120 via an interrupt to execute the task. For example, the content communicated by the memory manager (runtime) process on the host 200 to the simulation device 100 via a socket is as follows: The memory manager performs a write operation on the interrupt register of the agent module 110 in the simulation device 100 to write notification information about the task. After the notification information about the task is written to the interrupt register of the agent module 110, it interrupts the management module 120. After interrupting the management module 120, the agent module 110 executes its interrupt handling program (which belongs to the agent module 110's drive program). Since the interrupt handling program is in kernel mode (when a process is executed by a system call into kernel code, it is in kernel execution mode (kernel code), and at this time, it has the highest privilege level), it needs to notify the scheduler in user mode (when a process is executed by the user's own code, it is in user execution mode (i.e., user mode), and at this time, it has the lowest privilege level). For example, the scheduler in user mode can be notified in the form of a signal to receive a new task (i.e., a task sent from host 200).

[0055] For example, as shown in Figures 1A and 1B, the simulation device 100 may further include a software module 150, which includes an application program App and a virtual machine system (Guest OS). The application program App may include various tools, library files, a scheduling device, etc., and the virtual machine system may include an operating system kernel and drives.

[0056] For example, the management module 120 executes the operating system kernel, which may be, for example, the Linux® 5.2 kernel.

[0057] For example, the driver program for agent module 110 is loaded into the operating system kernel and executed; for instance, the driver program for agent module 110 is loaded into the kernel in the manner of a kernel module. A kernel module is a concept in operating systems, and driver programs are generally loaded by the operating system as kernel modules.

[0058] At least one embodiment of the present disclosure further provides a simulation system.

[0059] For example, as shown in Figure 2, the simulation system 1000 may include a simulation device 100, a host 200, and an object model 300. Regarding the communication method between the simulation device 100, the host 200, and the object model 300, refer to the description in the above-described embodiment for the simulation device 100; redundant explanations will not be repeated.

[0060] For example, the host 200 is configured to acquire tasks and send them to the simulation device 100. For example, the host 200 includes a memory manager, which communicates with the simulation device 100 to transmit tasks to the simulation device 100 and receives feedback information returned from the simulation device 100 after the object model 300 has processed the work information regarding the tasks.

[0061] For example, object model 300 is configured to process work information and obtain feedback information.

[0062] For example, in some embodiments, the object model 300 is used to simulate the functionality of a hardware accelerator (e.g., a neural network processor) and can be modeled using the SystemC language. SystemC is a modeling platform consisting of a set of C++ class libraries, to which a simulation kernel can be added to support hardware modeling at the system level, behavior description level, and register transfer level.

[0063] For example, the object model 300 can also be modeled using the Verlog language.

[0064] For example, the abstraction levels of the object model 300 are at least one of the following: algorithm level (ALM), system structure level (SAM), transaction level (TLM), and register transfer level (RTL).

[0065] For example, in some embodiments, the object model 300 may include execution units such as matrix execution units and vector execution units, memory-related units such as on-chip static random-access memory (SRAM) and SRAM controllers, and a microcontroller unit (MCU).

[0066] For example, in some embodiments, the object model 300 can model at least some of the read / save pipelines and arithmetic pipelines of a neural network processor and can statistically represent the average execution time for each operation in the neural network processor. For example, taking a CPU as an example, the arithmetic pipeline represents integer execution units, floating-point execution units, etc., and accordingly, the read / save pipeline represents I / O operations, and I / O operations represent LOAD / SAVE execution units, also commonly called LSUs.

[0067] For example, the average execution time of each operation can represent the number of execution cycles of each stage (for example, a stage represents a pipeline stage), i.e., the number of cycles required to perform each operation. For example, by statistically calculating the average execution time for each operation, it is possible to obtain a model with an accurate approximate cycle count.

[0068] To improve the execution efficiency of object model 300 as a cost model, object model 300 supports two modes: functional mode and performance mode. In performance mode, only the latency on the pipeline (for example, a pipeline represents a linear communication model of pipeline segments that exchange data) is simulated, and the actual operations corresponding to the read / storage pipeline and arithmetic pipeline are not performed. This allows for an accurate number of simulation cycles to be obtained in performance mode, and the execution speed is an order of magnitude faster than in functional mode.

[0069] In the embodiments of this disclosure, SystemC is used to model the delays in the read / storage pipeline and arithmetic pipeline of the object model 300, and the object model 300 satisfies the AI ​​compiler's demand for a high-performance cost model, thereby enabling the object model 300 to serve as the cost model for the AI ​​compiler, and as the cost model for the AI ​​compiler, it can significantly improve the inference time performance metrics of the current50 network on the chip, for example, by reducing the inference time.

[0070] At least one embodiment of the present disclosure further provides a simulation method applicable to a simulation system, the simulation system being the simulation system provided by any one embodiment of the present disclosure, for example, the simulation system 1000 shown in Figure 2.

[0071] Figure 3 is a schematic flowchart of a simulation method provided by at least one embodiment of the present disclosure.

[0072] As shown in Figure 3, this simulation method may include the following steps S10 to S13.

[0073] In step S10, the host sends the task.

[0074] In step S11, the simulation device receives and analyzes the task, and transmits the work information related to the task to the object model.

[0075] In step S12, the object model processes the work information and obtains feedback information.

[0076] In step S13, the simulation device provides feedback information to the host.

[0077] For example, taking the simulation system 1000 shown in Figure 2 as an example, step S10 is implemented by the host 200, in which step S10 the host 200 can receive a task from an external device and transmit the task to the simulation device 100. For example, in some embodiments, the task can be transmitted to the host 200 by a user via an external device.

[0078] For example, step S11 is performed by the simulation device 100, where, for example, the management module 120 in the simulation device 100 can analyze the task, and then the agent module 110 in the simulation device 100 transmits work information about the task to the object model 300.

[0079] For example, step S12 is implemented by the object model 300, in which step S12 the object model 300 can process work information to obtain feedback information, which can be transmitted to the simulation device 100. For example, the feedback information may include the results after the object model 300 has processed the work information, for example, if the task is to identify a target in an image, the feedback information may include the probability that the image contains that target.

[0080] For example, step S13 is performed by the simulation device 100, in which the simulation device 100 receives feedback information returned by the agent module 110 after the object model 300 has processed the work information.

[0081] Regarding the technical effects that the simulation method can achieve, you can refer to the relevant explanations in the above-described examples of the simulation apparatus and simulation system, and we will not repeat explanations of any overlapping parts.

[0082] At least one embodiment of the present disclosure further provides a simulation apparatus which may include one or more memories and one or more processors. The components of the above-mentioned simulation apparatus are illustrative and not limiting, and the simulation apparatus may have other components depending on the actual application needs, and the embodiments of the present disclosure are not specifically limited thereto.

[0083] For example, one or more memory locations are used to non-temporarily store computer executable instructions, and one or more processors are configured to execute the computer executable instructions. When the computer executable instructions are executed by one or more processors, they realize one or more steps in the simulation method described in any one embodiment of this disclosure. The specific implementation and related interpretations of each step of this simulation method can be found in the embodiments of the simulation method described above, and any overlapping parts are not described in detail here.

[0084] For example, the processor and memory can communicate with each other directly or indirectly.

[0085] For example, the processor and memory can communicate over a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. Communication between the processor and memory can also be achieved via a system bus, and this disclosure is not limited to this.

[0086] For example, the processor and memory can be located on the server side (or the cloud side).

[0087] For example, the processor can control other components in the simulation system to perform desired functions. The processor may be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc., and the processor may also be other forms of processing units having data processing and / or program execution capabilities, such as a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a tensor processing unit (TPU) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The central processing unit (CPU) may be an X86 or ARM architecture, etc.

[0088] For example, memory may be a computer-readable medium and may include any combination of one or more computer program products, the computer program products may include various forms of computer-readable storage media such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or high-speed cache memory (cache). Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions may be stored in the computer-readable storage medium, and the processor may execute the computer-readable instructions to realize various functions of the simulation device. The storage medium may further store various application programs and various data.

[0089] Regarding the technical effects that the simulation device can achieve, you can refer to the relevant explanations in the examples of the simulation method described above, and we will not repeat explanations of any overlapping parts.

[0090] Figure 4 is a schematic diagram of a non-temporary computer-readable storage medium provided by at least one embodiment of the present disclosure. For example, as shown in Figure 4, one or more computer-executable instructions 401 can be non-temporarily stored in the non-temporary computer-readable storage medium 40. For example, when executed by a processor, the computer-executable instructions 401 can perform one or more steps in a simulation method described in any one embodiment of the present disclosure.

[0091] For example, the non-temporary computer-readable storage medium 40 can be applied to the simulation apparatus. For example, the non-temporary computer-readable storage medium 40 may include memory in the simulation apparatus.

[0092] For example, the description of the non-temporary computer-readable storage medium 40 can be found in the description of the memory in the embodiment of the simulation device, and any overlapping parts will not be repeated.

[0093] Referring below to Figure 5, which shows a schematic diagram of the structure of an electronic device 500 suitable for realizing an embodiment of the present disclosure. The electronic device 500 may be a terminal device (e.g., a computer) or a processor, and can be used to perform the simulation method of the above embodiment. The electronic devices in the embodiment of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable media players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and wearable electronic devices, as well as fixed terminals such as digital TVs, desktop computers, and smart home devices. The electronic devices shown in Figure 5 are merely examples and do not limit the functions and scope of use of the embodiment of the present disclosure.

[0094] As shown in Figure 5, the electronic device 500 may include a processing unit (e.g., a central processing unit, an image processing unit, etc.) 501 which can perform various appropriate operations and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage device 508 into random access memory (RAM) 503. RAM 503 also stores various programs and data necessary to operate the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0095] Typically, devices such as input devices 506 including touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, and gyroscopes; output devices 507 including liquid crystal displays (LCDs), speakers, and vibrators; storage devices 508 including magnetic tapes and hard disks; and communication devices 509 can be connected to the I / O interface 505. The communication device 509 can enable the electronic device 500 to exchange data with other devices via wireless or wired communication. Figure 5 shows an electronic device 500 with various devices, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have been included instead.

[0096] In particular, according to embodiments of the present disclosure, the process described with reference to the flowchart above may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product which includes a computer program carried on a non-temporary computer-readable medium which includes program code for performing the method shown in the flowchart to perform one or more steps in the simulation method described above. In such embodiments, the computer program may be downloaded and installed from a network by a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing unit 501, the processing unit 501 can be made to perform the functions limited to the simulation method of embodiments of the present disclosure.

[0097] In the context of this disclosure, a computer-readable medium may be a tangible medium that contains or stores a program used by or in combination with an instruction execution system, apparatus, or device. A computer-readable medium may be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of more than these. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in combination with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signaling medium may include data signals propagated in the baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms and include, but are not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signaling medium may be any computer-readable medium other than a computer-readable storage medium, and such computer-readable signaling medium may transmit, propagate, or transmit programs used by or in combination with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium may be transmitted on any suitable medium, which includes, but is not limited to, wires, optical cables, RF (radio frequency), or any suitable combination thereof.

[0098] The computer-readable medium described above may be included in the electronic device described above, or it may exist independently without being incorporated into the electronic device.

[0099] Computer program code for performing the operations of the Disclosure can be organized in one or more programming languages ​​or a combination thereof, and such programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, and also include ordinary procedural programming languages ​​such as the C language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as an independent package, partially on the user's computer and partially on a remote computer, or fully on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer by any type of network (including a local area network (LAN) or a wide area network (WAN)), or it may be connected to an external computer (for example, connected via the Internet using an Internet service provider).

[0100] The flowcharts and block diagrams in the drawings illustrate the implementable architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of code containing one or more executable instructions for implementing a given logical function. In some alternative implementations, the functions attached to the blocks may be implemented in an order different from the order in which they are attached in the drawings. For example, two consecutively shown blocks may actually be executed essentially in parallel, or in reverse order depending on the function. Each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs a given function or operation, or by a combination of dedicated hardware and computer instructions.

[0101] The units described in the embodiments of this disclosure may be implemented in software form or in hardware form. For example, the names of the units are not necessarily limited to the units themselves.

[0102] In this specification, the functions described above can be performed by at least partially one or more hardware logic components. For example, non-limitingly, typical types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), and coupled-programmable logic circuits (CPLDs).

[0103] In a first embodiment, according to one or more embodiments of the present disclosure, a simulation apparatus for simulating a neural network processor includes: an agent module configured to communicate with an object model that simulates the neural network processor and to be an agent of the object model in the simulation apparatus; a management module configured to manage the simulation apparatus; and an interconnection module configured to communicate between the agent module and the management module, wherein the simulation apparatus receives a task transmitted from a host, the agent module transmits work information relating to the task to the object model, the agent module receives feedback information returned by the object model after processing the work information, and provides the feedback information to the host.

[0104] According to one or more embodiments of the present disclosure, the agent module and the host communicate using a socket, shared storage, and / or message queue method.

[0105] According to one or more embodiments of the present disclosure, the address space of the agent module includes an agent register space and a model space, the agent register space being used to define the registers of the neural network processor, and the registers of the neural network processor being located by memory addresses, and the model space being used to store the parameters of the neural network processor and the inputs and / or outputs relating to the task.

[0106] According to one or more embodiments of the present disclosure, the model space is mapped as a storage space in the object model.

[0107] According to one or more embodiments of the present disclosure, the model space is shared by the agent module and the host.

[0108] According to one or more embodiments of the present disclosure, the address space of the agent module further includes a configuration instruction space, which is used to store configuration instructions for the registers of the neural network processor and control instructions relating to the task.

[0109] According to one or more embodiments of the present disclosure, the agent module includes a plurality of base address register spaces, the plurality of base address register spaces include a first base address register space and a second base address register space, the agent register space corresponds to the first base address register space, and the configuration instruction space corresponds to the second base address register space.

[0110] According to one or more embodiments of the present disclosure, the configuration instruction space is mapped to a shared storage file shared by the agent module and the host, and the shared storage file is located on the host.

[0111] According to one or more embodiments of the present disclosure, the agent module is configured to provide the agent register space and the content in the model space to the object model based on the content in the configuration instruction space, to configure and schedule the object model, and to perform operation simulation.

[0112] According to one or more embodiments of the present disclosure, communication between the agent module and the object model is conducted using a message queue method.

[0113] According to one or more embodiments of the present disclosure, the management module runs an operating system kernel, and the agent module's driving program is loaded into the operating system kernel.

[0114] According to one or more embodiments of the present disclosure, the agent module includes an interrupt register, and the host writes notification information relating to the task to the interrupt register and notifies the management module in an interrupt manner to perform the task.

[0115] According to one or more embodiments of the present disclosure, the simulation apparatus further includes an input / output module, the input / output module and the host communicate via sockets to receive the task, and the input / output module is configured to transmit the task to the agent module or the management module.

[0116] According to one or more embodiments of the present disclosure, the simulation device is implemented by a virtual simulation platform, and the management module is implemented by a virtual central processing unit.

[0117] In a second embodiment, according to one or more embodiments of the present disclosure, a simulation system comprising a simulation apparatus described in any one embodiment of the present disclosure, the object model, and the host, wherein the host is configured to acquire the task and transmit the task to the simulation apparatus, and the object model is configured to process the work information and obtain the feedback information.

[0118] According to one or more embodiments of the present disclosure, the level of abstraction of the object model is the algorithm level, the system structure level, the transaction level, or the register transfer level.

[0119] According to one or more embodiments of the present disclosure, the object model models at least a portion of the read / storage pipelines and arithmetic pipelines of the neural network processor and statistically calculates the average execution time for each operation in the neural network processor.

[0120] In a third embodiment, according to one or more embodiments of the present disclosure, a simulation method applicable to a simulation system described in any one embodiment of the present disclosure, comprising the steps of: transmitting a task by a host; receiving and analyzing the task by a simulation device and transmitting work information relating to the task to an object model; processing the work information by the object model and obtaining feedback information; and providing the feedback information to the host by the simulation device.

[0121] In a fourth embodiment, according to one or more embodiments of the present disclosure, computer executable instructions are stored that, when executed by a processor, realize a simulation method described in any one embodiment of the present disclosure.

[0122] The above description is merely an explanation of preferred embodiments and the technical principles used in the present disclosure. As will be understood by those skilled in the art, the scope of the disclosure is not limited to technical solutions consisting of specific combinations of the above technical features, but should also encompass other technical solutions formed by any combination of the above technical features or their equivalents, without departing from the concept of the above disclosure. For example, it should encompass technical solutions formed by substituting the above features with (but not limited to) similar functional technical features disclosed in the present disclosure.

[0123] Furthermore, although the operations are described in a specific order, this should not be understood as requiring that these operations be performed in a specific or sequential order as shown. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although the above considerations include several specific implementation details, these should not be construed as limitations on the scope of this disclosure. Some features described in the context of individual embodiments may be further combined and implemented in a single embodiment. Conversely, various features described in the context of a single embodiment may be implemented in multiple embodiments individually or in any suitable subcombination.

[0124] Although this subject matter has already been described using language that specifies structural features and / or methodological logic, it should be understood that the subject matter limited in the attached claims is not limited to the specific features or operations described above. On the contrary, the specific features and operations described above are merely illustrative forms that realize the claims.

[0125] The following points need to be further explained regarding this disclosure.

[0126] (1) The drawings of the embodiments of this disclosure relate only to the structures relating to the embodiments of this disclosure, and other structures may refer to conventional designs.

[0127] (2) To the extent that they do not contradict each other, the embodiments and features of the embodiments of this disclosure can be combined with each other to obtain new embodiments.

[0128] The above description is merely a means of carrying out the invention of this disclosure, but the scope of protection of this disclosure is not limited thereto, and the scope of protection of this disclosure is based on the scope of protection of the claims below.

Claims

1. A simulation device for simulating a neural network processor, An agent module that communicates with an object model that simulates the neural network processor and is configured to act as an agent for the object model in the simulation device, A management module configured to manage the aforementioned simulation device, The interconnection module is configured to communicate between the agent module and the management module, The simulation device receives a task transmitted from the host, transmits work information related to the task to the object model via the agent module, receives feedback information returned by the object model after processing the work information via the agent module, and provides the feedback information to the host. The address space of the agent module includes the agent register space and the model space. The agent register space is used to define the registers of the neural network processor, and the registers of the neural network processor are located by memory addresses. The model space is a simulation device used to store the parameters of the neural network processor and the inputs and / or outputs related to the task.

2. The simulation apparatus according to claim 1, wherein communication between the agent module and the host is performed using a socket method, a shared storage method and / or a message queue method.

3. The simulation apparatus according to claim 1, wherein the model space is mapped as a memory space in the object model.

4. The simulation apparatus according to claim 1, wherein the model space is shared by the agent module and the host.

5. The address space of the agent module further includes a configuration instruction space, The simulation apparatus according to claim 1, wherein the setting instruction space is used to store setting instructions for the registers of the neural network processor and control instructions relating to the task.

6. The agent module includes a plurality of base address register spaces, the plurality of base address register spaces including a first base address register space and a second base address register space, The simulation apparatus according to claim 5, wherein the agent register space corresponds to the first base address register space, and the setting instruction space corresponds to the second base address register space.

7. The simulation apparatus according to claim 5, wherein the configuration command space is mapped to a shared storage file shared by the agent module and the host, and the shared storage file is located on the host.

8. The simulation apparatus according to claim 6, wherein the agent module is configured to provide the agent register space and the content in the model space to the object model based on the content in the setting instruction space, to set up the object model, schedule it, and perform operation simulation.

9. The simulation apparatus according to claim 1 or 2, wherein communication between the agent module and the object model is conducted using a message queue method.

10. The simulation apparatus according to claim 1 or 2, wherein the management module executes an operating system kernel, and the agent module's drive program is loaded into the operating system kernel.

11. The simulation apparatus according to claim 1 or 2, wherein the agent module includes an interrupt register, and the host writes notification information relating to the task to the interrupt register and notifies the management module in an interrupt manner to execute the task.

12. It further includes an input / output module, The input / output module and the host communicate using a socket method to receive the task. The simulation apparatus according to claim 1 or 2, wherein the input / output module is configured to transmit the task to the agent module or the management module.

13. The simulation apparatus according to claim 1 or 2, wherein the simulation apparatus is implemented by a virtual simulation platform, and the management module is implemented by a virtual central processing unit.

14. A simulation system comprising the simulation apparatus described in claim 1, the object model, and the host, The host is configured to acquire the task and transmit the task to the simulation device. The object model is a simulation system configured to process the work information and obtain the feedback information.

15. The simulation system according to claim 14, wherein the level of abstraction of the object model is the algorithm level, the system structure level, the transaction level, or the register transfer level.

16. The simulation system according to claim 14 or 15, wherein the object model models at least a portion of the read / storage pipeline and arithmetic pipeline of the neural network processor, and statistically calculates the average execution time for each operation in the neural network processor.

17. A simulation method applicable to the simulation system described in claim 14, The steps include sending a task by the aforementioned host, The simulation device receives and analyzes the task, and transmits work information related to the task to the object model. The steps include processing the work information using the object model and obtaining the feedback information, The steps include providing the feedback information to the host using the simulation device, A simulation method that includes this.

18. A non-temporary computer-readable storage medium storing computer-executable instructions that, when executed by a processor, realize the simulation method described in claim 17.

Images