Artificial intelligence chip pre-silicon verification system and method

Abstract

The invention provides an artificial intelligence chip pre-silicon verification system and method, and relates to the technical field of artificial intelligence chips, and the system comprises a model importing layer which is used for importing a current model; the model analysis layer is used for analyzing the current model to obtain a calculation graph; the distributed strategy configuration layer is used for configuring a distributed topology strategy of a hardware simulator of the artificial intelligence chip based on the distributed strategy of the current model, and adjusting a computational graph of the current model to enable the computational graph to be matched with the distributed topology strategy; the performance prediction layer is used for determining the operation performance data of the current model in the artificial intelligence chip based on the operation performance data of each operator; or determining a performance prediction result of the current model in the artificial intelligence chip based on the performance prediction result of each operator; and the operator implementation layer is used for controlling the hardware simulator to run each operator in the computational graph. According to the system and the method provided by the invention, the end-to-end whole-process verification from model input to result output is carried out on the artificial intelligence chip.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence chip, and particularly relates to an artificial intelligence chip silicon pre-verification system and method. BACKGROUND

[0002] Chip silicon pre-verification refers to that before a chip is taped out, a register transfer level (RTL) code and a field-programmable gate array (FPGA) prototype are comprehensively detected in function, timing and power consumption through simulation and automatic tools to ensure that the design meets the specifications. The chip silicon pre-verification is mainly based on a function simulator and various simulation circuit simulators to verify the behavior of the designed chip.

[0003] The chip silicon pre-verification in the related art mainly focuses on basic functions and operator levels. For example, precision verification can only be considered from the level of instructions and operators, and performance verification can only be analyzed one operator at a time. When this chip silicon pre-verification method is applied to an artificial intelligence chip, it is difficult to verify the precision and analyze the performance from the perspective of the whole model, the verification efficiency is low, and model-level problems may be hidden.

[0004] Therefore, how to improve the efficiency of artificial intelligence chip silicon pre-verification has become a technical problem to be solved in the industry. SUMMARY

[0005] The present application provides an artificial intelligence chip silicon pre-verification system and method, which is used to solve the technical problem of how to improve the efficiency of artificial intelligence chip silicon pre-verification.

[0006] The present application provides an artificial intelligence chip silicon pre-verification system, which comprises a model import layer, a model analysis layer, a distributed strategy configuration layer, a performance prediction layer and an operator implementation layer. The model import layer is used to import a current model. The model analysis layer is used to analyze the current model to obtain a computation graph of the current model. The distributed strategy configuration layer is used to configure a distributed topology strategy of a hardware simulator of an artificial intelligence chip based on a distributed strategy of the current model, and adjust the computation graph of the current model to adapt the computation graph to the distributed topology strategy. The hardware simulator is determined based on a design file of the artificial intelligence chip. The performance prediction layer is used to determine running performance data of the current model in the artificial intelligence chip based on running performance data of each operator in the hardware simulator, or determine a performance prediction result of the current model in the artificial intelligence chip based on a performance prediction result of each operator. The operator implementation layer is configured to control the hardware emulators to run the operators in the computation graph.

[0007] In some embodiments, the model importing layer is configured to: obtain model structure parameters, input configuration parameters, and model type parameters of the current model; import the current model from a model library based on the model structure parameters, the input configuration parameters, and the model type parameters; the model library registers a plurality of models.

[0008] In some embodiments, the model parsing layer is configured to: parse the current model to determine the operators in the current model; determine a computation graph of the current model based on the operators in the current model and data flow directions between the operators.

[0009] In some embodiments, the distributed strategy configuration layer is configured to: determine a distributed topology strategy based on a distributed strategy of the current model; the distributed topology strategy includes a plurality of hardware emulators of artificial intelligence chips; determine a collective communication operator to be inserted based on the operators run by the hardware emulators and connection relationships between the hardware emulators; insert the collective communication operator into the computation graph of the current model and adjust the computation graph of the current model.

[0010] In some embodiments, the performance prediction layer is configured to: obtain a performance counter file output by the hardware emulators; the performance counter file is configured to record running performance data of the operators in the hardware emulators; determine the running performance data of the operators based on the performance counter file; determine running performance data of the current model on the artificial intelligence chips based on the running performance data of the operators.

[0011] In some embodiments, the operator implementation layer is configured to: control the hardware emulators to run the operators in the computation graph to obtain precision verification results and performance verification results of the operators in the corresponding hardware emulators.

[0012] In some embodiments, the model type of the current model includes at least one of a computer vision model and a large language model.

[0013] The present application provides an artificial intelligence chip pre-silicon verification method, applied to an artificial intelligence chip pre-silicon verification system, comprising: analyzing the current model to obtain a computation graph of the current model; based on a distributed strategy of the current model, configuring a distributed topology strategy of a hardware simulator of the artificial intelligence chip, and adjusting the computation graph of the current model to adapt the computation graph to the distributed topology strategy; the hardware simulator is determined based on a design file of the artificial intelligence chip; controlling the hardware simulator to run each operator in the computation graph to obtain running performance data of each operator in the hardware simulator; based on the running performance data of each operator in the hardware simulator, determining running performance data of the current model on the artificial intelligence chip.

[0014] In some embodiments, after the parallel distributed strategy of the current model is used to configure a parallel computing architecture distributed topology strategy of a hardware simulator of the artificial intelligence chip and the computation graph of the current model is adjusted, the method further comprises: based on the performance prediction result of each operator, determining a performance prediction result of the current model on the artificial intelligence chip.

[0015] The present application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the artificial intelligence chip pre-silicon verification method when executing the computer program.

[0016] The artificial intelligence chip pre-silicon verification system and method provided by the present application comprises a model import layer, a model analysis layer, a distributed strategy configuration layer, a performance prediction layer, and an operator implementation layer. The model import layer is used to import a current model. The model analysis layer is used to analyze the current model to obtain a computation graph of the current model. The distributed strategy configuration layer is used to configure a distributed topology strategy of a hardware simulator of the artificial intelligence chip based on a distributed strategy of the current model, and adjust the computation graph of the current model to adapt the computation graph to the distributed topology strategy. The performance prediction layer is used to determine running performance data of the current model on the artificial intelligence chip based on running performance data of each operator in the hardware simulator, or determine a performance prediction result of the current model on the artificial intelligence chip based on a performance prediction result of each operator. The operator implementation layer is used to control the hardware simulator to run each operator in the computation graph. Through the hierarchical design, not only is the artificial intelligence chip pre-silicon verification performed from the operator level, but also the artificial intelligence chip pre-silicon verification is performed from the model as a whole. The end-to-end full-process verification of the artificial intelligence chip from the model input to the output result is realized, and the efficiency of the artificial intelligence chip pre-silicon verification is improved. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0018] In order to make the technical solutions of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0019] Figure 1 is one of the structural schematic diagram of the artificial intelligence chip silicon pre-verification system provided by the present application.

[0020] Figure 2 is a schematic diagram of distributed computing provided by the present application.

[0021] Figure 3 is the second structural schematic diagram of the artificial intelligence chip silicon pre-verification system provided by the present application.

[0022] Figure 4 is one of the flow schematic diagram of the artificial intelligence chip silicon pre-verification method provided by the present application.

[0023] Figure 5 is the second flow schematic diagram of the artificial intelligence chip silicon pre-verification method provided by the present application.

[0024] Figure 6 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0025] In order to make the technical solutions of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on these drawings.

[0026] It should be noted that the terms "first", "second", and the like in the present application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units or modules does not have to be limited to only those steps or units or modules clearly listed, but can include other steps or units or modules not clearly listed or inherent to these processes, methods, products or devices.

[0027] The silicon pre-chip verification in the related art includes precision verification and performance verification. For precision verification, it is usually in units of instructions and operators. At the instruction verification level, it mainly refers to verifying whether each hardware instruction can return the correct bit value (bit) for various inputs, and at the operator level, it mainly sets some acceptable precision thresholds for some commonly used operators, such as matrix multiply-accumulate operator (MMA), convolution operator (Conv), etc., to verify whether the behavior of the operator meets the expectation.

[0028] For performance verification, it is usually in units of operators. For some operators with performance requirements, such as matrix multiply-accumulate operators, etc., it can be verified whether the ideal performance can be achieved.

[0029] The current chip silicon pre-verification mainly focuses on the basic functions and operator level, and it is difficult to analyze the overall model precision and performance in various different scenarios. In particular, for artificial intelligence chips, due to the high cost and long cycle of tape-out, it is urgent to realize the full flow verification from model input to output result of the artificial intelligence chip, and to improve the efficiency of the artificial intelligence chip silicon pre-verification. The core goal is to ensure that the chip can correctly, stably and efficiently run the target model in the real application scenario, and meet the performance, power consumption, precision and other index requirements.

[0030] In order to solve the deficiencies of the related art, Figure 1 is one of the structural diagrams of the artificial intelligence chip silicon pre-verification system provided by the present application, as Figure 1 shown, the artificial intelligence chip silicon pre-verification system 100 includes a model import layer 110, a model analysis layer 120, a distributed strategy configuration layer 130, a performance prediction layer 140 and an operator implementation layer 150.

[0031] The model import layer is used to import the current model.

[0032] The model analysis layer is configured to analyze the current model to obtain a computation graph of the current model.

[0033] The distributed strategy configuration layer is configured to configure a distributed topology strategy of the hardware simulator of the artificial intelligence chip based on a distributed strategy of the current model, and adjust the computation graph of the current model to adapt the computation graph to the distributed topology strategy. The hardware simulator is determined based on a design file of the artificial intelligence chip.

[0034] The performance prediction layer is configured to determine running performance data of the current model on the artificial intelligence chip based on running performance data of each operator in the hardware simulator, or determine a performance prediction result of the current model on the artificial intelligence chip based on a performance prediction result of each operator.

[0035] The operator implementation layer is configured to control the hardware simulator to run each operator in the computation graph.

[0036] Specifically, the artificial intelligence chip silicon pre-verification system provided by the embodiments of the present application is suitable for silicon pre-verification of various artificial intelligence chips. The artificial intelligence chip can be a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a domain specific architecture (DSA), etc.

[0037] In terms of structure, the artificial intelligence chip silicon pre-verification system can include a model import layer, a model analysis layer, a distributed strategy configuration layer, a performance prediction layer, and an operator implementation layer connected in sequence.

[0038] The model import layer is mainly configured to import a current model. The current model is a model that needs to be used in silicon pre-verification of the artificial intelligence chip. The current model can include a computer vision model and a large language model, etc. The computer vision model (CVM) is a deep learning model used to process and understand image or video data. The large language model (LLM) is a deep learning model trained based on large-scale text data, which can understand and generate natural language.

[0039] Model analysis layer, responsible for converting the imported current model into a structured computational graph representation, so as to perform subsequent performance analysis, hardware mapping, simulation verification, etc. The model analysis layer analyzes the model structure of the current model, including analyzing the model layers, operators, parameters (weights, biases), input and output tensor shapes, etc., identifying the data flow path and computation dependency in the model, and then structuring the data flow path and computation dependency in the form of a graph to obtain a computational graph.

[0040] In the pre-silicon verification of artificial intelligence chips, the model is large in size and computationally intensive, and a single device (artificial intelligence chip) cannot bear it, so a distributed strategy is needed to distribute the model to multiple devices for running. The distributed strategy refers to a method or scheme for dividing and scheduling computing tasks in distributed computing or distributed computing in order to efficiently utilize multiple computing resources (such as multiple artificial intelligence chips). Common distributed strategies can include data parallelism (Data Parallelism, DP), model parallelism (Model Parallelism, MP), pipeline parallelism (Pipeline Parallelism, PP), and tensor parallelism (Tensor Parallelism, TP), etc.

[0041] Since it is pre-silicon verification, a hardware simulator is used. The hardware simulator is determined according to the design file of the artificial intelligence chip. The design file usually includes RTL code, etc. The hardware simulator will compile these design files into a model that can run on the simulator hardware, which can simulate the behavior of the chip, including executing instructions, processing data, and communicating with external interfaces, etc. In addition to the hardware simulator, there is a functional verification model, which is a software that simulates the instruction sequence behavior of hardware on a central processing unit (Central Processing Unit, CPU), and is divided into A model (architecture model) and C model (C language implementation of the functional model), etc.

[0042] The distributed topology strategy refers to the hardware and organizational structure designed to achieve distributed computing, which allows multiple computing units (such as artificial intelligence chips) to work simultaneously to improve computing efficiency and processing capacity.

[0043] The distributed strategy configuration layer configures the distributed topology strategy of the hardware simulator of the artificial intelligence chip according to the distributed strategy of the current model, so that it supports the corresponding multiple artificial intelligence chips for distributed computing. At the same time, the distributed strategy configuration layer adjusts the original single-device computational graph into a distributed computational graph suitable for multi-device distributed execution, including inserting communication nodes, dividing computing tasks, mapping to different devices, etc. The adjusted computational graph is adapted to the distributed topology strategy.

[0044] The performance prediction layer can realize two functions, one is to obtain the running performance data output by the hardware simulator, and the running performance data of the current model on the artificial intelligence chip can be analyzed according to the running performance data. The second is to calculate the performance of each operator to obtain the performance prediction result of each operator, and the performance prediction result of the current model on the artificial intelligence chip is obtained after being summarized. For example, the performance data of each operator can be combined according to the structure of the calculation graph to construct the performance prediction result of the entire model.

[0045] The operator implementation layer is used to call each operator, control the hardware simulator to run each operator in the calculation graph, register the forward cost function, the backward cost function and the like for each operator, respectively in the forward calculation and the backward calculation, and count the running cost of each operator (cost), record the running result (output tensor) and the performance data (such as execution time, resource occupation), compare the output of the hardware simulator with the expected result, and obtain the running performance data of each operator on the artificial intelligence chip, including the accuracy verification result and the performance verification result.

[0046] The model import layer is used to import the current model; the model analysis layer is used to analyze the current model to obtain the calculation graph of the current model; the distributed strategy configuration layer is used to configure the distributed topology strategy of the hardware simulator of the artificial intelligence chip based on the distributed strategy of the current model, and adjust the calculation graph of the current model to adapt the calculation graph to the distributed topology strategy; the performance prediction layer is used to determine the running performance data of the current model on the artificial intelligence chip based on the running performance data of each operator in the hardware simulator; or determine the performance prediction result of the current model on the artificial intelligence chip based on the performance prediction result of each operator; the operator implementation layer is used to control the hardware simulator to run each operator in the calculation graph; through the hierarchical design, not only the silicon pre-verification of the artificial intelligence chip is realized from the operator level (including the performance prediction result and the running performance data), but also the silicon pre-verification of the artificial intelligence chip is realized from the model as a whole, the end-to-end full-process verification of the artificial intelligence chip from the model input to the output result is realized, and the efficiency of the silicon pre-verification of the artificial intelligence chip is improved.

[0047] In some embodiments, the model import layer is used to: obtain model structure parameters, input configuration parameters and model type parameters of the current model; based on the model structure parameters, the input configuration parameters and the model type parameters, import the current model from a model library; the model library registers a plurality of models.

[0048] Specifically, the model library is a database or a collection of files that stores and manages multiple models, each of which has corresponding structure, input configuration, and type information. The information of each model (such as name, structure, input configuration, type, etc.) can be registered in the model library so that the system can identify and call it.

[0049] The model structure parameters refer to the network structure information of the model, such as the number of layers, the type of each layer (such as convolutional layer, fully connected layer), activation function, connection mode, etc. The input configuration parameters refer to the format and attributes of the model input data, such as the size, channel number, data type, preprocessing method, etc. of the input image. The model type parameters refer to the type of the model, including computer vision models and large language models, etc.

[0050] The model structure parameters, input configuration parameters, and model type parameters of the current model can be input in the model import layer, and the model import layer can match and import the current model in the model library according to the model structure parameters, input configuration parameters, and model type parameters.

[0051] The artificial intelligence chip pre-silicon verification system provided by the embodiments of the present application imports the current model through the model import layer, and provides a basis for subsequent model analysis and pre-silicon verification.

[0052] In some embodiments, the model analysis layer is configured to: analyze the current model to determine the operators in the current model; determine the computation graph of the current model based on the operators in the current model and the data flow between the operators.

[0053] Specifically, the operator is a basic computing unit in the model, such as a convolution operator, a matrix multiplication operator, an activation function operator, a normalization processing operator, etc.

[0054] The model analysis layer analyzes the current model layer by layer, extracts the type, parameters, input and output shapes, etc. of each layer, and identifies all operators used in the model. The node represents the operator or variable (such as input, weight), and the edge represents the data flow between the operators, and the computation graph representing the calculation process of the current model is obtained. The data flow between the operators can represent the data dependency relationship between the operators.

[0055] For example, the model analysis layer can analyze the attributes of the module (module) in torch (the core module of PyTorch framework) and the attributes of each node (node) therein, and then recombine these attributes into a graph structure, thereby constructing a computation graph.

[0056] The artificial intelligence chip silicon pre-validation system provided by the embodiment of the application extracts all operators and data dependency relationships thereof through a model analysis layer, constructs a calculation graph of the model, and provides a basis for subsequent model analysis and silicon pre-validation.

[0057] In some embodiments, the distributed policy configuration layer is configured to: determine a distributed topology policy based on the distributed policy of the current model; the distributed topology policy comprises hardware emulators of multiple artificial intelligence chips; determine a set communication operator to be inserted based on the operators running on each hardware emulator and the connection relationship between the hardware emulators; insert the set communication operator into the calculation graph of the current model, and adjust the calculation graph of the current model.

[0058] Specifically, the distributed policy configuration layer is responsible for converting a single-machine single-card model into a multi-machine multi-card model according to a distributed configuration. The distributed policy configuration layer can be configured manually or can perform a certain degree of optimal configuration search under a given resource.

[0059] The distributed policy is used to determine how to configure a single device to run the current model into multiple devices to run the current model. According to the distributed policy, a distributed topology policy can be determined. The distributed topology policy is composed of hardware emulators of multiple artificial intelligence chips.

[0060] Figure 2 is a schematic diagram of distributed computing provided by the application, as Figure 2 shown, the distributed policy is tensor distributed. Therefore, a linear transformation operator (linear) executed by a single device can be configured as a linear transformation operator executed by four devices. The input tensor is cut into four parts along a certain dimension, and each part is calculated on one device. Therefore, the distributed topology policy can configure hardware emulators of four artificial intelligence chips.

[0061] After determining the hardware simulator corresponding to each operator, the collective communication operator that needs to be inserted in the computation graph is determined according to the operators running in each hardware simulator and the connection relationship between each hardware simulator. The collective communication operator is a set of primitive operations used for collaborative communication and data synchronization between multiple devices in distributed computing. They are the basis for implementing data parallelism, model parallelism, pipeline parallelism and other distributed strategies, ensuring that the computing results on different devices can be efficiently and correctly aggregated or distributed. The collective communication operator includes a reduce operation operator (Reduce) and a gather operation operator (Gather). The reduce operation operator is to aggregate the data of multiple devices into one result (such as summation), and the gather operation operator is to splice the data of multiple devices into a larger result. They are used for gradient synchronization, result aggregation and other key operations in distributed training, ensuring the correctness and consistency of distributed computing.

[0062] The collective communication operator is inserted into the computation graph of the current model to adjust the computation graph of the current model. The adjusted computation graph is adapted to the distributed topology strategy.

[0063] The artificial intelligence chip silicon pre-verification system provided by the embodiment of the application determines the distributed computing architecture and adjusts the computation graph through the distributed strategy configuration layer, realizes accurate matching of the current model and the distributed architecture of the artificial intelligence chip, and lays a foundation for subsequent performance prediction and verification.

[0064] In some embodiments, the performance prediction layer is configured to: obtain a performance counter file output by the hardware simulator; the performance counter file is used to record the running performance data of each operator in the hardware simulator; determine the running performance data of each operator based on the performance counter file; determine the running performance data of the current model on the artificial intelligence chip based on the running performance data of each operator.

[0065] Specifically, the performance counter file (PFC) is a data file generated by the hardware simulator when running the model, which is used to record the running performance data of each operator in the hardware simulator, including execution time, throughput, resource utilization (such as computing unit, memory bandwidth), call frequency and other hardware events (such as cache hit rate, power consumption, etc.).

[0066] The performance prediction layer obtains the running performance data of each operator by reading the files, integrates the performance data of each operator according to the structure of the calculation graph, and predicts the running performance data of the entire model on the chip through performance modeling (such as accumulation, simulation, machine learning model, etc.), and outputs the performance indicators of the model level, such as end-to-end delay, throughput, resource utilization, power consumption and performance bottleneck analysis, to obtain the running performance data of the current model on the artificial intelligence chip.

[0067] The artificial intelligence chip silicon pre-verification system provided by the embodiment of the application, the performance prediction layer obtains the running performance data of each operator by reading the performance counter file generated by the hardware simulator, and predicts the running performance data of the entire model on the artificial intelligence chip according to the running performance data, thereby providing a basis for chip design optimization and model deployment.

[0068] In some embodiments, the operator implementation layer is configured to: Control the hardware simulator to run each operator in the calculation graph to obtain the accuracy verification result and the performance verification result of each operator in the corresponding hardware simulator.

[0069] Specifically, the operator implementation layer controls the hardware simulator to run each operator in the calculation graph, and simulates the overhead of each operator, including: (1) the overhead of loading data; (2) in forward calculation, each operator needs to register a forward cost function (costfunc), which can be implemented through different tools (provider); (3) in backward calculation, each operator needs to register a backward cost function (costfunc), which can be implemented through different tools (provider); (4) the overhead of each operator in the parameter update of the optimizer (Optimizer); (5) the overhead of the host, such as operator scheduling, memory allocation, copying the result back to the processor, etc.

[0070] The performance calculation method for one of the operators includes: (1) perfCalculaor (operator theoretical performance calculation tool), which is mainly used in matrix multiplication and convolution operators and attention operators, etc. Tensor core (tcore) operators, since the actual utilization rate of the tensor core is affected by many factors and is relatively complex, a separate tool is developed and implemented independently. The main logic is: calculate the possible strategy block situation of a tensor core operator; select the best strategy to calculate the actual operation and memory access; infer the theoretical performance according to the actual parameters of the hardware; (2) PFC (hardware information counter), which can compare the actual scene and the theoretical calculation from the perspective of hardware. This tool must be used under the hardware simulator, and the performance obtained is basically close to the performance of the actual chip. (3) Customized basic performance calculation method, mainly processing some vector core (vcore) or communication type calculation pure operator.

[0071] The operator implementation layer submits the calculation graph parsed by the model to the hardware emulator or Cmodel one by one according to nodes (operators), each operator is executed independently or in a dependent order, and the behavior of the operator on a real chip is simulated. The hardware emulator output is compared with the reference, the error index is counted, it is judged whether it is within the acceptable threshold, the precision verification result is obtained. The execution time, throughput, resource utilization, bandwidth occupation, etc. are recorded to generate performance counter data, and the performance verification result is obtained.

[0072] The artificial intelligence chip pre-silicon verification system provided by the embodiment of the application runs the operators in the calculation graph one by one on the hardware emulator, obtains the precision and performance data of each operator, and provides a basis for chip design correctness and performance evaluation.

[0073] Figure 3 is a structure diagram of the artificial intelligence chip pre-silicon verification system provided by the application, as shown in Figure 3 The system includes 5 layers of L0-L4. L0 is a model import layer, L1 is a model analysis layer, L2 is a distributed strategy configuration layer, L3 is a performance prediction layer, and L4 is an operator implementation layer.

[0074] For steps that need to perform operator calculation, the system can be configured to select execution on a central processing unit (CPU) or a graphics processing unit (GPU). According to the actual environment, it can be executed in a software model (Cmodel) written in C language or a hardware emulator (Emulator, EMU), or it can not be operated, but only performance prediction can be performed.

[0075] L0 layer is responsible for importing the current model. The current model is imported into the model library through registration. Any PyTorchModel or Graph JSON can be used. A new model only needs to add a small amount of code.

[0076] L1 layer is responsible for analyzing the current model to obtain the calculation graph of the current model. The obtained calculation graph is a single device computation graph.

[0077] L2 layer is responsible for distributed configuration. A distributed topology strategy of multiple devices is obtained according to the distributed strategy configuration, and the single device computation graph is adjusted to a multi-device computation graph.

[0078] The L3 layer is responsible for performing theoretical performance calculations of various types of operators, including Operator Cost, Host Cost, Optimzer Cost, and Other Cost. The L3 layer can also aggregate the running performance data of each operator output by the hardware simulator in the L4 layer to obtain the running performance data of the model.

[0079] The L4 layer is responsible for running each operator on the hardware simulator to obtain the results, including using PerfCalulator (Operator Theoretical Performance Calculator), PFC (Performance Counter File), and Other tools.

[0080] The artificial intelligence chip silicon pre-verification system in the above embodiments can perform hardware platform simulation and theoretical performance analysis, etc.

[0081] I. Applied to hardware platform simulation, the function and performance of the model can be verified in detail.

[0082] For example, the above system is used to compare the inference accuracy and performance of the EfficientNet (Efficient Neural Network) model, the process includes: (1) Configure the model structure and input of EfficientNet at the L0 layer. The model can be directly imported through third-party systems such as Transformers, or the model can be constructed using configuration and modified parameters.

[0083] (2) The L1 layer converts the EfficientNet model into a calculation graph of each operator.

[0084] (3) The L2 layer modifies the calculation graph to a calculation graph under the condition of multiple machines and multiple cards based on a distributed strategy. For the multi-card configuration of EfficientNet, the data parallelism in the inference stage is not required, so no additional operators are inserted, and the overall calculation graph remains unchanged.

[0085] (4) The L3 layer calls each operator in the system to run the model completely on the EMU server (or Cmodel running in CPU), obtain the output, and compare the accuracy.

[0086] (5) The L4 layer captures the PFC file of the output on the EMU server, and obtains the actual running operators and performance of the model by reading the PFC file.

[0087] II. Applied to theoretical performance analysis, the performance of a model can be quickly estimated.

[0088] For example, the above system is used to predict the theoretical performance of the llama3 model, the process includes: (1) In the L0 layer, the model structure and input of llama3 are configured.

[0089] (2) The L1 layer converts the llama model into a calculation graph of individual operators.

[0090] (3) The L2 layer modifies the calculation graph into a calculation graph in the case of multiple machines and multiple cards based on the configured distributed strategy. The configuration of llama is a mixture of data parallelism and tensor parallelism, so some Reduce operators and Gather operators need to be inserted.

[0091] (4) The L3 layer calls each operator in the framework, predicts the performance of each operator in the model running on the CPU, and adds necessary additional consumption such as loading input, launch overhead, etc.

[0092] (5) The L4 layer calculates the theoretical tensor core utilization rate based on different shapes, tiles, etc. according to the implementation, and calculates the theoretical performance according to the actual chip tensor core computing power. For other operators, it is divided into memory-intensive, computation-intensive, etc. According to the bandwidth and computing power of the chip, the system's built-in operator performance is used to calculate separately. Finally, the time consumption is added up to get the theoretical time consumption of the whole model.

[0093] The artificial intelligence chip pre-silicon verification system can verify the model on Cmodel or EMU according to the actual model application scenario at a small cost through hierarchical design, and perform theoretical calculation and verification of performance.

[0094] The method provided by the embodiment of the application will be described below. The method described below can be referred to each other with the device described above.

[0095] Figure 4 is one of the flowcharts of the artificial intelligence chip pre-silicon verification method provided by the application, as Figure 4 shown, the method is applied to the artificial intelligence chip pre-silicon verification system in the above embodiment, comprising: Step 410, analyzing the current model to obtain a calculation graph of the current model; Step 420, configuring a distributed topology strategy of a hardware simulator of the artificial intelligence chip based on the distributed strategy of the current model, and adjusting the calculation graph of the current model to adapt the calculation graph to the distributed topology strategy; the hardware simulator is determined based on the design file of the artificial intelligence chip; Step 430, controlling the hardware simulator to run each operator in the calculation graph to obtain running performance data of each operator in the hardware simulator; Step 440: Based on the performance data of each operator in the hardware simulator, determine the performance data of the current model on the artificial intelligence chip.

[0096] The AI ​​chip pre-silicon verification method provided in this invention, through a layered design, obtains the performance data of each operator through a hardware simulator, and summarizes them to obtain the performance data of the entire model on the AI ​​chip. This realizes end-to-end full-process verification of the AI ​​chip from model input to output results, and improves the efficiency of AI chip pre-silicon verification.

[0097] Figure 5 This is the second flowchart illustrating the pre-silicon verification method for artificial intelligence chips provided by this invention, as shown below. Figure 5 As shown, step 420 is followed by: Step 450: Based on the performance prediction results of each operator, determine the performance prediction results of the current model on the artificial intelligence chip.

[0098] The AI ​​chip pre-silicon verification method provided in this invention, through a hierarchical design, not only obtains the performance prediction results of each operator, but also summarizes them to obtain the performance prediction results of the entire model on the AI ​​chip, realizing end-to-end full-process verification of the AI ​​chip from model input to output results, thus improving the efficiency of AI chip pre-silicon verification.

[0099] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor, communications interface, and memory communicate with each other via the communications bus. The processor can invoke logical commands stored in the memory to execute the methods described in the above embodiments, for example: The current model is analyzed to obtain its computational graph. Based on the distributed strategy of the current model, the distributed topology strategy of the hardware simulator for the AI ​​chip is configured, and the computational graph of the current model is adjusted to adapt to the distributed topology strategy. The hardware simulator is determined based on the design documents of the AI ​​chip. The hardware simulator is controlled to run each operator in the computational graph to obtain the performance data of each operator in the hardware simulator. Based on the performance data of each operator in the hardware simulator, the performance data of the current model on the AI ​​chip is determined; or, based on the performance prediction results of each operator, the performance prediction results of the current model on the AI ​​chip are determined.

[0100] Furthermore, the logic commands in the memory described above can be realized in the form of a software function unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several commands for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0101] The processor in the electronic device provided by the embodiments of the present application can call the logic instructions in the memory to realize the above-mentioned method, and the specific implementation manners are consistent with the above-mentioned method implementation manners, and the same beneficial effects can be achieved, which will not be described here.

[0102] The embodiments of the present application further provide a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method provided by each of the above embodiments.

[0103] The specific implementation manners are consistent with the above-mentioned method implementation manners, and the same beneficial effects can be achieved, which will not be described here.

[0104] The embodiments of the present application provide a computer program product, which includes a computer program. The computer program is executed by a processor to implement the above-mentioned method.

[0105] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0106] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0107] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A pre-silicon verification system for artificial intelligence chips, characterized in that, It includes a model import layer, a model parsing layer, a distributed strategy configuration layer, a performance prediction layer, and an operator implementation layer; The model import layer is used to import the current model; The model parsing layer is used to parse the current model to obtain the computational graph of the current model; The distributed strategy configuration layer is used to configure the distributed topology strategy of the hardware simulator of the artificial intelligence chip based on the distributed strategy of the current model, and adjust the computation graph of the current model so that the computation graph is adapted to the distributed topology strategy. The hardware emulator is determined based on the design documents of the artificial intelligence chip; The performance prediction layer is used to determine the performance data of the current model on the artificial intelligence chip based on the performance data of each operator in the hardware simulator. Alternatively, the performance prediction result of the current model on the artificial intelligence chip can be determined based on the performance prediction results of each operator; The operator implementation layer is used to control the hardware simulator to run the various operators in the computation graph.

2. The AI ​​chip pre-silicon verification system according to claim 1, characterized in that, The model import layer is used for: Obtain the model structure parameters, input configuration parameters, and model type parameters of the current model; Based on the model structure parameters, the input configuration parameters, and the model type parameters, the current model is imported from the model library; the model library has multiple registered models.

3. The pre-silicon verification system for artificial intelligence chips according to claim 1, characterized in that, The model parsing layer is used for: The current model is analyzed to determine the operators in the current model; The computational graph of the current model is determined based on the operators in the current model and the data flow between the operators.

4. The pre-silicon verification system for artificial intelligence chips according to claim 1, characterized in that, The distributed policy configuration layer is used for: Based on the distributed strategy of the current model, the distributed topology strategy is determined; the distributed topology strategy includes a hardware simulator for multiple artificial intelligence chips. Based on the operators running in each hardware emulator and the connection relationships between each hardware emulator, determine the set communication operators that need to be inserted. The set communication operator is inserted into the computation graph of the current model to adjust the computation graph of the current model.

5. The pre-silicon verification system for artificial intelligence chips according to claim 1, characterized in that, The performance prediction layer is used for: Obtain the performance counter file output by the hardware emulator; the performance counter file is used to record the running performance data of each operator in the hardware emulator. Based on the performance counter file, determine the runtime performance data of each operator; Based on the performance data of each operator, the performance data of the current model on the artificial intelligence chip is determined.

6. The pre-silicon verification system for artificial intelligence chips according to claim 1, characterized in that, The operator implementation layer is used for: The hardware simulator is controlled to run each operator in the computation graph, and the accuracy verification results and performance verification results of each operator in the corresponding hardware simulator are obtained.

7. The pre-silicon verification system for artificial intelligence chips according to any one of claims 1 to 6, characterized in that, The model type of the current model includes at least one of computer vision models and large language models.

8. A method for pre-silicon verification of artificial intelligence chips, characterized in that, The artificial intelligence chip pre-silicon verification system according to any one of claims 1 to 7 comprises: The current model is analyzed to obtain the computational graph of the current model; Based on the distributed strategy of the current model, the distributed topology strategy of the hardware simulator of the artificial intelligence chip is configured, and the computation graph of the current model is adjusted to adapt the computation graph to the distributed topology strategy; the hardware simulator is determined based on the design documents of the artificial intelligence chip. The hardware simulator is controlled to run each operator in the computation graph to obtain the performance data of each operator in the hardware simulator. Based on the performance data of each operator in the hardware simulator, the performance data of the current model in the artificial intelligence chip is determined.

9. The pre-silicon verification method for artificial intelligence chips according to claim 8, characterized in that, After configuring the parallel computing architecture distributed topology strategy of the hardware simulator for the artificial intelligence chip based on the parallel distributed strategy of the current model, and adjusting the computation graph of the current model, the method further includes: Based on the performance prediction results of each operator, the performance prediction result of the current model on the artificial intelligence chip is determined.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the pre-silicon verification method for artificial intelligence chips as described in claim 8 or 9.

Images