Deep learning accelerator performance modeling system, performance testing method, generation method

Abstract

The application provides a deep learning accelerator performance modeling system, a performance testing method and a generation method. The deep learning accelerator performance modeling system comprises: a first input interface, which is used for converting an initial network model file into an intermediate network model file using a unified network model description format, and automatically analyzing the intermediate network model file according to a network description supported by a deep learning accelerator (DLA) performance modeling framework to obtain a target network modeling file adapted to the DLA performance modeling framework; a second input interface, which is used for inputting a hardware architecture configuration file, processing a network model corresponding to the target network modeling file using a virtual DLA hardware architecture after obtaining a virtual DLA hardware architecture; and an analysis model, which is used for analyzing the performance of a DLA corresponding to the virtual DLA hardware architecture according to a result of processing the network model using the virtual DLA hardware architecture, thereby solving the problems of a long hardware accelerator design cycle and high trial and error cost in the related art.

Description

Technical Field

[0001] This application relates to the field of modeling, and in particular to a performance modeling system, performance testing method, and generation method for deep learning accelerators. Background Technology

[0002] With the continuous development of artificial intelligence technology, neural network algorithms, represented by deep learning, are increasingly being widely applied in AI devices. Currently, due to the massive computational demands required for model training and inference in AI, traditional computing chips are insufficient to meet these demands due to the inherent characteristics of the algorithms and computations themselves. Accelerators are needed to speed up neural networks and improve their computational capabilities. Deep Learning Accelerators (DLAs) typically use specially customized, highly parallelized computing units to process specific network models, while employing relevant memory and microarchitecture technologies to optimize the overall processing efficiency. However, given the rapid development of neural network models today, designing dedicated hardware accelerators is time-consuming, inefficient, and slow in product iteration. Currently, no effective solutions have been proposed to address this technical challenge. Summary of the Invention

[0003] This application provides a deep learning accelerator performance modeling system, performance testing method, and generation method to solve one or more of the above-mentioned technical problems.

[0004] In a first aspect, embodiments of this application provide a deep learning accelerator modeling system, comprising: a first input interface for converting an initial network model file into an intermediate network model file using a unified network model description format, and automatically parsing the intermediate network model file according to the network description supported by the deep learning accelerator (DLA) performance modeling framework to obtain a target network modeling file adapted to the DLA performance modeling framework; a second input interface for inputting a hardware architecture configuration file, and after obtaining a virtual DLA hardware architecture, using the virtual DLA hardware architecture to process the network model corresponding to the target network modeling file; and an analysis model for analyzing the DLA performance corresponding to the virtual DLA hardware architecture based on the result of processing the network model using the virtual DLA hardware architecture.

[0005] Secondly, embodiments of this application provide a method for testing the performance of a deep learning accelerator, which is used to execute a target data stream using the aforementioned deep learning accelerator modeling system; and to test the performance of the DLA based on the output of the DLA performance modeling system.

[0006] Thirdly, embodiments of this application provide a method for generating a deep learning accelerator, comprising: testing a target DLA using the aforementioned deep learning accelerator performance testing method; and generating a DLA based on the hardware configuration file of the target DLA in the DLA performance modeling system when the performance of the target DLA meets preset conditions.

[0007] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method described in any of the above-mentioned embodiments.

[0008] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0009] Sixthly, embodiments of this application provide a computer program product, including computer instructions, which, when executed by a processor, implement the method described in any of the above-mentioned embodiments.

[0010] Compared with related technologies, this application has the following advantages:

[0011] The deep learning accelerator performance modeling system provided in this application includes: a first input interface for converting an initial network model file into an intermediate network model file using a unified network model description format, and automatically parsing the intermediate network model file according to the network description supported by the deep learning accelerator (DLA) performance modeling framework to obtain a target network modeling file adapted to the DLA performance modeling framework; a second input interface for inputting a hardware architecture configuration file, and after obtaining a virtual DLA hardware architecture, using the virtual DLA hardware architecture to process the network model corresponding to the target network modeling file; and an analysis model for analyzing the DLA performance corresponding to the virtual DLA hardware architecture based on the result of processing the network model using the virtual DLA hardware architecture. In other words, this application embodiment uses a software modeling method to simulate the performance of accelerators with different hardware architectures, solving the problems of long hardware accelerator design cycles and high trial-and-error costs. Specifically, the unified network model description format is used as an intermediate format in the above-mentioned DLA performance modeling system to align original networks from different sources, solving the problems in related technologies where different format types are used for the same network model on different platforms, making it difficult for the modeling framework to be compatible with all network formats, leading to the inability / inconvenience in processing the original model, and the problem that the original model contains a large amount of redundant information, making direct processing very inefficient. In addition, the embodiments of this application can automatically parse intermediate network model files to obtain network model files adapted to the DLA performance modeling framework, thereby improving network modeling efficiency and solving the problems of large errors and low efficiency caused by manually writing modelable operator nodes of network models in related technologies.

[0012] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0013] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0014] Figure 1 This illustration shows a scenario diagram of the deep learning accelerator performance modeling system provided in an embodiment of this application;

[0015] Figure 2 This paper shows a block diagram of the DLA performance modeling system provided in an embodiment of the present application;

[0016] Figure 3A block diagram (I) of the DLA performance modeling system provided in an embodiment of this application is shown;

[0017] Figure 4 A block diagram (II) of the DLA performance modeling system structure provided in an embodiment of this application is shown;

[0018] Figure 5 A flowchart of the deep learning accelerator performance testing method provided in an embodiment of this application is shown;

[0019] Figure 6 A flowchart of the deep learning accelerator generation method provided in an embodiment of this application is shown;

[0020] Figure 7 A structural block diagram of the deep learning accelerator performance testing device provided in an embodiment of this application is shown;

[0021] Figure 8 A block diagram of the deep learning accelerator generation device provided in the embodiments of this application is shown;

[0022] Figure 9 A block diagram of an electronic device used to implement embodiments of this application is shown. Detailed Implementation

[0023] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0024] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0025] The application scenarios for deep learning accelerator performance modeling involved in this application include, but are not limited to: hardware-friendly development of neural network algorithms, design of dedicated autonomous driving chips, and deployment of artificial intelligence (AI) network models. In these application scenarios, DLA design is a crucial step, and a user-friendly performance modeling system can largely solve problems such as long hardware accelerator design cycles and complex network structures that are difficult to map data flows. Specifically, DLA performance modeling simulates hardware execution network models (such as...) through abstract modeling in software. Figure 1As shown in the figure, theoretically relatively accurate hardware costs, performance, and execution efficiency are obtained. Typically, performance modeling frameworks run a "hardware-network-dataflow" test case in a very short time. Using appropriate modeling methods, sufficiently reliable accuracy and a sufficiently convenient testing process can be achieved. Efficient network modeling is a key area of ​​DLA performance modeling, but related technologies have several shortcomings: 1. Directly processing the original model is very inconvenient because the same network model uses different format types on different platforms, making it difficult for modeling frameworks to be compatible with all network formats; 2. Performance modeling usually does not include all operator types of the original network, so the original model contains a lot of redundant information, making direct processing very inefficient; 3. Some modeling frameworks use custom-formatted network descriptions internally, modeling the network by hand-writing network descriptions. This method is difficult to align with real-world networks, resulting in modeling errors.

[0026] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0027] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0028] This application provides a deep learning accelerator performance modeling system, such as... Figure 2 The diagram shown is a block diagram of a DLA performance modeling system according to an embodiment of this application, including:

[0029] The first input interface 202 is used to convert the initial network model file into an intermediate network model file using a unified network model description format, and automatically parse the intermediate network model file according to the network description supported by the Deep Learning Accelerator (DLA) performance modeling framework to obtain a target network model file adapted to the DLA performance modeling framework. It should be noted that the first input interface 202 can be an interface of the network modeling end in the DLA performance modeling system. The aforementioned network model can be a neural network model. The aforementioned unified network model description format can be a concise and clear unified format used to align network models from different sources to adapt to different modeling frameworks. Furthermore, in this embodiment, the front end of the unified network description is the initial network, and the back end is the network object modeled within the framework. As a unified intermediate format, it forms end-to-end network modeling, ensuring the reliability and accuracy of network modeling. Optionally, the aforementioned initial network model file can include: Open Neural Network Exchange (ONNX) files, open-source Python machine learning library PyTorch files, end-to-end open-source machine learning platform TensorFlow files, etc. In one possible implementation, the method of converting the initial network model file into an intermediate network model file using the Unified Network Model Description Format (UNMMDS) can include: writing the description information included in the initial network model file to the corresponding locations according to the requirements of the UNMMDS. The automated parsing of the intermediate network model file based on the network description supported by the Deep Learning Accelerator (DLA) performance modeling framework can be achieved by using a fast algorithm to automatically parse the intermediate network model file to obtain a target network modeling file adapted to the DLA performance modeling framework, rather than using a manually written .c or .py description file to model the network model in related technologies.

[0030] The second input interface 204 is used to input a hardware architecture configuration file. After obtaining the virtual DLA hardware architecture, this virtual DLA hardware architecture is used to process the network model corresponding to the target network modeling file. It should be noted that this second input interface 204 can be an interface of the hardware architecture modeling end. After inputting the hardware architecture configuration file, the second input interface 204 can generate a virtual DLA hardware architecture that can process the network model output by the first input interface 202. For example, it can accelerate the processing of the network model. The hardware architecture configuration file describes the technical details of the entire DLA. Users can customize all the details or fix the internal configuration information of the DLA; users can choose several fixed hardware design templates.

[0031] Analysis model 206 is used to analyze the DLA performance corresponding to the virtual DLA hardware architecture by processing the network model based on the results of the virtual DLA hardware architecture. Optionally, in this embodiment, analysis model 206 can analyze DLA hardware cost, performance, and execution efficiency, etc. That is, this embodiment can perform end-to-end testing of network-hardware deployment performance.

[0032] pass Figure 2 The system described includes: a first input interface for converting an initial network model file into an intermediate network model file using a unified network model description format, and automatically parsing the intermediate network model file according to the network description supported by the Deep Learning Accelerator (DLA) performance modeling framework to obtain a target network modeling file adapted to the DLA performance modeling framework; a second input interface for inputting a hardware architecture configuration file, and after obtaining a virtual DLA hardware architecture, using the virtual DLA hardware architecture to process the network model corresponding to the target network modeling file; and an analysis model for analyzing the DLA performance corresponding to the virtual DLA hardware architecture based on the results of processing the network model using the virtual DLA hardware architecture. In other words, this application embodiment uses a software modeling method to simulate the performance of accelerators with different hardware architectures, solving problems such as long hardware accelerator design cycles and high trial-and-error costs. Specifically, the unified network model description format is used as an intermediate format in the above-mentioned DLA performance modeling system, which can align original networks from different sources. This solves the problems in related technologies where different format types are used for the same network model on different platforms, making it difficult for the modeling framework to be compatible with all network formats, leading to the inability / inconvenience of processing the original model, and the problem that the original model contains a large amount of redundant information, making direct processing very inefficient. In addition, the embodiments of this application can automatically parse intermediate network model files to obtain network model files adapted to the DLA performance modeling framework, thereby improving network modeling efficiency and solving the problems of large errors and low efficiency caused by manually writing modelable operator nodes of network models in related technologies.

[0033] In one possible implementation, such as Figure 3As shown, the first input interface 202 may include a definition module 2022, used to define the unified network model description format. This definition module 2022 is also used to describe information from the entire network model to operators using a JSON file, according to the hierarchical structure of the divided network layers. The first level of the JSON file includes a list of input feature map names and a list of network layers for the entire network model. JavaScript Object Notation (JSON) is a lightweight data exchange format that can be used in various programming languages ​​for data transmission, storage, and exchange between the front-end and back-end. Optionally, the list of input feature map names contains several strings, each string representing the name of an input feature map. The list of network layers for the entire network model contains each layer of the entire network model, with each layer represented by a dictionary (the dictionary corresponds to the "attrs" field, which is a dict-type data structure). The front-end of the unified network model description is the initial network model, and the back-end is the network object modeled within the modeling framework. As a unified intermediate format, it forms end-to-end network modeling, ensuring the reliability and accuracy of network modeling.

[0034] A single network layer in the network layer list of the entire network model described above can include: a dictionary, an input list, a network layer name, and an output list. The following is a detailed description of the included information.

[0035] The dictionary described above uses corresponding fields to describe the features of this network layer for different operator types, including the operator's attribute parameters. For convolution, pooling, and deconvolution operators, at least the fields include groups(int), strides(list), dilation(list), and padding(list). It should be noted that while the operator's own attribute parameters are described here, not every operator has attribute parameters that need to be described.

[0036] The above input list includes a list of input feature maps for this network layer. The input feature maps are described using this dictionary. Each feature map includes a list of information describing its dimensions, information describing whether the feature map is a constant value, information describing the format of the dimensions, and the name of the feature map. It should be noted that the dimensions can be one to five, and the constant values ​​can be 0 or 1; for example, weight data can be 0 or 1. The format information describing the dimensions can be a string. For example, if the format is NCHW and the dimensions are [1, 30, 20, 20], then the dimensions N, C, H, and W have sizes of 1, 30, 20, and 20, respectively. Regarding the name information of the feature maps, if the output feature map (OFM) of layer x and the input feature map (IFM) of layer y have the same name, it indicates that there is a connection between the two layers.

[0037] The aforementioned network layer names include at least network layer information, wherein the network layer names are different throughout the entire network model. Optionally, in this embodiment, the network layer name is represented by a string, and the name needs to contain detailed information about the network layer, such as parameter counts, operator library, etc.

[0038] The aforementioned operator type can be a string separating the optional description and the operator name using a period, where the optional description is used to identify additional information about the operator. Optionally, in this embodiment, there can be multiple optional descriptions, separated by periods ".", with the last item being the operator name, represented by lowercase letters (the optional description field is arbitrary and will not be parsed, but such a field can be used to identify additional information about the operator, such as operator library information, quantization information, etc.). For example, assuming the optional description includes optional description 'a' and the operator name is 'b', then the corresponding operator type could be 'ab'; assuming the optional description includes optional descriptions a, b, and c, and the operator name is 'd', then the corresponding operator type could be 'abcd'.

[0039] The above output list includes all output feature maps of this network layer, wherein the description of the output feature map is the same as that of the input feature map.

[0040] The aforementioned network model parser 2024 is used to automatically parse the intermediate network model file according to the network description supported by the DLA performance modeling framework, to obtain the target network model file adapted to the DLA performance modeling framework. Optionally, as follows... Figure 4As shown, the network model parser 2024 includes: an initialization module 20242, a parsing module 20244, a first traversal module 20246, a second traversal module 20248, and an update module 20250. This network model parser 2024 obtains internal network modeling objects adapted to the modeling framework, greatly improving network modeling efficiency. Furthermore, the network model parser 2024 can set the operator types supported by the modeling framework, the mapping data flow for different operators, and whether activation functions are fused, etc., to ensure compatibility with different performance modeling frameworks. This contrasts with related technologies that rely entirely on manually written network model operator nodes for network modeling, which struggles to achieve a good match with network models deployed in real-world environments, leading to significant modeling errors and consequently, unreliable and untrustworthy performance modeling evaluation results. Additionally, manual processes are inefficient and time-consuming, especially with large networks.

[0041] Optionally, the aforementioned initialization module 20242 can be used to obtain information about each layer, input features, description boundaries of operator layers, intermediate variables required for initializing the network model in the intermediate network model file, mapping information (which may include any operator mapping information that needs to be applied in the modeling framework), and to set operator types that cannot be supported by the performance modeling framework (different frameworks support different ranges of operators, so this needs to be set during the initialization phase), and whether the activation function is fused with the previous linear operator for modeling (some modeling frameworks support fused calculation of the activation function and the previous convolutional layer, rather than modeling the activation function as a separate layer).

[0042] The aforementioned parsing module 20244 can be used to parse the input feature maps of the network model corresponding to the intermediate network model file. Optionally, when the network model has a single input, the feature maps are parsed directly; when the network model has multiple inputs, all feature maps are extracted, and all information in the description of the feature maps is parsed sequentially to generate the input feature list of the network model.

[0043] The aforementioned first traversal module 20246 can be used to traverse the network model and obtain the network topology. Optionally, the first traversal module 20246 can simplify the operators in the network model into multiple input feature map names, operator layer IDs, and multiple output feature map names; if the output feature map with ID X and the input feature map with ID Y have the same name, then it is determined that there is a sequential connection relationship between layer X and layer Y.

[0044] The aforementioned second traversal module 20248 can be used to traverse the network model again and parse the operator layers that need to be modeled. Optionally, the second traversal module 20248 can enter the corresponding network model parser to complete the parsing of the current operator layer for different operator types. The parsing information is all entries within a layer of the intermediate network model file. The information required to parse an operator layer is the corresponding operator information in the intermediate network model file and the mapping information of the current layer. If a fusion activation function is set in the initialization module, the information of the activation function is added to the layer before the current layer. For deconvolution operators, a corresponding parsing entry is set, and a deconvolution operator is parsed into multiple parallel convolution operators and an operator that rearranges the data (e.g., a contract operator) by decomposing the convolution kernel. The parsing of deconvolution operators is special. Other operators are parsed in a one-to-one correspondence. For example, convolution is parsed as convolution, and fully connected layers are parsed as fully connected layers. A deconvolution operator is parsed into multiple operators, not a single operator. Specifically, the aforementioned multiple parallel convolution operators can be (x_stride*y_stride) parallel convolution operators, where x_stride and y_stride are one of the attributes of the deconvolution operator: the deconvolution stride in the x and y directions, respectively.

[0045] The aforementioned update module 20250 can be used to update the connection relationships between operators to obtain the target network modeling file. Optionally, the update module 20250 can add inter-layer interconnection information to each layer of the target network modeling file; delete operators not supported by the performance modeling framework while maintaining the overall topology of the intermediate network model file; and update the layer ID information in the target network modeling file so that the target network modeling file is in a sequential state in the topology sorting.

[0046] In summary, this application proposes integrating a universal network modeling interface (i.e., the aforementioned first input interface) into the system-level performance modeling of deep learning accelerators. This interface integrates the initial network model and performance modeling framework end-to-end. It can align original network models from different sources and is compatible with different performance modeling frameworks, exhibiting good compatibility and versatility. By performing network modeling end-to-end, it eliminates the drawbacks of poor reliability and low accuracy in manually controlled network modeling. The unified network model description provides a unified network description format in the field of performance modeling, allowing different original network types to be aligned to this intermediate description format during modeling. This provides an effective solution for network modeling in complex situations. The unified neural network parser uses algorithms to quickly parse the network model description, improving network modeling efficiency.

[0047] Corresponding to the application scenarios and systems provided in the embodiments of this application, the embodiments of this application also provide a method for testing the performance of deep learning accelerators. For example... Figure 5 The diagram shown is a flowchart of a deep learning accelerator performance testing method according to an embodiment of this application. The method may include:

[0048] S502, using the aforementioned DLA performance modeling system, execute the target data stream. Optionally, the aforementioned DLA performance modeling system includes: a first input interface 202, used to convert an initial network model file into an intermediate network model file using a unified network model description format, and automatically parse the intermediate network model file according to the network description supported by the deep learning accelerator DLA performance modeling framework to obtain a target network modeling file adapted to the DLA performance modeling framework. It should be noted that the first input interface 202 can be an interface of the network modeling end in the DLA performance modeling system. The aforementioned network model can be a neural network model. The aforementioned unified network model description format can be a concise and clear unified format used to align network models from different sources to adapt to different modeling frameworks. In addition, in this embodiment, the front end of the unified network description is the initial network, and the back end is the network object modeled within the framework. As a unified intermediate format, it forms end-to-end network modeling, ensuring the reliability and accuracy of network modeling. Optionally, the initial network model file mentioned above may include: an Open Neural Network Exchange (ONNX) file, an open-source Python machine learning library (PyTorch) file, an end-to-end open-source machine learning platform (TensorFlow) file, etc. In one possible implementation, the method of converting the initial network model file into an intermediate network model file using a unified network model description format may include: writing the description information included in the initial network model file to the corresponding location according to the requirements of the unified network model description format. The automated parsing of the intermediate network model file based on the network description supported by the Deep Learning Accelerator (DLA) performance modeling framework may involve using a fast algorithm to automatically parse the intermediate network model file to obtain a target network modeling file adapted to the DLA performance modeling framework, rather than using a manually written .c or .py description file to model the network model in related technologies. The second input interface 204 is used to input a hardware architecture configuration file. After obtaining the virtual DLA hardware architecture, this virtual DLA hardware architecture is used to process the network model corresponding to the target network modeling file. It should be noted that the second input interface 204 may be an interface of the hardware architecture modeling end. After inputting the hardware architecture configuration file, the second input interface 204 can generate a virtual DLA hardware architecture that can process the network model output by the first input interface 202. For example, it can accelerate the network model. The hardware architecture configuration file describes the technical details of the entire DLA; users can customize all details or fix the internal configuration information of the DLA by selecting several fixed hardware design templates. The analysis model 206 is used to analyze the DLA performance corresponding to the virtual DLA hardware architecture based on the results of processing the network model using the virtual DLA hardware architecture.Optionally, in this embodiment, the analysis model 206 can analyze DLA hardware costs, performance, and execution efficiency. That is, this embodiment can perform end-to-end testing of network-hardware deployment performance.

[0049] S504. Based on the output of the DLA performance modeling system, the DLA performance is tested.

[0050] By using a unified network model description format as an intermediate format through S502-S504 above, original networks from different sources can be aligned. This solves the problems in related technologies where different format types are used for the same network model on different platforms, making it difficult for the modeling framework to be compatible with all network formats, leading to the inability / inconvenience of processing the original model, and the problem that the original model contains a large amount of redundant information, making direct processing very inefficient. In addition, the embodiments of this application can automatically parse intermediate network model files to obtain network modeling files adapted to the DLA performance modeling framework, improving network modeling efficiency and solving the problems of large errors and low efficiency caused by manually writing modelable operator nodes of network models in related technologies. Furthermore, the embodiments of this application can test network-hardware deployment performance end-to-end, greatly shortening the evaluation process and time in the field of deep learning accelerator performance modeling, improving modeling efficiency, and ensuring the reliability and accuracy of cost assessment.

[0051] Corresponding to the application scenarios and systems provided in the embodiments of this application, the embodiments of this application also provide a method for generating deep learning accelerators. For example... Figure 6 The diagram shown is a flowchart of a deep learning accelerator generation method according to an embodiment of this application, including:

[0052] S602, The target DLA is tested using the above-mentioned deep learning accelerator performance testing method;

[0053] S604, when the performance of the target DLA meets the preset conditions, a DLA is generated according to the hardware configuration file of the target DLA in the DLA performance modeling system. Optionally, in this embodiment, the preset conditions may be latency within a specified range or power consumption within a specified range, etc. In this embodiment, when the performance of the target DLA does not meet the preset conditions, the hardware architecture configuration file can be re-input through the second input interface of the deep learning accelerator performance modeling system. After obtaining a new virtual DLA hardware architecture, the network model is processed using the new virtual DLA hardware architecture, and the performance analysis results are obtained by analyzing the model to determine whether the new hardware architecture meets the preset conditions.

[0054] Through the above steps S602 to S604, the target DLA is tested using the DLA performance modeling system provided in this application embodiment. When the performance of the target DLA meets the preset conditions, the DLA can be generated according to the hardware configuration file of the target DLA in the DLA performance modeling system, that is, the design of the hardware DLA is realized, thereby solving the technical problems in the related technology of long design cycles, high trial and error costs, and slow product iteration speed of hardware accelerators in dedicated fields.

[0055] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a deep learning accelerator performance testing device. For example... Figure 7 The diagram shown is a structural block diagram of a deep learning accelerator performance testing device according to an embodiment of this application. The device may include:

[0056] The execution module 702 is used to execute the target data stream using the aforementioned DLA performance modeling system. Optionally, the aforementioned DLA performance modeling system includes: a first input interface 202, used to convert an initial network model file into an intermediate network model file using a unified network model description format, and automatically parse the intermediate network model file according to the network description supported by the deep learning accelerator DLA performance modeling framework to obtain a target network modeling file adapted to the DLA performance modeling framework. It should be noted that the first input interface 202 can be an interface of the network modeling end in the DLA performance modeling system. The aforementioned network model can be a neural network model. The aforementioned unified network model description format can be a concise and clear unified format used to align network models from different sources to adapt to different modeling frameworks. In addition, in this embodiment, the front end of the unified network description is the initial network, and the back end is the network object modeled within the framework. As a unified intermediate format, it forms end-to-end network modeling, ensuring the reliability and accuracy of network modeling. Optionally, the initial network model file mentioned above may include: an Open Neural Network Exchange (ONNX) file, an open-source Python machine learning library (PyTorch) file, an end-to-end open-source machine learning platform (TensorFlow) file, etc. In one possible implementation, the method of converting the initial network model file into an intermediate network model file using a unified network model description format may include: writing the description information included in the initial network model file to the corresponding location according to the requirements of the unified network model description format. The automated parsing of the intermediate network model file based on the network description supported by the Deep Learning Accelerator (DLA) performance modeling framework may involve using a fast algorithm to automatically parse the intermediate network model file to obtain a target network modeling file adapted to the DLA performance modeling framework, rather than using a manually written .c or .py description file to model the network model in related technologies. The second input interface 204 is used to input a hardware architecture configuration file. After obtaining the virtual DLA hardware architecture, this virtual DLA hardware architecture is used to process the network model corresponding to the target network modeling file. It should be noted that the second input interface 204 may be an interface of the hardware architecture modeling end. After inputting the hardware architecture configuration file, the second input interface 204 can generate a virtual DLA hardware architecture that can process the network model output by the first input interface 202. For example, it can accelerate the network model. The hardware architecture configuration file describes the technical details of the entire accelerator; users can customize all details or fix the accelerator's internal configuration information by selecting several fixed hardware design templates. The analysis model 206 is used to analyze the DLA performance corresponding to the virtual DLA hardware architecture based on the results of processing the network model using the virtual DLA hardware architecture.Optionally, in this embodiment, the analysis model 206 can analyze DLA hardware costs, performance, and execution efficiency. That is, this embodiment can perform end-to-end testing of network-hardware deployment performance.

[0057] The first test module 704 is used to test the performance of DLA based on the output of the DLA performance modeling system.

[0058] pass Figure 7 The apparatus shown uses a unified network model description format as an intermediate format, which can align original networks from different sources. This solves the problems in related technologies where different format types are used for the same network model on different platforms, making it difficult for the modeling framework to be compatible with all network formats, thus leading to the inability / inconvenience of processing the original model, and the problem that the original model contains a large amount of redundant information, making direct processing very inefficient. In addition, the embodiments of this application can automatically parse the intermediate network model file to obtain a network modeling file adapted to the DLA performance modeling framework, improving network modeling efficiency, and thus solving the problems of large errors and low efficiency caused by manually writing modelable operator nodes of the network model in related technologies. Furthermore, the embodiments of this application can test the network-hardware deployment performance end-to-end, which greatly shortens the evaluation process and time in the field of deep learning accelerator performance modeling, improves modeling efficiency, and ensures the reliability and accuracy of cost assessment.

[0059] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a deep learning accelerator generation apparatus. For example... Figure 8 The diagram shown is a structural block diagram of a deep learning accelerator generation device according to an embodiment of this application. The device may include:

[0060] The second test module 802 is used to test the target DLA using the aforementioned deep learning accelerator performance testing method;

[0061] Design module 804 is used to generate a DLA based on the hardware configuration file of the target DLA in the DLA performance modeling system when the performance of the target DLA meets preset conditions. Optionally, in this embodiment, the preset conditions may be latency within a specified range or power consumption within a specified range, etc. In this embodiment, when the performance of the target DLA does not meet the preset conditions, the hardware architecture configuration file can be re-input through the second input interface of the deep learning accelerator performance modeling system. After obtaining a new virtual DLA hardware architecture, the network model is processed using the new virtual DLA hardware architecture, and the performance analysis results are obtained by analyzing the model to determine whether the new hardware architecture meets the preset conditions.

[0062] pass Figure 8The apparatus shown uses the DLA performance modeling system provided in this application embodiment to test the target DLA. When the performance of the target DLA meets the preset conditions, the DLA can be generated according to the hardware configuration file of the target DLA in the DLA performance modeling system, that is, the design of the hardware DLA is realized, thereby solving the technical problems of long design cycles, high trial and error costs, and slow product iteration speed of hardware accelerators in related fields.

[0063] The functions of each device and system in the embodiments of this application can be found in the corresponding descriptions in the above methods, and they have corresponding beneficial effects, which will not be repeated here.

[0064] Figure 9 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 9 As shown, the electronic device includes a memory 901 and a processor 902. The memory 901 stores a computer program that can run on the processor 902. When the processor 902 executes the computer program, it implements the method described in the above embodiments. The number of memories 901 and processors 902 can be one or more.

[0065] The electronic device also includes:

[0066] The communication interface 903 is used to communicate with external devices and exchange and transmit data.

[0067] If the memory 901, processor 902, and communication interface 903 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0068] Optionally, in a specific implementation, if the memory 901, processor 902, and communication interface 903 are integrated on a single chip, then the memory 901, processor 902, and communication interface 903 can communicate with each other through an internal interface.

[0069] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0070] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0071] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0072] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0073] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0074] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0075] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0076] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0077] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0078] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0079] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0080] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0081] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A deep learning accelerator performance modeling system, comprising: The first input interface is used to convert the initial network model file into an intermediate network model file using a unified network model description format, and to automatically parse the intermediate network model file according to the network description supported by the Deep Learning Accelerator (DLA) performance modeling framework to obtain a target network model file adapted to the DLA performance modeling framework. The automatic parsing of the intermediate network model file according to the network description supported by the Deep Learning Accelerator (DLA) performance modeling framework includes: for different operator types, entering the corresponding network model parser to complete the parsing of the current operator layer. The information required to parse an operator layer is the corresponding operator information and the mapping information of the current layer in the intermediate network model file. The second input interface is used to input the hardware architecture configuration file. After obtaining the virtual DLA hardware architecture, the virtual DLA hardware architecture is used to process the network model corresponding to the target network modeling file. An analysis model is used to analyze the DLA performance corresponding to the virtual DLA hardware architecture by processing the network model based on the virtual DLA hardware architecture.

2. The system according to claim 1, wherein, The first input interface includes: The definition module is used to define the unified network model description format; A network model parser is used to automatically parse the intermediate network model file according to the network description supported by the DLA performance modeling framework, and obtain the target network model file adapted to the DLA performance modeling framework.

3. The system according to claim 2, wherein, The definition module is also used to describe information from the entire network model to the operators using JSON files, according to the hierarchical structure of the divided network layers. The first level of the JSON file includes a list of names of the input feature maps and a list of network layers of the entire network model.

4. The system according to claim 3, wherein, One network layer in the network layer list of the entire network model includes: A dictionary in which different operator types use corresponding fields to describe the features of this network layer, including the operator's attribute parameters. Among them, convolution operators, pooling operators, and deconvolution operators include at least grouping, stride list, hole list, and padding list. The input list includes a list of input feature maps for this network layer. The input feature maps are described by the dictionary. Each feature map includes list information describing the dimension size, information describing whether the feature map is a constant value, format information describing the dimension, and name information of the feature map. The network layer name includes at least network layer information, wherein the network layer names are different for the entire network model; Operator type, a string separating the optional description from the operator name by a dot, wherein the optional description is used to identify additional information about the operator; The output list includes all output feature maps of this network layer, wherein the output feature maps are described in the same way as the input feature maps.

5. The system according to claim 4, wherein, The network model parser includes: an initialization module, a parsing module, a first traversal module, a second traversal module, and an update module, wherein... The initialization module is used to obtain information of each level, input features, description boundaries of operator layers, intermediate variables and mapping information required for initializing the network model from the intermediate network model file, as well as to set operator types that cannot be supported by the performance modeling framework and whether the activation function is fused with the previous linear operator for modeling. The parsing module is used to parse the input feature map of the network model corresponding to the intermediate network model file; The first traversal module is used to traverse the network model to obtain the network topology. The second traversal module is used to traverse the network model again and parse the operator layers that need to be modeled; The update module is used to update the connection relationships between operators to obtain the target network modeling file.

6. The system according to claim 5, wherein the parsing module is further configured to... When the network model has a single input, the feature map is directly parsed. When the network model has multiple inputs, all feature maps are extracted, and all information in the description of the feature map is parsed in sequence to generate the input feature list of the network model.

7. The system according to claim 5, wherein the first traversal module is further configured to... The operators in the network model are simplified into multiple input feature map names, operator layer IDs, and multiple output feature map names; If the output feature map with ID X and the input feature map with ID Y have the same name, then it is determined that there is a connection between the X layer and the Y layer.

8. The system according to claim 5, wherein the second traversal module is further configured to... For different operator types, the corresponding network model parser completes the parsing of the current operator layer. The parsed information consists of all entries within a layer of the intermediate network model file; If a fusion activation function is set in the initialization module, the information of the activation function is added to the layer preceding the current layer. For deconvolution operators, a corresponding parsing entry is set, and a deconvolution operator is parsed into multiple parallel convolution operators and an operator that rearranges the data by decomposing the convolution kernel.

9. The system according to claim 5, wherein the updating module is further configured to... Add inter-layer interconnection information to each layer of the target network modeling file; Remove operators that are not supported by the performance modeling framework, while maintaining the overall topology of the intermediate network model file; Update the layer ID information in the target network modeling file so that the target network modeling file is in an ordered state in the topology sorting.

10. A method for testing the performance of a deep learning accelerator, comprising: Execute the target data stream using the DLA performance modeling system according to any one of claims 1-9; The DLA performance is tested based on the output of the DLA performance modeling system.

11. A method for generating deep learning accelerators, comprising: The target DLA was tested using the performance testing method described in claim 10; When the performance of the target DLA meets the preset conditions, the DLA is generated according to the hardware configuration file of the target DLA in the DLA performance modeling system.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 10-11.

13. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 10-11.

14. A computer program product comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the method of any one of claims 10-11.

Images