Locating neural network performance hotspots

Abstract

Implementations for locating performance hotspots include collecting sample data having instruction addresses, the sample data being for a neural network model, and determining instructions in the instruction addresses that are performance hotspots. The instructions of the sample data that are performance hotspots are mapped to locations in a low-level intermediate representation using a listing file. The locations of the low-level intermediate representation that are performance hotspots are mapped to operations in one or more high-level representations, one or more of the operations corresponding to the performance hotspots, using a mapping file, the mapping file being generated from compiling the neural network model.

Description

Background Technology

[0001] This invention generally relates to computer systems, and more specifically, to computer-implemented methods, computer systems, and computer program products configured and arranged to locate performance hotspots of neural networks.

[0002] Artificial neural networks (commonly referred to as neural networks) are computational systems inspired by the biological neural networks that make up the animal brain. Artificial neural networks are based on a collection of connection units, or nodes, called artificial neurons, which loosely mimic neurons in the biological brain. Each connection (like a synapse in the biological brain) can transmit a signal to other neurons. The artificial neuron that receives the signal then processes it and can signal back to the neurons connected to it. The "signal" at the connection is a real number, and the output of each neuron is calculated as a nonlinear function of the sum of its inputs. Connections are called edges. Neurons and edges typically have weights that adjust as learning progresses. Weights increase or decrease the signal strength at the connection. Neurons can have thresholds such that a signal is only sent when the aggregated signal crosses that threshold. Typically, neurons are aggregated into layers. Different layers can perform different transformations on their inputs. A signal may propagate from the first layer (input layer) through one or more hidden layers to the last layer (output layer) after passing through layers multiple times.

[0003] Neural networks can be extremely complex, comprising a large number of compiled instructions. Sometimes, performance hotspots may exist for one or more instructions. In computer science, performance hotspots are generally defined as regions in a computer program where a high percentage of instructions are executed and / or where the program spends the most time during execution. If a program is randomly interrupted, the program counter (a pointer to the next instruction to be executed) is often found to be within a specific range of instruction addresses, which may indicate that the code needs optimization. However, in neural network models, especially those compiled using modern compilers, it can be difficult to identify and / or locate performance hotspots that require optimization, thus necessitating improvements. Summary of the Invention

[0004] Embodiments of the present invention relate to a computer-implemented method for locating performance hotspots in a neural network for optimization. One non-limiting example computer-implemented method includes: collecting sample data having instruction addresses, the sample data being stored for use in a neural network model. The method includes: determining instructions at the instruction addresses that are performance hotspots; and mapping the instructions in the sample data that are performance hotspots to positions in a low-level intermediate representation using a listing file. Furthermore, the method includes: mapping the positions in the low-level intermediate representations that are performance hotspots to operations in one or more high-level representations using a mapping file, one or more of the operations corresponding to the performance hotspots, the mapping file being generated from compiling the neural network model.

[0005] This provides an improvement over known methods for identifying performance hotspots by efficiently determining and locating them at high-level intermediate representations associated with the neural network model. High-level representations are more easily read, understood, and modified by human users, enabling optimization of performance hotspots affecting the neural network model, thereby improving its performance. Furthermore, the improvements in one or more embodiments can be used to locate performance hotspots at any suitable level. One or more embodiments can find the largest hotspots, allowing for optimization of hotspot performance with minimal effort.

[0006] In addition to one or more features described above or below, or alternatively, in other embodiments of the invention, the method may include mapping the operations in the one or more high-level representations to nodes in the neural network model using the mapping file. Therefore, this improves the ability to identify which nodes in the neural network model are performance hotspots.

[0007] In addition to one or more features described above or below, or alternatively, in other embodiments of the invention, the method may include: determining which nodes in the neural network model represent the performance hotspots based on mappings from the one or more high-level representations, which are mapped from the low-level intermediate representations, which are mapped from the sample data. Thus, improvements advantageously track performance hotspots from more difficult machine-readable code (such as binary code) to operations at a level more easily read, understood, and modified by human users. Although compilation is a top-down process, one or more embodiments provide a bottom-up mapping process that uses mapping files to map relationships from low-level to high-level.

[0008] In addition to one or more features described above or below, or alternatively, in other embodiments of the invention, the method may include: classifying the performance hotspots at different levels of compiling the neural network model using the mapping file, the different levels including the low-level intermediate representations to the one or more high-level representations. Thus, the improvement advantageously provides options at different levels for optimizing performance hotspots in the neural network model.

[0009] In addition to one or more features described above or below, or alternatively, in other embodiments of the invention, the sample data further includes information from one or more counters.

[0010] In addition to one or more features described above or below, or as an alternative, in other embodiments of the invention, the performance hotspot includes one or more metrics that meet or exceed one or more thresholds.

[0011] In addition to one or more features described above or below, or as an alternative, in other embodiments of the invention, the method includes: determining one or more operations in the one or more high-level representations for optimization, thereby addressing the performance hotspots.

[0012] Other embodiments of the present invention implement the features of the above-described method in computer systems and computer program products.

[0013] Additional technical features and benefits are achieved through the technology of this invention. Embodiments and aspects of the invention are described in detail herein and are considered part of the claimed subject matter. For a better understanding, refer to the detailed description and accompanying drawings. Attached Figure Description

[0014] The details of the proprietary rights described herein are specifically pointed out and clearly claimed in the claims at the end of the specification. The foregoing and other features and advantages of embodiments of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, wherein:

[0015] Figure 1 A block diagram is depicted for use in conjunction with one or more embodiments of the present invention;

[0016] Figure 2 A block diagram depicting an example neural network model / architecture according to one or more embodiments of the present invention is provided;

[0017] Figure 3 A block diagram of a system for locating and optimizing performance hotspots in a neural network model according to one or more embodiments of the present invention is depicted.

[0018] Figure 4 A block diagram depicts an example architecture of a compiler for compiling a neural network model according to one or more embodiments of the present invention;

[0019] Figure 5 This is a flowchart of a computer implementation process for locating performance hotspots in a neural network model according to one or more embodiments of the present invention;

[0020] Figure 6 A block diagram depicting an example performance matrix showing relevant performance hotspots at different compilation levels according to one or more embodiments of the present invention;

[0021] Figure 7 A block diagram of a computer-implemented control flow for locating performance hotspots in a neural network model for optimization, according to one or more embodiments of the present invention, is depicted.

[0022] Figure 8 This is a flowchart of a computer-implemented method for locating performance hotspots in a neural network model and optimizing one or more performance hotspots according to one or more embodiments of the present invention;

[0023] Figure 9 A cloud computing environment according to one or more embodiments of the present invention is described; and

[0024] Figure 10 An abstract model layer is described according to one or more embodiments of the present invention. Detailed Implementation

[0025] One or more embodiments of the present invention provide a method, computer system, and computer program product for arranging and configuring a computer implementation for locating and optimizing performance hotspots in a neural network. One or more embodiments are configured to use a Multi-Level Intermediate Representation (MLIR) instruction address (IA) mapping file for a neural network model to address the problem of locating performance hotspots in neural network nodes (including locating performance hotspots at different compilation levels) in order to optimize neural network performance. According to one or more embodiments, the MLIR IA mapping file is used to map instruction address samples to name / position elements in an MLIR compilation that may include MLIR dialect operations and MLIR passes, as well as neural network nodes. The MLIR IA mapping file and MLIR dialects are used to annotate data flow graphs and MLIR elements using performance hotspot indexing information. Furthermore, one or more embodiments search for and find the hottest neural network operations, the hottest operations at each compiler level, and the hottest range of performance instructions.

[0026] Important practical questions exist in performance analysis, such as identifying which parts of a program take the most execution time (indicating actual performance problems to be addressed) and how to solve those problems. The code region that takes the most execution time (also known as Central Processing Unit (CPU) time) is called a performance hotspot. A performance hotspot can also be described as a code section that requires the most CPU usage per instruction over a given time period compared to other parts of the code, e.g., CPU percentage. Performance hotspots are the best locations for tuning and optimization because a small effort to make a performance hotspot faster can have a large performance improvement in a neural network model. Finding performance hotspots in a neural network model can be difficult. A single module without function-level symbols can exist in a neural network module, and there can be hot instructions distributed across ten thousand or more instructions (i.e., causing performance hotspots). Even if the hottest basic block pattern can be found, it is difficult to know which part to optimize.

[0027] As a technical solution and advantage for identifying and locating performance hotspots in improved neural network models, one or more embodiments are configured to locate which neural network nodes in the graph are performance hotspots and / or which operations in MLIR are performance hotspots, thereby locating neural network performance hotspots. One or more embodiments provide opportunities to optimize (i.e., improve) performance hotspots at different levels, including different levels of compiling the neural network model, such as the MLIP compilation level. It should be understood that performance hotspot, hotspot, hotspot information, hotspot index information, etc., can be used interchangeably to refer to one or more instructions and / or operations that, compared to other instructions / operations and / or compared to a predetermined threshold, use more execution time (i.e., CPU time) and / or require more CPU usage / percentage. By identifying performance hotspots in one of the low-level and / or high-level intermediate representations, optimization can be performed at any level to improve the execution of the neural network model, thereby improving the functionality of the computer system (itself) executing the neural network model. Identifying the performance hotspots to be optimized in the neural network model can reduce execution time (i.e., reduce runtime), reduce CPU usage, reduce memory usage, reduce bandwidth, etc.

[0028] Turn now Figure 1This document generally illustrates a computer system 100 according to one or more embodiments of the present invention. The computer system 100 may be an electronic computer framework comprising and / or employing any number of computing devices and networks utilizing various communication technologies and combinations thereof, as described herein. The computer system 100 may be readily scalable, extensible, and modular, with the ability to change to different services or reconfigure some features independently of others. The computer system 100 may be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, the computer system 100 may be a cloud computing node. The computer system 100 may be described in the general context of computer system executable instructions, such as program modules executed by the computer system. Typically, program modules may include routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement specific abstract data types. The computer system 100 may be practiced in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules may reside in local and remote computer system storage media, including memory storage devices.

[0029] like Figure 1 As shown, computer system 100 has one or more central processing units (CPUs) 101a, 101b, 101c, etc. (collectively or collectively referred to as processor 101). Processor 101 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. Processor 101 (also referred to as processing circuitry) is coupled to system memory 103 and various other components via system bus 102. System memory 103 may include read-only memory (ROM) 104 and random access memory (RAM) 105. ROM 104 is coupled to system bus 102 and may include a basic input / output system (BIOS) or a successor system such as a unified extensible firmware interface (UEFI), which controls specific basic functions of computer system 100. RAM is read-write memory coupled to system bus 102 for use by processor 101. System memory 103 provides temporary memory space for the operation of the instructions during operation. System memory 103 may include random access memory (RAM), read-only memory, flash memory, or any other suitable memory system.

[0030] Computer system 100 includes an input / output (I / O) adapter 106 and a communication adapter 107 coupled to a system bus 102. I / O adapter 106 may be a Small Computer System Interface (SCSI) adapter that communicates with a hard disk 108 and / or any other similar component. I / O adapter 106 and hard disk 108 are collectively referred to herein as mass storage 110.

[0031] Software 111 for execution on computer system 100 may be stored in mass storage 110. Mass storage 110 is an example of a tangible storage medium readable by processor 101, wherein software 111 is stored as instructions executed by processor 101 to cause computer system 100 to operate (as described herein with reference to the accompanying drawings). Examples of computer program products and the execution of such instructions are discussed in more detail here. Communication adapter 107 interconnects system bus 102 with network 112, which may be an external network enabling computer system 100 to communicate with other such systems. In one embodiment, a portion of system memory 103 and mass storage 110 jointly store an operating system, which may be a coordinating... Figure 1 The functions of the various components shown are available on any suitable operating system.

[0032] Additional input / output devices are shown connected to the system bus 102 via display adapter 115 and interface adapter 116. In one embodiment, adapters 106, 107, 115, and 116 may be connected to one or more I / O buses, which are connected to the system bus 102 via an intermediate bus bridge (not shown). A display 119 (e.g., a screen or display monitor) is connected to the system bus 102 via display adapter 115, which may include a graphics controller to improve the performance of graphics-intensive applications and video controllers. Keyboard 121, mouse 122, speakers 123, etc., may be interconnected to the system bus 102 via interface adapter 116, which may include, for example, a super I / O chip integrating multiple device adapters into a single integrated circuit. Suitable I / O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols such as Peripheral Component Interconnect (PCI) and Fast Peripheral Component Interconnect (PCIe). Therefore, as Figure 1 The computer system 100 configured therein includes processing capabilities in the form of a processor 101, storage capabilities including system memory 103 and mass storage 110, input devices such as a keyboard 121 and a mouse 122, and output capabilities including a speaker 123 and a display 119.

[0033] In some embodiments, the communication adapter 107 may use any suitable interface or protocol (such as an Internet Minicomputer System Interface) to send data. The network 112 may be a cellular network, radio network, wide area network (WAN), local area network (LAN), or the Internet. External computing devices may connect to the computer system 100 via the network 112. In some examples, the external computing device may be an external network server or a cloud computing node.

[0034] It should be understood that Figure 1 The block diagram is not intended to show that computer system 100 shall include Figure 1 Instead of all the components shown, computer system 100 may include Figure 1 Any suitable fewer or additional components not shown herein (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Furthermore, the embodiments described herein with respect to computer system 100 can be implemented with any suitable logic, wherein the logic mentioned herein may include any suitable hardware (e.g., processor, embedded controller, or application-specific integrated circuit, etc.), software (e.g., applications, etc.), firmware, or any suitable combination of hardware, software, and firmware in various embodiments.

[0035] Now for reference Figure 2 An example neural network model / architecture 200 according to one or more embodiments is illustrated. The neural network model / architecture 200 can be implemented using one or more software applications 111 on a computer system 100. In one or more embodiments, the computer system 100 may include one or more dedicated hardware components (such as accelerators) for use with the neural network model / architecture. The operation of the example neural network model 200 is now discussed. During feedforward operation, each input neuron in a set of input neurons 202 sends its corresponding input voltage in parallel to the corresponding row of weights 204. Each weight 204 has a settable resistance value such that a current output flows from the weight 204 to the corresponding hidden neuron 206 to represent the weighted input. The current output for a given weight is determined as I = V / r, where V is the input voltage from the input neuron 202 and r is the set resistance of the weight 204. The currents from each weight are summed column-wise and flow to the hidden neuron 206. A set of reference weights 207 have fixed resistances, and their outputs are combined to form a reference current provided to each hidden neuron 206. Because conductance values ​​can only be positive, some reference conductance is needed to encode positive and negative values ​​in the matrix. The current generated by weight 204 is continuously positive, therefore reference weight 207 is used to provide a reference current above which the current is considered positive, and below which the current is considered negative. In some embodiments, each weight array may include one or more reference weights with static resistance.

[0036] As an alternative to using reference weights 207, one or more embodiments may use a separate weight array 204 to capture negative values. Each method has its advantages and disadvantages. In some embodiments, using reference weights 207 is more efficient in terms of chip area, but the reference values ​​need to be closely matched to each other. In one or more embodiments, using a separate array for negative values ​​does not involve close matching because each value has a pair of weights to be compared with. However, the negative weight matrix method uses approximately twice the chip area compared to a single column of reference weights. Furthermore, the reference weight column generates currents that need to be copied to each neuron for comparison, while the negative matrix array directly provides a reference value for each neuron. In the negative array embodiment, the weights 204 of both the positive and negative arrays are updated, but this also increases the signal-to-noise ratio because each weight value is the difference between two conductance values. Both embodiments provide the same functionality in encoding negative values, and those skilled in the art will be able to choose the appropriate embodiment for their application.

[0037] Hidden neurons 206 use current from weight array 204 and reference weights 207 to perform some calculations. Hidden neurons 206 then output their own voltages to another weight array 207. This array performs in the same way, where a column of weights 204 receives voltages from their respective hidden neurons 206 to produce a weighted current output, which is added row by row and provided to output neuron 208.

[0038] It should be understood that any number of these levels can be achieved by inserting additional array layers and hidden neurons 206. It should also be noted that some neurons can be constant neurons 209 that provide a constant voltage to the array. Constant neurons 209 may be present between input neurons 202 and / or hidden neurons 206, and are used only during feedforward operations.

[0039] In one or more embodiments, during backpropagation, output neurons 208 provide voltage back-propagation across the array of weights 204. The output layer compares the generated network response with the training data and calculates the error. This error is applied to the array as a voltage pulse, wherein the pulse height and / or duration is modulated proportionally to the error value. In this example, a row of weights 204 receives voltage in parallel from the corresponding output neurons 208 and converts the voltage into a current, which is summed column-by-column to provide input to the hidden neurons 206. The hidden neuron 206 provides a weighted feedback signal combined with the derivative calculated from its feedforward, and stores the error value before outputting the feedback signal voltage to its corresponding row of weights 204. This backpropagation proceeds throughout the entire neural network model 200 until all hidden neurons 206 and input neurons 202 have stored the error value.

[0040] In one or more embodiments, during weight updates, input neuron 202 and hidden neuron 206 apply a first weight update voltage to feed forward through neural network model 200, while output neuron 208 and hidden neuron 206 apply a second weight update voltage to feed backward through neural network model 200. The combination of these voltages produces a state change within each weight 204, causing weight 204 to acquire a new resistance value. In this way, weights 204 can be trained to adapt neural network model 200 to errors in its processing. It should be noted that the three operating modes—feedforward, backpropagation, and weight update—do not overlap.

[0041] Figure 3 This is a block diagram of a system 300 for locating and optimizing performance hotspots in a neural network model according to one or more embodiments of the present invention. Figure 3 One or more computer systems 302 coupled to computer system 320 are depicted, communicating to exchange information as discussed herein. Computer system 320 may communicate with computer system 302 via wired and / or wireless networks. Components of computer system 100 may be used and / or integrated into computer system 302 and computer system 320. Software application 304 may be implemented as software 111 executing on one or more processors 101, such as... Figure 1 Similarly, as discussed in the paper, compiler 322, diagnostic tool 306, and neural network model 200 can be implemented using software 111 configured to execute on one or more processors 101.

[0042] Compiler 322 is used to compile neural network models, such as neural network model 200 or any other neural network model. Compiler 322 and neural network model 200 can execute in computer system 320, computer system 302, and / or both. Compiler 322 is a Multi-Level Intermediate (MLIR) compiler and / or uses the MLIR compiler framework. MLIR is a reusable and scalable modern compiler infrastructure. MLIR compilers can define dialects and optimization passes. Dialects serve as abstraction levels or intermediate representations, and optimization passes enable optimizations at the abstraction level or transformations between abstraction levels. Ready-to-use dialects exist in MLIR, such as llvm, std, scf, and affine. The llvm dialect is a low-level dialect. The llvm dialect wraps LLVM intermediate representation (IR) types and instructions into MLIR types and operations. The std dialect includes standard operations such as load, store, addi, addf, absf, and call. The scf dialect defines control flow operations such as for and if. Affine dialects provide abstractions for affine operations and analysis. Those skilled in the art will understand multi-level intermediate compilers.

[0043] Furthermore, compiler 322 may also be implemented using Open Neural Network Exchange (ONNX) as the format for representing the input model (e.g., neural network model 200) of compiler 322 in conjunction with MLIR. ONNX is an open-source, machine-independent format and is widely used for exchanging neural network models, as understood by those skilled in the art. Compiler 322 can be written using MLIR, a modern open-source compiler infrastructure for multi-level intermediate representations, and utilizes the LLVM project. The LLVM project is a compiler infrastructure that is a collection of modular and reusable compiler and toolchain technologies. Compiler 322 may be interchangeably referred to as the ONNX-MLIR compiler, the MLIR compiler, and / or simply the compiler.

[0044] The neural network model 200 may be written and / or generated in ONNX or ONNX format. The neural network model 200 may be an ONNX neural network model 200, which may be referred to as an ONNX model. The neural network model may also be referred to as an artificial intelligence (AI) model. The neural network model 200 may be executed, which includes (firstly) using compiler 322 to compile the instructions of the neural network model 200 into an executable format (e.g., executable file 334) for execution by computer system 302 and / or computer system 320 during runtime. In one or more embodiments, software application 304 may be used to cause and / or initiate the compilation of the neural network model 200 and the (subsequent) execution of the executable / compiled neural network model 200.

[0045] Figure 4 This is a block diagram of an example architecture of the ONNX-MLIR compiler when compiling an ONNX model (e.g., neural network model 200) according to one or more embodiments of the present invention. Figure 4 In this context, names prefixed with "--" are designated as passes. Figure 4In this model, the input is an ONNX model (e.g., neural network model 200), and the output is a library containing compiled code (e.g., executable file 334). The output library may contain an entry function called "_dyn_entry_point_main_graph", whose input and output are similar to the input and output of the ONNX model, respectively. To perform inference with the output library, users write their programs to call the entry function by passing the input and obtaining the result. While fewer or more dialects may exist, there are five main dialects in ONNX-MLIR: ONNX, KRNL, affine, std, and LLVM, organized into four levels of abstraction. The first level of abstraction is a high-level representation of ONNX operations. The first level of abstraction includes operations in the ONNX and std dialects, where the ONNX dialect is represented via, for example,... The script or importer of another script is used to automatically generate the code. The second level of abstraction includes the krnl, affine, and std dialects. The krnl dialect provides a representation suitable for loop optimization, which can easily perform affine transformations such as tiling, skew, and permutation. The krnl dialect is used as an intermediate dialect to efficiently reduce the ONNX dialect to low-level dialects (e.g., affine, std, and llvm). The third level of abstraction includes the affine and std dialects, where existing optimized traversals in MLIR can be freely applied. The fourth level of abstraction includes only the llvm dialect, which is used to prepare for generating bit code (i.e., binary code) (i.e., executable 334).

[0046] MLIR traversals exist for converting one dialect to another, and for optimizing in a specific dialect. A multi-traversal compiler is a compiler that processes a program's source code or abstract syntax tree multiple times. This contrasts with a single-traversal compiler, which traverses the program only once. Each traversal takes the result of the previous traversal as input and produces intermediate output. In this way, the (intermediate) code is improved traversally until the final traversal produces the final code. The ONNX dialect is converted to the KRNL dialect via the traversal '--convert ONNX to KML'. Then, the KRNL dialect (except for some of its operations) is converted to the affine and std dialects via the traversal '--convert KML to LLVM'. The remaining operations in the KRNL dialect, as well as operations in the affine and std dialects, are directly converted to LLVM instructions via the traversal '--convert KML to LLVM'. Figure 4The right side shows the optimized traversals that can be performed in shadow mode at each level of abstraction. It should be understood that the list of optimized traversals is not exhaustive. As described in this paper, MLIR provides a scalable framework for transforming operations using familiar concepts of compiler traversal. Enabling arbitrary traversals on any set of operations presents significant scaling challenges, as each transformation must potentially take into account the semantics of any operation. However, MLIR addresses this complexity by allowing the use of traits and interfaces to abstractly describe the semantics of operations, thus allowing transformations to perform more generalized operations on operations. Traits typically describe validation constraints on valid IR, enabling the capture and inspection of complex invariants.

[0047] Compiler 322 is configured to be in Figure 7 Each stage of the compilation / conversion process shown generates a mapping file 330. For example, Figure 3 The mapping file 330 includes: an ONNX intermediate representation (IR) mapping file for converting ONNX models (e.g., neural network model 200 and / or artificial intelligence models) into ONNX IR (code); a KRNL IR mapping file for converting ONNX IR (code) into KRNL IR (code); an affine IR mapping file for converting KRNL IR (code) into affine IR (code); and an LLVM IR mapping file for converting affine IR (code) into LLVM IR (code). The compiler 322 generates a listing file 332 for converting the LLVM IR (code) into an executable file 334 (also referred to as an executable file, library, binary code, assembly language, etc.) that can be read and run by the hardware / software of computer system 302 and / or computer system 320. Figure 7 As shown. Therefore, executable file 334 is run / executed by computer system 302 (and / or computer system 320), which corresponds to running / executing neural network model 200 to perform as also. Figure 7 The inferences shown are illustrated. For example, neural network model 200 can be designed to receive incoming data and classify the data into outputs. During runtime, computer system 302 generates a data file 340 containing various information, including instructions, instruction addresses, performance metrics, etc., which can be used to help improve neural network model 200, as discussed further herein.

[0048] Figure 5 This is a flowchart of a computer-implemented process 500 for locating performance hotspots in a neural network model according to one or more embodiments of the present invention. The computer-implemented process 500 can be used... Figure 2 The computer system 302 in the middle is used to execute. Figure 5The computer implementation process in 500 will be referenced. Figure 2 To describe.

[0049] In block 502, after the execution of generating data file 340, software application 304 is configured to collect sample data 352 from data file 340. Sample data 352 may be a portion of data file 340. In one or more embodiments, sample data 352 may include the entire data file 304. Data file 340 is generated as a result of executing a previously compiled neural network model 200. Data file 340 includes instructions, instruction addresses of instructions, performance metrics for each instruction and / or instruction address, etc. Performance metrics include cache misses, information from hardware counters, information from software counters, CPU usage, memory usage, etc. Sample data 352 includes all types of information included in data file 340 and the number of sample occurrences. Different computer architectures may have different numbers of counters. For example, an architecture may have a number of counters from 10 to 1000. In one or more embodiments, data file 340 may be an ETB file, such as... Figure 7 As described in the control flow.

[0050] In box 504, software application 304 is configured to identify and recognize performance hotspots and corresponding hot instructions in sample data 352. To identify performance hotspots, software application 304 may include and / or invoke one or more diagnostic tools 306 to identify instructions that meet and / or exceed predetermined thresholds used for metrics and / or combinations of metrics. A performance hotspot is an instruction, operation, and / or code that causes computer system 302 (and / or computer system 320) to meet and / or exceed predetermined thresholds used for metrics. For example, a performance hotspot may refer to identifying one or more instructions, operations, and / or code segments that meet and / or exceed one or a combination of predetermined thresholds related to CPU usage (e.g., percentage of processor usage), execution time (e.g., ticks, clock, etc.), cache misses, memory usage, etc. In some cases, even if an instruction, operation, and / or code does not meet and / or exceed any predetermined threshold, a performance hotspot may still exceed the average performance metric compared to other instructions in sample data 352. For example, a performance hotspot may be outside the statistical average of performance metrics and / or combinations of performance metrics. Software application 304 and / or diagnostic tool 306 can parse sample data 352 to determine performance metrics that meet and / or exceed one or more predetermined thresholds, along with associated instructions. Performance hotspot analysis can be manual, automatic, and / or semi-automatic. In one or more embodiments, experienced performance analysts and developers use analysis tools to traverse different levels of performance hotspots and identify the most valuable segments for optimization. Software application 304 and / or diagnostic tool 306 can employ known methods and techniques for software diagnostics and performance as understood by those skilled in the art.

[0051] In box 506, software application 304 is configured to map LLVMIR locations to instructions in sample data 352 using listing file 332, where the instructions are located at instruction addresses, for example, in memory 308. The instructions are similar to... Figure 4 The executable code discussed in section 334 is runnable. The runnable instructions are from... Figure 4 As noted in Figure 7 The LLVM IR described herein is compiled from it. Therefore, there is a mapping relationship between the LLVM IR and instruction addresses. This relationship is represented by a file called listing file 332. Listing file 332 is generated when the LLVM IR is compiled into an executable file 334, and the executable file 334 is executable code, binary code, machine code, etc., that can be stored in memory 308. The LLVM IR is the lowest level code that is not binary code. The listing file may include the following example columns from Table 1.

[0052] ***Table 1***

[0053] Instruction address | Symbol name | Source file | Line number

[0054] 0x1000|add()|math.c|13

[0055] ************

[0056] In box 508, software application 304 is configured to use mapping file 330 to map LLVM IR locations and information to their corresponding locations in the original neural network model 200. The neural network model 200 (e.g., the original AI model) employs multiple transformation stages to convert the neural network model 200 into final runnable code (e.g., an executable file 334). Each transformation stage has a mapping file, and combinations of these mapping files can be used to transform LLVM IR back into the neural network model 200 (e.g., the original AI model). Each mapping file 330 contains relationships and transformations from the current intermediate representation back to a previous high-level intermediate representation and from the highest-level intermediate representation back to the neural network model 200, such as... Figure 7 As shown, software application 304 can, for example, map input code to output code to locate LLVM IR positions in the original AI model. For example, software application 304 is configured to use the LLVM IR mapping file of mapping file 330 to map LLVM IR positions and information to relevant affine IR positions in the affine IR (code). Similarly, software application 304 is configured to use the affine IR mapping file of mapping file 330 to map affine IR positions and information to relevant krnl IR positions in the krnl IR (code). Likewise, software application 304 is configured to use the krnl IR mapping file of mapping file 330 to map krnl IR positions and information to relevant ONNX IR positions in the ONNX IR (code). Software application 304 is also configured to use the ONNX IR mapping file of mapping file 330 to map ONNX IR positions and information back to the ONNX model, which is the neural network model 200 (i.e., the AI ​​mode). It can be seen that throughout the MLIR compilation stages, in reverse order (e.g., with... Figure 4 The reverse order and Figure 7 The instructions are traced in the reverse order of the transformations in the MLIR compiler. The mappings in each individual mapping file (e.g., LLVM IR mapping file, affine IR mapping file, krnl IR mapping file, and onnx IR mapping file) are used throughout the MLIR compilation phase to trace the address and / or operation of each instruction.

[0057] In box 510, software application 304 is configured to generate and display the MLIR performance matrix step-by-step for hot instructions that have been mapped to different intermediate instructions. Figure 6This is a block diagram of an example MIIR performance matrix showing relevant performance hotspots at different compilation levels according to one or more embodiments of the present invention. The MLIR performance matrix can be displayed on a display such as display 119, allowing a user to quickly visualize which operations can be optimized to correct one or more hotspots. Software application 304 generates the MLIR performance matrix using the instruction addresses of sample data 352 combined with a mapping file 330 that may include instruction addresses. The performance matrix shows performance hotspots displayed at different levels using a dotted pattern, allowing the user to select and / or the software application 304 to select the optimal level (and operation) for optimization. For example, software application 304 may select optimization at LLVMIR, affine IR, krnl IR, and / or ONNX IR. In LLVM IR, affine IR, krnl IR, and / or ONNX IR, the selected optimization may include dialect operations and / or traversal. LLVM IR is closest to the instructions in sample data 352, and the performance matrix progresses upwards through ONNX IR. Neural network model 200 is not shown in the performance matrix; however, it should be understood that higher levels including nodes of neural network model 200 can be shown; some of these nodes will be performance hotspots. In one or more embodiments, software application 304 may be configured to optimize and / or invoke optimization tools (not shown) to optimize. Figure 6 The software application 304 can instruct the user to perform optimizations at one or more levels based on the performance hotspots depicted in the performance matrix. Some users may find it more familiar or comfortable to make modifications at specific levels. Furthermore, some levels can be more specific to fixing performance hotspots and / or aspects of performance hotspots without potentially causing unforeseen problems. For example, optimizing and / or fixing smaller or less complex operations at an intermediate level (while not optimizing other performance hotspots at the same level) can improve performance and cause unforeseen problems. However, optimizing nodes at a neural network model can cause potential problems when the neural network model is compiled. For example, after MLIR compilation, there are thousands of hot instructions and hot basic blocks distributed across millions of instructions. If these instructions are aggregated at higher levels, there may only be a few hot operations like "std.add" and "matrix bias add". In this way, the user can optimize these hot operations only through software tuning or hardware acceleration to achieve the greatest performance improvement with minimal optimization effort.

[0058] exist Figure 6In this example, the values ​​for operations at various compilation levels can be CPU usage, execution time, cache misses, etc., with higher values ​​indicating worse performance. The threshold for performance metrics could be, for example, 20%, so that each operation with a value of 20% or higher is a performance hotspot. By displaying the values ​​for operation names at each level (e.g., LLVM IR, Affine IR, KRNL IR, and / or ONNX IR), users can easily and quickly visualize where improvements and optimizations can be made. For example, based on the instruction address-to-name mapping provided by mapping file 330, discrete instruction address ranges 0x10f-0x120 and 0x204-x228 belong to... Figure 6 The "llvm.add" node is used. Then, samples at addresses 0x112, 0x120, 0x204, 0x208, etc., are aggregated into the llvm.add node. The aggregated sample count in llvm.add is 40% of the total sample count. Based on mapping nodes to llvm nodes using mapping file 330, the llvm.add node is transformed into a std.addf node. Therefore, all llvm.add sample numbers are aggregated into the std.addf node. The std.addf node is the hottest node in the affine traversal, such as... Figure 6 As shown.

[0059] Figure 7 This is a block diagram of a computer-implemented control flow for optimizing performance hotspots in a neural network model according to one or more embodiments of the present invention. In one or more embodiments, the computer-implemented control flow can be executed by computer system 302. In one or more embodiments, a portion of the computer-implemented control flow (such as the compilation and execution of neural network model 200) can be executed by computer system 320, and a portion (such as identifying and locating performance hotspots) can be executed by computer system 302. During compilation, the compilation process generates mapping IA files (e.g., as mapping file 330) and listing files (e.g., listing file 332). During execution, computer system 302 uses sampling tools such as collecting data files (e.g., data file 340). After execution by computer system 302, computer system 302 uses the mapping IA files, listing files, and data files to locate performance hotspots.

[0060] Figure 8 This is a flowchart of a computer implementation process 800 for locating performance hotspots in a neural network model and optimizing one or more performance hotspots according to one or more embodiments of the present invention. Figure 8 The computer-implemented process 800 in the middle can be by Figure 3 The system 300 is executed, and will refer to Figure 3 To describe.

[0061] In box 802, software application 304 on computer system 302 is configured to collect sample data (e.g., sample data 352) having instruction addresses (IA) for use in neural network model 200. Sample data 352 may come from data file 340 generated during execution of neural network model 200 (particularly during execution of compiled neural network model 200).

[0062] In block 804, software application 304 on computer system 302 is configured to identify instructions at instruction addresses that are performance hotspots. For example, software application 304 may include functionality and / or invoking / instructing one or more known diagnostic tools 306 configured to identify performance hotspots based on performance metrics.

[0063] At box 806, software application 304 on computer system 302 is configured to use listing file 332 to map instructions of sample data 352, which is a performance hotspot, to locations in a low-level intermediate representation. The low-level intermediate representation may be LLVM IR (code).

[0064] In box 808, software application 304 on computer system 302 is configured to use a mapping file (e.g., mapping file 330) to map the locations of low-level intermediate representations that serve as performance hotspots to operations in one or more high-level (intermediate) representations (e.g., affine IR, krn1IR, ONNX IR, etc.), one or more of which correspond to performance hotspots, the mapping file 330 being generated from the compiled neural network model 200.

[0065] Software application 304 on computer system 302 is configured to map operations in one or more high-level representations to nodes in neural network model 200 using mapping file 330. Software application 304 on computer system 302 is configured to determine which nodes in neural network model 200 represent performance hotspots based on mappings from one or more high-level (intermediate) representations mapped from low-level intermediate representations mapped from sample data 352.

[0066] Software application 304 on computer system 302 is configured to classify performance hotspots into different levels of the compiled neural network model 200 using mapping file 330. These different levels range from low-level intermediate representations to one or more high-level representations. The different levels may include LLVMIR, affine IR, krnl IR, and / or ONNX IR. Sample data (e.g., sample data 352) further includes information from one or more counters. Information regarding the execution of the neural network model 200 from hardware and / or software counters may be stored in data file 340 for which sample data has been collected. Performance hotspots include one or more (performance) metrics that meet and / or exceed one or more thresholds (e.g., performance metric thresholds). Software application 304 on computer system 302 is configured to determine one or more operations in one or more high-level representations for optimization to address performance hotspots.

[0067] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings described herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0068] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0069] The features are as follows:

[0070] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring manual interaction with the service provider.

[0071] Extensive network access: Capabilities are available on the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0072] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. Location independence has significance because consumers typically do not control or know the exact location of the resources provided, but can specify the location at a high level of abstraction (e.g., country, state, or data center).

[0073] Rapid Flexibility: In some cases, the ability to scale outwards and inwards quickly and flexibly can be provided. For consumers, the available capacity often appears unlimited and can be purchased in any quantity at any time.

[0074] Measurement services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both service providers and consumers.

[0075] The service model is as follows:

[0076] Software as a Service (SaaS): The capability offered to consumers is the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from various client devices through thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0077] Platform as a Service (PaaS): This provides consumers with the ability to deploy consumer-created or acquired applications onto cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environments.

[0078] Infrastructure as a Service (IaaS): This provides consumers with the capability to deliver processing, storage, networking, and other basic computing resources that enable them to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they do have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0079] The deployment model is as follows:

[0080] Private cloud: Cloud infrastructure operated solely by an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0081] Community cloud: Cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0082] Public cloud: Cloud infrastructure available to the general public or large industrial groups and owned by organizations that sell cloud services.

[0083] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported together (e.g., cloud bursting for load balancing between clouds).

[0084] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0085] Now for reference Figure 9 The diagram illustrates an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 to which local computing devices used by cloud consumers can communicate. These local computing devices include, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide Infrastructure as a Service, Platform as a Service, and Software as a Service without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 9 The types of computing devices 54A-N shown are for illustrative purposes only, and computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network and network-addressable connection (e.g., using a web browser).

[0086] Now for reference Figure 10 This demonstrates a cloud computing environment of 50 ( Figure 9 This provides a set of functional abstractions. It should be understood beforehand that... Figure 10 The components, layers, and functions shown are for illustrative purposes only, and embodiments of the invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0087] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a host 61; a server 62 based on a RISC (Reduced Instruction Set Computer) architecture; a server 63; a blade server 64; a storage device 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0088] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0089] In one example, management layer 80 may provide the following functionalities: Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 84 provides cloud resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 85 provides pre-scheduling and procurement of cloud resources, anticipating future requirements for those resources according to the SLA.

[0090] Workload layer 90 provides examples of functionalities that can leverage a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis and processing 94; transaction processing 95; and software applications implemented within workloads and functionalities (e.g., software application 304, compiler 322, neural network model 200, etc.) 96. Furthermore, software applications can work and / or integrate with resource provisioning 81.

[0091] Various embodiments of the invention are described herein with reference to the accompanying drawings. Alternative embodiments of the invention may be devised without departing from the scope thereof. In the following description and drawings, various connections and positional relationships (e.g., above, below, adjacent, etc.) are illustrated between elements. Unless otherwise stated, these connections and / or positional relationships may be direct or indirect, and the invention is not intended to be limiting in this respect. Therefore, coupling of entities may refer to direct or indirect coupling, and positional relationships between entities may be direct or indirect positional relationships. Furthermore, the various tasks and process steps described herein may be incorporated into a more comprehensive program or process with additional steps or functionality not described in detail herein.

[0092] One or more methods described herein can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logic functions based on data signals, application-specific integrated circuits (ASICs) having appropriately combined logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0093] For the sake of brevity, conventional techniques related to the manufacture and use of aspects of the present invention may or may not be described in detail herein. In particular, various aspects of the computing systems and specific computer programs used to implement the various technical features described herein are well known. Therefore, for the sake of brevity, many conventional implementation details are only briefly mentioned or omitted entirely herein, without providing well-known system and / or process details.

[0094] In some embodiments, various functions or actions may occur at a given location and / or in combination with the operation of one or more devices or systems. In some embodiments, a portion of a given function or action may be performed at a first device or location, and the remainder of the function or action may be performed at one or more additional devices or locations.

[0095] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0096] All the means or steps plus functional elements in the following claims are intended to include any structure, material, action, and equivalent for performing a function in combination with other claimed elements, as specifically claimed. This disclosure is presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of this disclosure. The embodiments were chosen and described in order to best explain the principles and practical application of this disclosure, and to enable others skilled in the art to understand the various embodiments of this disclosure with various modifications suitable for the particular intended use.

[0097] The diagrams described herein are illustrative. Numerous variations of the diagrams or steps (or operations) described herein may exist without departing from this disclosure. For example, actions may be performed in a different order, or actions may be added, deleted, or modified. Furthermore, the term "coupled" describes a signal path between two elements and does not imply a direct connection between elements without intermediate elements / connections. All such variations are considered part of this disclosure.

[0098] The following definitions and abbreviations are used to interpret the claims and specification. As used herein, the terms “comprising,” “including,” “having,” “containing,” or any other variations thereof are intended to cover a non-exclusive inclusion. For example, a composition, mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such compositions, mixtures, processes, methods, articles, or apparatus.

[0099] Additionally, the term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms "at least one" and "one or more" are understood to include any integer greater than or equal to one, i.e., one, two, three, four, etc. The term "multiple" should be understood to include any integer greater than or equal to two, i.e., two, three, four, five, etc. The term "connection" can include both indirect "connection" and direct "connection."

[0100] The terms “about,” “substantially,” “approximately,” and variations thereof are intended to include a degree of error associated with a measurement of a specific quantity based on the equipment available at the time of filing this application. For example, “about” may include a range of ±8%, 5%, or 2% of a given value.

[0101] Embodiments of the present invention may be systems, methods, and / or computer program products at any possible level of technical detail integration. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0102] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or recessed structures with instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0103] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0104] Computer-readable program instructions for performing the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including local area network (LAN) or wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing the status information of the computer-readable program instructions.

[0105] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0106] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0107] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational arrangements to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions indicated in the blocks may occur in a non-consecutive order as shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0109] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements relative to commercially available technologies, or to enable others skilled in the art to understand the embodiments described herein.

Claims

1. A computer-implemented method comprising: collecting sample data with instruction addresses, the sample data for a neural network model; determining instructions in the instruction addresses that are performance hotspots; mapping the instructions of the sample data that are performance hotspots to locations in a low-level intermediate representation using a listing file; and mapping the locations of the low-level intermediate representation that are performance hotspots to operations in one or more high-level representations using a mapping file, one or more of the operations corresponding to the performance hotspots, the mapping file generated from compiling the neural network model.

2. The computer-implemented method of claim 1, further comprising: mapping the operations in the one or more high-level representations to nodes in the neural network model using the mapping file.

3. The computer-implemented method of claim 1, further comprising: determining which nodes in the neural network model represent the performance hotspots based on mappings from the one or more high-level representations, the one or more high-level representations mapped from the low-level intermediate representation, the low-level intermediate representation mapped from the sample data.

4. The computer-implemented method of claim 1, further comprising: classifying the performance hotspots in different levels of compiling the neural network model using the mapping file, the different levels including the low-level intermediate representation to the one or more high-level representations.

5. The computer-implemented method of claim 1, wherein, the sample data further comprising information from one or more counters.

6. The computer-implemented method of claim 1, wherein, the performance hotspots comprising one or more metrics that meet or exceed one or more thresholds.

7. The computer-implemented method of claim 1, further comprising: determining one or more of the operations in the one or more high-level representations to optimize to address the performance hotspots.

8. A system comprising: a memory having computer-readable instructions; and one or more processors to execute the computer-readable instructions, the computer-readable instructions controlling the one or more processors to perform a process comprising: collecting sample data with instruction addresses, the sample data for a neural network model; determining instructions in the instruction addresses that are performance hotspots; mapping the instructions of the sample data that are performance hotspots to locations in a low-level intermediate representation using a listing file; and mapping the locations of the low-level intermediate representation that are performance hotspots to operations in one or more high-level representations using a mapping file, the mapping file generated from compiling the neural network model.

9. The system of claim 8, further comprising: mapping the operations in the one or more high-level representations to nodes in the neural network model using the mapping file.

10. The system of claim 8, further comprising: determining which nodes in the neural network model represent the performance hotspots based on mappings from the one or more high-level representations, the one or more high-level representations mapped from the low-level intermediate representation, the low-level intermediate representation mapped from the sample data.

11. The system of claim 8, further comprising: classifying the performance hotspots in different levels of compiling the neural network model using the mapping file, the different levels including the low-level intermediate representation to the one or more high-level representations.

12. The system of claim 8, wherein, the sample data further comprising information from one or more counters.

13. The system of claim 8, wherein, the performance hotspots comprising one or more metrics that meet or exceed one or more thresholds.

14. The system of claim 8, further comprising: determining one or more of the operations in the one or more high-level representations to optimize to address the performance hotspot.

15. A computer program product comprising a computer readable storage medium, the computer readable storage medium containing program instructions executable by one or more processors for causing the one or more processors to perform a process, the process comprising: collecting sample data with instruction addresses, the sample data for a neural network model; determining instructions in the instruction addresses that are performance hotspots; mapping the instructions of the sample data that are performance hotspots to locations in a low-level intermediate representation using a list file; and mapping the locations of the low-level intermediate representation to operations in one or more high-level representations using a mapping file, the mapping file generated from compiling the neural network model.

16. The computer program product of claim 15, further comprising: mapping the operations in the one or more high-level representations to nodes in the neural network model using the mapping file.

17. The computer program product of claim 15, further comprising: determining which nodes in the neural network model represent the performance hotspots based on the mapping from the one or more high-level representations, the one or more high-level representations mapped from the low-level intermediate representation, the low-level intermediate representation mapped from the sample data.

18. The computer program product of claim 15, further comprising: categorizing the performance hotspots in different levels of compiling the neural network model using the mapping file, the different levels including the low-level intermediate representation to the one or more high-level representations.

19. The computer program product of claim 15, wherein, the sample data further comprises information from one or more counters.

20. The computer program product of claim 15, wherein, the performance hotspots include one or more metrics that meet or exceed one or more thresholds.

Images