Instruction set optimization and compilation scheduling methods and systems for artificial intelligence chips

Abstract

This invention provides an instruction set optimization and compilation scheduling method and system for artificial intelligence chips, relating to the field of artificial intelligence technology. It includes constructing a cross-block instruction move legality determination mechanism by performing multi-level abstract modeling of the neural network computation graph, analyzing the spatiotemporal locality characteristics of instruction sequences to establish a data reuse pattern, pre-planning data migration paths and co-mapping them with the on-chip interconnect network, and finally generating target code and dynamically adjusting it. This invention can improve the computational efficiency of AI chips, reduce power consumption, optimize resource utilization, and enhance chip adaptability.

Description

Technical Field

[0001] This invention relates to artificial intelligence technology, and more particularly to instruction set optimization and compilation scheduling methods and systems for artificial intelligence chips. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning has achieved significant breakthroughs in fields such as computer vision and natural language processing. To meet the ever-increasing computational demands of neural network models, various dedicated AI chips have emerged. These chips typically employ heterogeneous computing architectures, containing multiple types of computing units and complex multi-level storage hierarchies to provide efficient neural network computing capabilities. However, the key to fully leveraging the performance of AI chips lies in compilation technology, which requires efficiently mapping high-level neural network models to underlying hardware resources.

[0003] Traditional compilation methods are primarily designed for general-purpose processors and cannot fully utilize the unique architectural characteristics of AI chips. Existing AI chip compilation technologies suffer from several major problems: First, current compilation technologies lack an effective semantic mapping mechanism between neural network computation graphs and chip instruction sets. This results in an inaccurate capture of computational semantics during operator-to-instruction conversion, leading to wasted computational resources and performance degradation. Second, existing methods often employ static scheduling strategies when dealing with heterogeneous computing units, failing to dynamically adjust instruction allocation based on runtime states. This makes it difficult to adapt to the optimal execution path under different computational modes, limiting the overall computational efficiency of the chip. Furthermore, traditional data management methods mainly rely on hardware-level caching mechanisms, lacking the ability to globally plan data movement during the compilation phase. This prevents optimization for specific data access patterns in neural network computations, leading to frequent data movement and severe memory access bottlenecks.

[0004] As neural network models become increasingly complex and artificial intelligence applications diversify, there is an urgent need for an instruction set optimization and compilation scheduling method for artificial intelligence chips. This method should be able to achieve precise mapping from the computation graph to the instruction level, support dynamic instruction scheduling between heterogeneous computing units, and optimize the movement of data between multi-level storage layers, thereby fully realizing the performance potential of artificial intelligence chips. Summary of the Invention

[0005] This invention provides an instruction set optimization and compilation scheduling method and system for artificial intelligence chips, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides an instruction set optimization and compilation scheduling method for artificial intelligence chips, comprising:

[0007] The neural network computation graph to be compiled and the architecture description information of the target artificial intelligence chip are obtained. Based on the architecture description information, the neural network computation graph is subjected to multi-level abstract modeling. The semantic expression of the operator layer is converted into the operation primitives of the instruction set architecture layer, thereby forming an intermediate representation containing dual constraints of control flow and data flow.

[0008] Based on the intermediate representation, a mechanism for determining the legality of instruction movement across basic blocks is constructed, and instructions are dynamically allocated by utilizing the functional complementarity between heterogeneous computing units, provided that the semantic equivalence of the program is satisfied.

[0009] By analyzing the spatiotemporal locality characteristics of instruction sequences, an abstract representation of data reuse patterns is established. During the compilation phase, the migration path of data between multiple storage levels is pre-planned, and the migration path is co-mapped with the communication topology of the on-chip interconnect network.

[0010] Target code is generated based on the migration path, the target code is executed on the target AI chip, and the abstract representation of the instruction movement legality determination mechanism and the data reuse mode is dynamically adjusted based on the performance feedback information during the execution process.

[0011] Based on the architecture description information, the neural network computation graph is subjected to multi-level abstract modeling, and the semantic expression of the operator layer is converted into the operation primitives of the instruction set architecture layer, thereby forming an intermediate representation containing both control flow and data flow constraints, including:

[0012] Semantic decomposition is performed on the operator nodes in the neural network computation graph to extract the computation kernel features and data access patterns of each operator node, and a hierarchical mapping model is constructed based on the instruction set semantics of heterogeneous computing units in the architecture description information.

[0013] Based on the intermediate abstraction layer in the hierarchical mapping model, dependency propagation analysis is performed on operator nodes with data flow dependencies, converting the tensor dependencies of the operator layer into computational mode dependencies, and further refining them into data read / write dependencies.

[0014] For the control flow structure in the neural network computation graph, an abstract representation of the control flow is established in the intermediate abstract layer. Based on the abstract representation, the conditional decision logic is associated with the data read / write dependency, so that the control flow constraints and data flow constraints form a unified execution constraint graph at the operation primitive layer. The data read / write dependency and the execution constraint graph are structurally encapsulated to form an intermediate representation.

[0015] Based on the intermediate abstraction layer in the hierarchical mapping model, dependency propagation analysis is performed on operator nodes with data flow dependencies. The tensor dependencies of the operator layer are converted into computational pattern dependencies, and further refined into data read / write dependencies, including:

[0016] The computational pattern representation of the intermediate abstract layer is extracted from the hierarchical mapping model, the data objects and their access attributes involved in the computational pattern representation are identified, and a tensor dependency propagation chain is constructed based on the tensor connection relationship between operator nodes in the neural network computation graph.

[0017] Traverse each path in the tensor dependency propagation chain, map the tensor production and consumption relationship between adjacent operator nodes on the path to the data object transfer relationship between the corresponding computational pattern representations, and concatenate the data object transfer relationship according to the path order to form a computational pattern dependency chain;

[0018] Based on the computational pattern dependency chain, the access sequence of data objects when each computational pattern is represented as an operation primitive sequence is analyzed. The execution order of read and write operations on the same data object in the operation primitive sequence is identified. Based on the data object transit relationship in the computational pattern dependency chain, the data object transit across computational patterns is refined into data read and write dependencies across operation primitives.

[0019] Based on the intermediate representation, a mechanism for determining the legality of instruction moves across basic blocks is constructed. Under the premise of satisfying program semantic equivalence, instructions are dynamically allocated by leveraging the functional complementarity between heterogeneous computing units, including:

[0020] The operation primitive sequence in the intermediate representation is divided into basic blocks, and a dependency constraint matrix across basic blocks is constructed based on the data read / write dependency and control dependency in the execution constraint graph.

[0021] Based on the dependency constraint matrix, the change in dependency direction of the candidate operation primitive to be moved from the source basic block to the target basic block is analyzed. When the dependency direction remains unchanged and does not cause a circular dependency, the move operation is determined to be a legal move.

[0022] For candidate operation primitives determined to be legitimate moves, a subset of heterogeneous computing units capable of executing the candidate operation primitives is identified based on the architecture description information. Within the subset of heterogeneous computing units, the computing unit that enables the candidate operation primitive to complete execution earliest is selected and dynamically allocated according to the performance characteristics and dependency strength of each heterogeneous computing unit.

[0023] An abstract representation of data reuse patterns is established by analyzing the spatiotemporal locality characteristics of instruction sequences. During the compilation phase, data migration paths between multiple storage levels are pre-planned, and these migration paths are collaboratively mapped with the communication topology of the on-chip interconnect network.

[0024] The spatiotemporal distribution characteristics of data access operations in the instruction sequence are analyzed. The access frequency and access interval of statistical objects are statistically analyzed in the time dimension and quantified as the temporal locality intensity. In the spatial dimension, the set of data objects with continuous addresses or fixed step sizes is identified and quantified as the spatial locality intensity. The temporal locality intensity and the spatial locality intensity are combined to construct an abstract representation of the data reuse pattern.

[0025] Based on the abstract representation of the data reuse pattern, the target storage level in the multi-level storage hierarchy is determined for each data object by the intensity of temporal locality, and the migration time from the source storage level to the target storage level is determined by the access interval, thereby planning the migration path of the data object between the multi-level storage hierarchy.

[0026] Based on the communication topology of the on-chip interconnect network in the architecture description information, a communication link that meets the data transmission bandwidth requirements is allocated to the migration path, and the migration operation of data objects with spatial proximity is mapped to the same communication link based on the spatial locality strength, thereby realizing the cooperative mapping between the migration path and the communication topology.

[0027] Target code is generated based on the migration path, the target code is executed on the target AI chip, and the abstract representation of the instruction move legality determination mechanism and the data reuse mode is dynamically adjusted based on performance feedback information during execution. This includes:

[0028] Based on the migration path, the sequence of operation primitives and data transfer operations in the intermediate representation are converted into the machine instruction sequence of the target artificial intelligence chip, and performance monitoring instructions are inserted into the machine instruction sequence to generate target code;

[0029] The target code is executed on the target artificial intelligence chip, and the execution latency and communication latency are collected through the performance monitoring instructions to generate performance feedback information.

[0030] Based on the execution latency in the performance feedback information, a bias analysis is performed on the dependency constraint matrix in the instruction move legality determination mechanism, and the dependency strength is updated. Based on the communication latency in the performance feedback information, an error analysis is performed on the temporal locality strength in the abstract representation of the data reuse pattern, and the temporal locality strength is updated. According to the updated dependency strength and the temporal locality strength, the instruction move legality determination mechanism and the abstract representation of the data reuse pattern are dynamically adjusted.

[0031] Based on the migration path, the sequence of operation primitives and data transfer operations in the intermediate representation are converted into a sequence of machine instructions for the target AI chip. Performance monitoring instructions are then inserted into this machine instruction sequence to generate target code, including:

[0032] Based on the target storage level and migration time planned in the migration path, the data transfer operation is converted into data transmission instructions for the target artificial intelligence chip, and the operation primitive sequence in the intermediate representation is converted into the calculation instructions of the target artificial intelligence chip. According to the migration time, the data transmission instructions and the calculation instructions are sequentially arranged to form a machine instruction sequence.

[0033] A pair of computation monitoring instructions is inserted before and after the execution position of the computation instruction in the machine instruction sequence, and a pair of transmission monitoring instructions is inserted before and after the execution position of the data transmission instruction in the machine instruction sequence. The pair of computation monitoring instructions and the pair of transmission monitoring instructions constitute a performance monitoring instruction.

[0034] The set of instructions containing the machine instruction sequence and the performance monitoring instructions is used as the target code.

[0035] A second aspect of this invention provides an instruction set optimization and compilation scheduling system for artificial intelligence chips, comprising:

[0036] The modeling unit is used to acquire the neural network computation graph to be compiled and the architecture description information of the target artificial intelligence chip, and to perform multi-level abstract modeling of the neural network computation graph based on the architecture description information, converting the semantic expression of the operator layer into the operation primitives of the instruction set architecture layer, thereby forming an intermediate representation containing dual constraints of control flow and data flow.

[0037] The allocation unit is used to construct a cross-basic block instruction move legality determination mechanism based on the intermediate representation, and dynamically allocate instructions by utilizing the functional complementarity between heterogeneous computing units, provided that the program semantic equivalence is satisfied.

[0038] The mapping unit is used to establish an abstract representation of the data reuse pattern by analyzing the spatiotemporal locality characteristics of the instruction sequence, pre-plan the migration path of data between multiple storage levels during the compilation stage, and coordinate the migration path with the communication topology of the on-chip interconnect network.

[0039] An adjustment unit is used to generate target code according to the migration path, execute the target code on the target artificial intelligence chip, and dynamically adjust the abstract representation of the instruction movement legality determination mechanism and the data reuse mode according to the performance feedback information during the execution process.

[0040] A third aspect of the present invention provides an electronic device, comprising:

[0041] processor;

[0042] Memory used to store processor-executable instructions;

[0043] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0044] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0045] The beneficial effects of this application are as follows:

[0046] By performing multi-level abstract modeling of the neural network computation graph, the semantics of the operator layer are converted into the operation primitives of the instruction set architecture layer, forming an intermediate representation containing dual constraints of control flow and data flow. This solves the problem of insufficient adaptability of traditional compilation methods when dealing with AI-specific computational patterns.

[0047] An innovative mechanism for determining the legality of instruction moves across basic blocks was constructed. Combined with the functional complementarity between heterogeneous computing units, dynamic instruction allocation was carried out, which effectively improved instruction-level parallelism and hardware resource utilization, and reduced execution latency.

[0048] By analyzing the spatiotemporal locality characteristics of instruction sequences to establish an abstract representation of data reuse patterns, and by pre-planning data migration paths during the compilation phase, the problems of high data handling overhead and high memory access latency in traditional methods are solved, significantly improving storage efficiency.

[0049] By co-mapping the data migration path with the communication topology of the on-chip interconnect network, data transmission bottlenecks are reduced and on-chip network bandwidth utilization is optimized. Dynamically adjusting the compilation strategy based on performance feedback during execution enables adaptive optimization, allowing the system to continuously adjust and improve according to actual operating conditions, further enhancing the overall performance and energy efficiency of the AI ​​chip. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the instruction set optimization and compilation scheduling method for artificial intelligence chips according to an embodiment of the present invention;

[0051] Figure 2 This is a flowchart illustrating the data dependency analysis process for neural network computation patterns in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0054] Figure 1 This is a flowchart illustrating the instruction set optimization and compilation scheduling method for artificial intelligence chips according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0055] The neural network computation graph to be compiled and the architecture description information of the target artificial intelligence chip are obtained. Based on the architecture description information, the neural network computation graph is subjected to multi-level abstract modeling. The semantic expression of the operator layer is converted into the operation primitives of the instruction set architecture layer, thereby forming an intermediate representation containing dual constraints of control flow and data flow.

[0056] Based on the intermediate representation, a mechanism for determining the legality of instruction movement across basic blocks is constructed, and instructions are dynamically allocated by utilizing the functional complementarity between heterogeneous computing units, provided that the semantic equivalence of the program is satisfied.

[0057] By analyzing the spatiotemporal locality characteristics of instruction sequences, an abstract representation of data reuse patterns is established. During the compilation phase, the migration path of data between multiple storage levels is pre-planned, and the migration path is co-mapped with the communication topology of the on-chip interconnect network.

[0058] Target code is generated based on the migration path, the target code is executed on the target AI chip, and the abstract representation of the instruction movement legality determination mechanism and the data reuse mode is dynamically adjusted based on the performance feedback information during the execution process.

[0059] In one optional implementation, the neural network computation graph is modeled using multi-level abstraction based on the architecture description information, converting the semantic expressions of the operator layer into operational primitives of the instruction set architecture layer, thereby forming an intermediate representation containing both control flow and data flow constraints, including:

[0060] Semantic decomposition is performed on the operator nodes in the neural network computation graph to extract the computation kernel features and data access patterns of each operator node, and a hierarchical mapping model is constructed based on the instruction set semantics of heterogeneous computing units in the architecture description information.

[0061] Based on the intermediate abstraction layer in the hierarchical mapping model, dependency propagation analysis is performed on operator nodes with data flow dependencies, converting the tensor dependencies of the operator layer into computational mode dependencies, and further refining them into data read / write dependencies.

[0062] For the control flow structure in the neural network computation graph, an abstract representation of the control flow is established in the intermediate abstract layer. Based on the abstract representation, the conditional decision logic is associated with the data read / write dependency, so that the control flow constraints and data flow constraints form a unified execution constraint graph at the operation primitive layer. The data read / write dependency and the execution constraint graph are structurally encapsulated to form an intermediate representation.

[0063] Semantic decomposition is performed on operator nodes in the neural network computation graph. Specifically, two types of key features are extracted for each operator node: computational kernel features and data access patterns. Computational kernel features include the type of mathematical operation performed by the operator (such as convolution, matrix multiplication, activation function, etc.), computational precision requirements, and parallelism characteristics; data access patterns describe how the operator accesses input and output tensors, including sequential access, step access, or random access patterns.

[0064] Based on feature extraction, a hierarchical mapping model is constructed by combining the instruction set semantics of heterogeneous computing units in the architecture description information. This mapping model consists of three layers: an operator semantic layer, an intermediate abstraction layer, and an instruction set architecture layer. The operator semantic layer describes the high-level operators defined by the neural network framework; the intermediate abstraction layer, as a bridging layer, contains architecture-independent computational patterns and data access primitives; and the instruction set architecture layer contains the specific instruction operations supported by the target hardware platform. For example, a convolution operator is mapped to a combination of several tensor traversal patterns and multiply-accumulate primitives, and then mapped to the SIMD instructions or tensor core instructions of the specific hardware.

[0065] Based on the constructed hierarchical mapping model, dependency propagation analysis is performed on operator nodes with data flow dependencies in the neural network computation graph to identify tensor dependencies between operators, i.e., which operators' output tensors are used as inputs by other operators. Then, an intermediate abstraction layer transforms these tensor-level dependencies into finer-grained computational pattern dependencies. For example, the dependency between two consecutive convolutional layers can be transformed into a production-consumption relationship of tensor elements within a specific region. Finally, the computational pattern dependencies are refined into data read-write dependencies, precisely describing the write-read order constraints of data elements in memory.

[0066] When processing the control flow structure of a neural network computation graph, it is necessary to establish an abstract representation of the control flow in an intermediate abstraction layer to identify control flow structures such as conditional branches (e.g., IF-THEN-ELSE structures) and loop structures (e.g., WHILE and FOR loops) in the computation graph. Then, a corresponding abstract representation is constructed for each control flow structure, including condition decision nodes, branch paths, and merging points.

[0067] By associating the abstract representation of control flow with the previously obtained data read / write dependencies, the unification of control flow constraints and data flow constraints is achieved. In this process, conditional decision logic is transformed into execution conditions for specific data read / write operations. For example, in an IF-THEN-ELSE structure, the evaluation result of the conditional expression determines which branch path's data read / write operation will be executed. Through this association, an execution constraint graph at the operation primitive level is formed, which fully describes the order of operation execution and the conditional constraints.

[0068] The data read / write dependencies and execution constraint graphs are encapsulated in a structured manner to form a unified intermediate representation. This intermediate representation adopts a directed graph structure, where nodes represent basic operation primitives and edges represent data dependencies or control dependencies. Each node contains attribute information such as operation type, input / output data description, and execution conditions; edges carry information such as dependency type (data dependency or control dependency) and dependency strength.

[0069] In practical applications, this method can be used for cross-platform deployment of deep learning models. For example, for image classification networks with branching structures, it first extracts the computational features and access patterns of each operator (such as convolution, pooling, and fully connected layers), and then constructs a mapping model based on the instruction set semantics of the target hardware (such as a specific GPU or NPU). Through dependency analysis, it identifies the data flow paths between convolutional layers and the execution constraints of different paths in conditional branches. The resulting intermediate representation contains both the execution constraints of the regular data processing paths and the execution conditions of different branches after conditional judgments, providing complete information for subsequent code generation and optimization.

[0070] This multi-level abstract modeling method successfully transformed the semantics of the operator layer into the operational primitives of the instruction set architecture layer, and constructed an intermediate representation of the neural network computation graph that simultaneously contains dual constraints of control flow and data flow, laying the foundation for the efficient execution of neural networks in heterogeneous computing environments.

[0071] In one optional implementation, based on the intermediate abstraction layer in the hierarchical mapping model, dependency propagation analysis is performed on operator nodes with data flow dependencies, converting the tensor dependencies of the operator layer into computational pattern dependencies, and refining them into data read / write dependencies, including:

[0072] The computational pattern representation of the intermediate abstract layer is extracted from the hierarchical mapping model, the data objects and their access attributes involved in the computational pattern representation are identified, and a tensor dependency propagation chain is constructed based on the tensor connection relationship between operator nodes in the neural network computation graph.

[0073] Traverse each path in the tensor dependency propagation chain, map the tensor production and consumption relationship between adjacent operator nodes on the path to the data object transfer relationship between the corresponding computational pattern representations, and concatenate the data object transfer relationship according to the path order to form a computational pattern dependency chain;

[0074] Based on the computational pattern dependency chain, the access sequence of data objects when each computational pattern is represented as an operation primitive sequence is analyzed. The execution order of read and write operations on the same data object in the operation primitive sequence is identified. Based on the data object transit relationship in the computational pattern dependency chain, the data object transit across computational patterns is refined into data read and write dependencies across operation primitives.

[0075] like Figure 2 As shown, the method includes:

[0076] In the process of neural network compilation and optimization, accurate analysis of the dependencies between operator nodes is crucial for achieving efficient execution. Based on the intermediate abstraction layer in the hierarchical mapping model, dependency propagation analysis can be performed on operator nodes with data flow dependencies, transforming high-level tensor dependencies into more specific data read / write dependencies.

[0077] The computational pattern representation of the intermediate abstraction layer is extracted from the hierarchical mapping model. Each operator node in the neural network computation graph, after being mapped to the intermediate abstraction layer, forms a specific computational pattern representation, which includes computational logic and data access patterns. For example, a convolution operator can be mapped to a nested loop computational pattern, containing data objects such as input feature maps, weights, and output feature maps. For each computational pattern, the involved data objects and their access attributes are identified, including the dimensional information of the data objects, access patterns (sequential access, random access, etc.), and read / write attributes (read-only, write-only, read / write). Subsequently, based on the tensor connections between operator nodes in the neural network computation graph, a tensor dependency propagation chain is constructed. This propagation chain describes how tensor data flows from one operator to another, forming a complete data flow path.

[0078] Taking a simple neural network segment as an example, suppose there is a convolution operator A, a batch normalization operator B, and a ReLU operator C connected in sequence. The constructed tensor dependency propagation chain is A→B→C, which means that the output tensor of the convolution flows to the batch normalization, and the output of the batch normalization flows to the ReLU.

[0079] Traversing each path in the tensor dependency propagation chain, the tensor production and consumption relationships between adjacent operator nodes are mapped to data object transfer relationships between corresponding computational patterns. For the path A→B above, the computational pattern of convolution operator A produces an output data object, which serves as the input data object for the computational pattern of batch normalization operator B. This data object transfer relationship can be represented by a triple (Source, DataObject, Target), where Source and Target are the source and target computational patterns, respectively, and DataObject is the transferred data object. After traversing the entire propagation chain, all data object transfer relationships are concatenated according to the path order to form a computational pattern dependency chain.

[0080] For multi-branch neural network architectures, such as the residual connections in ResNet, it is necessary to handle cases with multiple inputs and multiple outputs. This will form a complex dependency network. Topological sorting is used to ensure the correctness of the dependencies and avoid the formation of circular dependencies or missing dependencies.

[0081] Based on the established computational pattern dependency chain, we further analyze the access timing of data objects when each computational pattern is represented as a sequence of operation primitives. Computational patterns are typically decomposed into a series of basic operation primitives, such as memory allocation, data movement, and core computation. For each operation primitive, we clarify its access type (read / write) and access scope to data objects. For example, the convolutional computational pattern contains the following sequence of operation primitives: allocate memory for the output, read input data, read weights, perform multiply-accumulate operations, and write the output result.

[0082] Identifying the execution order of read and write operations on the same data object within a sequence of primitives is crucial, as it determines the data dependencies during actual execution. For example, if primitive P1 writes to data object D, and primitive P2 reads the same data object D, then P2 depends on P1 and must be executed only after P1 has finished. This read-after-write (RAW) dependency is the most common type of data dependency.

[0083] Based on the data object transit relationships in the computational pattern dependency chain, cross-computation pattern data object transit is refined into cross-operation primitive data read-write dependency relationships. Specifically, if the last write operation primitive PA_write of computational pattern A writes to data object D, and the first read operation primitive PB_read of computational pattern B reads the same data object D, then a read-write dependency edge PA_write→PB_read is established in the final dependency relationship.

[0084] In practical applications, this dependency analysis is crucial for instruction scheduling and memory optimization. For example, on resource-constrained edge devices, dependency analysis can minimize data migrations and reuse intermediate results, thereby reducing memory usage and computational latency. Furthermore, for hardware that supports parallel execution, this analysis can identify operations that can be executed in parallel, improving resource utilization and throughput.

[0085] This dependency propagation analysis method is applicable not only to feedforward neural networks, but also to complex network models such as RNNs containing recurrent structures. It handles recurrent dependencies through temporal unfolding, ensuring the correctness and efficiency of the optimization results.

[0086] In one optional implementation, a cross-block instruction move legality determination mechanism is constructed based on the intermediate representation, and, under the premise of satisfying program semantic equivalence, instructions are dynamically allocated by utilizing the functional complementarity between heterogeneous computing units, including:

[0087] The operation primitive sequence in the intermediate representation is divided into basic blocks, and a dependency constraint matrix across basic blocks is constructed based on the data read / write dependency and control dependency in the execution constraint graph.

[0088] Based on the dependency constraint matrix, the change in dependency direction of the candidate operation primitive to be moved from the source basic block to the target basic block is analyzed. When the dependency direction remains unchanged and does not cause a circular dependency, the move operation is determined to be a legal move.

[0089] For candidate operation primitives determined to be legitimate moves, a subset of heterogeneous computing units capable of executing the candidate operation primitives is identified based on the architecture description information. Within the subset of heterogeneous computing units, the computing unit that enables the candidate operation primitive to complete execution earliest is selected and dynamically allocated according to the performance characteristics and dependency strength of each heterogeneous computing unit.

[0090] The intermediate representation construction process converts the source program into a static, single-assignment three-address code sequence, with each instruction containing an opcode, operand, and result register fields. Basic block partitioning is achieved by identifying the leader instructions in the program, including the program entry point, the target address of jump instructions, and the instruction immediately following a conditional branch instruction. After the basic block boundaries are determined, each basic block is assigned an incrementing identifier, and the operation primitives within the basic block are numbered according to the program sequence. The execution constraint graph is constructed based on data flow analysis and control flow analysis. Data read / write dependencies are established through variable definition-use chains; when instruction A defines variable V and instruction B uses variable V, a data dependency edge is established between A and B. Control dependencies are determined through backward dominance analysis; when the execution of basic block A determines whether basic block B is executed, a control dependency edge is established between A and B.

[0091] The dependency constraint matrix is ​​constructed as an N×N Boolean matrix, where N is the total number of basic blocks in the program, and matrix elements M[i][j] represent the dependency relationship from basic block i to basic block j. The matrix filling process iterates through all dependency edges in the execution constraint graph. For data dependency edges, the matrix positions from the source basic block to the target basic block are set to true. For control dependency edges, the corresponding matrix positions are also marked. The dependency constraint matrix supports transitive closure computation, using the Floyd-Warshall algorithm to calculate indirect dependencies, ensuring that all dependency paths are captured.

[0092] The validity determination of candidate operation primitive moves analyzes the impact of moving operation primitives from the source basic block to the target basic block on the overall dependency structure. The analysis of changes in dependency direction is achieved by comparing the execution order of related operation primitives before and after the move. Before the move, candidate operation primitive O is located in the source basic block S, and the set of operation primitives that have dependencies on it is denoted as D. After the move, operation primitive O is located in the target basic block T. It is necessary to verify whether the relative execution order of each operation primitive in set D with O remains unchanged. The condition for the dependency direction to remain unchanged is: if operation primitive X was executed before O before the move, X is still executed before O after the move; if operation primitive Y was executed after O before the move, Y is still executed after O after the move.

[0093] Cyclic dependency detection is achieved by performing a depth-first search on the dependency constraint matrix. The move operation introduces new dependency edges into the dependency graph. The detection algorithm simulates the graph structure after adding these new dependency edges and verifies the existence of cycles through topological sorting. If the topological sorting cannot be completed or a back edge is detected, the move operation is determined to cause a cyclic dependency. The final determination of a valid move requires simultaneously satisfying two conditions: the dependency direction remains unchanged and no cyclic dependency is introduced. Candidate operation primitives that meet these conditions are marked as movable and enter the subsequent dynamic allocation process.

[0094] The architecture description information uses a hierarchical data structure to store the detailed characteristics of heterogeneous computing units. Each computing unit includes a functional characteristic description field, recording the supported instruction type set, data type compatibility, and special functional unit configurations. The instruction type set is represented by a bit vector, covering categories such as integer arithmetic, floating-point arithmetic, vector arithmetic, memory access, and branch jumps. The performance characteristic field records the execution latency, throughput, and power consumption parameters of various instructions. The execution latency is in processor clock cycles, ranging from 1 to 50 cycles. The throughput represents the number of similar instructions that can be executed per clock cycle, ranging from 0.25 to 4.0. The power consumption parameter is in milliwatts, representing the dynamic power consumption increment during instruction execution.

[0095] The heterogeneous computing unit subset identification process traverses all available computing units, determining the set of computing units capable of executing candidate operation primitives through instruction type matching. The matching algorithm compares the instruction type of the candidate operation primitive with the set of instruction types supported by each computing unit; when an intersection exists, the computing unit is added to the candidate subset. The matching process also considers data type compatibility and precision requirements to ensure that the computing units can correctly handle the data types involved in the operation primitives. After identification, the candidate subset contains all functionally capable computing units that can execute the operation primitives, and the subset size is typically 20% to 80% of the total number of computing units.

[0096] The performance characteristic evaluation calculates the expected completion time for each candidate computing unit to execute the operation primitive. The calculation process considers basic execution latency, resource contention impact, and pipeline scheduling overhead. Basic execution latency is obtained from the architecture description information and corresponds to the standard execution cycle number for the operation primitive type. Resource contention impact is determined by analyzing the current load of the computing unit; a latency penalty factor is introduced when the load rate exceeds a threshold. Pipeline scheduling overhead considers resource conflicts between the operation primitive and scheduled instructions, calculating additional waiting cycles using a conflict detection matrix. The final completion time equals the basic latency plus the contention latency and scheduling latency.

[0097] Dependency strength calculation is based on the importance of candidate operation primitives in the program's critical path. The critical path is determined using a longest path algorithm in the dependency graph, and the path length is equal to the cumulative execution latency of all operation primitives on that path. The dependency strength of a candidate operation primitive is equal to the length of the longest dependency path through that primitive. The calculation process uses a dynamic programming algorithm, calculating the longest successor path length for each operation primitive from the bottom up. The dependency strength value ranges from the execution latency of a single instruction to the length of the entire program's critical path, typically ranging from 5 to 200 clock cycles.

[0098] Dynamic allocation decisions select the optimal computing unit based on a multi-objective optimization strategy. Optimization objectives include earliest completion time, resource utilization balancing, and power consumption control. Earliest completion time is directly measured by the expected completion time, selecting the computing unit with the shortest completion time. When multiple computing units have similar completion times, resource utilization balancing is considered, prioritizing the computing unit with the lightest current load. Power consumption control is achieved by setting a power consumption budget constraint; allocation schemes exceeding the budget are excluded. The final allocation decision uses a weighted scoring mechanism: completion time has a weight of 0.6, load balancing has a weight of 0.3, and power consumption control has a weight of 0.1. The computing unit with the lowest score is selected to execute candidate operation primitives, and the allocation result updates the computing unit status and operation primitive scheduling information.

[0099] In one optional implementation, an abstract representation of the data reuse pattern is established by analyzing the spatiotemporal locality characteristics of the instruction sequence. During the compilation phase, the migration path of data between multiple storage levels is pre-planned, and the migration path is cooperatively mapped with the communication topology of the on-chip interconnect network, including:

[0100] The spatiotemporal distribution characteristics of data access operations in the instruction sequence are analyzed. The access frequency and access interval of statistical objects are statistically analyzed in the time dimension and quantified as the temporal locality intensity. In the spatial dimension, the set of data objects with continuous addresses or fixed step sizes is identified and quantified as the spatial locality intensity. The temporal locality intensity and the spatial locality intensity are combined to construct an abstract representation of the data reuse pattern.

[0101] Based on the abstract representation of the data reuse pattern, the target storage level in the multi-level storage hierarchy is determined for each data object by the intensity of temporal locality, and the migration time from the source storage level to the target storage level is determined by the access interval, thereby planning the migration path of the data object between the multi-level storage hierarchy.

[0102] Based on the communication topology of the on-chip interconnect network in the architecture description information, a communication link that meets the data transmission bandwidth requirements is allocated to the migration path, and the migration operation of data objects with spatial proximity is mapped to the same communication link based on the spatial locality strength, thereby realizing the cooperative mapping between the migration path and the communication topology.

[0103] The spatiotemporal distribution feature analysis of data access operations in the instruction sequence is achieved through a combination of static program analysis and dynamic profile information collection. The static analysis phase traverses the intermediate representation of the program, identifying all instructions involving memory access, including load instructions, store instructions, and array access instructions. For each data access operation, the accessed memory address, access type, loop level, and estimated execution frequency are recorded. Dynamic profile information is obtained by instrumenting data during program execution to collect actual access sequences, recording the timestamp, memory address, and access size for each access. The spatiotemporal distribution feature extraction module processes the collected access sequences, establishing a mapping relationship between data objects and access operations. Data objects are identified by memory address ranges or variable names.

[0104] Access frequency statistics in the time dimension employ a sliding window mechanism, with a window size of 1000 clock cycles and a sliding step size of 100 clock cycles. Within each time window, the number of accesses to each data object is counted, and the access frequency is equal to the number of accesses divided by the window size. Access intervals are calculated based on the time difference between two consecutive accesses to the same data object. All access intervals are recorded, and statistical characteristic values, including minimum interval, maximum interval, average interval, and standard deviation, are calculated. Temporal locality strength is quantified using a weighted average method, with weight parameters including access frequency (0.6) and interval stability (0.4). Interval stability is calculated as the ratio of the interval standard deviation to the average interval; a smaller ratio indicates a more regular access pattern. Temporal locality strength ranges from 0 to 1, with higher values ​​indicating stronger temporal locality.

[0105] Spatial contiguous address identification is achieved through address sequence analysis. The algorithm traverses all access addresses of each data object, detecting the existence of contiguous address patterns. The criteria for contiguous address determination are that the difference between adjacent access addresses is less than or equal to the data object size, and the length of the contiguous access sequence is greater than or equal to 3 addresses. Fixed step size detection analyzes the difference sequence of access address sequences; when the difference value remains constant across multiple consecutive accesses, it is determined to be a fixed step size pattern. The step size range is an integer multiple of the data object size, with typical step sizes including 4 bytes, 8 bytes, 16 bytes, and 64 bytes. Spatial locality intensity quantization is based on the coverage and step size regularity of the contiguous access sequence. Coverage equals the number of contiguous access addresses divided by the total number of access addresses, and step size regularity is calculated using the coefficient of variation of the step size value. Spatial locality intensity ranges from 0 to 1, and the calculation formula is coverage multiplied by 0.7 plus step size regularity multiplied by 0.3.

[0106] The abstract representation of the data reuse pattern uses a multi-dimensional feature vector, which includes six dimensions: temporal locality strength, spatial locality strength, access frequency, average access interval, principal step size, and data object size. Feature vector normalization ensures a consistent range for each dimension; the normalization method uses maximum-minimum scaling to map all feature values ​​to the 0-1 range. The abstract representation also includes semantic labels for data objects, with label types including array elements, structure fields, local variables, and global variables. Semantic labels are determined through symbol table information and type inference, providing auxiliary information for subsequent storage level allocation. The abstract representation data structure is defined as a structure containing object identifiers, feature vectors, semantic labels, and lifecycle information fields.

[0107] The multi-level storage hierarchy comprises five levels: registers, L1 cache, L2 cache, L3 cache, and main memory. Each storage level has different capacity, access latency, and bandwidth characteristics. Register capacity ranges from 64 to 256 registers, with an access latency of 1 clock cycle. L1 cache capacity ranges from 32 kilobytes to 64 kilobytes, with an access latency of 2 to 3 clock cycles. L2 cache capacity ranges from 256 kilobytes to 1 megabyte, with an access latency of 8 to 15 clock cycles. L3 cache capacity ranges from 8 megabytes to 32 megabytes, with an access latency of 20 to 40 clock cycles. Main memory capacity is in the gigabyte range, with an access latency of 100 to 300 clock cycles. The target storage level is determined based on a comprehensive evaluation of temporal locality strength and data object size.

[0108] A temporal locality strength threshold is used for storage tier allocation decisions. Data objects with a strength value greater than 0.8 are allocated to registers or L1 cache, those with a strength value between 0.5 and 0.8 are allocated to L2 cache, those with a strength value between 0.2 and 0.5 are allocated to L3 cache, and those with a strength value less than 0.2 are retained in main memory. Data object size constraints ensure that the allocated storage tier has sufficient capacity; when the target tier's capacity is insufficient, it is downgraded to the next lower storage tier. Capacity constraint checks employ an optimal fit algorithm, maintaining available capacity information for each storage tier and prioritizing allocation to the tier with capacity closest to the requirement.

[0109] The migration timing is determined based on a comparative analysis of access intervals and storage tier access latency. When the average access interval of a data object is less than 10 times the access latency of the current storage tier, a migration operation to an upper tier is triggered. The target tier for upward migration is the highest tier whose access latency is less than half the average access interval. Downward migration is triggered when the data object has not been accessed at the current tier for a preset time, which is equal to 100 times the access latency of the current tier. The migration timing calculation considers a balance between migration costs and access benefits. Migration costs include data transfer time and the capacity occupancy of the target tier, while access benefits are calculated based on the cumulative reduction in access latency.

[0110] The migration path planning employs a hierarchical path search algorithm. This algorithm maintains a multi-level storage hierarchy topology graph, where nodes represent storage levels, edges represent migration paths, and edge weights represent migration costs. Path search starts from the source storage level and uses breadth-first search to find the shortest path to the target level. Path cost calculation includes transmission latency, bandwidth usage, and capacity allocation costs. Transmission latency equals the data size divided by the transmission bandwidth; bandwidth usage affects the available bandwidth for subsequent migration operations. Capacity allocation costs are determined by the capacity utilization rate of the target level; higher utilization rates result in higher costs. The migration path includes the sequence of intermediate nodes, transmission time estimates, and required bandwidth resources.

[0111] The communication topology of the on-chip interconnect network is represented by a graph structure. Nodes in the graph represent storage or processing units, and edges represent communication links. Edge attributes include bandwidth capacity, latency characteristics, and reliability parameters. The communication topology supports various network structures, including bus, ring, mesh, tree, and fat-tree topologies. Each topology has different bandwidth distribution and latency characteristics: bus topologies share bandwidth, ring topologies have uniform latency, and mesh topologies offer good scalability. Topology information is read from an architecture description file, which supports both JSON and XML formats and includes a list of nodes, a list of links, and a performance parameter table.

[0112] The migration path communication link allocation employs a bandwidth-aware path mapping algorithm. This algorithm analyzes the bandwidth requirements of each migration operation, where the bandwidth requirement equals the data size divided by the allowed transmission time. The link allocation process traverses each edge of the migration path, checking if the available bandwidth meets the requirements. When available bandwidth is insufficient, the algorithm searches for alternative paths or adjusts the transmission time. A link reservation mechanism ensures that allocated bandwidth is not occupied by other operations during the migration operation; reservation information includes start time, duration, and bandwidth quantity. Link utilization is monitored and maintained in real-time to track bandwidth usage; when utilization exceeds a threshold, load balancing is triggered.

[0113] Spatial proximity is determined based on the similarity of memory address distribution and access patterns of data objects. Address proximity is determined by calculating the degree of overlap in the address ranges of data objects; objects with an overlap greater than 50% are considered spatially proximity. Access pattern similarity is calculated by comparing the access sequences of data objects, using a sequence similarity algorithm to calculate a similarity score. Objects with a similarity score greater than 0.7 are considered patternly proximity. Migration operations of spatially proximity data objects are preferentially mapped to the same communication link to reduce link switching overhead and improve transmission efficiency. Same-link mapping employs a time multiplexing mechanism, allocating multiple migration operations to different time periods on the same link according to their chronological order.

[0114] Cooperative mapping enables unified management of migration paths and communication link allocation through a migration scheduler. The scheduler maintains three core data structures: a migration queue, a link status table, and a mapping table. The migration queue is sorted by priority, calculated based on the temporal locality and access urgency of data objects. The link status table records the real-time usage of each communication link, including current load, available bandwidth, and reserved time period. The mapping table records the correspondence between migration operations and communication links, supporting dynamic adjustments and conflict detection. The goal of cooperative mapping is to maximize storage hierarchy utilization efficiency and minimize communication network congestion.

[0115] In one optional implementation, target code is generated based on the migration path, the target code is executed on the target AI chip, and the abstract representation of the instruction move legality determination mechanism and the data reuse mode is dynamically adjusted based on performance feedback information during execution.

[0116] Based on the migration path, the sequence of operation primitives and data transfer operations in the intermediate representation are converted into the machine instruction sequence of the target artificial intelligence chip, and performance monitoring instructions are inserted into the machine instruction sequence to generate target code;

[0117] The target code is executed on the target artificial intelligence chip, and the execution latency and communication latency are collected through the performance monitoring instructions to generate performance feedback information.

[0118] Based on the execution latency in the performance feedback information, a bias analysis is performed on the dependency constraint matrix in the instruction move legality determination mechanism, and the dependency strength is updated. Based on the communication latency in the performance feedback information, an error analysis is performed on the temporal locality strength in the abstract representation of the data reuse pattern, and the temporal locality strength is updated. According to the updated dependency strength and the temporal locality strength, the instruction move legality determination mechanism and the abstract representation of the data reuse pattern are dynamically adjusted.

[0119] Based on the migration path, the sequence of operation primitives and data transfer operations in the intermediate representation are converted into a sequence of machine instructions for the target AI chip. Specifically, for each operation primitive in the intermediate representation, it is mapped to a corresponding machine instruction according to the instruction set architecture of the target AI chip. For example, for convolution operation primitives, depending on whether the target chip supports dedicated convolution instructions, the mapping is selected as a single convolution instruction or decomposed into multiple basic computation instructions. For data transfer operations, they are converted into corresponding load, store, or data transfer instructions according to the memory hierarchy and data transfer mechanism of the target chip.

[0120] After generating the machine instruction sequence, performance monitoring instructions are inserted at critical instruction locations. These monitoring instructions mainly fall into two categories: execution latency monitoring instructions and communication latency monitoring instructions. Execution latency monitoring instructions are typically inserted before and after computationally intensive operations to record the number of clock cycles required for the operation to execute; communication latency monitoring instructions are inserted before and after data transmission operations to record the time required for data to move between different memory levels. For example, if the target chip supports a timestamp counter, instructions to read the timestamp counter can be inserted before and after the execution of critical instructions, and latency data can be obtained by calculating the difference.

[0121] After generating the instruction sequence and monitoring instruction insertion, the generated machine instruction sequence is assembled into target code that can be executed on the target AI chip. This typically includes steps such as adding necessary code segment markers, adjusting memory alignment, and generating a symbol table, ultimately outputting a binary file or a directly loadable instruction stream that conforms to the target chip's executable format.

[0122] The generated target code is executed on the target AI chip. During execution, performance monitoring instructions are inserted as described above to collect execution latency and communication latency data. Execution latency data mainly records the actual execution time of various computational instructions, while communication latency data records the data transfer time between different memory levels. This data is collected in the form of timestamp sequences or counter differences and further processed into structured performance feedback information, including key information such as instruction type, execution context, and execution time.

[0123] After collecting performance feedback, a deviation analysis is performed on the dependency constraint matrix in the instruction move legality determination mechanism based on execution latency data. The dependency constraint matrix represents the dependencies between instructions and their strength, where the matrix element values ​​represent the dependency strength. Deviation analysis calculates the deviation value by comparing the actual execution latency between instructions with the latency predicted by the dependency constraint matrix. If the deviation between the actual execution latency and the predicted latency of a pair of instructions exceeds a preset threshold, the corresponding dependency strength in the current dependency constraint matrix is ​​considered inaccurate and needs to be updated.

[0124] When updating dependency strength, a weighted average method is used to fuse the newly observed dependency strength value with the historical dependency strength: Updated dependency strength = Historical dependency strength × (1 - Update weight) + New observed dependency strength × Update weight. The update weight can be dynamically adjusted based on the number of observed samples and the confidence level to ensure the stability and adaptability of the update process.

[0125] Error analysis is performed on the temporal locality intensity in the abstract representation of data multiplexing patterns based on communication delay data. Temporal locality intensity represents the degree to which data is repeatedly accessed within a short period of time and is a key characteristic of data multiplexing patterns. Error analysis calculates the error value by comparing the actual observed data revisit interval with the interval predicted by the temporal locality model. If the error between the actual revisit interval and the predicted interval of a data access pattern exceeds a set threshold, the current temporal locality intensity is considered inaccurate and needs adjustment.

[0126] The updated temporal locality intensity is also calculated using a weighted method: Updated temporal locality intensity = Historical temporal locality intensity × (1 - Adjustment factor) + New observation temporal locality intensity × Adjustment factor. The adjustment factor is dynamically calculated based on the size and stability of the observation sample to ensure insensitivity to outliers while responding promptly to stable trends.

[0127] Based on the updated dependency strength and temporal locality strength, the abstract representations of the instruction move legality determination mechanism and the data reuse pattern are dynamically adjusted. For the instruction move legality determination mechanism, the legality of instruction move operations is re-evaluated using the updated dependency constraint matrix. This reveals that previously invalid instruction moves are now valid, or previously valid moves are now invalid. For the abstract representation of the data reuse pattern, the optimal data prefetching strategy and cache replacement strategy are recalculated based on the updated temporal locality strength, optimizing the timing and retention time of data movement between different memory levels.

[0128] Through the aforementioned dynamic adjustment mechanism, instruction scheduling and data management strategies can be continuously optimized based on the actual operating characteristics of the target AI chip, reducing performance loss caused by insufficient understanding of chip characteristics during migration and improving the execution efficiency of the neural network on the target chip. This mechanism is particularly suitable for neural network deployment scenarios targeting new AI chips, effectively addressing challenges such as incomplete chip documentation and unclear performance characteristics.

[0129] In one optional implementation, the sequence of operation primitives and data transfer operations in the intermediate representation are converted into a sequence of machine instructions for the target AI chip according to the migration path, and performance monitoring instructions are inserted into the machine instruction sequence to generate target code, including:

[0130] Based on the target storage level and migration time planned in the migration path, the data transfer operation is converted into data transmission instructions for the target artificial intelligence chip, and the operation primitive sequence in the intermediate representation is converted into the calculation instructions of the target artificial intelligence chip. According to the migration time, the data transmission instructions and the calculation instructions are sequentially arranged to form a machine instruction sequence.

[0131] A pair of computation monitoring instructions is inserted before and after the execution position of the computation instruction in the machine instruction sequence, and a pair of transmission monitoring instructions is inserted before and after the execution position of the data transmission instruction in the machine instruction sequence. The pair of computation monitoring instructions and the pair of transmission monitoring instructions constitute a performance monitoring instruction.

[0132] The set of instructions containing the machine instruction sequence and the performance monitoring instructions is used as the target code.

[0133] The migration path information is parsed, which includes the planning of the target storage level and the migration time. The target storage level typically includes different levels such as on-chip memory, cache, and global memory, while the migration time defines the specific point in time when data is transferred between different storage levels. The parsed migration information will guide the subsequent instruction translation process.

[0134] Based on the analyzed migration path, the data movement operations are converted into data transfer instructions for the target AI chip. For example, a DMA transfer instruction is generated for data migration from global memory to on-chip memory; a load instruction is generated for data migration from on-chip memory to a computing unit. During the conversion, the hardware characteristics of the target chip need to be considered, such as DMA controller configuration and transfer size limitations. Each converted data transfer instruction includes information such as source address, destination address, and transfer size.

[0135] The sequence of operation primitives in the intermediate representation is converted into computational instructions for the target AI chip. These primitives include matrix multiplication, convolution, activation functions, etc., and need to be mapped according to the instruction set supported by the target chip. For example, matrix multiplication primitives are converted into GEMM instructions, and convolution operations are converted into specific convolution instructions. During the conversion process, the characteristics of the target chip's computational units, such as SIMD width and tensor core configuration, must also be considered to optimize instruction parameters and maximize computational efficiency.

[0136] Based on the migration time, data transmission instructions and computation instructions are sequentially arranged to form a machine instruction sequence. This sequential arrangement must ensure that data dependencies are satisfied; that is, before a computation instruction is executed, the required data has already been loaded into the corresponding location via data transmission instructions. Furthermore, to improve execution efficiency, data transmission and computation instructions can be executed as overlappingly as possible without disrupting dependencies, thus achieving parallel computation and communication.

[0137] After the basic instruction sequence is generated, performance monitoring instructions are inserted. Pairs of computation monitoring instructions are inserted before and after the execution positions of computation instructions in the machine instruction sequence. These pairs typically include a start monitoring instruction and an end monitoring instruction. The start monitoring instruction records the current timestamp or starts a performance counter, while the end monitoring instruction records the end timestamp or reads the performance counter value. These instruction pairs are used to accurately measure the execution time and resource consumption of computational operations.

[0138] Transmission monitoring instruction pairs are inserted before and after the execution position of the data transmission instruction in the machine instruction sequence. These pairs are used to monitor performance indicators of data transmission operations, such as transmission time and bandwidth utilization. By using start and end monitoring instructions in pairs, relevant performance data for a single data transmission can be accurately captured.

[0139] Based on the inserted calculation monitoring command pairs and transmission monitoring command pairs, a complete performance monitoring system is constructed. This system includes functions for collecting, summarizing, and analyzing performance data. For example, a dedicated data structure can be designed to store the performance data from each monitoring point and generate a detailed performance report after the program execution is complete.

[0140] The instruction set, containing machine instruction sequences and performance monitoring instructions, is integrated into target code. This integration process requires ensuring the integrity and correctness of the instruction sequences, including correctly setting jump relationships and label positions between instructions. Simultaneously, necessary metadata, such as symbol tables and debugging information, needs to be generated for subsequent debugging and optimization.

[0141] In practical applications, the target code requires further linking and packaging. For example, for virtual machine-based AI chips, the instruction sequence needs to be packaged into a binary file of a specific format; for chips that directly run machine code, a loadable target file needs to be generated. Through these processes, executable code that can be run directly on the target AI chip is finally formed.

[0142] The target code generated using the above method not only correctly implements the functions of the original computational task, but also has built-in performance monitoring capabilities, facilitating performance analysis and optimization for developers. Furthermore, because the instruction generation process fully considers the characteristics of the target chip, the generated code can efficiently utilize chip resources to achieve optimal performance.

[0143] A second aspect of this invention provides an instruction set optimization and compilation scheduling system for artificial intelligence chips, comprising:

[0144] The modeling unit is used to acquire the neural network computation graph to be compiled and the architecture description information of the target artificial intelligence chip, and to perform multi-level abstract modeling of the neural network computation graph based on the architecture description information, converting the semantic expression of the operator layer into the operation primitives of the instruction set architecture layer, thereby forming an intermediate representation containing dual constraints of control flow and data flow.

[0145] The allocation unit is used to construct a cross-basic block instruction move legality determination mechanism based on the intermediate representation, and dynamically allocate instructions by utilizing the functional complementarity between heterogeneous computing units, provided that the program semantic equivalence is satisfied.

[0146] The mapping unit is used to establish an abstract representation of the data reuse pattern by analyzing the spatiotemporal locality characteristics of the instruction sequence, pre-plan the migration path of data between multiple storage levels during the compilation stage, and coordinate the migration path with the communication topology of the on-chip interconnect network.

[0147] An adjustment unit is used to generate target code according to the migration path, execute the target code on the target artificial intelligence chip, and dynamically adjust the abstract representation of the instruction movement legality determination mechanism and the data reuse mode according to the performance feedback information during the execution process.

[0148] A third aspect of the present invention provides an electronic device, comprising:

[0149] processor;

[0150] Memory used to store processor-executable instructions;

[0151] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0152] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0153] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An instruction set optimization and compilation scheduling method for artificial intelligence chips, characterized in that, include: The neural network computation graph to be compiled and the architecture description information of the target artificial intelligence chip are obtained. Based on the architecture description information, the neural network computation graph is subjected to multi-level abstract modeling. The semantic expression of the operator layer is converted into the operation primitives of the instruction set architecture layer, thereby forming an intermediate representation containing dual constraints of control flow and data flow. Based on the intermediate representation, a mechanism for determining the legality of instruction movement across basic blocks is constructed, and instructions are dynamically allocated by utilizing the functional complementarity between heterogeneous computing units, provided that the semantic equivalence of the program is satisfied. By analyzing the spatiotemporal locality characteristics of instruction sequences, an abstract representation of data reuse patterns is established. During the compilation phase, the migration path of data between multiple storage levels is pre-planned, and the migration path is co-mapped with the communication topology of the on-chip interconnect network. Target code is generated based on the migration path, the target code is executed on the target AI chip, and the abstract representation of the instruction movement legality determination mechanism and the data reuse mode is dynamically adjusted based on the performance feedback information during the execution process.

2. The method according to claim 1, characterized in that, Based on the architecture description information, the neural network computation graph is subjected to multi-level abstract modeling, and the semantic expression of the operator layer is converted into the operation primitives of the instruction set architecture layer, thereby forming an intermediate representation containing both control flow and data flow constraints, including: Semantic decomposition is performed on the operator nodes in the neural network computation graph to extract the computation kernel features and data access patterns of each operator node, and a hierarchical mapping model is constructed based on the instruction set semantics of heterogeneous computing units in the architecture description information. Based on the intermediate abstraction layer in the hierarchical mapping model, dependency propagation analysis is performed on operator nodes with data flow dependencies, converting the tensor dependencies of the operator layer into computational mode dependencies, and further refining them into data read / write dependencies. For the control flow structure in the neural network computation graph, an abstract representation of the control flow is established in the intermediate abstract layer. Based on the abstract representation, the conditional decision logic is associated with the data read / write dependency, so that the control flow constraints and data flow constraints form a unified execution constraint graph at the operation primitive layer. The data read / write dependency and the execution constraint graph are structurally encapsulated to form an intermediate representation.

3. The method according to claim 2, characterized in that, Based on the intermediate abstraction layer in the hierarchical mapping model, dependency propagation analysis is performed on operator nodes with data flow dependencies. The tensor dependencies of the operator layer are converted into computational pattern dependencies, and further refined into data read / write dependencies, including: The computational pattern representation of the intermediate abstract layer is extracted from the hierarchical mapping model, the data objects and their access attributes involved in the computational pattern representation are identified, and a tensor dependency propagation chain is constructed based on the tensor connection relationship between operator nodes in the neural network computation graph. Traverse each path in the tensor dependency propagation chain, map the tensor production and consumption relationship between adjacent operator nodes on the path to the data object transfer relationship between the corresponding computational pattern representations, and concatenate the data object transfer relationship according to the path order to form a computational pattern dependency chain; Based on the computational pattern dependency chain, the access sequence of data objects when each computational pattern is represented as an operation primitive sequence is analyzed. The execution order of read and write operations on the same data object in the operation primitive sequence is identified. Based on the data object transit relationship in the computational pattern dependency chain, the data object transit across computational patterns is refined into data read and write dependencies across operation primitives.

4. The method according to claim 1, characterized in that, Based on the intermediate representation, a mechanism for determining the legality of instruction moves across basic blocks is constructed. Under the premise of satisfying program semantic equivalence, instructions are dynamically allocated by leveraging the functional complementarity between heterogeneous computing units, including: The operation primitive sequence in the intermediate representation is divided into basic blocks, and a dependency constraint matrix across basic blocks is constructed based on the data read / write dependency and control dependency in the execution constraint graph. Based on the dependency constraint matrix, the change in dependency direction of the candidate operation primitive to be moved from the source basic block to the target basic block is analyzed. When the dependency direction remains unchanged and does not cause a circular dependency, the move operation is determined to be a legal move. For candidate operation primitives determined to be legitimate moves, a subset of heterogeneous computing units capable of executing the candidate operation primitives is identified based on the architecture description information. Within the subset of heterogeneous computing units, the computing unit that enables the candidate operation primitive to complete execution earliest is selected and dynamically allocated according to the performance characteristics and dependency strength of each heterogeneous computing unit.

5. The method according to claim 1, characterized in that, An abstract representation of data reuse patterns is established by analyzing the spatiotemporal locality characteristics of instruction sequences. During the compilation phase, data migration paths between multiple storage levels are pre-planned, and these migration paths are collaboratively mapped with the communication topology of the on-chip interconnect network. The spatiotemporal distribution characteristics of data access operations in the instruction sequence are analyzed. The access frequency and access interval of statistical objects are statistically analyzed in the time dimension and quantified as the temporal locality intensity. In the spatial dimension, the set of data objects with continuous addresses or fixed step sizes is identified and quantified as the spatial locality intensity. The temporal locality intensity and the spatial locality intensity are combined to construct an abstract representation of the data reuse pattern. Based on the abstract representation of the data reuse pattern, the target storage level in the multi-level storage hierarchy is determined for each data object by the intensity of temporal locality, and the migration time from the source storage level to the target storage level is determined by the access interval, thereby planning the migration path of the data object between the multi-level storage hierarchy. Based on the communication topology of the on-chip interconnect network in the architecture description information, a communication link that meets the data transmission bandwidth requirements is allocated to the migration path, and the migration operation of data objects with spatial proximity is mapped to the same communication link based on the spatial locality strength, thereby realizing the cooperative mapping between the migration path and the communication topology.

6. The method according to claim 1, characterized in that, Target code is generated based on the migration path, the target code is executed on the target AI chip, and the abstract representation of the instruction move legality determination mechanism and the data reuse mode is dynamically adjusted based on performance feedback information during execution. This includes: Based on the migration path, the sequence of operation primitives and data transfer operations in the intermediate representation are converted into the machine instruction sequence of the target artificial intelligence chip, and performance monitoring instructions are inserted into the machine instruction sequence to generate target code; The target code is executed on the target artificial intelligence chip, and the execution latency and communication latency are collected through the performance monitoring instructions to generate performance feedback information. Based on the execution latency in the performance feedback information, a bias analysis is performed on the dependency constraint matrix in the instruction move legality determination mechanism, and the dependency strength is updated. Based on the communication latency in the performance feedback information, an error analysis is performed on the temporal locality strength in the abstract representation of the data reuse pattern, and the temporal locality strength is updated. According to the updated dependency strength and the temporal locality strength, the instruction move legality determination mechanism and the abstract representation of the data reuse pattern are dynamically adjusted.

7. The method according to claim 6, characterized in that, Based on the migration path, the sequence of operation primitives and data transfer operations in the intermediate representation are converted into a sequence of machine instructions for the target AI chip. Performance monitoring instructions are then inserted into this machine instruction sequence to generate target code, including: Based on the target storage level and migration time planned in the migration path, the data transfer operation is converted into data transmission instructions for the target artificial intelligence chip, and the operation primitive sequence in the intermediate representation is converted into the calculation instructions of the target artificial intelligence chip. According to the migration time, the data transmission instructions and the calculation instructions are sequentially arranged to form a machine instruction sequence. A pair of computation monitoring instructions is inserted before and after the execution position of the computation instruction in the machine instruction sequence, and a pair of transmission monitoring instructions is inserted before and after the execution position of the data transmission instruction in the machine instruction sequence. The pair of computation monitoring instructions and the pair of transmission monitoring instructions constitute a performance monitoring instruction. The set of instructions containing the machine instruction sequence and the performance monitoring instructions is used as the target code.

8. An instruction set optimization and compilation scheduling system for artificial intelligence chips, used to implement the method of any one of claims 1-7, characterized in that, include: The modeling unit is used to acquire the neural network computation graph to be compiled and the architecture description information of the target artificial intelligence chip, and to perform multi-level abstract modeling of the neural network computation graph based on the architecture description information, converting the semantic expression of the operator layer into the operation primitives of the instruction set architecture layer, thereby forming an intermediate representation containing dual constraints of control flow and data flow. The allocation unit is used to construct a cross-basic block instruction move legality determination mechanism based on the intermediate representation, and dynamically allocate instructions by utilizing the functional complementarity between heterogeneous computing units, provided that the program semantic equivalence is satisfied. The mapping unit is used to establish an abstract representation of the data reuse pattern by analyzing the spatiotemporal locality characteristics of the instruction sequence, pre-plan the migration path of data between multiple storage levels during the compilation stage, and coordinate the migration path with the communication topology of the on-chip interconnect network. An adjustment unit is used to generate target code according to the migration path, execute the target code on the target artificial intelligence chip, and dynamically adjust the abstract representation of the instruction movement legality determination mechanism and the data reuse mode according to the performance feedback information during the execution process.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images