Sparse matrix dataflow search method and apparatus based on hardware compiler

Abstract

The application relates to the technical field of computing data processing, in particular to a sparse matrix data flow search method and device based on a hardware compiler, wherein the method comprises the following steps: transforming intermediate linear combinations, multiplication items and output accumulation modes of an initial data flow scheme to obtain a candidate data flow scheme satisfying preset semantic equivalence conditions; determining a delay time of the candidate data flow scheme according to a target hardware compiler, and performing random walk search in the candidate data flow scheme with the delay time as an optimization target until a preset termination condition is satisfied to obtain a data flow search result of a target sparse matrix multiplication. Therefore, the problems that, due to the fact that related technologies only take the number of multiplications as an evaluation index, the physical execution cost under a sparse mode cannot be accurately described, the scheme obtained by search has a large deviation from actual hardware execution performance, and it is difficult to fully exert the computing performance of sparse matrix multiplication and the utilization efficiency of hardware resources and the like are solved.

Description

Technical Field

[0001] This application relates to the field of computational data processing technology, and in particular to a sparse matrix data stream search method and apparatus based on a hardware compiler. Background Technology

[0002] In related technologies, sparse matrix multiplication search methods typically aim to minimize the number of multiplications, thereby reducing the theoretical computational complexity by decreasing the number of intermediate multiplication terms involved in the matrix multiplication process.

[0003] However, the relevant technologies only use the number of multiplications as an evaluation metric, which cannot accurately characterize the physical execution cost in sparse mode. This results in a large deviation between the searched solutions and the actual hardware execution performance, making it difficult to fully utilize the computational performance and hardware resource utilization efficiency of sparse matrix multiplication. Summary of the Invention

[0004] This application provides a sparse matrix data stream search method and apparatus based on a hardware compiler to solve the problem in related technologies that only use the number of multiplications as an evaluation index, which leads to an inability to accurately characterize the physical execution cost in sparse mode, resulting in a large deviation between the searched solution and the actual hardware execution performance, making it difficult to fully utilize the computational performance of sparse matrix multiplication and the efficiency of hardware resource utilization.

[0005] The first aspect of this application provides a sparse matrix data flow search method based on a hardware compiler, comprising the following steps: determining an initial data flow scheme for a target sparse matrix multiplication; transforming at least one of the intermediate linear combinations, multiplication terms, and output accumulation methods of the initial data flow scheme to obtain candidate data flow schemes that satisfy preset semantic equivalence conditions; determining the delay time of the candidate data flow schemes according to the target hardware compiler, and performing a random walk search in the candidate data flow schemes with the delay time as the optimization objective until a preset termination condition is met, so as to obtain the data flow search result for the target sparse matrix multiplication.

[0006] By using the above techniques and taking the execution latency time evaluated by the target hardware compiler as the optimization target, the random walk search process can be driven. The search direction can be continuously corrected by the actual performance feedback of the target hardware, so that the data flow scheme obtained by the search can take into account both computational complexity and actual execution efficiency, thereby improving the computational performance of sparse matrix multiplication and the utilization of hardware resources.

[0007] Optionally, in one embodiment of this application, determining the delay time of a candidate data stream scheme according to the target hardware compiler includes: obtaining the hash key of the candidate data stream scheme; filtering the candidate data stream scheme based on the hash key to obtain a first candidate scheme that meets the preset structural conditions; and determining the delay time of the first candidate scheme according to the target hardware compiler.

[0008] By using the above technical means, a mapping storage between candidate data flow schemes and compilation evaluation results is established based on hash keys. During the search process, the corresponding compilation results and performance evaluation information can be queried according to the hash keys. The evaluation results of the evaluated candidate data flow schemes can be directly reused, reducing the overhead of repeated compilation.

[0009] Optionally, in one embodiment of this application, determining the delay time of the first candidate scheme according to the target hardware compiler includes: converting the first candidate scheme into a second candidate scheme that meets preset unified conditions; determining invalid input items, invalid intermediate items, and invalid output accumulation items of the second candidate scheme according to the target sparse mask; determining a third candidate scheme according to the invalid input, invalid intermediate items, and invalid output accumulation; calling the target hardware compiler to convert the third candidate scheme into a target instruction sequence or hardware scheduling result; and determining the delay time according to the target instruction sequence or hardware scheduling result.

[0010] By using the above techniques to delete invalid input items, invalid intermediate items, and invalid output accumulation items based on the target mask, the computation graph can retain only the valid nodes and data dependencies involved in the target matrix multiplication calculation, reducing redundant calculations and invalid data transmission, lowering computational resource consumption, and improving the efficiency of data flow scheme search, compilation mapping, and target hardware execution.

[0011] Optionally, in one embodiment of this application, screening candidate data flow schemes includes: determining a first screening scheme based on at least one of the following: addition nodes, output depth, resource requirements, and data dependencies of the candidate data flow schemes; determining a second screening scheme based on the local change regions of the candidate data flow schemes; recording the local structural features corresponding to candidate data flow schemes that satisfy at least one of preset compilation failure conditions, preset hardware resource overrun conditions, or preset scheduling timeout conditions, and determining a third screening scheme based on the local structural features; and screening candidate data flow schemes based on the first screening scheme, the second screening scheme, and the third screening scheme.

[0012] By employing the above technical means, candidate data flow schemes are screened to retain those with optimization potential before calling the target hardware compiler, thereby reducing compilation resource consumption and performance evaluation time, and improving compilation evaluation efficiency.

[0013] Optionally, in one embodiment of this application, the calculation formula for the optimization target is: , in, Indicates the optimization objective. This represents the cycle-level latency given by the compiler and performance simulator. Indicates the candidate data stream scheme,M Represents a sparse mask. H This indicates the target hardware constraints.

[0014] By employing the above technical means, optimization targets are established based on execution latency, resource overruns, and compilation failures. This approach balances execution efficiency, resource constraints, and compilation feasibility, reduces invalid searches and redundant evaluations, improves the accuracy of data flow scheme searches, and yields data flow schemes adapted to the target hardware.

[0015] A second aspect of this application provides a sparse matrix data stream search apparatus based on a hardware compiler, comprising: a determination module for determining an initial data stream scheme for a target sparse matrix multiplication; a transformation module for transforming at least one of the intermediate linear combinations, multiplication terms, and output accumulation methods of the initial data stream scheme to obtain candidate data stream schemes that satisfy preset semantic equivalence conditions; and a search module for determining the delay time of the candidate data stream schemes according to the target hardware compiler, and performing a random walk search in the candidate data stream schemes with the delay time as the optimization objective until a preset termination condition is met to obtain the data stream search result for the target sparse matrix multiplication.

[0016] Optionally, in one embodiment of this application, the search module includes: an acquisition unit for acquiring the hash key of a candidate data stream scheme; a filtering unit for filtering the candidate data stream scheme based on the hash key to obtain a first candidate scheme that meets preset structural conditions; and a determination unit for determining the delay time of the first candidate scheme according to the target hardware compiler.

[0017] Optionally, in one embodiment of this application, the determining unit includes: a first conversion subunit, configured to convert a first candidate scheme into a second candidate scheme that satisfies a preset unified condition; a first determining subunit, configured to determine invalid input items, invalid intermediate items, and invalid output accumulation items of the second candidate scheme based on a target sparse mask; a second determining subunit, configured to determine a third candidate scheme based on invalid inputs, invalid intermediate items, and invalid output accumulation; a second conversion subunit, configured to call a target hardware compiler to convert the third candidate scheme into a target instruction sequence or hardware scheduling result; and a third determining unit, configured to determine a delay time based on the target instruction sequence or hardware scheduling result.

[0018] Optionally, in one embodiment of this application, the filtering unit includes: a first filtering subunit, configured to determine a first filtering scheme based on at least one of the addition nodes, output depth, resource requirements, and data dependencies of the candidate data flow scheme; a second filtering subunit, configured to determine a second filtering scheme based on the local change regions of the candidate data flow scheme; a third filtering subunit, configured to record the local structural features corresponding to the candidate data flow scheme that satisfy at least one of the preset compilation failure conditions, preset hardware resource over-limit conditions, or preset scheduling timeout conditions, and determine a third filtering scheme based on the local structural features; and a fourth filtering subunit, configured to filter the candidate data flow schemes based on the first filtering scheme, the second filtering scheme, and the third filtering scheme.

[0019] Optionally, in one embodiment of this application, the calculation formula for the optimization target is: , in, Indicates the optimization objective. This represents the cycle-level latency given by the compiler and performance simulator. Indicates the candidate data stream scheme, M Represents a sparse mask. H This indicates the target hardware constraints.

[0020] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the sparse matrix data stream search method based on a hardware compiler as described in the above embodiments.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described hardware compiler-based sparse matrix data stream search method.

[0022] A fifth aspect of this application provides a computer program product, including a computer program that, when executed, is used to implement the hardware compiler-based sparse matrix data stream search method described above.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1This is a flowchart of a sparse matrix data stream search method based on a hardware compiler, according to an embodiment of this application. Figure 2 This is a block diagram of a sparse matrix data stream search device based on a hardware compiler, according to an embodiment of this application. Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0025] Figure label: 10-Sparse matrix data stream search device based on hardware compiler; 100-Determining module, 200-Transformation module, 300-Search module; 301-Memory, 302-Processor, 303-Communication interface. Detailed Implementation

[0026] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0027] The following describes a sparse matrix data stream search method and apparatus based on a hardware compiler, according to embodiments of this application, with reference to the accompanying drawings. Addressing the technical problem mentioned in the background art, where using only the number of multiplications as an evaluation metric fails to accurately characterize the physical execution cost in sparse mode, resulting in a significant deviation between the searched solution and the actual hardware execution performance, and hindering the full utilization of the computational performance and hardware resource efficiency of sparse matrix multiplication, this application provides a sparse matrix data stream search method based on a hardware compiler. This method aims to maintain the input-output mapping relationship of the target sparse matrix multiplication while automatically determining an efficient data stream scheme adapted to the target hardware by performing equivalent transformations and hardware-aware search on the initial data stream scheme, comprehensively considering compilation scheduling, resource consumption, and actual execution latency. This reduces invalid computation and data access overhead, thereby improving the execution performance of sparse matrix multiplication.

[0028] Specifically, Figure 1 This is a flowchart illustrating a sparse matrix data stream search method based on a hardware compiler, provided in an embodiment of this application.

[0029] like Figure 1 As shown, this hardware compiler-based sparse matrix data flow search method includes the following steps: In step S101, the initial data flow scheme for the target sparse matrix multiplication is determined.

[0030] As a concrete example, the target sparse matrix multiplication can be represented as C = A × B, where at least one of the matrices A, B, or C has a structured or unstructured sparse pattern. An initial data flow scheme S can be represented as a three-stage computation process: Input linear combination stage: Based on the coefficient matrices U and V, the effective elements of A and B are linearly combined to obtain several input combination terms; Intermediate multiplication stage: Multiply the corresponding input combination terms pairwise to obtain the intermediate multiplication term. ; Output accumulation stage: Based on the coefficient matrix W, the intermediate multiplication terms are linearly combined into the elements of the output matrix C.

[0031] Where R represents the number of intermediate multiplication terms, which corresponds to the rank in fast matrix multiplication. However, in the embodiments of this application, R is not used as the sole optimization objective, but rather as one of the structural properties or constraints of the candidate data stream.

[0032] For a given sparse mask M, embodiments of this application can delete or mask input, multiplication, and output terms that are guaranteed to be zero when parsing the data flow scheme, and generate a sparse-aware computation graph G(S,M). This computation graph includes input nodes, addition nodes, multiplication nodes, negation nodes, and output nodes, and preserves the data dependencies between nodes, providing a structured computational representation for subsequent data flow scheme transformation, candidate scheme search, and target hardware compilation and scheduling.

[0033] In step S102, at least one of the intermediate linear combination, multiplication terms, and output accumulation methods of the initial data flow scheme is transformed to obtain a candidate data flow scheme that satisfies the preset semantic equivalence condition.

[0034] Transforming the intermediate linear combination method in the initial data flow scheme can involve reconstructing the original linear combination terms, which were directly calculated by multiple input elements, into a new combination of elements; transforming the multiplication terms can involve decomposing the original multiple independent multiplication terms into different rank-one calculation terms; transforming the output accumulation method can involve adjusting the accumulation order or accumulation topology of multiple intermediate multiplication terms, etc., without specific restrictions.

[0035] The presupposed semantic equivalence condition means that the same input data can produce the same output result, that is, the input-output mapping relationship of the target matrix multiplication remains unchanged.

[0036] The embodiments of this application can perform local transformations on the initial data flow scheme to generate candidate data flow schemes that are semantically equivalent to the initial data flow scheme, thus providing a candidate space for the search of the target data flow scheme.

[0037] As one possible implementation, embodiments of this application can employ the flip operation as a fundamental local transformation of the equivalent data stream space. Given two rank terms in scheme S: , , If the two combinations share the same linear combination along a certain factor dimension, a cross term O can be constructed, and a new combination of two rank terms can be obtained by adding or subtracting this cross term. The new combination is consistent with the original combination in terms of tensor sum, therefore the corresponding matrix multiplication calculation satisfies the semantic equivalence condition.

[0038] More generally, the flip operation can be performed on any one of the shared dimensions of the three factors A, B, and C. Its function is to change the intermediate linear combinations, multiplication terms, and output accumulation methods without altering the overall matrix multiplication tensor, thereby obtaining a candidate data flow scheme S'.

[0039] In the embodiments of this application, flipping can be used not only to find solutions with lower rank, but also to search for data flows that are better executed by the hardware. That is, even if the rank is exactly the same before and after flipping, as long as the post-compilation latency is different, the flip still has search value.

[0040] In step S103, the delay time of the candidate data flow scheme is determined according to the target hardware compiler, and a random walk search is performed in the candidate data flow scheme with the delay time as the optimization target until the preset termination condition is met, so as to obtain the data flow search result of the target sparse matrix multiplication.

[0041] In the embodiments of this application, random walk search refers to using the current data flow scheme as the search state, generating new candidate data flow schemes by randomly selecting and setting transformation operations, and evaluating the candidate schemes based on the execution latency fed back by the target hardware compiler to iteratively update the search state and gradually obtain a data flow scheme that meets the optimization objective. The preset termination condition can be the random walk length, time budget, or convergence condition, etc., and is not specifically limited here.

[0042] It can be noted that the target hardware may include, but is not limited to, CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), NPU (Neural Processing Unit), ASIC (Application Specific Integrated Circuit), etc.

[0043] In this embodiment, the candidate data flow scheme can be compiled according to the compiler corresponding to the target hardware, the candidate data flow scheme can be mapped to the target hardware execution resources, the corresponding scheduling information and execution code can be generated, and the delay time of the candidate data flow scheme can be determined according to the scheduling information, resource occupation information and hardware execution feedback information.

[0044] Furthermore, embodiments of this application can optimize by minimizing the execution latency of candidate data flow schemes on the target hardware. The latency of each candidate data flow scheme is determined based on the scheduling results output by the target hardware compiler and the hardware execution feedback. The target data flow scheme with the minimum latency is then determined from multiple candidate data flow schemes to obtain the data flow search result. This allows the determined data flow scheme to be adapted to the computing unit, storage structure, and data transmission characteristics of the target hardware, reducing invalid computation and data transfer overhead, and improving the execution performance of sparse matrix multiplication.

[0045] Optionally, in one embodiment of this application, determining the delay time of a candidate data stream scheme according to the target hardware compiler includes: obtaining the hash key of the candidate data stream scheme; filtering the candidate data stream scheme based on the hash key to obtain a first candidate scheme that meets the preset structural conditions; and determining the delay time of the first candidate scheme according to the target hardware compiler.

[0046] Understandably, since compiler evaluation is more expensive than pure algebraic evaluation, caching and filtering mechanisms are introduced to cache the compilation results, scheduling information and performance metrics of evaluated candidate dataflow schemes, and to reuse existing evaluation results in subsequent search processes. At the same time, low-performance candidate schemes are eliminated in advance according to filtering conditions, thereby reducing the number of invalid compilations and accelerating the search process for the target dataflow scheme.

[0047] The preset structural conditions can be that the addition nodes, output depth, resource requirements, etc. all meet the corresponding requirements, and no specific restrictions are imposed here.

[0048] First, in this embodiment, the candidate data stream scheme can be normalized to generate a unique or nearly unique hash key, and the candidate data stream scheme can be determined based on the hash key. If it has been evaluated, the cache latency is read directly; if not, the latency of the candidate data stream scheme is evaluated.

[0049] Secondly, embodiments of this application can screen candidate data flow schemes. For example, embodiments of this application can quickly screen candidate data flow schemes based on factors such as the number of adder nodes, output accumulation depth, number of storage accesses, and hardware resource requirements, eliminating obviously inferior schemes. This reduces the number of candidate schemes in the subsequent compilation and evaluation stage, saves compilation resources and performance evaluation overhead, and improves the efficiency of data flow scheme search.

[0050] Subsequently, embodiments of this application can compile the selected candidate data stream schemes according to the compiler corresponding to the target hardware to determine their delay time.

[0051] As one possible implementation, in one embodiment of this application, determining the delay time of a first candidate scheme according to a target hardware compiler includes: converting the first candidate scheme into a second candidate scheme that satisfies a preset unified condition; determining invalid input items, invalid intermediate items, and invalid output accumulation items of the second candidate scheme according to a target sparse mask; determining a third candidate scheme according to the invalid input, invalid intermediate items, and invalid output accumulation; invoking the target hardware compiler to convert the third candidate scheme into a target instruction sequence or hardware scheduling result; and determining the delay time according to the target instruction sequence or hardware scheduling result.

[0052] It can be noted that the preset uniform conditions can be uniform U / V / W coefficient matrices, expression files, computation graphs, etc.

[0053] First, the embodiments of this application can convert S' into a unified intermediate representation, such as a U / V / W coefficient matrix, expression file, computation graph, or IR (Intermediate Representation), to achieve a standardized expression of the data flow scheme, thereby eliminating the differences between different algorithm representations and enabling different data flow schemes to be compared, transformed, and optimized within a unified representation space.

[0054] Secondly, according to the sparse pattern M, the embodiments of this application can determine the validity of the input nodes, intermediate computation nodes, and output accumulation nodes of the intermediate representation to delete invalid inputs, invalid intermediate terms, and invalid output accumulations. At the same time, the embodiments of this application can perform common subexpression elimination and addition tree reconstruction on the intermediate representation, such as the computation graph, to merge duplicate computation nodes, reduce redundant computations, and adjust the topology of the output accumulation path to reduce the computation dependency depth and improve the parallel execution capability and hardware resource utilization efficiency of the computation graph.

[0055] Subsequently, embodiments of this application can invoke the target hardware compiler to convert the sparse-aware computation graph into a target instruction sequence or hardware scheduling result, and complete operator binding, register allocation, on-chip memory allocation, VLIW (Very Long Instruction Word) packing and pipeline scheduling; further, embodiments of this application can invoke an instruction-level simulator or compiler profiler to obtain the cycle-accurate latency of the candidate scheme on the target hardware based on the target instruction sequence or hardware scheduling result. Finally, embodiments of this application can feed back metrics such as latency, instruction length, critical path, bank (memory) conflict, resource usage, and compilation failure information to the search engine.

[0056] In the embodiments of this application, the optimization objective can be defined as: , in, Indicates the optimization objective. This represents the cycle-level latency given by the compiler and performance simulator. Indicates the candidate data stream scheme, M Represents a sparse mask. H This represents the target hardware constraint. The primary objective in this objective function is always physical execution latency, not algebraic rank.

[0057] In some cases, in one embodiment of this application, the candidate data flow scheme is screened, including: determining a first screening scheme based on at least one of the addition nodes, output depth, resource requirements, and data dependencies of the candidate data flow scheme; determining a second screening scheme based on the local change regions of the candidate data flow scheme; recording the local structural features corresponding to the candidate data flow scheme that satisfy at least one of the preset compilation failure conditions, preset hardware resource overrun conditions, or preset scheduling timeout conditions, and determining a third screening scheme based on the local structural features; and screening the candidate data flow schemes according to the first screening scheme, the second screening scheme, and the third screening scheme.

[0058] Specifically, firstly, this application embodiment filters obviously inferior schemes before calling the compiler, such as schemes with too many addition nodes, too large output depth, resource requirements exceeding the hardware limit, or containing unschedulable dependency loops, so as to screen candidate data flow schemes through structural pruning.

[0059] Secondly, for local changes caused by only one flip, the unchanged computation graph, intermediate representation and compilation optimization results in the previous candidate scheme are reused through incremental evaluation. Only the compilation mapping and performance evaluation information corresponding to the changed part are updated to reduce the repeated compilation process and reduce the evaluation overhead of the candidate scheme.

[0060] Furthermore, embodiments of this application may record local structures that cause compilation failure, resource exhaustion, or scheduling timeout, in order to reduce their sampling probability in subsequent searches.

[0061] The embodiments of this application can first use the above-mentioned fast structural indicators for pre-screening, and then perform a complete cycle-accurate compilation evaluation on promising candidate dataflow schemes, which can... Hardware feedback is continuously used in level random walks, without having to perform a full, cost-effective compilation of all candidate data stream schemes.

[0062] In actual implementation, the embodiments of this application can use a random walk search method based on the flip operation to traverse the equivalent data stream space. Let the length of the random walk be... In one embodiment The random walk search process may include the following steps: 1) Initial matrix multiplication data flow scheme S 0 is used as the current solution, and the initial latency is obtained through compiler evaluation; 2) In the current plan S t Enumerate or sample the set of valid flips F( S t ); 3) From F( S t Randomly select a flip operation from the given information to generate candidate solutions S'. 4) Perform semantic correctness checks and sparse mask validity checks on S'; 5) Call the compiler and performance simulator to obtain Score(S'); 6) If Score(S') is better than the current solution or the globally optimal solution, then accept S' and update the optimal record; 7) If the score(S') is not better than the current solution, then accept the candidate according to the set probability, annealing strategy or diversity strategy to avoid getting trapped in local optima; 8) Repeat the above process until the random walk length is reached. Time budget or convergence conditions.

[0063] In one embodiment, the acceptance criteria may take the following form: If L(S') <L(S t If ), then accept S'; If L(S') ≥ L( S t ), then with probability exp(-(L(S')-L( S t Accept S'; Here, τ is a temperature parameter that can be gradually decreased with the number of search iterations. Alternatively, ε-greedy algorithms, multi-starting-point random walks, or retaining a top-k (top K optimal candidates) candidate pool can be used to enhance search coverage.

[0064] The following specific example further illustrates the sparse matrix data stream search method based on a hardware compiler according to embodiments of this application. The algorithm flow of embodiments of this application may include the following steps: Input: Initial data flow scheme S 0, sparse mask M, target hardware description H, compiler C, random walk length Accept strategy A.

[0065] Output: Low-latency data stream solution S best and its compiled instruction sequence Inst best .

[0066] The specific process is as follows: 1) S 0 is converted into a sparse perceptual computational graph G( S 0,M); 2) Call the C compiler to generate the instruction sequence, and obtain L( ) through the profiler. S 0); 3) Order S best = S 0, L best =L( S 0); 4) For t=1,…, implement: From the current plan S t Sample a flip from the set of valid flips; 5) Execute the flip to obtain candidate solution S'; 6) Perform correctness checks, sparse mask checks, and structural pruning on S'; 7) If S' already exists in the cache, read the cache latency; otherwise, call the compiler C and profiler to get L(S') and write it to the cache; 8) Based on L(S') and acceptance strategy A, decide whether to use S' as the next current option: If L(S') < L best Then update S best and L best t, and save the corresponding instruction sequence.

[0067] Returns S_best, L_best, Inst_best, and the search log.

[0068] In practical implementation, in an embodiment for sparse attention or sparse linear layers of a large language model, this application embodiment first provides a sparse mask of the weight matrix or attention matrix and generates an initial fast matrix multiplication data stream. The searcher performs a flip random walk on this data stream to generate a series of semantically equivalent sparse data stream candidates.

[0069] Each candidate is converted into a compiler IR and fed into the hardware compiler. The compiler generates a sequence of instructions executable by the target accelerator and obtains the execution cycle count for that candidate using `inst_profiler`. The searcher does not use the rank of the candidate solutions as the primary sorting criterion, but rather uses the cycle-accurate latency returned by the profiler as the optimization target. The final output solution may not be the rank-minimum solution, but it will have lower actual instruction length, pipeline pauses, and execution cycle count on the target hardware.

[0070] In the embodiments of this application, AlphaTensor (a reinforcement learning-based tensor decomposition search algorithm) optimizes the number of multiplications, while the sparse matrix data stream search method based on a hardware compiler optimizes the physical execution latency measured by the compiler.

[0071] The sparse matrix data flow search method based on a hardware compiler proposed in this application transforms the intermediate linear combinations, multiplication terms, and output accumulation methods of the initial data flow scheme to obtain candidate data flow schemes that satisfy preset semantic equivalence conditions. Then, using the latency of the target hardware compiler as the optimization target to drive random walk search, it can explore more data flow implementations adapted to the target hardware while maintaining the overall computational semantics of matrix multiplication. Compared to methods that optimize only based on theoretical multiplication counts or algebraic complexity, this scheme comprehensively considers compiler scheduling, memory access, computational resource allocation, and hardware execution characteristics, effectively reducing actual execution latency and improving the computational efficiency and hardware resource utilization of sparse matrix multiplication. Simultaneously, by avoiding getting trapped in local optima through random walk search, it enhances the ability to discover optimal data flow schemes, thereby obtaining data flow implementation schemes that balance algorithm complexity and hardware execution performance.

[0072] Next, referring to the accompanying drawings, a sparse matrix data stream search device based on a hardware compiler, according to an embodiment of this application, is described.

[0073] Figure 2 This is a block diagram of a sparse matrix data stream search device based on a hardware compiler according to an embodiment of this application.

[0074] like Figure 2 As shown, the sparse matrix data stream search device 10 based on a hardware compiler includes: a determination module 100, a transformation module 200, and a search module 300.

[0075] The determination module 100 is used to determine the initial data flow scheme for the target sparse matrix multiplication.

[0076] The transformation module 200 is used to transform at least one of the intermediate linear combination, multiplication terms and output accumulation methods of the initial data flow scheme to obtain a candidate data flow scheme that satisfies the preset semantic equivalence conditions.

[0077] The search module 300 is used to determine the delay time of the candidate data flow scheme according to the target hardware compiler, and to perform a random walk search in the candidate data flow scheme with the delay time as the optimization target until the preset termination condition is met, so as to obtain the data flow search result of the target sparse matrix multiplication.

[0078] Optionally, in one embodiment of this application, the search module 300 includes: an acquisition unit, a filtering unit, and a determination unit.

[0079] The acquisition unit is used to acquire the hash key of the candidate data stream scheme.

[0080] The filtering unit is used to filter candidate data stream schemes based on hash keys to obtain the first candidate scheme that meets the preset structural conditions.

[0081] The determination unit is used to determine the delay time of the first candidate scheme based on the target hardware compiler.

[0082] Optionally, in one embodiment of this application, the determining unit includes: a first conversion subunit, a first determining subunit, a second determining subunit, a second conversion subunit, and a third determining unit.

[0083] The first conversion subunit is used to convert the first candidate scheme into a second candidate scheme that meets the preset unified conditions.

[0084] The first determining subunit is used to determine the invalid input items, invalid intermediate items, and invalid output accumulation items of the second candidate scheme based on the target sparse mask.

[0085] The second determining subunit is used to determine the third candidate scheme by accumulating invalid inputs, invalid intermediate terms, and invalid outputs.

[0086] The second conversion subunit is used to call the target hardware compiler to convert the third candidate scheme into a target instruction sequence or hardware scheduling result.

[0087] The third determining unit is used to determine the delay time based on the target instruction sequence or hardware scheduling result.

[0088] Optionally, in one embodiment of this application, the filtering unit includes: a first filtering subunit, a second filtering subunit, a third filtering subunit, and a fourth filtering subunit.

[0089] The first screening subunit is used to determine the first screening scheme based on at least one of the following: the addition node, output depth, resource requirements, and data dependencies of the candidate data flow scheme.

[0090] The second screening subunit is used to determine the second screening scheme based on the local variation region of the candidate data stream scheme.

[0091] The third screening subunit is used to record the local structural features of candidate data flow schemes that meet at least one of the preset compilation failure conditions, preset hardware resource over-limit conditions, or preset scheduling timeout conditions, and to determine the third screening scheme based on the local structural features.

[0092] The fourth filtering subunit is used to filter candidate data stream schemes according to the first filtering scheme, the second filtering scheme, and the third filtering scheme.

[0093] Optionally, in one embodiment of this application, the calculation formula for the optimization target is: , in, Indicates the optimization objective. This represents the cycle-level latency given by the compiler and performance simulator. Indicates the candidate data stream scheme, M Represents a sparse mask. H This indicates the target hardware constraints.

[0094] It should be noted that the foregoing explanation of the embodiment of the sparse matrix data stream search method based on a hardware compiler also applies to the sparse matrix data stream search device based on a hardware compiler in this embodiment, and will not be repeated here.

[0095] The sparse matrix data stream search device based on a hardware compiler proposed in this application transforms the intermediate linear combination, multiplication terms, and output accumulation method of the initial data stream scheme to obtain candidate data stream schemes that satisfy preset semantic equivalence conditions. Then, the random walk search is driven by the latency of the target hardware compiler as the optimization target. This allows for the exploration of more data stream implementations adapted to the target hardware while maintaining the overall computational semantics of matrix multiplication. Compared with methods that optimize only based on theoretical multiplication counts or algebraic complexity, this scheme can comprehensively consider compilation scheduling, memory access, computational resource allocation, and hardware execution characteristics, effectively reducing actual execution latency and improving the computational efficiency and hardware resource utilization of sparse matrix multiplication.

[0096] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 301, the processor 302, and the computer program stored on the memory 301 and capable of running on the processor 302.

[0097] When the processor 302 executes the program, it implements the sparse matrix data stream search method based on the hardware compiler provided in the above embodiments.

[0098] Furthermore, electronic devices also include: Communication interface 303 is used for communication between memory 301 and processor 302.

[0099] The memory 301 is used to store computer programs that can run on the processor 302.

[0100] The memory 301 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0101] If the memory 301, processor 302, and communication interface 303 are implemented independently, then the communication interface 303, memory 301, and processor 302 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0102] Optionally, in a specific implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface.

[0103] Processor 302 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0104] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described sparse matrix data stream search method based on a hardware compiler.

[0105] This application also provides a computer program product, including a computer program that can run computer instructions. When the computer instructions are executed by a processor, they implement the sparse matrix data stream search method based on a hardware compiler provided in this application.

[0106] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0107] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0108] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0109] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0110] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0111] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0113] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A sparse matrix data stream search method based on a hardware compiler, characterized in that, Includes the following steps: Determine the initial data flow scheme for the target sparse matrix multiplication; Transform at least one of the intermediate linear combination, multiplication terms, and output accumulation methods of the initial data flow scheme to obtain a candidate data flow scheme that satisfies a preset semantic equivalence condition; The delay time of the candidate data flow scheme is determined by the target hardware compiler, and a random walk search is performed on the candidate data flow scheme with the delay time as the optimization target until a preset termination condition is met, so as to obtain the data flow search result of the target sparse matrix multiplication.

2. The method according to claim 1, characterized in that, The step of determining the delay time of the candidate data stream scheme based on the target hardware compiler includes: Obtain the hash key of the candidate data stream scheme; Based on the hash key, the candidate data stream schemes are filtered to obtain a first candidate scheme that meets the preset structural conditions; The delay time of the first candidate scheme is determined according to the target hardware compiler.

3. The method according to claim 2, characterized in that, The step of determining the delay time of the first candidate scheme according to the target hardware compiler includes: The first candidate solution is converted into a second candidate solution that meets the preset unified conditions; The invalid input items, invalid intermediate items, and invalid output accumulation items of the second candidate scheme are determined based on the target sparse mask. A third candidate solution is determined by accumulating the invalid inputs, invalid intermediates, and invalid outputs. The target hardware compiler is invoked to convert the third candidate scheme into a target instruction sequence or hardware scheduling result; The delay time is determined based on the target instruction sequence or the hardware scheduling result.

4. The method according to claim 2, characterized in that, The process of filtering the candidate data stream schemes includes: The first screening scheme is determined based on at least one of the following: addition node, output depth, resource requirements, and data dependencies of the candidate data stream scheme; A second screening scheme is determined based on the local variation region of the candidate data stream scheme; Record the local structural features corresponding to the candidate data flow scheme that meet at least one of the preset compilation failure conditions, preset hardware resource limit conditions, or preset scheduling timeout conditions, and determine the third screening scheme based on the local structural features; The candidate data stream schemes are filtered according to the first filtering scheme, the second filtering scheme, and the third filtering scheme.

5. The method according to claim 1, characterized in that, The formula for calculating the optimization objective is: ， in, Indicates the optimization objective. This represents the cycle-level latency given by the compiler and performance simulator. Indicates the candidate data stream scheme, M Represents a sparse mask. H This indicates the target hardware constraints.

6. A sparse matrix data stream search device based on a hardware compiler, characterized in that, include: The determination module is used to determine the initial data flow scheme for the target sparse matrix multiplication; The transformation module is used to transform at least one of the intermediate linear combination, multiplication terms and output accumulation methods of the initial data flow scheme to obtain a candidate data flow scheme that satisfies the preset semantic equivalence condition. The search module is used to determine the delay time of the candidate data flow scheme according to the target hardware compiler, and to perform a random walk search in the candidate data flow scheme with the delay time as the optimization target until a preset termination condition is met, so as to obtain the data flow search result of the target sparse matrix multiplication.

7. The apparatus according to claim 6, characterized in that, The search module includes: The acquisition unit is used to acquire the hash key of the candidate data stream scheme; The filtering unit is used to filter the candidate data stream schemes based on the hash key to obtain a first candidate scheme that meets the preset structural conditions. A determining unit is configured to determine the delay time of the first candidate scheme based on the target hardware compiler.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, the processor executing the program to implement the sparse matrix data stream search method based on a hardware compiler as described in any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the hardware compiler-based sparse matrix data stream search method as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the sparse matrix data stream search method based on a hardware compiler as described in any one of claims 1-5.

Images