Operator tiling parameter determination method based on evolutionary calculation and hardware-in-loop feedback

Abstract

The operator Tiling parameter determination method based on evolutionary calculation and hardware-in-the-loop feedback provided by the embodiments of the present disclosure comprises: determining a search space model of Tiling parameters according to architecture parameters of a target hardware platform and an operator calculation mode corresponding to the target hardware platform; determining an initial candidate parameter set corresponding to the Tiling parameters according to the search space model; determining a target compilation file according to each initial candidate parameter vector, and obtaining a target fitness value of the target hardware platform running the target compilation file; based on an evolutionary calculation strategy, iteratively evolving the initial candidate parameter set according to the target fitness value of the target hardware platform under each initial candidate parameter vector, and determining a next group of candidate parameter set in the search space model until an iteration condition is met. The efficiency of determining the Tiling parameters is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology and related technical fields, specifically to a method for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback. Background Technology

[0002] In the field of high-performance computing, operators (or kernels) are the smallest units that perform specific mathematical calculations during AI model training and inference, such as matrix multiplication (GEMM), attention mechanisms, and layer normalization. The execution efficiency of operators on hardware is highly dependent on the configuration of the tiling parameter—tiling refers to dividing large-scale computational tasks into small blocks suitable for on-chip cache processing and orchestrating the computation and data transport pipelines of these small blocks.

[0003] Currently, the industry mainly relies on manual experience-based tuning, exhaustive grid search, and cost model-based static prediction methods for configuring Tiling parameters. However, manual experience-based tuning is highly dependent on system experts and is inefficient, exhaustive grid search has low search efficiency, and cost model-based static prediction methods are limited by model accuracy, resulting in significant deviations between predicted and actual performance.

[0004] Given the problems with existing technologies, there is an urgent need for a method to determine operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback. Summary of the Invention

[0005] In view of this, the embodiments described herein provide a method for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback, which improves the efficiency of determining Tiling parameters.

[0006] In a first aspect, according to the content of this disclosure, a method for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback is provided, including: Based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform, the search space model of the Tiling parameter is determined. The architecture parameters include at least the processor number information, the first cache capacity information, the second cache capacity information, the memory bandwidth information, and the computing unit type information. Based on the search space model, an initial candidate parameter set corresponding to the Tiling parameter is determined, wherein the initial candidate parameter set includes multiple initial candidate parameter vectors; Based on each of the initial candidate parameter vectors, the target compilation file is determined, and the target fitness value for running the target compilation file on the target hardware platform is obtained; Based on the target fitness value of the target hardware platform under each initial candidate parameter vector, the initial candidate parameter set is iteratively evolved based on the evolutionary computation strategy, and the next set of candidate parameters is determined in the search space model until the iteration condition is met.

[0007] In some embodiments of this disclosure, determining the search space model for the Tiling parameter based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform includes: Obtain the architecture parameters and operator computation modes of the target hardware platform; Based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform, the dimensions and constraints of each Tiling parameter are determined. The Tiling parameters include at least the block size, pipeline depth, double buffer switch, loop unrolling factor, vectorization width, and on-chip cache allocation strategy. The dimensions and constraints of each Tiling parameter constitute the search space model.

[0008] In some embodiments of this disclosure, determining the initial candidate parameter set corresponding to the Tiling parameter based on the search space model includes: Based on the Bayesian optimization algorithm, Latin hypersquare sampling is used to select a first initial candidate parameter vector for a first number of objectives from the search space model; or, Based on the genetic algorithm, a second initial candidate parameter vector is randomly selected from the search space model to determine the second target number.

[0009] In some embodiments of this disclosure, determining the target compilation file based on each of the initial candidate parameter vectors and obtaining the target fitness value for the target hardware platform running the target compilation file includes: Based on the initial candidate parameter vectors, determine the operator source code file; The compiler on the target hardware platform is invoked to compile the operator source code file, generate a target compiled file, and send the target compiled file to the target compilation platform so that the target compiled file can be run on the target hardware platform. Obtain the fitness sequence of the target hardware platform running the target compiled file, and determine the target fitness value corresponding to the target hardware platform running the target compiled file based on the fitness value sequence.

[0010] In some embodiments of this disclosure, determining the target fitness value corresponding to the target compiled file running on the target hardware platform based on the fitness value sequence includes: Based on the fitness value sequence, the fitness values ​​in the fitness value sequence are sorted from smallest to largest; The median of the sorted fitness value sequence is selected as the target fitness value for the target hardware platform running the target compiled file.

[0011] In some embodiments of this disclosure, the evolutionary computation strategy includes a Bayesian optimization strategy; The step of iteratively evolving the initial candidate parameter set based on the fitness values ​​of the target hardware platform under each initial candidate parameter vector, using an evolutionary computation strategy, to determine the next set of candidate parameters includes: Using the initial candidate parameter vector minus the target fitness value as the observation data, a Gaussian process surrogate model is fitted, wherein the Gaussian process surrogate model outputs the predicted mean and predicted variance of the first candidate parameter vector; The next set of candidate parameters is determined based on the predicted mean, predicted variance, and acquisition function of each first candidate parameter vector.

[0012] In some embodiments of this disclosure, the evolutionary computation strategy includes a genetic algorithm; The step of iteratively evolving the initial candidate parameter set based on the fitness values ​​of the target hardware platform under each initial candidate parameter vector, using an evolutionary computation strategy, to determine the next set of candidate parameters includes: Based on the target fitness value corresponding to each initial candidate parameter vector, sort the initial candidate parameter vectors according to the target fitness value, and select the first number of initial candidate parameter vectors from the sorted initial candidate parameter vectors as the first candidate parameter set; The first candidate parameter set is processed based on the tournament selection mechanism to obtain the second candidate parameter set; The second candidate parameter set is processed based on the cross operation to obtain the third candidate parameter set; The third candidate parameter set is processed based on the mutation operation to obtain the fourth candidate parameter set; The fourth candidate parameter set is processed based on the constraint repair operation to obtain the next set of candidate parameters.

[0013] Secondly, according to this disclosure, an operator Tiling parameter determination device based on evolutionary computation and hardware-in-the-loop feedback is provided, comprising: The search space model determination module is used to determine the search space model of the Tiling parameter based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform. The architecture parameters include at least processor number information, first cache capacity information, second cache capacity information, memory bandwidth information and computing unit type information. The initial candidate parameter set determination module is used to determine the initial candidate parameter set corresponding to the Tiling parameter according to the search space model, wherein the initial candidate parameter set includes multiple initial candidate parameter vectors; The fitness value determination module is used to determine the target compilation file based on each of the initial candidate parameter vectors, and to obtain the target fitness value of the target hardware platform running the target compilation file; The target candidate parameter determination module is used to iteratively evolve the initial candidate parameter set based on the target fitness value of the target hardware platform under each initial candidate parameter vector, and determine the next set of candidate parameters in the search space model until the iteration conditions are met.

[0014] Thirdly, according to this disclosure, a computer device is provided, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any of the first aspects.

[0015] Fourthly, according to this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described in any of the first aspects.

[0016] The operator Tiling parameter determination method based on evolutionary computation and hardware-in-the-loop feedback provided in this disclosure first determines the search space model of the Tiling parameters according to the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform. Then, based on the search space model, the initial candidate parameter set corresponding to the Tiling parameters is determined. Next, based on each initial candidate parameter vector, the target compilation file is determined, and the target fitness value of the target hardware platform running the target compilation file is obtained. Finally, based on the target fitness value of the target hardware platform under each initial candidate parameter vector, the initial candidate parameter set is iteratively evolved based on the evolutionary computation strategy, and the next set of candidate parameter sets is determined in the search space model until the iteration conditions are met. By constructing a closed-loop automatic search mechanism of "candidate parameter generation → hardware execution compilation → performance index collection → feedback-driven adjustment of search direction", the near-globally optimal Tiling parameter configuration on the target hardware platform is found within a limited number of search iterations.

[0017] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. It should be understood that the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure, wherein: Figure 1 This is a flowchart illustrating a method for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback, provided in an embodiment of this disclosure. Figure 2 This is a schematic diagram of the structure of an operator Tiling parameter determination device based on evolutionary computation and hardware-in-the-loop feedback provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure.

[0019] In the accompanying diagram, markers with the same last two digits correspond to the same elements. It should be noted that the elements in the diagram are schematic and not drawn to scale. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are also within the scope of protection of this disclosure.

[0021] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, the statement of “connecting” or “coupling” two or more parts together shall mean that these parts are directly joined together or joined through one or more intermediate components.

[0022] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of the phrase "embodiment" in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] In this article, the terms "or" or "or" are merely a way of describing the relationship between related objects, indicating that there can be three kinds of relationships. For example, A or B can mean that only A exists, both A and B exist, or only B exists.

[0024] Furthermore, in all embodiments of this disclosure, terms such as “first” and “second” are used only to distinguish one component (or part of a component) from another component (or another part of a component).

[0025] In the description of this application, unless otherwise stated, "multiple" means two or more (including two), and similarly, "multiple groups" means two or more (including two groups).

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0027] To address the problems existing in the prior art, this disclosure provides a method for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback. Figure 1 This is a flowchart illustrating a method for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback, as provided in an embodiment of this disclosure. Figure 1 As shown, the operator Tiling parameter determination method based on evolutionary computation and hardware-in-the-loop feedback includes: S110. Determine the search space model for the Tiling parameters based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform.

[0028] The architecture parameters include at least the number of processors, the capacity of the first cache, the capacity of the second cache, the memory bandwidth, and the type of computing unit.

[0029] Specifically, once the target hardware platform is determined, its corresponding architecture parameters are fixed. These parameters include the number of processors, the first cache capacity, the second cache capacity, memory bandwidth, and the type of computing unit. Furthermore, the target hardware platform supports various operator computation modes, including element-wise operator computation, reduction operator computation, matrix multiplication operator computation, and attention mechanism operator computation.

[0030] Specifically, the number of processors in the architecture parameters of the target hardware platform determines the parallel granularity of the target hardware platform. For example, during optimization, the total computing tasks (Tile blocks) need to be reasonably allocated to all processors. If the Tile block is too large, the processor may be idle; if the Tile block is too small, the scheduling overhead of the processor will increase dramatically. The first cache capacity information refers to the capacity information corresponding to the first cache. The first cache is a high-speed temporary storage device that is closest to the processor, with limited capacity (usually a few hundred KB) but extremely fast. The total amount of data in a single Tile block must be less than the cache capacity corresponding to the first cache; otherwise, the data cannot be loaded, which will lead to repeated data transfer from the slow cache, resulting in a sharp drop in performance. Therefore, the Block size (Mx) in the Tiling parameter is crucial. The product of KxN is strictly constrained by the cache capacity corresponding to the first cache. The second cache capacity information is the capacity information corresponding to the second cache. The second cache is closer to the computing unit than the first cache, and its capacity is extremely small (usually tens of KB). It is used for input / output buffering of a specific computing unit. The second cache capacity information determines the granularity of micro-tiling in the pipeline and affects the allocation of vectorization width and register pressure. The memory bandwidth information reflects the data exchange rate between the external video memory (such as HBM) and the internal memory of the chip corresponding to the target hardware platform. The unit is usually GB / s. During optimization, the choice of pipeline depth and double buffering strategy depends on the memory bandwidth information. The computing unit type refers to the specialized hardware circuit inside the chip responsible for performing different mathematical operations.

[0031] The computation process of the target hardware platform varies under different computing modes. Among them, the element-wise (such as Add, ReLU) operator computation mode has consistent input and output shapes, no data dependency, no reduction dimension, and extremely low computational intensity (memory-intensive). The reduction (such as the mean squared error of Sum, LayerNorm) operator computation mode has a reduction axis that needs to be accumulated (such as summing the last dimension), and the data is dependent on the reduction axis, requiring cross-thread communication. The matrix multiplication (GEMM) operator computation mode has a clear three-dimensional loop of M (batch), N (output feature), and K (reduction accumulation). The attention mechanism operator computation mode includes multiple stages, with the Q×K^T part belonging to matrix multiplication (reduction dimension is Head_Dim), the Softmax part belonging to exponential reduction, and finally weighted summation.

[0032] The block dimensions differ depending on the operator computation mode. Specifically, the element-wise operator computation mode only needs to divide the total data volume (one-dimensional linear block) and set Block_Size (the number of elements processed per core); the matrix multiplication operator computation mode needs to divide into M and N dimensions (for spatial parallelism, allocating to different cores), and also divide into K dimensions (for temporal block, batching data into the cache), thus generating dimensions including Block_M, Block_N, and Block_K; the attention mechanism operator computation mode involves a complex 4D tensor, dividing into Batch_Size (batch size), Q_Seq_Length (query sequence length), and Head_Dim (head dimension). In addition, for the intermediate result matrix S = Q The K^T segmentation must take into account the sequence length dimension.

[0033] After determining the architecture parameters and operator computation mode of the target hardware platform, the search space model for the Tiling parameters is determined.

[0034] In a specific implementation, the search space model for the Tiling parameters is determined based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform. This includes: obtaining the architecture parameters and operator computation mode of the target hardware platform; and determining the dimensions and constraints of each Tiling parameter based on the architecture parameters and operator computation mode corresponding to the target hardware platform. The Tiling parameters include at least the block size, pipeline depth, double buffer switch, loop unrolling factor, vectorization width, and on-chip cache allocation strategy. The dimensions and constraints of each Tiling parameter constitute the search space model.

[0035] Specifically, the constraints on the Block size (including one-dimensional or multi-dimensional block size) in the Tiling parameter are related to the first cache capacity information; the constraints on the pipeline depth are related to the memory bandwidth information; the constraints on the double buffer switch are related to the remaining cache capacity information of the first cache; the constraints on the loop unrolling factor are related to the heap depth of the physical registers of the target hardware platform; the constraints on the vectorization width are related to the bit width of the target hardware platform; and the constraints on the on-chip cache allocation strategy are related to the first cache capacity information.

[0036] The dimension of the Tiling parameter is related to the operator computation mode. If the operator computation mode is element-wise operator computation mode, then the Tiling parameter has a one-dimensional dimension, including Block_Size; if the operator computation mode is matrix multiplication operator computation mode, then the Tiling parameter has a three-dimensional dimension, including Block_M, Block_N, and Block_K; if the operator computation mode is attention mechanism operator computation mode, then the Tiling parameter has a four-dimensional dimension, including Batch, Seq_Q, Seq_K, and Head_Dim.

[0037] Furthermore, it should be noted that the existence of certain Tiling parameters is determined by the operator computation mode, but the mathematical form of their constraints is determined by the architectural parameters of the target hardware platform. For example, the existence of the loop unrolling factor parameter depends on whether there is an expandable inner loop inside the operator—this is determined by the operator computation mode (the inner product loop of GEMM can be unrolled, while element-wise loops do not need to be unrolled). However, the maximum value that the loop unrolling factor can take is entirely determined by the depth of the physical register file of the target hardware platform, that is, by the architectural parameters of the target hardware platform.

[0038] It is understandable that the dimension of the Tiling parameter is determined by the operator computation mode, and the value range, step size, and constraints of the Tiling parameter are determined by the architecture parameters of the target hardware platform; the dimension, value range, step size, and constraints of each Tiling parameter together constitute the search space model.

[0039] S120. Based on the search space model, determine the initial candidate parameter set corresponding to the Tiling parameter.

[0040] The initial candidate parameter set includes multiple sets of initial candidate parameter vectors.

[0041] After determining the search space model of the Tiling parameters in step S110, multiple sets of initial candidate parameter vectors are determined in the search space model.

[0042] In a specific implementation, the initial candidate parameter set corresponding to the Tiling parameter is determined according to the search space model, including: selecting a first initial candidate parameter vector set with a first number of objectives in the search space model based on the Bayesian optimization algorithm and using Latin hypersquare sampling; or, randomly selecting a second initial candidate parameter vector with a second number of objectives in the search space model based on the genetic algorithm.

[0043] Specifically, based on the Bayesian optimization algorithm, Latin hypersquare sampling is used to search for 2×D sets of first initial candidate parameter vectors in the search space model, where D is the parameter dimension of the Tiling parameter; or based on the genetic algorithm, a second initial candidate parameter vector of a second target number is randomly generated, where the second target number is determined based on the population size in the search space model.

[0044] The first candidate parameter vector or the second candidate parameter vector constitutes the initial candidate parameter set.

[0045] In this embodiment of the disclosure, under the Bayesian optimization algorithm, Latin hypersquare sampling is used to select a first initial candidate parameter vector group with a first number of targets in the search space model. The technical effect is as follows: (1) To maximize the spatial coverage uniformity of the first initial candidate parameter vector group, Latin hypersquare sampling performs equal-probability stratification within the value interval of each parameter dimension and ensures that each stratified interval is sampled only once. This results in a maximized dispersion distribution of sample points throughout the parameter space, given a total sample size of only 2×D (where D is the parameter dimension). This mechanism effectively avoids the problem of local clustering or large-area blanking of sample points that may be caused by random sampling, thereby obtaining global prior information about the objective function within a limited sampling budget.

[0046] (2) Achieving a linear growth relationship between sampling overhead and dimension; the sample size of Latin hypersquare sampling grows linearly with the parameter dimension DD, rather than exponentially with grid search. This characteristic enables the present invention to establish a high-quality initial surrogate model with extremely low initial evaluation cost when facing high-dimensional Tiling parameter spaces (such as four-dimensional block parameters of attention mechanisms), significantly reducing the startup cost of hardware-in-the-loop evaluation.

[0047] (3) Ensuring unbiased prior construction for the Gaussian process surrogate model. Uniformly covered initial samples provide an unbiased mean prior and a stable basis for kernel function hyperparameter estimation for the Gaussian process. Compared with the anisotropic prediction variance caused by random sampling, Latin hypersquare sampling enables the surrogate model to have reliable initial prediction capabilities in all regions of the search space, thereby avoiding systematic deviation of subsequent sampling directions due to initial information bias.

[0048] In this embodiment of the genetic algorithm, a random generation strategy is used to construct a second initial candidate parameter vector for a second target number. The technical effect of this method is as follows: (1) Maximize population genetic diversity. The completely random generation method makes the initial individuals present a discrete distribution in the parameter space. Each candidate parameter combination maintains sufficient differences in value. This diversity provides rich genetic material for subsequent selection, crossover and mutation operations, effectively preventing the population from falling into local optima in the early stage of evolution due to gene convergence (i.e., premature convergence), and improving the global convergence ability of the algorithm in the entire space.

[0049] (2) Extremely low initial population construction overhead. Random generation only requires calling a pseudo-random number generator to sample parameters, without involving any objective function evaluation or surrogate model fitting. The computational complexity is O(P×D), and the upper limit of P (100) is much smaller than the single compilation execution overhead of hardware-in-the-loop evaluation. This low-overhead characteristic allows more computational budget to be allocated to subsequent iterative evolution stages, optimizing the resource allocation efficiency of the overall tuning task.

[0050] (3) Natural fit with the "survival of the fittest" mechanism of genetic algorithms. The core operations of genetic algorithms—tournament selection and elite retention—naturally possess the ability to automatically eliminate low-quality initial individuals. Even if some individuals in the initial random population are assigned the worst fitness due to violation of capacity constraints or compilation failure, the selection mechanism can effectively exclude them in the first generation of evolution without affecting the overall evolutionary direction of the population. This characteristic makes the random generation strategy highly robust in the genetic algorithm framework.

[0051] S130. Based on each initial candidate parameter vector, determine the target compilation file and obtain the target fitness value for running the target compilation file on the target hardware platform.

[0052] In a specific implementation, the target compiled file is determined based on each initial candidate parameter vector, and the fitness value of the target compiled file running on the target hardware platform is obtained. This includes: determining the operator source code file based on each initial candidate parameter vector; calling the compilation tool of the target hardware platform to compile the operator source code file, generating the target compiled file, and sending the target compiled file to the target compilation platform for running on the target hardware platform; obtaining the fitness value sequence of the target compiled file running on the target hardware platform, and determining the target fitness value corresponding to the target compiled file running on the target hardware platform based on the fitness value sequence.

[0053] First, based on each initial candidate parameter vector, a complete operator source code file that can be recognized by the target compilation tool is generated. Then, by calling the compilation tool of the target hardware platform, the operator source code file is compiled to generate a binary executable target compilation file. The target hardware platform then calls and runs the target compilation file to obtain the computation time series (i.e., fitness value series) of the target hardware platform under the candidate parameter vector. Finally, based on the fitness value series, the target fitness value corresponding to the target hardware platform running the target compilation file is determined.

[0054] Specifically, based on the fitness value sequence, the fitness values ​​in the fitness value sequence are sorted from smallest to largest; the median of the sorted fitness value sequence is selected as the target fitness value corresponding to the target compiled file running on the target hardware platform.

[0055] In the above embodiments, when the target hardware platform's compilation tool is called to compile the target source code file and generate the target compiled file, if the target hardware platform fails to compile the target compiled file, the target fitness value corresponding to the target compiled file is a preset fitness value.

[0056] S140. Based on the target fitness value of the target hardware platform under each initial candidate parameter vector, the initial candidate parameter set is iteratively evolved based on the evolutionary computation strategy, and the next set of candidate parameters is determined in the search space model until the iteration conditions are met.

[0057] As a specific implementable method, evolutionary computation strategies include Bayesian optimization strategies; Based on the fitness values ​​of the target hardware platform under each initial candidate parameter vector, the initial candidate parameter set is iteratively evolved using an evolutionary computation strategy to determine the next set of candidate parameters. This includes: using the initial candidate parameter vector minus the target fitness value as observation data, fitting a Gaussian process surrogate model, wherein the Gaussian process surrogate model outputs the predicted mean and predicted variance of any unsampled candidate parameter vector in the search space model; and determining the next set of candidate parameters based on the predicted mean, predicted variance, and acquisition function of any unsampled candidate parameter vector in the search space model.

[0058] Specifically, firstly, an observation dataset is constructed, consisting of vectors of initial candidate parameters and their corresponding target fitness values. The constructed observation dataset satisfies the following:

[0059] in, This is the initial candidate parameter vector for the i-th group (i.e., a complete set of Tiling parameter configurations in the search space model). Let be the target fitness value corresponding to the i-th initial candidate parameter vector.

[0060] Then, using the observation dataset Based on this, Gaussian process regression is used to construct the objective function. The probabilistic surrogate model, the Gaussian process is composed of the mean function Sum of covariance functions Complete characterization:

[0061] The covariance function is used to measure the similarity between any two sets of candidate parameter vectors in the search space model.

[0062] After fitting, for any unsampled candidate parameter point The Gaussian process surrogate model outputs its predicted mean. and prediction variance .

[0063] Then, based on the above steps, the average value is output. and prediction variance The acquisition function is calculated to guide the selection of the next candidate parameter vector, where the acquisition function is expressed as:

[0064] in, For the Gaussian process surrogate model at candidate parameter points The predicted mean at that location, For the Gaussian process surrogate model at candidate parameter points The prediction variance at the location is κ, which is the exploration-utilization balance coefficient and a real number greater than zero. It is used to control the weight relationship between the algorithm's tendency to explore unknown regions and its tendency to utilize known good value regions during the search process.

[0065] It should be noted that the κ value is initially set to 2.0, and decreases by 0.1 every 10 iterations until it reaches a minimum value of 0.2; alternatively, the exponential decay formula κ(t) = ×exp(-αt), where The initial value is α (2.0~3.0), the decay rate is α (0.01~0.05), and t is the current iteration number.

[0066] Since the fitness value in this embodiment is the kernel execution time (the smaller the better), the smaller the acquisition function value, the better the comprehensive evaluation of the candidate parameter points.

[0067] After determining the LCB acquisition function for each unsampled candidate parameter point, the next set of candidate parameters can be determined based on the LCB acquisition function value of each unsampled candidate parameter point.

[0068] As another specific implementation method, evolutionary computation strategies include genetic algorithms; Based on the fitness values ​​of the target hardware platform under each initial candidate parameter vector, the initial candidate parameter set is iteratively evolved using an evolutionary computation strategy to determine the next set of candidate parameters, including: Based on the target fitness values ​​corresponding to each initial candidate parameter vector, the initial candidate parameter vectors are sorted according to their target fitness values, and the first candidate parameter vectors with the first number of targets are selected from the sorted initial candidate parameter vectors as the first candidate parameter set. The first candidate parameter set is processed based on the tournament selection mechanism to obtain the second candidate parameter set. The second candidate parameter set is processed based on the crossover operation to obtain the third candidate parameter set. The third candidate parameter set is processed based on the mutation operation to obtain the fourth candidate parameter set. The fourth candidate parameter set is processed based on the constraint repair operation to obtain the next set of candidate parameter sets.

[0069] That is, through the above steps of the genetic algorithm, the next set of candidate parameters is determined. In the process of determining the next set of candidate parameters based on the genetic algorithm, the parameters such as crossover and mutation in the genetic algorithm are preset in advance.

[0070] It should be noted that the evolutionary computation strategy of the genetic algorithm can be performed simultaneously with the Bayesian optimization strategy, or the evolutionary computation strategy of the genetic algorithm can be performed after the Bayesian optimization strategy. This disclosure does not specifically limit this.

[0071] Furthermore, in the above embodiments, the iteration conditions include: (1) Reach the maximum number of iterations; (2) No better parameter combination than the current optimal combination was found in N consecutive iterations; (3) The current target fitness value has reached the user's preset fitness value.

[0072] This disclosure provides a method for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback. First, a search space model for the Tiling parameters is determined based on the architecture parameters of the target hardware platform and the corresponding operator computation mode. Then, based on the search space model, an initial set of candidate parameters corresponding to the Tiling parameters is determined. Next, based on each initial candidate parameter vector, a target compilation file is determined, and the target fitness value of the target hardware platform running the target compilation file is obtained. Finally, based on the target fitness value of the target hardware platform under each initial candidate parameter vector, the initial candidate parameter set is iteratively evolved using an evolutionary computation strategy, and the next set of candidate parameters is determined in the search space model until the iteration conditions are met. By constructing a closed-loop automatic search mechanism of "candidate parameter generation → hardware execution compilation → performance index collection → feedback-driven adjustment of search direction," the near-globally optimal Tiling parameter configuration on the target hardware platform is discovered within a limited number of search iterations. Compared to manual tuning and grid search, this significantly improves the efficiency and reliability of parameter determination.

[0073] Based on the above embodiments, this disclosure also provides an operator Tiling parameter determination device based on evolutionary computation and hardware-in-the-loop feedback. Figure 2 This is a schematic diagram of the structure of an operator Tiling parameter determination device based on evolutionary computation and hardware-in-the-loop feedback provided in an embodiment of this disclosure, as shown below. Figure 2 As shown, the operator Tiling parameter determination device based on evolutionary computation and hardware-in-the-loop feedback includes: The search space model determination module 210 is used to determine the search space model of the Tiling parameter based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform. The architecture parameters include at least processor number information, first cache capacity information, second cache capacity information, memory bandwidth information and computing unit type information. The initial candidate parameter set determination module 220 is used to determine the initial candidate parameter set corresponding to the Tiling parameter according to the search space model, wherein the initial candidate parameter set includes multiple initial candidate parameter vectors; The fitness value determination module 230 is used to determine the target compilation file based on each of the initial candidate parameter vectors, and to obtain the target fitness value of the target hardware platform running the target compilation file; The target candidate parameter determination module 240 is used to iteratively evolve the initial candidate parameter set based on the target fitness value of the target hardware platform under each initial candidate parameter vector, and determine the next set of candidate parameters in the search space model until the iteration conditions are met.

[0074] This disclosure provides an operator Tiling parameter determination device based on evolutionary computation and hardware-in-the-loop feedback. First, a search space model for the Tiling parameters is determined based on the architecture parameters of the target hardware platform and the corresponding operator computation mode. Then, based on the search space model, an initial candidate parameter set corresponding to the Tiling parameters is determined. Next, based on each initial candidate parameter vector, a target compilation file is determined, and the target fitness value of the target hardware platform running the target compilation file is obtained. Finally, based on the target fitness value of the target hardware platform under each initial candidate parameter vector, the initial candidate parameter set is iteratively evolved using an evolutionary computation strategy, and the next set of candidate parameters is determined in the search space model until the iteration conditions are met. By constructing a closed-loop automatic search mechanism of "candidate parameter generation → hardware execution compilation → performance index collection → feedback-driven adjustment of search direction," the device discovers a near-globally optimal Tiling parameter configuration on the target hardware platform within a limited number of search iterations. Compared to manual tuning and grid search, this significantly improves the efficiency and reliability of parameter determination.

[0075] In some embodiments of this disclosure, determining the search space model for the Tiling parameter based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform includes: Obtain the architecture parameters and operator computation modes of the target hardware platform; Based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform, the dimensions and constraints of each Tiling parameter are determined. The Tiling parameters include at least the block size, pipeline depth, double buffer switch, loop unrolling factor, vectorization width, and on-chip cache allocation strategy. The dimensions and constraints of each Tiling parameter constitute the search space model.

[0076] In some embodiments of this disclosure, determining the initial candidate parameter set corresponding to the Tiling parameter based on the search space model includes: Based on the Bayesian optimization algorithm, Latin hypersquare sampling is used to select a first initial candidate parameter vector for a first number of objectives from the search space model; or, Based on the genetic algorithm, a second initial candidate parameter vector is randomly selected from the search space model to determine the second target number.

[0077] In some embodiments of this disclosure, determining the target compilation file based on each of the initial candidate parameter vectors and obtaining the target fitness value for the target hardware platform running the target compilation file includes: Based on the initial candidate parameter vectors, determine the operator source code file; The compiler on the target hardware platform is invoked to compile the operator source code file, generate a target compiled file, and send the target compiled file to the target compilation platform so that the target compiled file can be run on the target hardware platform. Obtain the fitness sequence of the target hardware platform running the target compiled file, and determine the target fitness value corresponding to the target hardware platform running the target compiled file based on the fitness value sequence.

[0078] In some embodiments of this disclosure, determining the target fitness value corresponding to the target compiled file running on the target hardware platform based on the fitness value sequence includes: Based on the fitness value sequence, the fitness values ​​in the fitness value sequence are sorted from smallest to largest; The median of the sorted fitness value sequence is selected as the target fitness value for the target hardware platform running the target compiled file.

[0079] In some embodiments of this disclosure, the evolutionary computation strategy includes a Bayesian optimization strategy; The step of iteratively evolving the initial candidate parameter set based on the fitness values ​​of the target hardware platform under each initial candidate parameter vector, using an evolutionary computation strategy, to determine the next set of candidate parameters includes: Using the initial candidate parameter vector minus the target fitness value as the observation data, a Gaussian process surrogate model is fitted, wherein the Gaussian process surrogate model outputs the predicted mean and predicted variance of the first candidate parameter vector; The next set of candidate parameters is determined based on the predicted mean, predicted variance, and acquisition function of each first candidate parameter vector.

[0080] In some embodiments of this disclosure, the evolutionary computation strategy includes a genetic algorithm; The step of iteratively evolving the initial candidate parameter set based on the fitness values ​​of the target hardware platform under each initial candidate parameter vector, using an evolutionary computation strategy, to determine the next set of candidate parameters includes: Based on the target fitness value corresponding to each initial candidate parameter vector, sort the initial candidate parameter vectors according to the target fitness value, and select the first number of initial candidate parameter vectors from the sorted initial candidate parameter vectors as the first candidate parameter set; The first candidate parameter set is processed based on the tournament selection mechanism to obtain the second candidate parameter set; The second candidate parameter set is processed based on the cross operation to obtain the third candidate parameter set; The third candidate parameter set is processed based on the mutation operation to obtain the fourth candidate parameter set; The fourth candidate parameter set is processed based on the constraint repair operation to obtain the next set of candidate parameters.

[0081] This application also provides a computer device, please refer to 3 for details. Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0082] The computer device includes a memory 510 and a processor 520 that are interconnected via a system bus. It should be noted that only a computer device with components 510-520 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components may be implemented alternatively. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0083] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0084] The memory 510 includes at least one type of readable storage medium, including non-volatile memory or volatile memory, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. RAM may include static RAM or dynamic RAM. In some embodiments, the memory 510 may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory 510 may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card equipped on the computer device. Of course, the memory 510 may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory 510 is typically used to store the operating system and various application software installed on the computer device, such as the program code of the method described above. In addition, the memory 510 may also be used to temporarily store various types of data that have been output or will be output.

[0085] The processor 520 is typically used to perform the overall operation of a computer device. In this embodiment, the memory 510 is used to store program code or instructions, including computer operation instructions. The processor 520 is used to execute the program code or instructions stored in the memory 510 or to process data, such as program code that runs the methods described above.

[0086] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus system can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0087] Another embodiment of this application also provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads computer-readable program code stored in the computer-readable medium, enabling the processor to execute the functional actions specified in each step or combination of steps in the above method; and to generate means for implementing the functional actions specified in each block or combination of blocks in the block diagram.

[0088] Computer-readable media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any suitable combination thereof, wherein the memory is used to store program code or instructions, the program code including computer operation instructions, and the processor is used to execute the program code or instructions of the above-described methods stored in the memory.

[0089] The definitions of memory and processor can be found in the description of the foregoing computer device embodiments, and will not be repeated here.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0091] In the various embodiments of this application, the functional units or modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0092] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] Unless otherwise expressly indicated by the context, the singular form of words used herein and in the appended claims includes the plural form, and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Similarly, the terms “comprising” and “including” shall be interpreted as including rather than exclusively. Likewise, the terms “including” and “or” shall be interpreted as including unless such interpretation is expressly prohibited herein. Where the term “example” is used herein, particularly when it follows a set of terms, the “example” is merely exemplary and illustrative and should not be considered exclusive or extensive.

[0094] Further aspects and scope of adaptation become apparent from the description provided herein. It should be understood that various aspects of this application may be implemented individually or in combination with one or more other aspects. It should also be understood that the descriptions and specific embodiments herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0095] Several embodiments of this disclosure have been described in detail above. However, it is obvious that those skilled in the art can make various modifications and variations to the embodiments of this disclosure without departing from the spirit and scope of this disclosure. The scope of protection of this disclosure is defined by the appended claims.

Claims

1. A method for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback, characterized in that, include: Based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform, the search space model of the Tiling parameter is determined. The architecture parameters include at least the processor number information, the first cache capacity information, the second cache capacity information, the memory bandwidth information, and the computing unit type information. Based on the search space model, an initial candidate parameter set corresponding to the Tiling parameter is determined, wherein the initial candidate parameter set includes multiple initial candidate parameter vectors; Based on each of the initial candidate parameter vectors, the target compilation file is determined, and the target fitness value for running the target compilation file on the target hardware platform is obtained; Based on the target fitness value of the target hardware platform under each initial candidate parameter vector, the initial candidate parameter set is iteratively evolved based on the evolutionary computation strategy, and the next set of candidate parameters is determined in the search space model until the iteration condition is met.

2. The method according to claim 1, characterized in that, The step of determining the search space model for the Tiling parameters based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform includes: Obtain the architecture parameters and operator computation modes of the target hardware platform; Based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform, the dimensions and constraints of each Tiling parameter are determined. The Tiling parameters include at least the block size, pipeline depth, double buffer switch, loop unrolling factor, vectorization width, and on-chip cache allocation strategy. The dimensions and constraints of each Tiling parameter constitute the search space model.

3. The method according to claim 1, characterized in that, The step of determining the initial candidate parameter set corresponding to the Tiling parameter based on the search space model includes: Based on the Bayesian optimization algorithm, Latin hypersquare sampling is used to select a first initial candidate parameter vector for a first number of objectives from the search space model; or, Based on the genetic algorithm, a second initial candidate parameter vector is randomly selected from the search space model to determine the second target number.

4. The method according to claim 1, characterized in that, The step of determining the target compilation file based on each of the initial candidate parameter vectors and obtaining the target fitness value for the target hardware platform running the target compilation file includes: Based on the initial candidate parameter vectors, determine the operator source code file; The compiler on the target hardware platform is invoked to compile the operator source code file, generate a target compiled file, and send the target compiled file to the target compilation platform so that the target compiled file can be run on the target hardware platform. Obtain the fitness sequence of the target hardware platform running the target compiled file, and determine the target fitness value corresponding to the target hardware platform running the target compiled file based on the fitness value sequence.

5. The method according to claim 4, characterized in that, The step of determining the target fitness value corresponding to the target compiled file running on the target hardware platform based on the fitness value sequence includes: Based on the fitness value sequence, the fitness values ​​in the fitness value sequence are sorted from smallest to largest; The median of the sorted fitness value sequence is selected as the target fitness value for the target hardware platform running the target compiled file.

6. The method according to claim 1, characterized in that, The evolutionary computation strategy includes a Bayesian optimization strategy; The step of iteratively evolving the initial candidate parameter set based on the fitness values ​​of the target hardware platform under each initial candidate parameter vector, using an evolutionary computation strategy, to determine the next set of candidate parameters includes: Using the initial candidate parameter vector - target fitness value as observation data, a Gaussian process surrogate model is fitted, wherein the Gaussian process surrogate model outputs the predicted mean and predicted variance of any unsampled candidate parameter vector in the search space model; The next set of candidate parameters is determined based on the predicted mean, predicted variance, and acquisition function of any unsampled candidate parameter vector in the search space model.

7. The method according to claim 6, characterized in that, The evolutionary computation strategy includes genetic algorithms; The step of iteratively evolving the initial candidate parameter set based on the fitness values ​​of the target hardware platform under each initial candidate parameter vector, using an evolutionary computation strategy, to determine the next set of candidate parameters includes: Based on the target fitness value corresponding to each initial candidate parameter vector, sort the initial candidate parameter vectors according to the target fitness value, and select the first number of initial candidate parameter vectors from the sorted initial candidate parameter vectors as the first candidate parameter set; The first candidate parameter set is processed based on the tournament selection mechanism to obtain the second candidate parameter set; The second candidate parameter set is processed based on the cross operation to obtain the third candidate parameter set; The third candidate parameter set is processed based on the mutation operation to obtain the fourth candidate parameter set; The fourth candidate parameter set is processed based on the constraint repair operation to obtain the next set of candidate parameters.

8. A device for determining operator Tiling parameters based on evolutionary computation and hardware-in-the-loop feedback, characterized in that, include: The search space model determination module is used to determine the search space model of the Tiling parameter based on the architecture parameters of the target hardware platform and the operator computation mode corresponding to the target hardware platform. The architecture parameters include at least processor number information, first cache capacity information, second cache capacity information, memory bandwidth information and computing unit type information. The initial candidate parameter set determination module is used to determine the initial candidate parameter set corresponding to the Tiling parameter according to the search space model, wherein the initial candidate parameter set includes multiple initial candidate parameter vectors; The fitness value determination module is used to determine the target compilation file based on each of the initial candidate parameter vectors, and to obtain the target fitness value of the target hardware platform running the target compilation file; The target candidate parameter determination module is used to iteratively evolve the initial candidate parameter set based on the target fitness value of the target hardware platform under each initial candidate parameter vector, and determine the next set of candidate parameters in the search space model until the iteration conditions are met.

9. A computer device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images