A Joint Constraint-Driven Code Generation and Combinatorial Tuning Method
By combining constraint-driven methods based on operator shape, hardware architecture, and scheduling strategies, we can generate legal and efficient code, solving the problems of long generation time and insufficient optimization effect in existing technologies, and achieving more efficient code generation and optimization.
Patent Information
- Application Number
- CN202611576269.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-13
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies fail to effectively combine constraints of operator shape, hardware architecture, and scheduling strategy when generating code for custom matrix or vector accelerators, resulting in long generation times, difficult filtering, and insufficient optimization effects.
By acquiring and normalizing operator shapes, hardware architectures, and basic scheduling descriptions, candidate scheduling combinations are constructed. Based on these descriptions, a validity check is performed, and after deduplication, a representative program is generated to ensure that the candidate programs meet hardware and operator constraints, thereby optimizing target metrics such as cycle time and throughput.
It improves the efficiency and optimization of code generation, avoids generating unsuitable candidate programs, reduces repeated generation and measurement, and improves hardware utilization.
Smart Images

Figure CN123114481A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of software code technology, and in particular to a method for joint constraint-driven code generation and combinatorial optimization. Background Technology
[0002] On custom matrix or vector accelerators, the same matrix multiplication or convolution operator can employ various loop orders, block sizes, data residency methods, and loading methods. Whether a candidate program can execute correctly and is worth tuning depends not only on the scheduling policy itself but also on the hardware architecture and operator shape. The hardware architecture determines the available instructions, various length limits, register capacity, and concurrency resources; the operator shape determines the number of loops, tail blocks, padding boundaries, and the range of data reuse that can be achieved; the scheduling policy determines the loop organization, block size, residency depth, data layout, and configuration location. All three conditions must be met simultaneously for the scheduling selection to be transformed into a valid and effective operator program.
[0003] To address the aforementioned needs, existing technologies primarily employ the following solutions: Fixed template and parameterized code generation schemes, where vendors or compilers pre-provide several manually validated loop templates. The closest template is matched based on operator type and data type, and block size, expansion factor, etc., are then filled into the template and concatenated to generate code. Analytical modeling schemes, which establish analytical performance models based on memory access volume, data reuse distance, and computational cost to estimate memory access and computational costs under different block parameters. Based on this, the theoretically optimal block size is directly selected without requiring candidate-by-candidate testing. Heuristic selection schemes based on static features, which construct scoring functions using statically statistical indicators such as the number of loaded instructions, static code volume, number of nested loops, or theoretical number of instructions. Candidate scheduling is ranked, and only a small number of top-ranked candidates are compiled and validated, thereby reducing tuning overhead.
[0004] However, template-based and parameterized generation processes simply concatenate code according to scheduling parameters, failing to incorporate register capacity, length limits, and concurrency resource limits as construction-phase constraints. This easily leads to programs exceeding hardware capacity, with problems often only emerging during compilation or runtime, sometimes even resulting in malfunctioning code. Furthermore, they cannot determine whether a strategy is effective under the current form. For example, when the number of loops is insufficient or the dimension is inadequate to support resident reuse, the scheduling parameters, while nominally effective, may actually degenerate into non-reusability or introduce redundant padding. Moreover, existing solutions only compare the scheduling parameters themselves, failing to identify that different parameter combinations ultimately generate identical loop structures, resident forms, or load sharing relationships. This results in equivalent candidates being repeatedly generated and tested, wasting search overhead. Summary of the Invention
[0005] This application provides a joint constraint-driven code generation and combinatorial optimization method to address the technical problems of existing compilers and generation templates lacking support for joint constraints and performance filtering of operator shape, hardware architecture, and scheduling strategy, as well as program deduplication. This makes it difficult to perform joint optimization and filtering among multiple types of constraints, resulting in long operator generation and optimization times, difficult filtering, and insufficient optimization effects.
[0006] This application provides a method for joint constraint-driven code generation and combinatorial optimization, including: Obtain and normalize the description parameters; the description parameters include: operator shape description, hardware architecture description, and basic scheduling description; Based on the described parameters, candidate scheduling combinations are constructed according to preset information. The preset information includes: the source of the scheduling candidate values, the set of parameters that can be independently valued, the strategies that must change in groups, and the upper limit of the number of candidates. The candidate scheduling combinations are obtained by bounded expansion of the scheduling fields in the basic scheduling description within the range of candidate values. The candidate scheduling combinations are checked based on the candidate constraints to determine the legal scheduling combinations; the candidate constraints include: operator shape constraints, hardware architecture constraints, and scheduling policy constraints. Based on the legal scheduling combinations, candidate programs and equivalent programs are determined; The candidate and equivalent programs are deduplicated to determine the representative program; Based on the target hardware architecture, operator shape, and target metrics, a scheduling scheme and corresponding operator program are determined from the representative program; the target metrics are cycle time, throughput, energy consumption, code size, or a combination of multiple metrics.
[0007] In some embodiments, the operator shape constraints for matrix multiplication include matrix size parameters, whether the input matrix is transposed, and the input and accumulation data types; for two-dimensional convolution, they include batch size, number of input channels, number of output channels, input height and width, convolution kernel height and width, stride, padding, dilation rate, number of groups, and data type; the operator shape constraints are used to determine the loop boundary, actual length of the tail block, effective input of the padding region, number of output blocks that can be processed simultaneously, and the range of data reuse that can be formed; The hardware architecture constraints include various length limits, number of data channels, number of source registers, number of weight registers, number of matrix registers, number of accumulator registers, maximum number of instructions in transit, and target instruction support. These hardware architecture constraints are used for block determination, register allocation, instruction selection, and candidate validity checks. The scheduling policy constraints include reduced block size, output block size, number of output blocks residing simultaneously, loop order, weight layout, register multiples of consecutive data types, static expansion or runtime loop, configuration instruction location, prefetching method, number of buffers, candidate value range of fields, policy pairing relationship, and data dependency conditions.
[0008] In some embodiments, the step of checking the candidate scheduling combinations based on candidate constraints to determine legal scheduling combinations includes: The candidate scheduling combinations are checked according to the candidate constraints. If the candidate scheduling combination meets the candidate constraints, then the candidate scheduling combination is a valid scheduling combination. If the candidate scheduling combination does not meet the candidate constraint conditions, then the candidate scheduling combination is an illegal scheduling combination and the candidate constraint conditions that the illegal scheduling combination does not meet are recorded.
[0009] In some embodiments, the step of determining candidate programs based on the legal scheduling combinations includes: Determine whether the scheduling parameters corresponding to the legal scheduling combination are triggered by the current operator shape, whether the policy pairing conditions are met, and whether a new loop organization, residence depth, load sharing relationship, register allocation, or target instruction is formed. If so, then perform default value completion, tail block pruning, loop normalization, and instruction selection operations on the basic scheduling description corresponding to the legal scheduling combination to generate candidate programs.
[0010] In some embodiments, the method further includes: When the basic scheduling description corresponding to the legal scheduling combination is completed with default value completion, tail block pruning, loop normalization and instruction selection to obtain the same program, and the same loop, register allocation and target instruction are obtained, then an equivalent program is generated.
[0011] In some embodiments, the method further includes: When the operator is matrix multiplication, the program is generated according to the output row block, output column resident window and reduction block. In one output column resident window, independent accumulators are allocated to multiple output groups and their partial sums are maintained. When the input matrix source block is processed in the reduction loop, the matrix source block is loaded once and calculated with the weight blocks of each output group in the output column resident window respectively. After the reduction is completed, it is written back to each output row block respectively. In this case, when the cyclic sequence causes the same reduction block to cover multiple output groups and reside in multiple output groups at the same time, a shared matrix source block is formed.
[0012] In some embodiments, the method further includes: When the operator is a two-dimensional convolution, multiple output rows or output regions are retained. A source register and accumulator are allocated to each output region. Weights are loaded once for the same convolution kernel position and input channel block. Then, the input windows corresponding to each output region are loaded and the accumulation is completed. At the filling boundary, the set of effective output regions is calculated according to the current convolution kernel position. Input loading and calculation are generated for the effective regions. When the number of output channels exceeds the preset number, the instruction body of the output channel group is generated and executed repeatedly after updating the weight address and output address at runtime.
[0013] In some embodiments, the step of deduplicating the candidate programs and equivalent programs to determine the representative program includes: The basic scheduling descriptions corresponding to the candidate programs and equivalent programs are deduplicated; the deduplication process includes: eliminating description duplication caused by field order, explicit default values, blanks, or equivalent syntax; Calculate a summary of the normalized content of the candidate program and the equivalent program; Based on the summary, a representative procedure is determined.
[0014] In some embodiments, the method further includes: The representative program is subjected to functional checks, compilation, and actual execution to obtain functional correctness results, execution cycle, and hardware wait indicators. The hardware wait indicators include: source data loading times, weight loading times, configuration times, code size, instruction cache wait, data cache wait, and wait caused by insufficient dependent resources.
[0015] In some embodiments, the method further includes: The results of the representative program are associated with the corresponding equivalent candidate scheduling combinations, and each candidate scheduling combination is recorded as an illegal candidate, a legal but not actually effective equivalent candidate, a candidate in which the program has changed but there is no performance gain, a candidate in which the program has changed and there is a gain, a candidate in which the function fails, and a candidate in which the performance is rolled back.
[0016] This application provides a method for joint constraint-driven code generation and combinatorial optimization, comprising: acquiring and normalizing description parameters; the description parameters including: operator shape description, hardware architecture description, and basic scheduling description; constructing candidate scheduling combinations based on the description parameters and according to preset specified information; the preset specified information including: the source of scheduling candidate values, the set of parameters that can be independently valued, strategies that must change in groups, and an upper limit on the number of candidates; the candidate scheduling combination is obtained by bounded expansion of the scheduling field in the basic scheduling description within the candidate value range; checking the candidate scheduling combination according to the candidate constraint conditions to determine a legal scheduling combination; the candidate... The selection of constraints includes: operator shape constraints, hardware architecture constraints, and scheduling policy constraints; based on the legal scheduling combinations, candidate programs and equivalent programs are determined; the candidate programs and equivalent programs are deduplicated to determine representative programs; according to the target hardware architecture, operator shape, and target indicators, a scheduling scheme and corresponding operator programs are determined from the representative programs; the target indicators are periodicity, throughput, energy consumption, code size, or a combination of multiple indicators, so that the three types of constraints—hardware architecture, operator shape, and scheduling policy—directly participate in candidate construction, legality checking, and target code generation, avoiding the inclusion of parameter combinations that are not applicable to the current hardware or shape as valid candidates. Attached Figure Description
[0017] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart of the joint constraint-driven code generation and combinatorial optimization method in this application; Figure 2 This is a flowchart illustrating the determination of whether changes in scheduling parameters actually alter the program in this application. Figure 3 This is a schematic diagram illustrating how a matrix source block is shared by two resident output groups in matrix multiplication. Figure 4 This is a diagram illustrating how a single weight loading in a direct convolution can be shared by multiple resident output rows. Detailed Implementation
[0019] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.
[0020] For example, general matrix multiplication can be represented as the sum of the product of the corresponding row of the input matrix and the corresponding column of the weight matrix for each element in the output matrix; two-dimensional convolution can be represented as the sum of the product of the input window and the convolution kernel in the input channel and kernel space. The mathematical expression only specifies the calculation result and does not specify the loop order, block size, memory layout, register usage, or instruction execution order.
[0021] Custom matrix or vector accelerators typically feature different types of source registers, weight registers, matrix registers, and accumulator registers, along with multiple length fields, data channels, dedicated load instructions, and in-transit instruction limits. Meanwhile, different operator shapes alter the number of loops, tail block size, effective boundary range, and the number of output blocks that can be processed simultaneously. Therefore, code generation must simultaneously address hardware resources, operator shapes, and scheduling policies to ensure that the generated program is both executable and accurately reflects the scheduling choices.
[0022] For example, when the actual output contains only one output group, changing the "number of allowed resident output groups" from 1 to 2 will not create a second consumer; when the loop still completes one output group at a time, even if the register capacity allows multiple resident output groups, input sharing will not occur. Changing parameter values does not necessarily mean that the target program has changed.
[0023] Common implementation methods include fixed-template code generation, manual target assembly, and general automatic tuning. Fixed templates are suitable for a small number of known shapes, but hardware constants, boundary handling, and scheduling rules are easily mixed in the same template; manual assembly requires handling addresses, registers, dependencies, and tail blocks simultaneously, which is prone to errors; general automatic tuning can propose and measure multiple candidates, but when applied to custom accelerators, it still requires additional description of multiple length fields, physical register occupancy, special instructions, and the effective conditions between strategies. If the three types of constraints are not uniformly incorporated into candidate selection and code generation, it is difficult to guarantee that candidates simultaneously possess legitimacy, practical effectiveness, and repeatability.
[0024] To address the technical problems of existing compilers and template generators lacking support for joint constraints and performance filtering based on operator shape, hardware architecture, and scheduling strategies, as well as program deduplication, making it difficult to perform joint optimization and filtering among multiple types of constraints, resulting in long operator generation and optimization times, difficult filtering, and insufficient optimization effects, this application provides a joint constraint-driven code generation and combinatorial optimization method. The following is a description of the joint constraint-driven code generation and combinatorial optimization method: like Figure 1 The diagram shown is a flowchart of the joint constraint-driven code generation and combinatorial optimization method in this application.
[0025] This application provides a method for joint constraint-driven code generation and combinatorial optimization, including the following steps: S100: Obtain and normalize the description parameters; the description parameters include: operator shape description, hardware architecture description and basic scheduling description.
[0026] For example, the operator shape constraint for matrix multiplication includes matrix size parameters (M, N, K), whether the input matrix is transposed, and the input and accumulation data types; for two-dimensional convolution, it includes batch number, number of input channels, number of output channels, input height and width, convolution kernel height and width, stride, padding, dilation rate, number of groups, and data type; the operator shape constraint is used to determine the loop boundary, the actual length of the tail block, the effective input of the padding region, the number of output blocks that can be processed simultaneously, and the range of data reuse that can be formed.
[0027] The hardware architecture constraints include various length limits, number of data channels, number of source registers, number of weight registers, number of matrix registers, number of accumulator registers, maximum number of instructions in transit, and target instruction support. These hardware architecture constraints are used for block determination, register allocation, instruction selection, and candidate validity checks.
[0028] The scheduling policy constraints include reduced block size, output block size, number of output blocks residing simultaneously, loop order, weighted layout, register multiples for consecutive data types, static expansion or runtime loop, configuration instruction location, prefetching method, number of buffers, candidate value range for fields, policy pairing relationships, and data dependency conditions. The scheduling policy must not change the mathematical definition of the operator itself.
[0029] The scheduling strategy is a method for determining the loop, block, data retention location, and execution order without changing the calculation results.
[0030] S200: Based on the description parameters, construct candidate scheduling combinations according to preset specified information; the preset specified information includes: the source of scheduling candidate values, the set of parameters that can be independently valued, the strategies that must be changed in groups, and the upper limit of the number of candidates; the candidate scheduling combination is obtained by bounded expansion of the scheduling field in the basic scheduling description within the candidate value range.
[0031] Specifically, candidate values for scheduling can be proposed by R&D personnel, rule bases, historical results, or search algorithms. The current prototype groups independently changeable fields into independent parameter sets and expands these candidate values through bounded combinations; fields that must be changed simultaneously are grouped into group strategies. This process automatically constructs combinations within a predefined candidate range, rather than exhaustively enumerating all possible instruction programs.
[0032] Specifically, the same field cannot be given two different values simultaneously by both independent parameters and grouping strategies. The system sets an upper limit on the number of candidates and sets the allowed fields, value ranges, and pairing rules for different operators and hardware. For example, when a convolution configuration is moved to the outer loop of the runtime output channel, that runtime loop must already be enabled; double-buffered prefetching must simultaneously provide a prefetch switch, two sets of physical buffers, and dependency protection before overwrite.
[0033] S300: Based on the candidate constraints, the candidate scheduling combinations are checked to determine the legal scheduling combinations; the candidate constraints include: operator shape constraints, hardware architecture constraints, and scheduling policy constraints. For each candidate scheduling combination, operator shape checks, hardware architecture checks, and scheduling policy checks are performed sequentially to eliminate illegal combinations that exceed operator boundaries, hardware length and register capacity, instruction support capabilities, and data dependency requirements.
[0034] Specifically, the step of checking the candidate scheduling combinations based on the candidate constraints to determine the legal scheduling combinations includes the following sub-steps: S310: Check the candidate scheduling combination according to the candidate constraint conditions. If the candidate scheduling combination meets the candidate constraint conditions, then the candidate scheduling combination is a valid scheduling combination.
[0035] S320: If the candidate scheduling combination does not meet the candidate constraint conditions, then the candidate scheduling combination is an illegal scheduling combination and the candidate constraint conditions that the illegal scheduling combination does not meet are recorded.
[0036] Specifically, each candidate undergoes three types of checks before the generation process. The operator shape check confirms the validity of the blocks, cycle boundaries, tail blocks, and padding regions; the hardware architecture check confirms that the instruction type, various lengths, register counts, and concurrent resources meet the requirements; and the scheduling policy check confirms that field values, policy pairings, and data dependencies are valid. If any check fails, the candidate is deemed illegal, and the reason is recorded.
[0037] Wherein, the number of bits required for weighting = reduction block × input bit width × number of hardware channels; the number of bits required for matrix source = number of matrix rows × reduction block × input bit width; the number of bits required for accumulation = output block × accumulation bit width × number of hardware channels.
[0038] The number of output blocks (small chunks of output data computed and stored in registers at one time) resident simultaneously (data loaded into registers but not overwritten or written back, reserved for subsequent calculations), the multiple of registers occupied by a logical data group consecutively, and the number of buffers all determine the number of active registers. Taking direct convolution as an example, when T output rows are resident simultaneously, matrix source registers and accumulators need to be reserved for each output row, and the weight registers used in the column directions of the convolution kernel need to be included. If any type exceeds the corresponding number of physical registers, the candidate is directly rejected. Passing the validity check only indicates that the candidate can be generated and executed; whether the scheduling parameters actually change the program still needs to be determined by subsequent actual effect assessment.
[0039] S400: Based on the legal scheduling combinations, determine the candidate program and the equivalent program.
[0040] The step of determining candidate programs based on the legal scheduling combinations includes the following sub-steps: S410: Determine whether the scheduling parameters corresponding to the legal scheduling combination are triggered by the current operator shape, whether the policy pairing conditions are met, and whether a new loop organization, residence depth, load sharing relationship, register allocation, or target instruction is formed.
[0041] S420: If so, then perform default value completion, tail block pruning, loop normalization and instruction selection operations on the basic scheduling description corresponding to the legal scheduling combination to generate candidate programs.
[0042] For example, each valid scheduling combination is first converted into operator index and reduction computation relationships, that is, clarifying the index correspondence between output dimension, reduction dimension, input, weight, and output; then micro-operations such as configuration, zeroing, loading, computation, and write-back are generated, physical registers and control fields are allocated, target instructions are selected, and runtime loops are generated as needed, ultimately forming a complete candidate program. The same operator shape, hardware architecture, scheduling description, and generator version should yield the same program content.
[0043] The scheduling fields affect the program as follows: the loop order changes the relative positions of the reduction loop and the output block loop; the resident depth changes the number of simultaneously active output blocks and physical register allocation; the memory layout determines whether to use continuous loading or step loading; the runtime loop determines whether the instruction body is statically copied or repeated during execution; the configuration location determines whether the same configuration is repeatedly executed in the inner layer; the block size and register multiple together determine the actual length, register span, and tail block processing.
[0044] Specifically, changes in scheduling parameters do not necessarily lead to changes in the target program. This application, under the same operator shape and hardware architecture, normalizes and generates programs for the modified schedule and the baseline schedule respectively, and checks whether the changed fields meet the shape triggering conditions, hardware execution conditions, and policy pairing conditions. The specific process is as follows: Figure 2 As shown.
[0045] First, the operator shape may not trigger the parameters. For example, if there is only one output group in the actual output, changing the number of allowed resident output groups from 1 to 2 will not result in an actual resident number of 1; or, after processing the tail block, two different block parameters will result in the same actual length.
[0046] Second, candidates may not meet hardware capacity, instruction support, or data dependency requirements. Such candidates are illegal and should be rejected before code generation, rather than being interpreted as having zero scheduling effect.
[0047] Third, a particular strategy may lack supporting strategies, thus failing to create a new program structure. For example, simply increasing the dwell depth while the loop still completes output blocks one by one will not allow multiple output blocks to survive simultaneously; merely changing the loop order while only dwelling one output block will not allow a second consumer to share the input.
[0048] Fourth, different scheduling descriptions, after default value completion, tail block pruning, loop normalization, and instruction selection, may result in the same loop, register allocation, and target instruction. In this case, they should be considered equivalent programs, indicating that the parameter has not actually taken effect under the current three types of constraints, but this does not mean that the code generation method has failed, nor does it mean that the strategy is invalid under other operator shapes or supporting strategies.
[0049] The method further includes the following steps: S430: When the basic scheduling description corresponding to the legal scheduling combination is completed with default value completion, tail block pruning, loop normalization and instruction selection to obtain the same program, and the same loop, register allocation and target instruction are obtained, then an equivalent program is generated.
[0050] The system only considers a candidate as a new program candidate for functional verification and actual measurement when the candidate is valid, the action condition is triggered by the operator shape, the strategy pairing is valid, and the normalized program content changes relative to the baseline.
[0051] The method further includes the following steps: S440: When the operator is matrix multiplication, the program is generated according to the output row block, the output column resident window, and the reduction block (a segment of data processed once in the K direction of matrix multiplication or the input channel direction of convolution). Within an output column resident window, independent accumulators are allocated to multiple output groups and their partial sums are maintained. When the input matrix source block is processed in a reduction loop, the matrix source block is loaded once and calculated with the weight blocks of each output group in the output column resident window. After the reduction is completed, it is written back to each output row block.
[0052] In this case, when the cyclic sequence causes the same reduction block to cover multiple output groups and reside in multiple output groups at the same time, a shared matrix source block is formed.
[0053] Specifically, for matrix multiplication, the system generates procedures based on output row blocks, output column resident windows, and reduction blocks. Within an output column resident window, independent accumulators are allocated to multiple output groups, and their partial sums are preserved. When the reduction loop processes a source block of an input matrix, the source block is loaded only once, and then calculated separately with the weight blocks of each output group within the resident window. After reduction is complete, the results are written back to each output block.
[0054] When the cyclic order still completes output groups one by one, even if two output groups are allowed to reside, no sharing will occur; when the cyclic order causes the same reduce block to cover multiple output groups, but only one output group resides, there is no second consumer. Only when the cyclic order and residency depth are combined can a matrix source block be shared by multiple output groups, such as... Figure 3 As shown.
[0055] For M, N, or K tails that are not divisible by the block size, the system uses the number of remaining elements as the actual length; for different weighted memory layouts, the system selects the corresponding contiguous load or straddle load instruction. Each resident output group uses an independent accumulator, and when multiple physical registers are occupied consecutively, they are allocated in group alignment.
[0056] The method further includes the following steps: S450: When the operator is a two-dimensional convolution, multiple output rows or output regions are retained, a source register and an accumulator are allocated to each output region, weights are loaded once for the same convolution kernel position and input channel block, the input windows corresponding to each output region are loaded and accumulated, and the set of effective output regions is calculated at the filling boundary according to the current convolution kernel position, and input loading and calculation are generated for the effective regions.
[0057] When the number of output channels exceeds the preset number, the instruction body of the output channel group is generated and executed repeatedly after updating the weight address and output address at runtime.
[0058] Specifically, the baseline approach to direct convolution typically performs reduction row by row, repeatedly loading weights from the same kernel location when adjacent output rows use weights from the same kernel position. This invention, however, preserves multiple output rows or other output regions, allocating independent source registers and accumulators to each output region, such as... Figure 4 As shown, weights are loaded only once for the same convolution kernel position and input channel block, and then the input windows corresponding to each output region are loaded and accumulated.
[0059] At the padding boundaries, not all resident output lines have valid input coordinates. The system calculates the set of valid output regions based on the current convolution kernel position, generating input loading and computation only for valid regions, thus balancing boundary correctness and data reuse within the internal regions.
[0060] For cases with a large number of output channels, the system can generate only one instruction body for the output channel group, update the weight address and output address at runtime, and execute it repeatedly. If a configuration remains unchanged throughout the entire runtime loop, the configuration is moved to the outer loop; if the tail block or boundary causes the configuration length to change, the necessary local configuration is retained.
[0061] S500: Perform deduplication on the candidate programs and equivalent programs (programs generated by different scheduling descriptions but with the same normalized loops, register allocations and instruction contents) to determine the representative program.
[0062] Specifically, the step of deduplicating the candidate programs and equivalent programs to determine the representative program includes the following sub-steps: S510: Perform deduplication processing on the basic scheduling descriptions corresponding to the candidate programs and equivalent programs; the deduplication processing includes: eliminating description duplication caused by field order, explicit default values, blanks or equivalent writing.
[0063] S520: Calculate a summary of the normalized content of the candidate program and the equivalent program.
[0064] S530: Based on the summary, determine the representative procedure.
[0065] Specifically, the system first normalizes the scheduling descriptions, eliminating duplicates caused by field order, explicit default values, blank spaces, or equivalent syntax. After code generation, a summary is calculated for the normalized content of the complete candidate program. When different scheduling combinations generate the same program, only one stable representative is selected for compilation and actual measurement, while the remaining combinations retain their correspondence with the representative program.
[0066] It's worth noting that different scheduling combinations generating the same program typically stem from two causes: one is description-level equivalence, such as differences in field order, explicit default values, or fields not used by the current operator; the other is action-level equivalence, meaning parameter changes are not triggered by the operator shape or lack a corresponding strategy, ultimately failing to alter loops, residency, sharing relationships, and instruction sequences. This does not indicate that the optimization method has failed, but rather that the strategy has not actually taken effect under the current hardware architecture, operator shape, and other scheduling conditions. The system groups them into the same representative program to avoid redundant compilation and measurement.
[0067] The method further includes the following steps: S540: Perform functional checks, compilation, and actual execution on the representative program to obtain functional correctness results, execution cycle, and hardware wait indicators; the hardware wait indicators include: source data loading times, weight loading times, configuration times, code size, instruction cache wait, data cache wait, and wait caused by insufficient dependent resources.
[0068] Specifically, the representative program can be executed on an RTL simulator, a cycle-accurate model, an FPGA, or an actual chip. It should at least record functional correctness and execution cycle time, and can also record source data load counts, weight load counts, configuration counts, code size, instruction cache waits, data cache waits, and waits caused by insufficient dependent resources.
[0069] The method further includes the following steps: S550: Associate the result of the representative program back into the corresponding equivalent candidate scheduling combination, and record each candidate scheduling combination as an illegal candidate, a legal but not actually effective equivalent candidate, a candidate whose program has changed but has no performance benefit, a candidate whose program has changed and has gained benefits, a candidate whose function has failed, and a candidate whose performance has been rolled back.
[0070] The system distinguishes and records: illegal candidates, legal but ineffective equivalent candidates, candidates whose programs have changed but offer no performance benefit, candidates whose programs have changed and offer benefits, and candidates whose functions have failed or whose performance has regressed. This avoids confusing "no new program generated" with "a new program generated but with the same performance." In the GEMM (Generalized Matrix Multiplication) implementation, simply changing the loop order or simply increasing the dwell depth does not create input sharing; only when both are combined does it reduce matrix source loading and achieve significant benefits.
[0071] S600: Based on the target hardware architecture, operator shape, and target metrics, determine the scheduling scheme and corresponding operator program from the representative program; the target metrics are cycle time, throughput, energy consumption, code size, or a combination of multiple metrics.
[0072] This application provides a joint constraint-driven code generation and combinatorial optimization method, using operator shape description, hardware architecture description, and scheduling strategy description as three types of inputs. The operator shape description answers "what needs to be calculated and what boundaries exist," the hardware architecture description answers "what the target hardware can accommodate and execute," and the scheduling strategy description answers "how to organize this computation." The system first constructs and filters candidate schedules under the three types of constraints, then determines whether the scheduling changes have the actual conditions for effectiveness, and subsequently deterministically generates candidate programs, identifies equivalent programs, and performs functional verification and actual measurement. The current prototype uses a bounded combination method where candidate values are pre-defined by developers or rule documents, rather than exhaustively listing all possible programs; other candidate proposal methods must also enter the same constraint checking and code generation process.
[0073] This application provides a method for joint constraint-driven code generation and combinatorial optimization, and specific embodiments are as follows: Example 1: GEMM Loop Sequence and Output Resident Combination. The workload is M=1, N=8, K=16, the input and weights are 8-bit integers, the accumulated result is a 32-bit integer, and the hardware has 4 computing channels. The loop organization of MNK and MKN is compared, as well as the simultaneous residence of one or two output groups. Although MNK / Resident 2 changes the residence parameters, the loop still completes one output group at a time; the two output groups are not simultaneously active and cannot share the matrix source. Although MKN / Resident 1 changes the loop relationship, only one output group retains a partial sum, and there is no second consumer. Only MKN / Resident 2 simultaneously satisfies the loop organization and residence conditions, allowing one matrix source block to be used by both output groups. The matrix source loading is reduced from 4 times to 2 times, the length configuration instruction is reduced from 10 times to 6 times, and the cycle time is reduced from 2^22 to 162.
[0074] Example 2: Direct Convolution with Multiple Output Row Retention. For 3×3, 5×5, and 7×7 direct convolutions with a stride of 1, maintaining the input-output space size, we compared simultaneously retaining 1 and 4 output rows. A single weight loading can support a maximum of 4 valid output rows for computation. The actual cycle times decreased from 16567 to 15026, from 52971 to 49437, and from 128128 to 120447, respectively, with all three convolutional kernels achieving positive returns.
[0075] Example 3: Scheduling parameters changed, but program unchanged. When an operator shape contains only one actual output group in the target dimension, the number of allowed resident output groups is changed from 1 to 2. The actual resident number during code generation is still taken as the smaller value between the candidate value and the remaining number of output groups, resulting in 1. At this time, register allocation, loading, computation, and write-back instructions are all the same as the baseline. The system marks the candidate as valid but does not actually take effect, and reuses the measurement results of the baseline program instead of interpreting it as a code generation failure or ineffective performance optimization.
[0076] The current prototype primarily implements GEMM with 8-bit integer input and 32-bit integer accumulation, as well as typical direct convolution with a stride of 1 that preserves spatial dimensions. Other data types, strides, padding methods, grouped convolutions, and larger search methods are optional implementations and are not considered to be fully implemented in the current prototype.
[0077] This application provides a method for joint constraint-driven code generation and combinatorial optimization, which has the following beneficial effects: 1. Three types of constraints—hardware architecture, operator shape, and scheduling strategy—directly participate in candidate construction, validity checks, and target code generation, avoiding the use of parameter combinations that are not applicable to the current hardware or shape as valid candidates.
[0078] 2. By combining loop organization with output dwell, a single input or weight load can be used by multiple computations, thereby reducing redundant data handling and configuration.
[0079] 3. Before generation, exclude illegal candidates that exceed the length, register, or policy pairing conditions, and mark legal candidates that do not actually change the program as "ineffective" to avoid confusing the three situations of illegal, ineffective, and no performance benefit.
[0080] 4. Deduplicatively analyze the scheduling description and the actual program; when different parameter combinations result in the same program after shape trimming, strategy reduction or code generation, only one representative program is measured.
[0081] 5. By evaluating candidates based on functional correctness, actual cycle time, cache wait, and dependency wait, it can identify situations where individual strategies do not form a shared structure or only take effect after combination, and can also retain negative results of performance fallback.
[0082] For example, representative results of the current prototype are as follows: In GEMM, both the cycle order and the simultaneous residence of two output groups are 222 cycles when changed individually, but are reduced to 162 cycles when used together, an improvement of approximately 27.0%; matrix source loading is reduced from 4 times to 2 times, and length configuration instructions are reduced from 10 times to 6 times. In direct convolution, simultaneously residence of 4 output rows improves performance by approximately 9.3%, 6.7%, and 6.0% on 3×3, 5×5, and 7×7 convolution kernels, respectively, compared to residence of 1 output row, as shown in Table 1 below.
[0083] Table 1: Actual period corresponding to different operators / conditions
[0084] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.
Claims
1. A method for joint constraint-driven code generation and combinatorial optimization, characterized in that, include: Obtain and normalize the description parameters; The description parameters include: operator shape description, hardware architecture description, and basic scheduling description; Based on the described parameters, candidate scheduling combinations are constructed according to preset information. The preset information includes: the source of the scheduling candidate values, the set of parameters that can be independently valued, the strategies that must change in groups, and the upper limit of the number of candidates. The candidate scheduling combinations are obtained by bounded expansion of the scheduling fields in the basic scheduling description within the range of candidate values. The candidate scheduling combinations are checked based on the candidate constraints to determine the legal scheduling combinations; the candidate constraints include: operator shape constraints, hardware architecture constraints, and scheduling policy constraints. Based on the legal scheduling combinations, candidate programs and equivalent programs are determined; The candidate and equivalent programs are deduplicated to determine the representative program; Based on the target hardware architecture, operator shape, and target metrics, a scheduling scheme and corresponding operator program are determined from the representative program; the target metrics are cycle time, throughput, energy consumption, code size, or a combination of multiple metrics.
2. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The operator shape constraints for matrix multiplication include matrix size parameters, whether the input matrix is transposed, and the data types of input and accumulation; for two-dimensional convolution, they include batch size, number of input channels, number of output channels, input height and width, convolution kernel height and width, stride, padding, dilation rate, number of groups, and data type; the operator shape constraints are used to determine the loop boundary, actual length of the tail block, effective input of the padding region, number of output blocks that can be processed simultaneously, and the range of data reuse that can be formed; The hardware architecture constraints include various length limits, number of data channels, number of source registers, number of weight registers, number of matrix registers, number of accumulator registers, maximum number of instructions in transit, and target instruction support. These hardware architecture constraints are used for block determination, register allocation, instruction selection, and candidate validity checks. The scheduling policy constraints include reduced block size, output block size, number of output blocks residing simultaneously, loop order, weight layout, register multiples of consecutive data types, static expansion or runtime loop, configuration instruction location, prefetching method, number of buffers, candidate value range of fields, policy pairing relationship, and data dependency conditions.
3. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The step of checking the candidate scheduling combinations based on the candidate constraints to determine the legal scheduling combinations includes: The candidate scheduling combinations are checked according to the candidate constraints. If the candidate scheduling combination meets the candidate constraints, then the candidate scheduling combination is a valid scheduling combination. If the candidate scheduling combination does not meet the candidate constraint conditions, then the candidate scheduling combination is an illegal scheduling combination and the candidate constraint conditions that the illegal scheduling combination does not meet are recorded.
4. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The step of determining candidate programs based on the legal scheduling combinations includes: Determine whether the scheduling parameters corresponding to the legal scheduling combination are triggered by the current operator shape, whether the policy pairing conditions are met, and whether a new loop organization, residence depth, load sharing relationship, register allocation, or target instruction is formed. If so, then perform default value completion, tail block pruning, loop normalization, and instruction selection operations on the basic scheduling description corresponding to the legal scheduling combination to generate candidate programs.
5. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The method further includes: When the basic scheduling description corresponding to the legal scheduling combination is completed with default value completion, tail block pruning, loop normalization and instruction selection to obtain the same program, and the same loop, register allocation and target instruction are obtained, then an equivalent program is generated.
6. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The method further includes: When the operator is matrix multiplication, the program is generated according to the output row block, output column resident window and reduction block. In one output column resident window, independent accumulators are allocated to multiple output groups and their partial sums are maintained. When the input matrix source block is processed in the reduction loop, the matrix source block is loaded once and calculated with the weight blocks of each output group in the output column resident window respectively. After the reduction is completed, it is written back to each output row block respectively. In this case, when the cyclic sequence causes the same reduction block to cover multiple output groups and reside in multiple output groups at the same time, a shared matrix source block is formed.
7. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The method further includes: When the operator is a two-dimensional convolution, multiple output rows or output regions are retained. A source register and accumulator are allocated to each output region. Weights are loaded once for the same convolution kernel position and input channel block. Then, the input windows corresponding to each output region are loaded and the accumulation is completed. At the filling boundary, the set of effective output regions is calculated according to the current convolution kernel position. Input loading and calculation are generated for the effective regions. When the number of output channels exceeds the preset number, the instruction body of the output channel group is generated and executed repeatedly after updating the weight address and output address at runtime.
8. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The step of deduplicating the candidate programs and equivalent programs to determine the representative program includes: The basic scheduling descriptions corresponding to the candidate programs and equivalent programs are deduplicated; the deduplication process includes: eliminating description duplication caused by field order, explicit default values, blanks, or equivalent syntax; Calculate a summary of the normalized content of the candidate program and the equivalent program; Based on the summary, a representative procedure is determined.
9. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The method further includes: The representative program is subjected to functional checks, compilation, and actual execution to obtain functional correctness results, execution cycle, and hardware wait indicators. The hardware wait indicators include: source data loading times, weight loading times, configuration times, code size, instruction cache wait, data cache wait, and wait caused by insufficient dependent resources.
10. The method for joint constraint-driven code generation and combinatorial optimization according to claim 1, characterized in that, The method further includes: The results of the representative program are associated with the corresponding equivalent candidate scheduling combinations, and each candidate scheduling combination is recorded as an illegal candidate, a legal but not actually effective equivalent candidate, a candidate in which the program has changed but there is no performance gain, a candidate in which the program has changed and there is a gain, a candidate in which the function fails, and a candidate in which the performance is rolled back.
