A GPU code parallelism analysis and optimization method and system

Abstract

The application discloses a GPU code parallelism analysis and optimization method and system, first constructs a multi-dimensional program graph containing statements, variables, constants, control nodes and three types of directed edges, collects the runtime step sequence of the loop variable through non-intrusive debugger insertion and carries out timing and scalar fusion coding, adopts single-head cross attention to accurately inject dynamic execution characteristics into the variable node corresponding to the program graph, encodes the structural prior through a two-stage relationship graph convolution network, then propagates dynamic information, fuses static and dynamic graph level representations through adaptive gating based on prediction entropy, and jointly predicts parallelism types and optimization strategies based on gradient isolation hierarchical decision head. The application can effectively realize adaptive fusion of program static structure and dynamic execution evidence, improve the parallelism inference accuracy of irregular memory access loops, and is suitable for application scenarios such as automatic parallelization of serial programs and automatic generation of GPU codes.

Description

Technical Field

[0001] This invention relates to the fields of high-performance computing (HPC), program analysis and deep learning, specifically to a method and system for GPU code parallelism analysis and optimization that integrates program graph and execution stride features, which can be applied to scenarios such as automatic parallelization of serial programs and automatic generation of GPU code. Background Technology

[0002] With the rapid development of high-performance computing technology, the demand for computing performance in fields such as scientific computing, engineering simulation, artificial intelligence, and financial modeling continues to grow. Graphics processors, with their powerful parallel computing capabilities, have become the core hardware platform for accelerating large-scale computing tasks. However, a large amount of existing high-performance computing code is still written in serial C / C++, unable to directly utilize the parallel computing resources of GPUs. How to automatically identify parallelizable loops in serial programs and convert them into efficient GPU parallel code is one of the core problems in the compiler and high-performance computing fields.

[0003] Loop parallelism inference is a crucial step in automatic parallelization, aiming to determine whether a program's loops can be executed in parallel across multiple threads or devices. Traditional loop parallelism inference methods primarily rely on static compiler analysis based on polyhedral models, exemplified by polyhedral compilers such as Pluto and PPCG. These methods use affine transformation theory to symbolically model and transform the data dependencies of loops. This type of method has strong optimization capabilities for affine loops, automatically inferring loop parallelism and generating parallel code. However, the polyhedral model only applies to affine loops. For loops containing pointer indirection or irregular subscripts, static dependency analysis conservatively reports "dependencies exist," leading to the incorrect rejection of many actually parallelizable loops.

[0004] With the development of deep learning technology, learning methods based on program graphs and graph neural networks have been applied to parallel inference tasks. These methods represent the abstract syntax tree and dependency graph of a program as graphs and use graph neural networks to learn the classification of loop parallelism. However, these methods only utilize the static structural features of the source code, remaining completely invisible to runtime memory access patterns, thus limiting their generalization ability on loops with complex memory access patterns. Subsequent work has called large language models for inference based on static graph representations, but it still cannot handle dependency ambiguities that require execution-level evidence to resolve, and the inference results are unstable due to reliance on LLM generation.

[0005] Parallelization tools based on dynamic analysis collect data dependency information through runtime trajectories, enabling observation of memory access behavior during actual program execution. However, dynamic analysis results are highly dependent on specific input patterns and lack generalization ability for uncovered input behaviors; they also lack a mechanism for integration with the program's static structure and are insufficient in distinguishing and modeling multiple types of dependencies.

[0006] Existing patents in the field of automatic CUDA code generation and optimization, such as CN2025101279072 which constructs a multi-memory-level data flow graph and performs performance-driven memory block optimization, CN202111464906 which extracts loop structures from serial code and performs traditional static data dependency analysis, and CN202510761772 which incorporates a triple verification feedback mechanism with a large language model as the core engine, either rely on pure static symbol analysis, which cannot handle pointer alias ambiguity, or rely on LLM to infer optimization strategies from text without introducing runtime memory access traces. None of these solutions achieve a precise fusion of the program's static structure and dynamic execution evidence at the node level.

[0007] Therefore, there is an urgent need for a loop parallelism representation learning method that can simultaneously encode program syntax structure, typed data dependencies, and runtime execution stride features. This method would enable the model to fully utilize prior knowledge of program structure and resolve dependency ambiguities through actual execution evidence, thereby improving the accuracy and coverage of loop parallelism inference. Summary of the Invention

[0008] The purpose of this invention is to provide a method and system for GPU code parallelism analysis and optimization, which is a method and system for learning loop parallelism representation that can simultaneously encode program syntax structure, typed data dependencies and runtime execution stride characteristics.

[0009] The technical solution adopted by the method of the present invention is: a GPU code parallelism analysis and optimization method, comprising the following steps: Step 1: Obtain the serial loop code to be analyzed, construct a multi-dimensional program graph containing three types of nodes: statement nodes, variable nodes, and constant nodes, as well as three types of edges, and insert control nodes at the serial loop exit point and branch confluence point; Step 2: Perform non-intrusive instrumentation on the serial loop, collect the runtime memory address sequence of the serial loop variables, and calculate the step difference sequence and scalar features; Step 3: Encode the stride difference sequence and the scalar features, and fuse them to obtain the variable-level dynamic embedding; Step 4: Complete the structural prior encoding of the multidimensional program graph to obtain the static embedding of each node, and obtain the static graph-level representation through attention pooling; use the static embedding of variable nodes as the query and the dynamic embedding as the key and value, perform single-head cross-attention calculation, and update the variable node embedding through residual connections; Step 5: For the updated multidimensional program graph, propagate the dynamic information of the variable nodes along various edges to all nodes in the graph, and obtain a dynamic graph-level representation through attention pooling; Step 6: Calculate the entropy value based on the parallel category prediction probability distribution output by the parallel classification head of the static graph-level representation obtained in Step 4. Map the entropy value to the gating weight through a learnable affine transformation. Then, weightedly fuse the static graph-level representation and the dynamic graph-level representation to generate a fused graph-level representation vector. Step 7: Based on the fused graph-level representation vector, jointly predict the parallelism type of the loop and the multidimensional optimization strategy.

[0010] As a preliminary step, in step 1, the serial loop code is parsed to generate an abstract syntax tree; based on the abstract syntax tree, three types of nodes are identified: statement nodes, variable nodes, and constant nodes; control nodes are inserted at loop exit points and branch confluence points; syntactic edges are constructed to connect statements with their referenced or defined variable nodes; typed data dependency edges are constructed and labeled according to dependency type as RAW (Read-After-Write), WAR (Read-After-Write), and WAW (Multiple-Write-WAW); control flow edges are constructed to connect control nodes with the statement nodes they govern; and a multi-dimensional program graph is formed by the three types of nodes, control nodes, and the three types of directed edges.

[0011] As a preliminary step, in step 2, the serial loop is non-intrusively instrumented using the GDB debugger. The specific implementation process is as follows: a GDB breakpoint script is automatically generated, conditional breakpoints are set at the loop entry point and memory access instruction, the memory access address and loop iteration counter are recorded when the breakpoint is triggered, and the step sequence is automatically generated by parsing the log file after the program exits. The entire process does not require modification of the source code.

[0012] As a preliminary step, in step 2, under K typical input configurations, each loop iteration is performed T times, and the element-level memory address of each memory access variable in the loop body is recorded successively to form an address sequence; the address sequence is differentially calculated to obtain the stride difference sequence; based on the stride difference sequence, four-dimensional scalar features are extracted, including stride regularity, aliasing degree, stride mean and cross-iteration reuse degree; where K and T are preset values.

[0013] Let variables In the The memory address of the next iteration is Then the stride difference sequence and four-dimensional scalar characteristics The calculation formula is as follows: ; ; ; ; ; in, This indicates variance calculation. Indicates the first Variables in the next iteration The set of write addresses Indicates the first Variables in the next iteration The set of read addresses.

[0014] As a preliminary step, in step 3, the stride difference sequence is input into a bidirectional long short-term memory network (BiLSTM) to extract temporal features. The four-dimensional scalar features are input into a multilayer perceptron (MLP) to extract statistical features. The two features are concatenated and then linearly projected to generate a variable-level dynamic embedding. ; ; ; in, Represents the stride difference sequence. This represents a four-dimensional scalar eigenvector. This represents a vector concatenation operation. and These represent the weight matrix and bias term of the linear projection layer, respectively. Representing variables Dynamic embedding.

[0015] As a preliminary step, in step 4, a first-layer relational graph convolutional network (RGCN) is applied to the multidimensional procedural graph to obtain the static structure embedding of each node. ;in, Represents a node The initial feature vector, Represents a set of edge types. Represents a node By type The set of neighboring nodes connected by the edge. Represents the normalization constant. Representation type The corresponding weight matrix, This represents the node self-loop weight matrix. Represents a non-linear activation function. Represents a node neighboring nodes The initial feature vector; The full graph node embeddings output by the first-layer RGCN are subjected to attention-based global pooling, and the importance weight of each node is calculated and weighted summed to obtain a static graph-level representation. , ;in, and This represents the attention weight parameter. Represents a node Static attention weights, Indicates a loop Static graph-level representation, For a set of nodes, Represents the attention weight vector transpose; For each variable node in the multidimensional program graph Static embeddings output from the first-layer RGCN For the query vector, dynamically embedding the variable Simultaneously serving as both key and value vectors, the single-head cross-attention score is calculated, and the variable node embeddings are updated via residual connections to obtain the updated variable node embeddings. , , ;in, , , These represent the linear projection matrices of the query, key, and value, respectively. Represents the attention dimension. Indicates attention weights. This indicates the output of attention.

[0016] As a preliminary step, in step 5, a second-layer relational graph convolutional network (RGCN) is applied to the updated multidimensional program graph. The updated variable embeddings are propagated across the entire graph along syntactic edges, three types of data dependency edges, and control flow edges, allowing execution information to spread from variable nodes to statement nodes, constant nodes, and control nodes, resulting in dynamically enhanced full-graph node embeddings. ; ; in, Represents a non-linear activation function. Represents a set of edge types. Represents a node By type The set of neighboring nodes connected by the edge. Represents the normalization constant. Representation type The corresponding weight matrix, Represents a node The feature vector updated after cross attention, This represents the node self-loop weight matrix. Represents a node Neighbors Feature vector updated after cross-attention; Embedding of dynamically enhanced full graph nodes Attention-based global pooling is performed, the importance weight of each node is calculated and weighted summation is performed to obtain a dynamic graph-level representation, which is used in the subsequent adaptive gating fusion step based on prediction entropy. ; ; in, For a set of nodes, and This represents the attention weight parameter. Represents a node Attention weights Indicates a loop The dynamic graph-level representation.

[0017] As a preliminary step, in step 6, the static branch prediction entropy in, Indicates a loop The static branch prediction entropy value, This represents a set of parallelism categories, including independent parallelism, reducing parallelism, and serial parallelism. This indicates that static branching is based on loop code. For parallelism categories The predicted probability, Indicates the category of parallelism. This represents the input loop code; Entropy value is obtained through learnable affine transformation. Mapped to gate weights A larger gating weight indicates a higher contribution of dynamic evidence to the fusion result; among which, and For learnable scalar parameters, This represents the Sigmoid activation function; The dynamic and static graph-level representations are weighted and summed using gating weights and their complements, respectively, to generate a fused graph-level representation vector. ;in, and These represent static graph-level representation and dynamic graph-level representation, respectively. This represents the fused graph-level representation vector; when dynamic instrumentation is not feasible, the gating weights are... Setting it to zero degenerates the fused graph-level representation into a purely static graph-level representation, but the system can still output effective parallel prediction results.

[0018] As a preliminary step, in step 7, based on the fused graph-level representation vector... After parallelism classification and Softmax activation, the output probability distribution of cyclic parallelism types is obtained, including three categories: independent parallelism, reduced parallelism, and serial parallelism.

[0019] in, and The weight matrix and bias terms of the parallel classification head are represented. Represents the probability distributions of the three types of parallelism; After gradient stopping gate isolation to isolate parallel classification gradients, independent linear layers and sigmoid activations are applied to the fused graph-level representation, outputting a five-dimensional binary optimization policy vector, corresponding to five policies: memory merging, warp reduction, block partitioning, shared memory, and pipelined optimization.

[0020] in, and This represents the weight matrix and bias terms of the policy prediction head. This indicates that the gradient has stopped.

[0021] The technical solution adopted by the system of this invention is: a GPU code parallelism analysis and optimization system, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, enable the one or more processors to implement the GPU code parallelism analysis and optimization method.

[0022] Compared with the prior art, the beneficial effects of the present invention include: (1) This invention constructs a multidimensional program graph containing statement nodes, variable nodes, constant nodes and control nodes and three types of directed edges to finely encode program structure and typed data dependencies. It can simultaneously infer the syntactic structure of the program, the three types of data conflict semantics (RAW / WAR / WAW) and control flow constraints in the same graph representation. It overcomes the defect of existing methods that use only a single AST view, resulting in incomplete expression of dependency semantics, and provides richer structural priors for parallel inference.

[0023] (2) This invention collects the runtime step sequence of loop variables by instrumenting a non-intrusive debugger and performs timing and scalar fusion encoding. The real memory access behavior of the loop can be observed without modifying the source code. This breaks through the blind spot of pure static analysis on pointer aliases and non-affine subscripts, enabling the model to obtain runtime execution evidence and significantly improving the ability to judge the parallelism of irregular memory access loops.

[0024] (3) The present invention uses single-head cross attention to accurately inject dynamic execution features into the variable nodes corresponding to the program graph, thereby achieving accurate alignment of dynamic execution evidence and static program graph at the node level, avoiding feature confusion caused by global splicing, enabling execution information to accurately correct the representation of the corresponding node according to the variable granularity, while retaining structural priors through residual connections to ensure that the injection process does not destroy the semantics of the original graph structure.

[0025] (4) This invention encodes the structural prior and then propagates the dynamic information through a two-stage relational graph convolutional network. It also fuses the static and dynamic graph-level representations through adaptive gating based on prediction entropy. The two-stage design ensures that the structural prior is fully established before the dynamic information is injected, thus avoiding interference of the static structure encoding by the dynamic signal. The prediction entropy gating mechanism reduces the weight of dynamic evidence when the static prediction confidence is high and increases the weight of dynamic evidence when the static prediction uncertainty is high, thereby achieving adaptive fusion of static and dynamic information and effectively resolving dependency ambiguity.

[0026] (5) The present invention uses a gradient-isolated hierarchical decision head to jointly predict the parallelism type and optimization strategy, thereby achieving adaptive fusion of the static structure of the program and the dynamic execution evidence. By blocking the gradient interference of the optimization strategy prediction on the parallelism representation learning through gradient stopping operation, the two prediction tasks are decoupled and trained, ensuring that the learning of the parallelism representation is not polluted by the optimization strategy supervision signal. At the same time, the two tasks of parallelism classification and optimization strategy prediction are completed with a single model, thereby improving the overall efficiency of the system.

[0027] (6) This invention significantly improves the accuracy of parallelism inference for non-affine loops and pointer-intensive loops, and has high practical application value. Attached Figure Description

[0028] The technical solutions of the present invention will be further illustrated below using embodiments and specific implementation methods. In addition, some accompanying drawings are used in the description of the technical solutions. Those skilled in the art can obtain other drawings and the intent of the present invention from these drawings without any creative effort.

[0029] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the GDB non-invasive stud insertion and stride sequence acquisition process according to an embodiment of the present invention; Figure 3 This is a flowchart of the two-stage fusion representation learning process according to an embodiment of the present invention. Detailed Implementation

[0030] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0031] Please see Figure 1 This embodiment provides a GPU code parallelism analysis and optimization method, which includes the following steps: Step 1: Obtain the serial loop code to be analyzed, construct a multi-dimensional program graph containing three types of nodes: statement nodes, variable nodes, and constant nodes, as well as three types of edges, and insert additional control nodes at the serial loop exit point and branch confluence point; In one implementation, serial C / C++ loop code to be analyzed is obtained. This serial loop code includes, but is not limited to, program code containing iterative computation structures such as scientific computing loops, engineering simulation loops, and data processing loops. A lightweight parser is used to parse the loop code, generating an abstract syntax tree. This lightweight parser can be a tree-sitter or other syntax analysis tool that does not require a complete compilation environment.

[0032] Based on the abstract syntax tree, three types of nodes are identified: statement nodes, variable nodes, and constant nodes. Statement nodes correspond to each executable statement within the loop body, including assignment statements, function call statements, and conditional branch statements; variable nodes correspond to each memory variable accessed within the loop body, including array variables, pointer variables, and scalar variables; constant nodes correspond to literals and compile-time constants appearing within the loop body. Control nodes are additionally inserted at loop exit points and branch confluence points to represent key nodes in the loop control flow. Three types of directed edges are constructed: syntactic edges connect statements to variable nodes they reference or define, encoding the syntactic structure of the program; typed data dependency edges are labeled into three categories based on dependency type: RAW (read-before-write dependency), WAR (read-before-write dependency), and WAW (multiple-write dependency), with these dependency types extracted from the source code through symbolic analysis; control flow edges connect control nodes to the statement nodes they govern, encoding the control dependencies between branches and loops. The statement nodes, variable nodes, constant nodes, control nodes, and the three types of edges together constitute a multi-dimensional program graph.

[0033] Specifically, let the input loop code be... The constructed multidimensional program graph is as follows The set of nodes and the set of edges are respectively:

[0034]

[0035] in, Represents a set of statement nodes. Represents a set of variable nodes. Represents a set of constant nodes. Represents the set of control nodes; Represents the set of syntactic edges. Represents a set of typed data dependency edges. This represents the set of control flow edges. Through the heterogeneous structure design of the multidimensional program graph, the model can simultaneously infer the syntactic structure of the program, different types of data conflicts, and control flow constraints, providing rich structural priors for subsequent parallel inference.

[0036] Step 2: Perform non-intrusive instrumentation on the serial loop using a debugger, collect the runtime memory address sequence of the serial loop variables, and calculate the step difference sequence and scalar features; In one implementation, please see Figure 2 The target loop is instrumented non-intrusively using the GDB debugger. This non-intrusive instrumentation does not require modification of the source code or recompilation of the program; it only utilizes the debugger's breakpoints and watchpoints to observe memory access behavior. Execution under typical input configurations iterations. The value can be 3 or any other number that can cover different input modes. The value of can be 50 or other iterations that allow for sufficient observation of memory access patterns. The element-level memory address of each memory access variable within the loop body is recorded sequentially, forming an address sequence.

[0037] The address sequence is differentially calculated to obtain the stride difference sequence, which represents the change in memory address of the same variable between adjacent iterations. Four-dimensional scalar features are calculated: stride regularity, aliasing degree, stride mean, and cross-iteration reuse. Stride regularity, calculated as the inverse normalized step variance, measures the consistency of memory access strides across iterations; a higher value indicates more regular strides. Aliasing degree represents the overlap rate of write operation address sets across different iterations; a value of 0 indicates the absence of write aliases and is a key signal for determining the parallelizability of a loop. The stride mean is the average address difference between adjacent iterations. Cross-iteration reuse is the proportion of the intersection between the read operation address set and the previous iteration's write address set.

[0038] Specifically, let variables be defined. In the The memory address of the next iteration is The formulas for calculating the stride difference sequence and the four-dimensional scalar features are as follows: ; ; ; ; ; in, This indicates variance calculation. Indicates the first Variables in the next iteration The set of write addresses Indicates the first Variables in the next iteration The set of read addresses. Through the above four-dimensional scalar characteristics, the key features that quantify the memory access behavior of loop variables can be described.

[0039] When dynamic instrumentation is not feasible or the user chooses to skip dynamic acquisition, the variable-level dynamic embedding is set to an all-zero vector. At this time, the cross-attention module degenerates into an identity mapping. At the same time, the gating weight of the entropy-guided fusion module is set to zero, and the fused graph-level representation degenerates into a pure static graph-level representation. The system smoothly degrades to a pure static parallelism representation learning method based on a multi-dimensional program graph, and can still output effective parallelism prediction results.

[0040] Step 3: Encode the stride difference sequence using a bidirectional long short-term memory network, encode the scalar features using a multilayer perceptron, and fuse them to obtain a variable-level dynamic embedding; In one implementation, the stride difference sequence for each input mode is input into a bidirectional long short-term memory (BiLSTM) network to obtain a stride temporal embedding. The BiLSTM network can simultaneously utilize forward and backward temporal context information to capture temporal dependencies in the stride sequence. A four-dimensional scalar feature vector is input into a multilayer perceptron (MLP) to obtain a scalar feature embedding. The temporal embedding and the scalar feature embedding are concatenated and projected onto a unified dimension through a linear layer to obtain a dynamic embedding of the variables. This dynamic fusion encoding process can be represented as: ; ; ; in, Represents the stride difference sequence. This represents a four-dimensional scalar eigenvector. This represents a vector concatenation operation. and These represent the weight matrix and bias term of the linear projection layer, respectively. Representing variables Dynamic embedding.

[0041] Step 4: Apply the first layer of relational graph convolutional network to the multidimensional program graph to complete the structural prior encoding, obtain the static embedding of each node, and obtain the static graph-level representation through attention pooling; use the static embedding of variable nodes as the query and the dynamic embedding as the key and value, perform single-head cross-attention calculation, and update the variable node embedding through residual connections; Please see Figure 3 In one implementation, before performing cross-attention injection, a first-layer relational graph convolutional network (RGCN) is applied to the multidimensional procedural graph to obtain the static structural embeddings of each node. The relational graph convolutional network can learn independent transformation parameters for different types of edges, thereby effectively modeling multiple types of relationships in heterogeneous graphs. The computation process of the first-layer RGCN can be represented as follows: ; in, Represents a node The initial feature vector, Represents a set of edge types. Represents a node By type The set of neighboring nodes connected by the edge. Represents the normalization constant. Representation type The corresponding first-layer weight matrix, This represents the node self-loop weight matrix. Represents a non-linear activation function. Represents a node neighboring nodes The initial feature vector; The full-graph node embeddings output by the first-layer RGCN are subjected to attention-based global pooling. The importance weight of each node is calculated and summed in a weighted manner to obtain a static graph-level representation, which is then used by the entropy-guided fusion module. The calculation process of attention pooling can be represented as follows: ; ; in, and This represents the attention weight parameter. Represents a node Static attention weights, Indicates a loop Static graph-level representation. For a set of nodes, Represents the attention weight vector transpose; For each variable node in the program graph Static embeddings output from the first-layer RGCN For the query vector, dynamically embedding the variable Simultaneously serving as both key and value vectors, the single-head cross-attention score is calculated, and the variable node embeddings are updated via residual connections. The calculation process of cross-attention can be represented as: ; ; ; in, , , These represent the linear projection matrices of the query, key, and value, respectively. Represents the attention dimension. Indicates attention weights. Indicates attention output, This represents the updated variable node embedding. Through residual joins, both prior structural information and dynamic execution evidence are preserved, achieving precise node-level fusion. For statement nodes, constant nodes, and control nodes, their first-level RGCN output remains unchanged.

[0042] Step 5: Apply a second layer of relational graph convolutional network to the updated multidimensional program graph to propagate the dynamic information of variable nodes along various edges to all nodes in the graph, and obtain a dynamic graph-level representation through attention pooling; In one implementation, after execution features have been injected into variable nodes, a second-layer relational graph convolutional network (RGCN) is applied to the program graph. The updated variable embeddings are propagated across the entire graph along syntactic edges, three types of data dependency edges, and control flow edges, allowing execution information to spread from variable nodes to statement nodes, constant nodes, and control nodes, resulting in dynamically enhanced full-graph node embeddings. The second-layer RGCN uses an independent set of parameters, and the computation process can be represented as follows: ; in, Represents a non-linear activation function. Represents a set of edge types. Represents a node By type The set of neighboring nodes connected by the edge. Represents the normalization constant. Representation type The corresponding weight matrix, Represents a node Feature vector updated after cross-attention; This represents the node self-loop weight matrix. Represents a node Neighbors Feature vector updated after cross-attention; Through a two-stage RGCN design, the first stage establishes structural priors, and the second stage propagates dynamic information to ensure that execution features can effectively spread along the structural dependencies of the program, thereby enabling all types of nodes in the entire graph to obtain enhanced dynamic execution evidence.

[0043] The dynamically enhanced full-graph node embeddings are subjected to attention-based global pooling. The importance weight of each node is calculated and summed in a weighted manner to obtain a dynamic graph-level representation, which is then used in the subsequent adaptive gating fusion step based on prediction entropy. The calculation process of attention pooling can be represented as follows: ; ; in, and This represents the attention weight parameter. Represents a node Attention weights Indicates a loop The dynamic graph-level representation.

[0044] Step 6: Calculate the entropy value based on the parallel category prediction probability distribution output by the parallel classification head of the static graph-level representation obtained in Step 4. Map the entropy value to the gating weight through a learnable affine transformation. Then, weightedly fuse the static graph-level representation and the dynamic graph-level representation to generate a fused graph-level representation vector. In one implementation, the entropy value is calculated based on the predicted probability distribution of the parallelism category by the static branch. A higher entropy value indicates greater uncertainty in the static analysis regarding the dependency. The calculation process of the static branch prediction entropy can be expressed as follows:

[0045] in, Indicates a loop The static branch prediction entropy value, This represents a set of parallelism categories, including independent parallelism, reducing parallelism, and serial parallelism. This indicates that static branching is based on loop code. For parallelism categories The predicted probability, Indicates the category of parallelism. This represents the input loop code; The entropy value is mapped to a gate weight through a learnable affine transformation. The larger the gate weight, the greater the contribution of dynamic evidence to the fusion result.

[0046] The dynamic and static graph-level representations are weighted and summed using gating weights and their complements respectively to generate a fused graph-level representation vector:

[0047] in, and For learnable scalar parameters, This represents the Sigmoid activation function. and These represent static graph-level representation and dynamic graph-level representation, respectively. This represents the fused graph-level representation vector. When dynamic instrumentation is not feasible, the gating weights are... Setting it to zero degenerates the fused graph-level representation into a purely static graph-level representation, but the system can still output effective parallel prediction results.

[0048] Step 7: Based on the fused graph-level representation vector, jointly predict the parallelism type of the loop and the multidimensional optimization strategy.

[0049] In one implementation, based on the fused graph-level representation vector, after passing through a parallelism classification head and Softmax activation, the probability distribution of cyclic parallelism types is output, including three categories: independent parallelism, reduced parallelism, and serial parallelism. ; in, and The weight matrix and bias terms of the parallel classification head are represented. This represents the probability distribution of the three types of parallelism.

[0050] After isolating the parallel classification gradients by gradient-stopping gating, independent linear layers and sigmoid activations are applied to the fused graph-level representation, outputting a five-dimensional binary optimization policy vector, corresponding to five policies: memory merging, warp reduction, block partitioning, shared memory, and pipelined optimization. ; in, and This represents the weight matrix and bias terms of the policy prediction head. This indicates a gradient-stopping operation. Through gradient isolation, it ensures that the prediction of the optimization policy does not interfere with the learning of the parallel representation, thus achieving decoupled training of the two tasks.

[0051] The parallelism types and optimization strategies described are used to guide the generation of GPU parallel code, specifically including the following 26 selectable parallelization optimization methods: I. Optimization methods for independent parallel loops: Output 1: Parallelism type: Independent parallelism; Policy vector: [1;0;0;0;0]; Policy combination: Memory merging.

[0052] Optimization method: The memory access step size of this loop is fixed and equal to the data type width, with adjacent iterations accessing adjacent memory addresses. A memory merging strategy is adopted, arranging threads to access global memory at consecutive addresses, utilizing the GPU hardware's memory merging mechanism to reduce the number of memory transactions and improve memory bandwidth utilization.

[0053] Output 2: Parallelism type: Independent parallelism; Policy vector: [1;0;0;1;0]; Policy combination: Memory merging; Shared memory.

[0054] Optimization Method: The total amount of data in this loop is known and extremely small, and can be completely contained within the shared memory of a single thread block. It also exhibits data reuse and sequential memory access characteristics. A memory merging and shared memory collaborative strategy is adopted. First, data is loaded from global memory to shared memory using a merging method, and subsequent calculations are performed from shared memory. The memory merging mechanism is used to improve loading efficiency, and the low latency of shared memory is used to accelerate data reuse.

[0055] Output 3: Parallelism type: Independent parallelism; Policy vector: [1;0;1;0;0]; Policy combination: Memory merging; Blocking.

[0056] Optimization Method: This loop involves a large amount of data and accesses contiguous addresses. A memory merging and block-based collaborative strategy is adopted, decomposing the computational task into fixed-size data blocks. Within each block, threads access global memory using contiguous addresses. Memory merging reduces memory transactions, while block-based access improves data locality and cache hit rate.

[0057] Output 4: Parallelism type: Independent parallelism; Policy vector: [1;0;0;0;1]; Policy combination: Memory merging; Pipeline.

[0058] Optimization Method: This loop involves streaming data processing with continuous memory access patterns and overlapping memory access and computation. In this scenario, the pipeline strategy adaptively maps to a host-side multi-stream pipeline, using overlapping streams for kernel execution and data transfer. Multiple CUDA streams are used in the CUDA code to alternately execute data transfer and computation, with threads accessing memory at contiguous addresses using a merging mechanism. Overlapping memory access and computation within the pipeline hides global memory access latency.

[0059] Output 5: Parallelism type: Independent parallelism; Policy vector: [1;0;1;1;0]; Policy combination: Memory merging; Blocking; Shared memory.

[0060] Optimization Method: This loop is computationally intensive, with large data volumes, continuous data access, and intra-block reuse after partitioning. A collaborative strategy of memory merging, partitioning, and shared memory is adopted. The computational task is partitioned and then loaded from global memory to shared memory using a merging method. Intra-block computations are then read from shared memory. Memory merging improves loading efficiency, partitioning controls data size, and shared memory accelerates intra-block reuse.

[0061] Output 6: Parallelism type: Independent parallelism; Policy vector: [1;0;0;1;1]; Policy combination: Memory merging; Shared memory; Pipeline.

[0062] Optimization Method: The total amount of data in this loop is known and extremely small, and can be completely contained within the shared memory of a single thread block. It also exhibits multi-stage data dependencies and continuous memory access characteristics. In this scenario, the pipeline strategy adaptively maps to an asynchronous pipeline on the device side, utilizing a hardware-level asynchronous copy mechanism to hide global memory loading latency. A shared memory double-buffering mechanism is employed to overlap computation and data loading at different stages, with memory access merging utilized during the loading stage. Instruction-level pipelines are used to hide loading latency, and shared memory accelerates reuse.

[0063] Output 7: Parallelism type: Independent parallelism; Policy vector: [1;0;1;0;1]; Policy combination: Memory merging; Blocking; Pipeline.

[0064] Optimization Method: This loop involves large amounts of data, continuous access, and overlapping memory access and computation. In this scenario, the pipeline strategy adaptively maps to a host-side multi-stream pipeline, using overlapping streams for kernel execution and data transfer. Data is chunked in the CUDA code, and data transfer and computation are performed across different buffers through overlapping multiple CUDA streams. Threads access memory at contiguous addresses using a merging mechanism. Chunking controls data volume, merging memory accesses improves bandwidth, and the pipeline overlaps memory access and computation.

[0065] Output 8: Parallelism type: Independent parallelism; Policy vector: [1;0;1;1;1]; Policy combination: Memory merging; Blocking; Shared memory; Pipeline.

[0066] Optimization Method: This loop is a large-scale, computationally intensive loop with continuous data access, requiring block partitioning, intra-block reuse, and overlapping loading and computation. In this scenario, the pipeline strategy adaptively maps to a device-side shared-memory double-buffered asynchronous pipeline, utilizing a hardware-level asynchronous copy mechanism to hide global memory loading latency. In the CUDA code, after block partitioning, shared-memory double buffering is used, and data is asynchronously loaded from global memory to shared memory using a merging method. Block partitioning controls the scale, merging memory accesses improves bandwidth, shared memory reuse hides pipeline latency.

[0067] Output 9: Parallelism type: Independent parallelism; Policy vector: [0;0;1;1;0]; Policy combination: Blocking; Shared memory.

[0068] Optimization method: After the loop is divided into blocks, data is reused within the memory. A collaborative strategy of block division and shared memory is adopted, loading each data block into shared memory, and all threads within the block read data from shared memory for computation. Block division is used to control the data size so that sub-blocks adapt to the shared memory capacity, and shared memory accelerates reuse within blocks and reduces repeated loading of global memory.

[0069] Output 10: Parallelism type: Independent parallelism; Policy vector: [0;0;1;1;1]; Policy combination: Blocking; Shared memory; Pipeline.

[0070] Optimization Method: This loop involves large amounts of data requiring block partitioning, intra-block reuse necessitates shared memory, and loading and computation can overlap. In this scenario, the pipeline strategy adaptively maps to a device-side shared-memory double-buffered asynchronous pipeline, utilizing a hardware-level asynchronous copy mechanism to hide global memory loading latency. In the CUDA code, data is partitioned and double-buffered using shared memory; one buffer is used for the current block's computation, while the other asynchronously loads the next block. Block partitioning controls data size, shared memory accelerates intra-block reuse, and the pipeline hides global memory loading latency.

[0071] Output 11: Parallelism type: Independent parallelism; Policy vector: [0;0;0;1;1]; Policy combination: Shared memory; Pipeline.

[0072] Optimization Method: The total amount of data in this loop is known and extremely small, and can be completely contained within the shared memory of a single thread block. However, there are multi-stage data dependencies. In this scenario, the pipeline strategy adaptively maps to an asynchronous pipeline on the device side, utilizing a hardware-level asynchronous copy mechanism to hide global memory loading latency. A shared memory double-buffering mechanism is employed to overlap multi-stage data dependency computations. Instruction-level pipelines are used to hide global memory latency, and shared memory accelerates data reuse, preventing idle computation units.

[0073] Output 12: Parallelism type: Independent parallelism; Policy vector: [0;0;0;1;0]; Policy combination: Shared memory.

[0074] Optimization Method: The total amount of data in this loop is known and extremely small, and can be completely contained within the shared memory of a single thread block. However, data reuse exists, with the same data being accessed by multiple threads. A shared memory strategy is adopted, loading frequently accessed data from global memory into shared memory, and then reading it from shared memory for subsequent calculations. The low latency of shared memory is leveraged to accelerate data reuse and reduce the number of global memory accesses.

[0075] Output 13: Parallelism type: Independent parallelism; Policy vector: [0;0;0;0;1]; Policy combination: Pipeline.

[0076] Optimization Method: The memory access and computation ratio in this loop are balanced, leaving room for overlap between data prefetching and computation. In this scenario, the pipeline strategy is adaptively mapped to a host-side multi-stream pipeline, using overlapping streams for kernel execution and data transfer. Multiple CUDA streams are used in the CUDA code to alternately execute data transfer and computation. By leveraging pipelined overlap of memory access and computation, the impact of global memory access latency on computation units is hidden.

[0077] II. Optimization methods for reducing parallel loops: Output 14: Parallelism type: Reduction Parallelism; Policy vector: [0;1;0;0;0]; Policy combination: Warp Reduction.

[0078] Optimization Method: This loop involves small-scale reduction operations, and data can be directly loaded into registers by the thread. A Warp reduction strategy is adopted, utilizing register-level data exchange within the thread bundle to complete the reduction operation. Since shared memory is not enabled, the independent local reduction results of each Warp need to be written to global memory atomically to achieve cross-Warp aggregation, or the Warp reduction result can be made equivalent to the block reduction result by limiting the thread block size.

[0079] Output 15: Parallelism type: Reduction Parallelism; Policy vector: [1;1;0;0;0]; Policy combination: Memory Merge; Warp Reduction.

[0080] Optimization Method: This loop involves a reduction operation, and the data access pattern before reduction is contiguous address. A combined strategy of memory merging and Warp reduction is adopted. First, data is continuously loaded from global memory into registers using a merging method, and then the reduction is completed using register-level data exchange within the thread block. Since shared memory is not enabled, the independent local reduction results of each Warp need to be written to global memory through atomic operations to complete cross-Warp aggregation, or the Warp reduction result can be made equivalent to the block reduction result by limiting the thread block size. Memory access merging is used to reduce the number of memory transactions, and Warp reduction avoids the serialization of atomic operations.

[0081] Output 16: Parallelism type: Reduction Parallelism; Policy vector: [0;1;0;1;0]; Policy combination: Warp Reduction; Shared memory.

[0082] Optimization Method: This loop involves multi-step reduction operations, requiring multiple accesses to intermediate results. A collaborative strategy of Warp reduction and shared memory is adopted. Warp-level reduction is completed using register-level data exchange within the thread bundle, and the local reduction results of each Warp are temporarily stored in shared memory. The first Warp then reads the results to complete the secondary reduction. The low latency of shared memory accelerates the temporary storage of intermediate results, and Warp reduction avoids the serialization of atomic operations, reducing the number of global memory write-backs.

[0083] Output 17: Parallelism type: Reduction Parallelism; Policy vector: [1;1;0;1;0]; Policy combination: Memory Merge; Warp Reduction; Shared Memory.

[0084] Optimization Method: This loop involves multi-step reduction with continuous data access and intermediate results need to be cached. A collaborative strategy of memory merging, Warp reduction, and shared memory is adopted. First, data is loaded from global memory into registers using a merging method. Warp-level reduction is completed using register-level data exchange within a thread bundle. Local results from each Warp are temporarily written to shared memory, and the first Warp completes the second reduction. Memory merging improves loading efficiency, shared memory storage of intermediate results reduces global memory write-backs, and Warp reduction avoids serialization.

[0085] Output 18: Parallelism type: Reduction Parallelism; Policy vector: [1;1;0;0;1]; Policy combination: Memory Merge; Warp Reduction; Pipeline.

[0086] Optimization Method: This loop involves reduction operations on multiple sets of contiguously stored data, which can be piped together. In this scenario, the pipeline strategy is adaptively mapped to a host-side multi-stream pipeline, with overlapping kernel execution and data transfer between streams. In the CUDA code, each reduction set is assigned to a different CUDA stream. Data transfer and reduction computation are overlapped through asynchronous transmission. Threads in the kernel access memory at contiguous addresses using a merging mechanism, and reduction is completed within a group using register-level data exchange within the thread bundle. Since shared memory is not enabled, the independent local reduction results of each warp need to be written to global memory atomically to achieve cross-warp aggregation, or the warp reduction result can be made equivalent to a block reduction result by limiting the thread block size. Pipelining is used to overlap memory access and computation for multiple reduction sets.

[0087] Output 19: Parallelism type: Reduction Parallelism; Policy vector: [0;1;0;1;1]; Policy combination: Warp Reduction; Shared memory; Pipeline.

[0088] Optimization Method: This loop involves multi-step reduction, requiring intermediate results to be cached and allowing overlap between reduction stages. In this scenario, the pipeline strategy adaptively maps to an asynchronous pipeline on the device side, utilizing a hardware-level asynchronous copy mechanism to hide global memory loading latency. A shared memory double-buffering mechanism is employed to overlap multi-stage reduction computations. Warp-level reduction is completed using register-level data exchange within thread bundles, with shared memory temporarily storing intermediate results, and the instruction-level pipeline hiding loading latency.

[0089] Output 20: Parallelism type: Reduction Parallelism; Policy vector: [1;1;0;1;1]; Policy combination: Memory Merge; Warp Reduction; Shared Memory; Pipeline.

[0090] Optimization Method: This loop involves small-scale, multi-step reduction. The total data volume is extremely small and can be fully accommodated by a single block. Data access is continuous, intermediate results need to be cached, and multi-stage reductions can overlap. In this scenario, the pipeline strategy is adaptively mapped to an asynchronous pipeline on the device side, utilizing a hardware-level asynchronous copy mechanism to hide global memory loading latency. A shared memory double-buffering mechanism is used to overlap reduction calculations at different cascaded stages, with memory access merging used during the loading stage. Warp-level reduction is completed using register-level data exchange within the thread bundle, with shared memory temporarily storing intermediate results. The multi-stage instruction pipeline hides loading latency.

[0091] Output 21: Parallelism type: Reduction Parallelism; Policy vector: [0;1;1;1;0]; Policy combination: Warp Reduction; Blocking; Shared Memory.

[0092] Optimization Method: This loop involves a very large-scale reduction operation, requiring multi-level reduction and non-contiguous memory access. A collaborative strategy of Warp reduction, block partitioning, and shared memory is adopted. First, block partitioning controls the size of the reduction. Each thread directly loads data from global memory into registers, utilizing register-level data exchange within the thread to complete Warp-level local reduction. Each Warp only writes the final local result it holds to shared memory for temporary storage (significantly saving shared memory bandwidth and capacity). The first Warp reads from shared memory to complete the secondary reduction within the block. Finally, the results of each block are written to global memory to complete the inter-block reduction. Block partitioning controls the reduction size, shared memory temporarily stores intermediate results, and Warp reduction avoids serialization.

[0093] Output 22: Parallelism type: Reduction Parallelism; Policy vector: [1;1;1;1;0]; Policy combination: Memory Merge; Warp Reduction; Blocking; Shared Memory.

[0094] Optimization Method: This loop involves large-scale reduction operations with continuous data access, requiring block partitioning and caching of intermediate results. A collaborative strategy of memory merging, Warp reduction, block partitioning, and shared memory is adopted. First, the data is partitioned. Sub-blocks are loaded from global memory to shared memory using a merging method. Warp-level local reduction is completed using register-level data exchange within a thread bundle. After each Warp's local results are written to shared memory, the first Warp performs a second reduction. Finally, the results of each block are written to global memory to complete inter-block reduction. Memory merging improves loading efficiency, block partitioning controls the reduction scale, shared memory temporarily stores intermediate results, and Warp reduction avoids serialization.

[0095] Output 23: Parallelism type: Reduction Parallelism; Policy vector: [0;1;1;1;1]; Policy combination: Warp Reduction; Blocking; Shared Memory; Pipeline.

[0096] Optimization Method: This loop involves ultra-large-scale, multi-group reduction operations, requiring multi-level reduction, intermediate caching, and multiple pipelines. In this scenario, the pipeline strategy adaptively maps to a device-side shared-memory double-buffered asynchronous pipeline, utilizing a hardware-level asynchronous copy mechanism to hide global memory loading latency. Large-scale data is divided into blocks for control, using shared-memory double buffering. Warp-level local reduction is completed using register-level data exchange within thread bundles. The local results of each Warp are written to shared memory, then aggregated again by the first Warp, and finally, the block results are written to global memory to complete inter-block reduction. Block division controls the reduction scale, shared memory temporarily stores intermediate results, the device-side asynchronous pipeline hides loading latency, and Warp reduction avoids serialization.

[0097] Output 24: Parallelism type: Reduction Parallelism; Policy vector: [1;1;1;1;1]; Policy combination: Memory Merge; Warp Reduction; Blocking; Shared Memory; Pipeline.

[0098] Optimization Method: This loop involves complex reduction operations, requiring continuous data access, block partitioning, caching of intermediate results, and multiple pipelined sets. In this scenario, the pipeline strategy adaptively maps to a device-side shared-memory double-buffered asynchronous pipeline, utilizing a hardware-level asynchronous copy mechanism to hide global memory loading latency. All five optimization strategies are comprehensively adopted: block partitioning to control data size, loading data into shared memory using a merging method, utilizing register-level data exchange within thread bundles to complete local reduction, writing local results from each Warp to shared memory followed by secondary aggregation by the first Warp, and finally writing the final block result to global memory to complete inter-block reduction. The overall approach uses a double-buffered pipeline to overlap memory access and computation. This leverages memory merging to improve bandwidth, block partitioning to control size, shared memory to accelerate reuse, Warp reduction to avoid serialization, and the pipeline to hide loading latency.

[0099] Output 25: Parallelism type: Reduction Parallelism; Policy vector: [1;1;1;0;0]; Policy combination: Memory Merge; Warp Reduction; Blocking.

[0100] Optimization Method: This loop performs block reduction operations with continuous data access, and intermediate results can reside in registers. A collaborative strategy of memory merging, Warp reduction, and block division is employed. A grid-stride loop is used for block control, continuously loading data from global memory in a merging manner. Warp-level local reduction is completed using register-level data exchange within thread bundles. Since shared memory is not enabled, the independent local reduction results of each Warp need to be written to global memory atomically to achieve cross-Warp aggregation, or the Warp reduction result can be made equivalent to the block reduction result by limiting the thread block size. Block division controls the reduction scale, merging memory access improves loading efficiency, and Warp reduction avoids serialization.

[0101] Output 26: Parallelism type: Reduction Parallelism; Policy vector: [1;1;1;0;1]; Policy combination: Memory Merge; Warp Reduction; Blocking; Pipeline.

[0102] Optimization Method: This loop involves multi-group block reduction operations with continuous data access, pipelining capabilities for each group, and relatively small block sizes. In this scenario, the pipeline strategy is adaptively mapped to a host-side multi-stream pipeline, using overlapping streams for kernel execution and data transfer. In the CUDA code, each group of reductions is assigned to different CUDA streams. Each group uses a grid-stride loop for block control, overlapping data transfer and reduction computation across groups via asynchronous transmission. Threads in the kernel access memory at contiguous addresses using a merging mechanism, and within a group, register-level data exchange within the thread bundle completes warp-level reduction. Since shared memory is not enabled, the independent local reduction results of each warp need to be written to global memory atomically to achieve cross-warp aggregation, or the warp reduction result can be made equivalent to a block reduction result by limiting the thread block size. Pipelining is used to overlap memory access and computation for multiple reduction groups.

[0103] The GPU code parallelism analysis and optimization method and system provided in this invention constructs a multi-dimensional program graph containing statement nodes, variable nodes, constant nodes, control nodes, and three types of directed edges to finely encode program structure and typed data dependencies. It collects the runtime stride sequence of loop variables through non-intrusive debugger instrumentation and performs temporal and scalar fusion encoding. Single-head cross-attention is used to accurately inject dynamic execution features into the corresponding variable nodes of the program graph. A two-stage relational graph convolutional network first encodes structural priors and then propagates dynamic information. Adaptive gating based on prediction entropy fuses static and dynamic graph-level representations, and a gradient-isolated hierarchical decision head jointly predicts the parallelism type and optimization strategy. This invention can effectively achieve adaptive fusion of static program structure and dynamic execution evidence, significantly improving the accuracy of parallelism inference for non-affine loops and pointer-intensive loops. It is applicable to application scenarios such as automatic parallelization of serial programs, automatic GPU code generation, and compiler optimization decisions.

[0104] The invention will be further illustrated below through specific experiments.

[0105] Systematic evaluation was conducted on the standard high-performance computing parallelism annotation dataset (OMP_Serial). This dataset contains 32,570 loop samples, covering affine loops, non-affine loops, and pointer-intensive loops, divided into training, validation, and test sets in an 8:1:1 ratio. To support dynamic feature learning, instrumentation was performed on 4,000 samples in the training set and 500 samples in the validation set. Instrumentation was also completed on the complete test set (3,257 samples). Experiments were conducted on a workstation equipped with an Intel i7-14700KF CPU, an NVIDIA RTX 4090 GPU, and 64GB of RAM. C code and CUDA code were compiled using GCC 11.4 and NVCC 12.4, respectively, with the large language model inference temperature set to 0.3. Model training used the Adam optimizer with a learning rate of 0.001 and a weight decay of 1×10⁻⁶. -5 The batch size is 32, the training is up to 100 rounds, early stopping is performed based on the validation set Macro-F1, all experiments use a fixed random seed of 42, the sequence length T=50, and the number of input patterns K=3.

[0106] This invention was comprehensively compared with polyhedral compilation methods, learning methods, and large language model methods, and the results are shown in Table 1.

[0107] Table 1 Experimental Results

[0108] IND indicates independent parallelism, RED indicates reduction parallelism, and SEQ indicates serialism. Optimization strategy prediction includes five categories: memory merging (MC), warp reduction (WR), block division (TL), shared memory (SM), and pipeline (ST). Polyhedral compilation methods do not support optimization strategy prediction and are marked with "—".

[0109] This invention achieves a parallelism detection accuracy of 96.4% and a Macro-F1 score of 96.7% on the test set, outperforming the most powerful language model baseline GPT-4.1 (94.4%) by 2.0 percentage points and AutoParLLM (91.4%) by 5.0 percentage points. In the optimization policy prediction task, the Macro-F1 score reaches 0.831, outperforming GPT-4.1 (0.761) by 7.0 percentage points, demonstrating that large language models cannot reliably infer optimization policies dependent on runtime memory access information solely from source code text. All large language models outperform those using only static or dynamic signals under a combined static and dynamic signal input configuration, proving that static structural features and dynamic execution stride features are complementary and indispensable.

[0110] The test set was analyzed hierarchically by loop type. The polyhedral compilation method achieved F1 scores of only 0.147 to 0.178 on non-affine loops and only 0.155 to 0.239 on pointer-intensive loops, almost completely failing. This invention achieves optimal performance on affine, non-affine, and pointer-intensive loops, verifying the effectiveness of incorporating dynamic execution stride features for irregular memory access loops.

[0111] The test set was divided into a static and dynamic prediction consistency group (73.9%, 2,407 samples, average prediction entropy). =0.31) and the inconsistent group (26.1%, 850 samples, average prediction entropy) In the inconsistent group, the prediction accuracy of static graph-level representation alone was only 71.3%, and that of dynamic graph-level representation alone was only 68.5%. This invention, through adaptive gating fusion based on prediction entropy, achieved 86.2%, exceeding the accuracy by +14.9pp and +17.7pp respectively. The average prediction entropy of the inconsistent group was significantly higher than that of the consistent group, verifying the design assumption that static prediction entropy reliably identifies structurally ambiguous samples, and proving that adaptive gating based on prediction entropy can effectively resolve dependency ambiguities between static and dynamic signals.

[0112] End-to-end code generation was evaluated on 79 samples, including Polybench (30 affine loops), Rodinia (19 irregular high-performance computing cores), and 30 non-parallel loops. The results are shown in Table 2. Table 2 CUDA code generation performance

[0113] The overall accuracy of this invention reaches 86.1%, outperforming the most powerful language model baseline (74.7%) by 11.4 percentage points (pp); the relative speedup reaches 81.0%. Regarding the correct rejection of non-parallel loops, this invention correctly rejected 26 / 30 non-parallel loops, the highest among all methods. It correctly processed 17 / 19 samples on the irregular loop (Rodinia subset), while PPCG correctly processed 0 samples on this subset, further validating the effectiveness of this invention in handling irregular loops.

[0114] To verify the contribution of each core module of this invention, a systematic ablation experiment was conducted, and the results are shown in Table 3: Table 3 Ablation Experiment Results

[0115] Ablation experiments show that: removing the stride pattern encoding module leads to a 4.3 percentage point decrease in parallelism detection accuracy and a 6.0 percentage point decrease in the Macro-F1 score for optimized policy prediction, demonstrating the key contribution of the dynamic stride features acquired through non-invasive instrumentation to parallelism inference; randomly shuffling the stride difference sequence leads to a 3.5 percentage point decrease in accuracy, proving that the invention utilizes the real cross-iteration temporal memory access patterns rather than statistical pseudo-correlation; removing the adaptive gating fusion module based on prediction entropy leads to a 1.4 percentage point decrease in accuracy, demonstrating the necessity of adaptive gating for the fusion of static and dynamic information; removing the gradient isolation mechanism leads to a 0.4 percentage point decrease in the Macro-F1 score for optimized policy prediction, demonstrating the importance of gradient isolation for decoupling the training of parallelism classification and optimized policy prediction tasks.

[0116] It should be understood that the embodiments described above are only some, not all, of the embodiments of the present invention. Furthermore, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0117] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for GPU code parallelism analysis and optimization, characterized in that, Includes the following steps: Step 1: Obtain the serial loop code to be analyzed, construct a multi-dimensional program graph containing statement nodes, variable nodes, constant nodes, and three types of edges, and insert control nodes at the serial loop exit point and branch confluence point; Step 2: Perform non-intrusive instrumentation on the serial loop using the GDB debugger, collect the runtime memory address sequence of the serial loop variables, and calculate the step difference sequence and scalar features; Step 3: Based on bidirectional long short-term memory network and multilayer perceptron, the stride difference sequence is encoded, the scalar features are encoded, and fused to obtain variable-level dynamic embedding; Step 4: Based on the relational graph convolutional network, the structure prior encoding of the multidimensional program graph is completed to obtain the static embedding of each node and the static graph-level representation; Use statically embedded variable nodes as queries and dynamically embedded nodes as keys and values ​​to update variable node embeddings; Step 5: For the updated multidimensional program graph, propagate the dynamic information of the variable nodes along various edges to all nodes in the graph, and obtain a dynamic graph-level representation through attention pooling; Step 6: Calculate the entropy value based on the parallel category prediction probability distribution output by the static graph-level representation, map the entropy value to the gating weight through a learnable affine transformation, and weight and fuse the static graph-level representation and the dynamic graph-level representation to generate a fused graph-level representation vector; Step 7: Based on the fused graph-level representation vector, jointly predict the parallelism type and multi-dimensional optimization strategy of the loop; the parallelism type includes independent parallelism and reduction parallelism; the optimization strategy includes five types of GPU optimization strategies: memory merging, warp reduction, block partitioning, shared memory, and pipeline.

2. The GPU code parallelism analysis and optimization method according to claim 1, characterized in that: In step 1, the serial loop code is parsed to generate an abstract syntax tree (AST). Based on the AST, three types of nodes are identified: statement nodes, variable nodes, and constant nodes. Control nodes are inserted at loop exit points and branch confluence points. Syntactic edges are constructed to connect statements with their referenced or defined variable nodes. Typed data dependency edges are constructed and labeled according to dependency type as RAW (Read-Only), WAR (Read-Only), and WAW (Multiple Writes). Control flow edges are constructed to connect control nodes with the statement nodes they govern. The three types of nodes, control nodes, and the three types of directed edges constitute a multidimensional program graph.

3. The GPU code parallelism analysis and optimization method according to claim 1, characterized in that: In step 2, the serial loop is instrumented non-intrusively using the GDB debugger. The specific implementation process is as follows: a GDB breakpoint script is automatically generated, conditional breakpoints are set at the loop entry point and memory access instruction, the memory access address and loop iteration counter are recorded when the breakpoint is triggered, and the step sequence is automatically generated by parsing the log file after the program exits. The entire process does not require modification of the source code.

4. The GPU code parallelism analysis and optimization method according to claim 1, characterized in that: In step 2, under each of the K typical input configurations, T loop iterations are performed, and the element-level memory address of each memory access variable in the loop body is recorded one by one to form an address sequence; The stride difference sequence is obtained by performing differential calculation on the address sequence; Four-dimensional scalar features, including stride regularity, are extracted based on stride difference sequences. Alias Mean stride Reusability across iterations Where K and T are preset values; Let variables In the The memory address of the next iteration is Then the stride difference sequence and four-dimensional scalar characteristics The calculation formula is as follows: ； ； ； ； ； in, This indicates variance calculation. Indicates the first Variables in the next iteration The set of write addresses Indicates the first Variables in the next iteration The set of read addresses.

5. The GPU code parallelism analysis and optimization method according to claim 1, characterized in that: In step 3, the stride difference sequence is input into a bidirectional long short-term memory network (BiLSTM) to extract temporal features. The four-dimensional scalar features are input into a multilayer perceptron (MLP) to extract statistical features. The two feature streams are concatenated and then linearly projected to generate a variable-level dynamic embedding. ; ； ； ； in, Represents the stride difference sequence; This represents a four-dimensional scalar eigenvector, including stride regularity. Alias Mean stride Reusability across iterations ; This represents a vector concatenation operation; and These represent the weight matrix and bias term of the linear projection layer, respectively; Representing variables Dynamic embedding.

6. The GPU code parallelism analysis and optimization method according to claim 1, characterized in that: In step 4, the first layer of the relational graph convolutional network RGCN is applied to the multidimensional program graph to obtain the static structure embedding of each node. ;in, Represents a node The initial feature vector, Represents a set of edge types. Represents a node By type The set of neighboring nodes connected by the edge. Represents the normalization constant. Representation type The corresponding weight matrix, This represents the node self-loop weight matrix. Represents a non-linear activation function. Represents a node neighboring nodes The initial feature vector; The full graph node embeddings output by the first-layer RGCN are subjected to attention-based global pooling, and the importance weight of each node is calculated and weighted summed to obtain a static graph-level representation. , ;in, and This represents the attention weight parameter. Represents a node Static attention weights, Indicates a loop Static graph-level representation, For a set of nodes, Represents the attention weight vector transpose; For each variable node in the multidimensional program graph Static embeddings output from the first-layer RGCN For the query vector, dynamically embedding the variable Simultaneously serving as both key and value vectors, the single-head cross-attention score is calculated, and the variable node embeddings are updated via residual connections to obtain the updated variable node embeddings. , , ;in, , , These represent the linear projection matrices of the query, key, and value, respectively. Represents the attention dimension. Indicates attention weights. This indicates the output of attention.

7. The GPU code parallelism analysis and optimization method according to claim 1, characterized in that: In step 5, a second-layer relational graph convolutional network (RGCN) is applied to the updated multidimensional program graph. The updated variable embeddings are propagated throughout the entire graph along syntactic edges, three types of data dependency edges, and control flow edges. This allows execution information to spread from variable nodes to statement nodes, constant nodes, and control nodes, resulting in dynamically enhanced full-graph node embeddings. : ； in, Represents a non-linear activation function. Represents a set of edge types. Represents a node By type The set of neighboring nodes connected by the edge. Represents the normalization constant. Representation type The corresponding weight matrix, Represents a node The feature vector updated after cross attention, This represents the node self-loop weight matrix. Represents a node Neighbors Feature vector updated after cross-attention; Embedding of dynamically enhanced full graph nodes Attention-based global pooling is performed, the importance weight of each node is calculated and weighted summation is performed to obtain a dynamic graph-level representation, which is used in the subsequent adaptive gating fusion step based on prediction entropy. ； ； in, For a set of nodes, and This represents the attention weight parameter. Represents a node Attention weights Indicates a loop The dynamic graph-level representation.

8. The GPU code parallelism analysis and optimization method according to claim 1, characterized in that: In step 6, the static branch prediction entropy in, Indicates a loop The static branch prediction entropy value, This represents a set of parallelism categories, including independent parallelism, reducing parallelism, and serial parallelism. This indicates that static branching is based on loop code. For parallelism categories The predicted probability, Indicates the category of parallelism. This represents the input loop code; Entropy value is obtained through learnable affine transformation. Mapped to gate weights A larger gating weight indicates a higher contribution of dynamic evidence to the fusion result; among which, and For learnable scalar parameters, This represents the Sigmoid activation function; The dynamic and static graph-level representations are weighted and summed using gating weights and their complements, respectively, to generate a fused graph-level representation vector. ;in, and These represent static graph-level representation and dynamic graph-level representation, respectively. This represents the fused graph-level representation vector; when dynamic instrumentation is not feasible, the gating weights are... Setting it to zero degenerates the fused graph-level representation into a purely static graph-level representation, but the system can still output effective parallel prediction results.

9. The GPU code parallelism analysis and optimization method according to any one of claims 1-8, characterized in that: In step 7, based on the fused graph-level representation vector After parallelism classification and Softmax activation, the output probability distribution of cyclic parallelism types is obtained, including three categories: independent parallelism, reduced parallelism, and serial parallelism. in, and The weight matrix and bias terms of the parallel classification head are represented. Represents the probability distributions of the three types of parallelism; After gradient stopping gate isolation to isolate parallel classification gradients, independent linear layers and sigmoid activations are applied to the fused graph-level representation, outputting a five-dimensional binary optimization policy vector, corresponding to five policies: memory merging, warp reduction, block partitioning, shared memory, and pipelined optimization. in, and This represents the weight matrix and bias terms of the policy prediction head. This indicates that the gradient has stopped.

10. A GPU code parallelism analysis and optimization system, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the GPU code parallelism analysis and optimization method as described in any one of claims 1 to 9.

Images