Parallel thread virtualization and simt semantic degradation execution method and system

Abstract

The application provides a parallel thread virtualization and SIMT semantic degradation execution method and system, and belongs to the technical field of parallel computing. The application comprises the following steps: receiving a parallel source program and identifying kernel data; extracting parallel semantics from the kernel data and constructing an intermediate representation; constructing an activity mask based on the intermediate representation to indicate an effective logical thread set in a strip by using a vector length independent striping mapping; splitting host and device program instructions according to the kernel data; performing vector predication on a conditional branch to realize independent execution of each logical thread; intercepting a kernel start call through a pad layer on the host side, serializing kernel parameters into a command descriptor, writing the command descriptor into a lock-free ring queue of shared memory, and notifying the device to execute through a doorbell mechanism; and the device end firmware analyzes the command descriptor, configures execution resources, and jumps to the generated kernel code for execution. The application realizes efficient and transparent degradation of a parallel thread model.

Description

Technical Field

[0001] This invention belongs to the field of parallel computing technology, and in particular relates to a parallel thread virtualization and SIMT semantic degradation execution method and system. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid growth in computing power demands from workloads such as artificial intelligence training and inference, big data analysis, and scientific computing, heterogeneous computing architectures (host CPUs and accelerators working together) have become mainstream. Existing general-purpose graphics processors are typically equipped with hardware-level warp scheduling, branching and diverging processing, and multi-level storage systems, and are deeply integrated with mature software ecosystems, forming a complete software closed loop from programming models to compilers, runtimes, and mathematical libraries.

[0004] Meanwhile, new parallel computing chips designed for energy efficiency and area optimization are increasingly adopting domain-specific custom instruction sets and integrating hardware structures such as scalable vector computation units and matrix multiplication acceleration units. However, they often improve computational density by simplifying hardware control logic, thus lacking the hardware thread bundle scheduler and branch convergence unit of traditional GPUs. These architectural differences result in existing programs in the single-instruction multiple-thread (SIMT) ecosystem being difficult to run directly on these chips. Users need to rewrite or manually modify them into specific assembly / non-mainstream languages, leading to high migration costs and difficulty in achieving native optimization levels.

[0005] While existing solutions can achieve compatibility to some extent with PTX-level dynamic translation, they typically suffer from problems such as high simulation overhead and difficulty in fully utilizing the hardware characteristics of custom vectors / matrices. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention proposes a parallel thread virtualization and SIMT semantic degradation execution method and system. Targeting custom instruction set parallel computing chips, and addressing the lack of traditional GPU hardware thread bundle schedulers, this invention achieves transparent compatibility with the SIMT ecosystem and maximizes the utilization of underlying vector / matrix hardware computing power on custom instruction set parallel computing chips lacking traditional GPU hardware thread bundle schedulers. This is achieved through "logical thread group" abstraction, vector length-independent modeling, control flow masking for full-path execution, and lock-free circular queue-based host orchestration runtime. This invention enables efficient and transparent degradation mapping of the SIMT parallel thread model to a custom SIMD / matrix instruction set without altering or with minimal modification to the upper-level source code, and provides a supporting low-latency host-device collaborative runtime solution.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, the present invention discloses a method for parallel thread virtualization and SIMT semantic degradation execution, comprising:

[0009] Receives parallel source programs described using the SIMT programming model and identifies kernel data;

[0010] Parallel semantics are extracted from the kernel data and an intermediate representation is constructed. Based on the intermediate representation, an activity mask is constructed using a vector length-independent striping mapping to indicate the set of valid logical threads within the stripe. Host and device program instructions are split according to the kernel data.

[0011] Based on the program instructions, the conditional branches are vectorized into predicates to enable each logical thread to execute independently.

[0012] Based on the intermediate representation, generate device-side code containing vector instructions and matrix acceleration instructions;

[0013] On the host side, the kernel boot call is intercepted through the pad layer, the kernel parameters are serialized into command descriptors, written into a lock-free circular queue in shared memory, and the device is notified to execute through a doorbell mechanism; the device firmware parses the command descriptors, configures execution resources, and jumps to the generated kernel code for execution.

[0014] Secondly, this invention discloses a parallel thread virtualization and SIMT semantic degradation execution system, comprising:

[0015] The data acquisition module is configured to receive parallel source programs described using the SIMT programming model and identify kernel data;

[0016] The instruction mapping module is configured to: extract parallel semantics from the kernel data and construct an intermediate representation; construct an activity mask based on the intermediate representation using a vector length-independent striping mapping to indicate the set of valid logical threads within the stripe; and split host and device-side program instructions according to the kernel data.

[0017] The predicate processing module is configured to: vectorize the conditional branches based on the program instructions to enable each logical thread to execute independently;

[0018] A vector and matrix execution module is configured to generate device-side code containing vector instructions and matrix acceleration instructions based on the intermediate representation;

[0019] The launch execution module is configured to: intercept kernel startup calls on the host side through a pad layer, serialize kernel parameters into command descriptors, write them to a lock-free circular queue in shared memory, and notify the device to execute via a doorbell mechanism; the device firmware parses the command descriptors, configures execution resources, and jumps to the generated kernel code for execution.

[0020] Thirdly, the present invention discloses an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, complete the steps of the above-mentioned parallel thread virtualization and SIMT semantic degradation execution method.

[0021] Fourthly, the present invention discloses a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-mentioned parallel thread virtualization and SIMT semantic degradation execution method.

[0022] Fifthly, the present invention discloses a computer program product, the computer program product comprising executable instructions stored in a computer-readable storage medium;

[0023] When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the above-described parallel thread virtualization and SIMT semantic degradation execution method is completed.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] Low performance overhead: Semantic refactoring during the compilation phase replaces instruction-level simulation, and through memory access merging, vector predicates and matrix unit utilization, the generated code can approach the peak performance of the underlying hardware.

[0026] Good adaptability and portability: Through vector length-independent modeling and runtime VL adaptive striping, the same binary can run on chip models with different vector widths, protecting software investment and supporting hardware upgrades.

[0027] Strong ecosystem compatibility: Through logical thread virtualization and pad layer API interception, existing SIMT ecosystem programs can be transparently compiled and run on custom instruction set chips, significantly reducing migration costs.

[0028] Lower latency task launch: Through a shared memory lock-free circular queue and doorbell notification, low-latency command issuance and high-throughput task orchestration between host and device are achieved, which is suitable for short kernel and frequent launch scenarios.

[0029] Semantic correctness is verifiable: Through mask stacks, software barriers, and atomic visibility constraints, definite equivalent semantics are provided for control flow divergence and synchronization primitives, which facilitates formal verification and engineering implementation.

[0030] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0031] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0032] Figure 1 This is a schematic diagram of the parallel thread virtualization and SIMT semantic degradation execution method described in Embodiment 1 of the present invention. Detailed Implementation

[0033] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0034] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0035] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0036] Terminology Explanation:

[0037] (1) Logical Thread: A thread instance in the SIMT programming model.

[0038] (2) Micro-Thread: A vector lane of the device-side vector execution unit, used to carry a logical thread in a certain vector execution.

[0039] (3) Logical Thread Group (LTG): A collection of W logical threads used to map to one or more vector executions.

[0040] (4) LTG size W: The number of logical threads contained in an LTG, which is a configurable parameter.

[0041] (5) Vector Length (VL): The number of lanes that the hardware vector execution unit executes in parallel at one time, which can be queried by the device or issued by the host.

[0042] (6) Execution Mask M: A bit vector of length VL, where 1 indicates that the logical thread of the corresponding lane is active.

[0043] (7) Striping / Strip-mining: When W>VL, the LTG is split into T stripes, and VL logical threads are executed each time and the effective lane is selected by mask.

[0044] (8) Number of stripes T: T=ceil(W / VL), which represents the number of vector executions required to complete one LTG.

[0045] (9) Thread linear number tid: A one-dimensional logical thread index obtained by mapping threadIdx / blockIdx, etc., used for indexing and address calculation.

[0046] (10) Scalar Evolution: Symbolic derivation of index / loop variables to extract the affine relation from tid to address / iteration space.

[0047] (11) Intermediate Representation (IR): The compiler’s internal representation, used to explicitly express thread hierarchy, control flow and data flow and to carry optimizations.

[0048] (12) Vectorization: elevates the scalar operations of logical threads to vector operations, so that each lane simulates a logical thread.

[0049] (13) Vector-Length Agnostic (VLA): Generates a code framework that is not bound to a fixed VL, and executes adaptively after injecting the VL at runtime.

[0050] (14) Coalescing: When addresses are contiguous or have regular spans, generate vector loading / storage to merge accesses and improve bandwidth utilization.

[0051] (15) Gather / Scatter: Generates index vector loading / storage for non-contiguous address patterns; can degenerate into a masked scalar loop if the hardware does not support it.

[0052] (16) Divergence: Different logical threads within the same LTG take different values ​​for branch conditions, resulting in inconsistent paths.

[0053] (17) Predication: Converts the control flow into full-path sequential execution and masked write-back, using the mask to control side effects.

[0054] (18) Mask Stack: Saves / restores the mask context of nested branches or loops, and is used to restore the pre-entry mask at the merging point.

[0055] (19) Software Barrier: A synchronization mechanism implemented with shared memory and atomic operations, used to map synchronization primitives such as barriers within thread blocks.

[0056] (20) Lock-free Ring + Doorbell: The host writes the command descriptor to a shared memory ring queue, triggers the doorbell to notify the device to parse and start the kernel execution.

[0057] Example 1

[0058] In one or more embodiments, to address the need for parallel semantic extraction, control flow divergence, and correct synchronization primitives when directly mapping SIMT programs to SIMD execution during the compilation phase, a parallel thread virtualization and SIMT semantic degradation execution method is disclosed, such as... Figure 1 The steps shown are as follows, but are not limited to these steps:

[0059] Step S1: Receive the parallel source program described using the SIMT programming model and identify the kernel data.

[0060] Source program reception and kernel data identification: This function receives parallel source programs described using the SIMT programming model and identifies kernel data such as kernel functions, thread index expressions, thread block / grid dimensions, and synchronization primitives. The thread index expressions include threadIdx, blockIdx, blockDim, and their combinations.

[0061] Step S2: Extract parallel semantics from the kernel data and construct an intermediate representation, model logical thread groups; construct an activity mask based on the intermediate representation using a vector length-independent striping mapping to indicate the set of valid logical threads within the stripe, and split host and device program instructions according to the kernel data.

[0062] Extracting parallel semantics and constructing intermediate representations:

[0063] First, scalar evolution and data dependency analysis are performed on the thread index expression at the compiler front-end or middle-end, elevating thread-level parallelism to an analyzable multi-dimensional iterative space;

[0064] Secondly, an intermediate representation (IR) with a three-level structure of grid / thread block / thread is constructed, and the private and shared states of each logical thread are explicitly expressed in the intermediate representation IR. Specifically, the construction of the intermediate representation includes: generating a control flow graph (CFG) and a static single assignment form (SSA) for kernel functions; adding three-level attribute fields of grid / thread block / logical thread to the IR; marking variables according to their storage space as logical thread private variables, thread block shared variables, and global variables; and using the thread index tid as an explicit IR value to participate in subsequent address calculations and vectorization transformations.

[0065] The modeling logic thread group is specifically a virtualized modeling parallel thread group, including:

[0066] The device-side hardware vector channel is abstracted as a micro-thread, and the upper-level logical thread is abstracted as a logical thread. Specifically, each vector lane is defined as a micro-thread with lane_id ∈ [0, VL-1]. The linear number tid of the logical thread is mapped to tid = k•VL + lane_id + ltg_base, where ltg_base is the starting tid of the logical thread group. An execution mask M is used to control whether the lane participates in the operation and write-back, thereby simulating a logical thread on the vector execution unit.

[0067] Furthermore, a Logical Thread Group (LTG) is defined, such that one logical thread group corresponds to one thread bundle in the SIMT, and the size W of the LTG is further defined as a configurable parameter. The thread bundle can be a hardware-level thread bundle such as a Warp.

[0068] The vector length-independent striping mapping specifically involves generating a device-side code framework that is independent of the hardware vector length (VL) during the compilation phase. This VL-independent device-side code framework includes: an outer loop structure with stripe k as the loop variable, and an execution mask M used to generate the k-th stripe. k The mask calculation instruction sequence, in M k A controlled sequence of vector loading / computation / storage instructions, and a mechanism for injecting VL values ​​at runtime and updating M accordingly. k A parameterized expression for the address offset k•VL. The hardware vector length is queried from the device or issued by the host at runtime, and the number of stripes is calculated.

[0069]

[0070] In the formula, is the stripe count; is the ceiling function; is the size of the logical thread group LTG (the number of logical threads included); is the hardware vector length, i.e., the number of lanes.

[0071] For the k-th stripe (0 ≤ k < T), construct an active mask to indicate the set of valid logical threads within the stripe, enabling adaptive execution of the same binary on different VL chip models. Construct the mask M for the k-th stripe k including: calculating g = k • VL + lane_id for each lane_id; setting the corresponding bit to 1 when g < W, otherwise to 0; and further ANDing with the thread block boundary condition mask to obtain the final execution mask M k which is the active mask.

[0072] Splitting the host and device - side program instructions according to kernel data includes region vectorization and host - device splitting, specifically:

[0073] According to the host scheduling strategy, generate host - side multi - thread control code for the grid - level / thread - block - level scheduling and task distribution part; preferably, generate host - side multi - thread control code according to a preset host scheduling strategy (task division and emission at the thread - block granularity); where the host threads fetch thread - block tasks from the pending emission queue and construct command descriptors to write into the circular queue.​​​​​​​​​​​As an implementation, this embodiment presents a process of semantic extraction and vectorization mapping for a typical SIMT core. Taking the vector addition core as an example: each logical thread calculates an element y[i] = a[i] + b[i]. In the SIMT source program, the index i is usually obtained by combining the thread block index blockIdx, the dimension size blockDim of the thread block, and the threadIdx within the thread block. Specifically:

[0078] Step S201, the compiler identifies the thread index expression and performs scalar evolution analysis to obtain the affine relationship of i with respect to the linear number tid of the logical thread: i = base + tid, where base is calculated from the thread block number.

[0079] Step S202, construct the LTG size W (for example, 32), and query the hardware vector length VL at runtime. If VL ≥ W, then one vector execution covers the entire LTG; if VL < W, then calculate T = ceil(W / VL), and generate a mask for each stripe k , such that the corresponding lane is valid only when (k • VL + lane) < W, where k is the stripe number, lane is the vector channel number, and M k is the execution mask for the k-th stripe, VL is the hardware vector length, W is the LTG size, and T is the number of stripe executions.

[0080] Step S203, lift the scalar variables a[i], b[i], y[i] to vector memory accesses: when a and b are continuous in the tid dimension, generate the continuous vector loads vA = VLOAD(a + base + k • VL), vB = VLOAD(b + base + k • VL); and generate the vector addition vY = vA + vB; finally, write back according to the mask through VSTORE(y + base + k • VL, vY, M k ). Here, a, b, and y are the input array and output array respectively, and a[i], b[i], y[i] represent the elements corresponding to the subscripts; VLOAD(addr) represents loading the vector register value from the continuous address starting from addr; VSTORE(addr, v, M) represents writing the vector v back to the corresponding position of addr in the lanes where the mask M is 1 (the lanes with mask 0 do not produce write-back side effects).

[0081] Step S204, when detecting discontinuous addresses (such as indirect indexing a[idx[tid]]), generate the Gather / Scatter form: vIdx = VLOAD(idx + base + k • VL), vA = VGATHER(a, vIdx), and use VSCATTER(y, vIdx, vY, M kIf the hardware does not support VGATHER / VSCATTER, it degenerates into a masked scalar loop, maintaining memory access order constraints consistent with SIMT thread-by-thread (this can be achieved by inserting memory barriers in weakly consistent scenarios). Here, idx is an index array (one index per thread); vIdx is an index vector; VGATHER samples from the base array according to the index vector; VSCATTER writes back according to the index vector; and vY is the vectorized y-result vector.

[0082] Through the above-mentioned SIMT to custom SIMD instruction mapping based on logical thread groups, each vector instruction sequence on the device side simulates VL logical threads, and the host side only needs to distribute tasks at the thread block granularity, thereby achieving SIMT semantic equivalent execution without a hardware Warp scheduler.

[0083] Step S3: Based on the program instructions, the conditional branches are vectorized into predicates to enable each logical thread to execute independently.

[0084] In this embodiment, the masking and linearization of control flow divergence involves vector predication of conditional branches, converting the control flow into full-path sequential execution and dynamic mask writing. Specifically, this includes: maintaining the current execution mask M; and calculating the sub-mask when encountering branch condition C.

[0085]

[0086]

[0087] In the formula, , For true and false path submasks; The current execution mask before entering this branch; The predicate vector obtained after vectorization of the branch conditions; For bitwise AND; This is a bitwise NOT.

[0088] Among them, the conditional branches are identified by the control flow graph (CFG) constructed in step S2; the execution mask M is initially taken as the activity mask M of the corresponding strip. k This is used to indicate the set of lanes that are currently active during vector execution. The execution mask is M constructed during striping in step S2. k When entering a branch, the "current mask M" is initially set to M. k Subsequent nested branches are updated using a mask stack.

[0089] In respectively and The corresponding path is executed under the mask, and the previous mask is restored at the rendezvous point; for nested branches, the mask context is saved / restored through the mask stack, thereby simulating the independent execution of each logical thread in the absence of a hardware branch convergence unit.

[0090] Preferably, this embodiment proposes software semantics for implementing synchronization and communication primitives, including: mapping thread synchronization / fence primitives (such as intra-thread block barriers) in the SIMT program to software barriers based on shared memory and atomic operations; ensuring the visibility and timing correctness of different logical threads before and after the synchronization point through atomic counters, arrival flags, and memory fences. Mapping the intra-thread block barrier to a software barrier: allocating an atomic counter cnt and a generation flag gen in shared memory for each thread block; when a logical thread reaches the barrier, performing an atomic increment of cnt and executing the memory fence; when cnt reaches W (or the sum of the effective thread counts in the stripe), the last arriving thread resets cnt and flips gen; the remaining threads spin and wait for gen to change before continuing execution, thereby ensuring that writes before the barrier are visible to reads after the barrier and maintaining SIMT semantic consistency.

[0091] As one implementation, the masked full path execution of control flow divergence is further described.

[0092] When the SIMT kernel contains conditional branches, the following methods are used:

[0093] When the tid-th logical thread satisfies p[tid] greater than 0, it executes foo(a) to obtain y; otherwise, it executes bar(b) to obtain y. In SIMT hardware, this branch will cause divergence within the warp and is managed by the hardware convergence unit.

[0094] In this invention, the compiler performs linearization and predicate-based processing on branches, with the core steps as follows:

[0095] Step S301: Calculate the branch condition vector vC = (vP>0), and AND it with the current execution mask M to obtain M. true = M∧vC,M false =M∧¬vC. Where vP is a vector composed of p[tid] of the logical threads corresponding to each lane; vC is the predicate vector obtained from the comparison operation; M true With M false These represent the sets of active lanes for the true path and the false path, respectively.

[0096] Step S302: Expand the true and false paths in the original control flow into a single linear code block in sequence: first in the mask M true The vector instruction sequence of the path foo is executed, and then in the mask M falseThe vector instruction sequence for executing the bar path is used. All write-back instructions carry a corresponding mask to prevent unwanted lane writes.

[0097] Step S303: At the branch merging point, restore the execution mask to the execution mask M before entering the branch. For nested branches, use a mask stack to save the entry mask of each level, and pop the stack to restore it at the merging point; optionally, the compiler performs common subexpression elimination and rearrangement on instructions without side effects in the mutual exclusion path to reduce redundancy in full-path execution.

[0098] Step S304: When a loop exists and the loop condition is inconsistent across different lanes, convert the loop into a mask-driven "least common multiple loop" form: maintain the loop activity mask M. loop The circulation body is in M loop The following steps are executed, updating M in each iteration. loop It exits when all values ​​are 0, thus ensuring that different lanes can exit after different number of iterations without affecting the correctness of other lanes.

[0099] The above method avoids generating explicit jumps and complex convergence logic, enabling control flow to be executed on custom vector hardware with deterministic mask semantics, consistent with the per-thread semantics of SIMT.

[0100] Step S4: Generate device-side code containing vector instructions and matrix acceleration instructions based on the intermediate representation.

[0101] Generate custom instruction set code and matrix unit utilization: Generate custom vector instructions based on vectorized IR in the compilation backend, and rewrite them as matrix acceleration unit instructions or built-in function calls when matrix multiplication and addition, block matrix operations and other calculation modes are detected; retain vector instruction generation for parts that cannot be matched to ensure correct functionality.

[0102] Specifically, the intermediate representation is the IR constructed and quantized in step S2. The compilation backend performs operator pattern matching on the IR: when matrix multiplication / addition / block matrix multiplication (e.g., the multiply-addition-accumulate chain of GEMM block decomposition) is identified, the operand block, dimension parameter and step size information are extracted, and the corresponding vector instruction sequence is rewritten as matrix acceleration unit instructions (or equivalent built-in function calls); when alignment, dimension or data type constraints are not met, the vector instruction generation is kept as a fallback path to ensure correct functionality.

[0103] Step S5: On the host side, the kernel startup call is intercepted through the pad layer, the kernel parameters are serialized into command descriptors, written into a lock-free circular queue in shared memory, and the device is notified to execute through a doorbell mechanism; the device firmware parses the command descriptors, configures execution resources, and jumps to the generated kernel code for execution.

[0104] Runtime Shim Layer and Task Launch: On the host side, a shim layer compatible with the existing runtime library binary interface (ABI) is provided to transparently intercept APIs such as memory allocation, data copying, and kernel startup; information such as kernel entry pointer, parameters, grid / thread block configuration, and VL is serialized into command descriptors, written into a lock-free circular command queue in the host-device shared memory, and notified to the device to execute through a doorbell mechanism.

[0105] Minimalist firmware parsing and low-latency startup on the device side: The device firmware polls or responds to doorbell interrupts to read the command queue; parses the command descriptor, configures the vector length (VL) and execution resources; performs zero-copy jump to the device kernel entry point for execution; after execution, it writes back the completion status or event flag for the host to query / callback.

[0106] Preferably, the interception of library-level and routing operators includes: symbolic interception of calls to upper-level mathematical libraries / operator libraries, redirecting standard library calls to basic mathematical libraries (such as GEMM and FFT) that are manually optimized for custom instruction sets; and supporting automatic compilation and routing of open-source CUDA operator source code, so that key operators can be continuously optimized and iterated with the compiler.

[0107] As one implementation method, this embodiment addresses the challenge of achieving low-latency host-to-device task launch in the absence of device-side hardware schedulers and complex operating system support. It proposes a host orchestration runtime based on a shared memory lock-free circular queue, specifically including:

[0108] Step S501: Provide a dynamic link library on the host side that is compatible with the existing runtime ABI. This library intercepts parameters when the application calls the kernel startup API and constructs a command descriptor (CMD). The CMD contains at least: kernel entry address / symbol ID, parameter buffer address and length, grid dimension, thread block dimension, shared memory configuration, expected LTG size W, device-side VL configuration strategy (fixed / adaptive), and event handles.

[0109] Step S502: During host runtime, the CMD is written to a lock-free circular queue in the host-device shared memory area. The circular queue can adopt a producer-consumer model: the host maintains a write pointer head, and the device maintains a read pointer tail; during the write process, the head is updated atomically, and a memory fence is used to ensure that the CMD content is visible to the device before the head.

[0110] Step S503: The host notifies the device via the doorbell mechanism: The doorbell can be implemented by writing a queue head snapshot to the device's MMIO register, triggering a lightweight interrupt, or writing an event file descriptor (fd). To avoid jitter caused by frequent doorbell rings, multiple CMDs can be batch-aggregated before triggering the doorbell.

[0111] Step S504: The device runs a simplified firmware or a runtime loop: when the doorbell is triggered or polling finds that head ≠ tail, read the CMD; after parsing, configure the VL and set the execution context; perform necessary address relocation of the parameters (this can be omitted if a unified virtual address is used), and then jump to the kernel entry point for execution.

[0112] Step S505: After the kernel execution is complete, the device writes back the completion flag and return code, and advances the tail; the host can obtain the completion status through polling, event callback or blocking wait, to achieve the same synchronization semantics as the existing heterogeneous runtime.

[0113] This embodiment uses a shared memory lock-free queue and doorbell notification. While ensuring concurrency safety, this embodiment reduces system calls and lock overhead, is suitable for short kernel and high-frequency transmission scenarios, and can work on the device side without an operating system or lightweight firmware.

[0114] As one implementation, the present invention is applicable to heterogeneous systems comprising a host-side processor and a device-side parallel computing chip, wherein the device-side chip includes at least: a variable-length vector execution unit, a vector predicate / mask operation unit, a matrix multiplication and addition acceleration unit (optional), on-chip shared memory or explicitly managed memory (optional), and a mappable memory region shared with the host (optional).

[0115] Example 2

[0116] In one or more embodiments, a parallel thread virtualization and SIMT semantic degradation execution system is disclosed, specifically including:

[0117] The data acquisition module is configured to receive parallel source programs described using the SIMT programming model and identify kernel data;

[0118] The instruction mapping module is configured to: extract parallel semantics from the kernel data and construct an intermediate representation; construct an activity mask based on the intermediate representation using a vector length-independent striping mapping to indicate the set of valid logical threads within the stripe; and split host and device-side program instructions according to the kernel data.

[0119] The predicate processing module is configured to: vectorize the conditional branches based on the program instructions to enable each logical thread to execute independently;

[0120] A vector and matrix execution module is configured to generate device-side code containing vector instructions and matrix acceleration instructions based on the intermediate representation;

[0121] The launch execution module is configured to: intercept kernel startup calls on the host side through a pad layer, serialize kernel parameters into command descriptors, write them to a lock-free circular queue in shared memory, and notify the device to execute via a doorbell mechanism; the device firmware parses the command descriptors, configures execution resources, and jumps to the generated kernel code for execution.

[0122] Example 3

[0123] This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they complete the steps of the above-described parallel thread virtualization and SIMT semantic degradation execution method.

[0124] Example 4

[0125] This embodiment provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the above-described parallel thread virtualization and SIMT semantic degradation execution method.

[0126] Example 5

[0127] This embodiment provides a computer program product including executable instructions, which are computer instructions; the executable instructions are stored in a computer-readable storage medium. When the processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the electronic device performs the parallel thread virtualization and SIMT semantic degradation execution method described above in this embodiment.

[0128] The steps and methods involved in Embodiments 2 to 5 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0129] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0130] The above description is only a preferred embodiment of the present invention. Although the specific implementation of the present invention has been described in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the present invention.

Claims

1. A method for parallel thread virtualization and SIMT semantic degradation execution, characterized in that, include: Receives parallel source programs described using the SIMT programming model and identifies kernel data; Parallel semantics are extracted from the kernel data and an intermediate representation is constructed. Based on the intermediate representation, an activity mask is constructed using a vector-length-independent striping map to indicate the set of valid logical threads within a stripe. Host and device-side program instructions are split according to the kernel data. Specifically, the construction of the activity mask using a vector-length-independent striping map to indicate the set of valid logical threads within a stripe involves: During the compilation phase, a device-side code framework independent of the hardware vector length is generated; at runtime, the hardware vector length is queried by the device or issued by the host, and the number of striping operations is calculated. In the formula, For the number of striping operations; It is a rounding function; Size of the logical thread group; The length of the hardware vector; For the k-th stripe, construct an activity mask to indicate the set of valid logical threads within that stripe, enabling the same binary code to be adaptively executed on different hardware vector length chip models; Based on the program instructions, the conditional branches are vectorized into predicates to enable each logical thread to execute independently. Based on the intermediate representation, generate device-side code containing vector instructions and matrix acceleration instructions; On the host side, the kernel boot call is intercepted through the pad layer, the kernel parameters are serialized into command descriptors, written into a lock-free circular queue in shared memory, and the device is notified to execute through a doorbell mechanism; the device firmware parses the command descriptors, configures execution resources, and jumps to the generated kernel code for execution.

2. The parallel thread virtualization and SIMT semantic degradation execution method as described in claim 1, characterized in that, The extraction of parallel semantics and construction of intermediate representations includes: Perform scalar evolution and data dependency analysis on thread index expressions at the compiler front-end or mid-end, elevating thread-level parallelism to an analyzable multidimensional iterative space; Construct an intermediate representation containing a three-level structure of grid / thread block / thread, and explicitly define the private and shared states of each logical thread in the intermediate representation.

3. The parallel thread virtualization and SIMT semantic degradation execution method as described in claim 1, characterized in that, The step of splitting host and device-side program instructions based on the kernel data includes: Based on the host orchestration strategy, the grid-level / thread block-level scheduling and task distribution parts are generated into host-side multi-threaded control code; The numerical computation part in the kernel is converted into device-side vector code using full function vectorization or region vectorization, so that each vector channel simulates a logical thread; and the private scalar variables of the logical thread are promoted to vector registers.

4. The parallel thread virtualization and SIMT semantic degradation execution method as described in claim 1, characterized in that, The method of vectorizing the conditional branches based on the program instructions to enable independent execution of each logical thread is as follows: Maintain the current execution mask, and calculate the submask when encountering branch condition C: In the formula, , For true and false path submasks; The current execution mask before entering this branch; This is the predicate vector obtained after vectorization of the conditional branch; For bitwise AND; For bitwise NOT; The corresponding paths are executed under the sub-masks respectively, and the execution mask before the branch is restored at the merging point; for nested branches, the mask context is saved / restored through the mask stack, thereby simulating the independent execution of each logical thread in the absence of a hardware branch convergence unit.

5. The parallel thread virtualization and SIMT semantic degradation execution method as described in claim 1, characterized in that, The host orchestration runtime based on a shared-memory lock-free circular queue specifically includes: Step S501: Provide a dynamic link library compatible with the existing runtime ABI on the host side; construct a command descriptor; Step S502: During host operation, the command descriptor is written to a lock-free circular queue in the host-device shared memory area; the circular queue adopts a producer-consumer model: the host maintains a write pointer, and the device maintains a read pointer; the write process updates the host-maintained write pointer through atomic operations and uses a memory fence to ensure that the content of the command descriptor is visible to the device before the host-maintained write pointer. Step S503: The host notifies the device via the doorbell mechanism; Step S504: The device runs a simplified firmware or a runtime loop: when the doorbell is triggered or polling finds that the host maintenance write pointer and the device maintenance read pointer are different, the command descriptor is read; after parsing, the hardware vector length is configured and the execution context is set; the parameters are relocated to the address, and then the execution jumps to the kernel entry point. Step S505: After the kernel execution is complete, the device writes back the completion flag and return code, and advances the device maintenance read pointer; the host obtains the completion status and achieves synchronization semantics consistent with the existing heterogeneous runtime.

6. A parallel thread virtualization and SIMT semantic degradation execution system, characterized in that, include: The data acquisition module is configured to receive parallel source programs described using the SIMT programming model and identify kernel data; The instruction mapping module is configured to: extract parallel semantics from the kernel data and construct an intermediate representation; construct an activity mask based on the intermediate representation using a vector-length-independent striping map to indicate the set of valid logical threads within a stripe; and split host and device-side program instructions according to the kernel data. Specifically, the construction of the activity mask using a vector-length-independent striping map to indicate the set of valid logical threads within a stripe involves: During the compilation phase, a device-side code framework independent of the hardware vector length is generated; at runtime, the hardware vector length is queried by the device or issued by the host, and the number of striping operations is calculated. In the formula, For the number of striping operations; It is a rounding function; Size of the logical thread group; The length of the hardware vector; For the k-th stripe, construct an activity mask to indicate the set of valid logical threads within that stripe, enabling the same binary code to be adaptively executed on different hardware vector length chip models; The predicate processing module is configured to: vectorize the conditional branches based on the program instructions to enable each logical thread to execute independently; A vector and matrix execution module is configured to generate device-side code containing vector instructions and matrix acceleration instructions based on the intermediate representation; The launch execution module is configured to: intercept kernel startup calls on the host side through a pad layer, serialize kernel parameters into command descriptors, write them to a lock-free circular queue in shared memory, and notify the device to execute via a doorbell mechanism; the device firmware parses the command descriptors, configures execution resources, and jumps to the generated kernel code for execution.

7. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the parallel thread virtualization and SIMT semantic degradation execution method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the parallel thread virtualization and SIMT semantic degradation execution method as described in any one of claims 1-5.

9. A computer program product, characterized in that, The computer program product includes executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, it implements the parallel thread virtualization and SIMT semantic degradation execution method according to any one of claims 1-5.

Images