A hardware semaphore synchronization apparatus and method for heterogeneous computing architectures

Abstract

The application discloses a hardware semaphore synchronization device and method for a heterogeneous computing architecture. The device comprises a plurality of computing cores, an instruction distribution unit, an asynchronous processing unit group, a shared module, a high-speed local memory and an interrupt controller; the instruction distribution unit identifies the type of instructions issued by the computing cores and forwards the instructions to the asynchronous processing unit group and the shared module according to the type; the asynchronous processing unit group executes asynchronous tasks after receiving the instructions; the shared module allocates semaphore storage space in the high-speed local memory after receiving the instructions and executes semaphore synchronization control to generate an interrupt request; and the interrupt controller sends the interrupt request to a target computing core to wake up a blocked thread in the target computing core. The application implements semaphore synchronization logic in a full hardware mode, does not depend on an operating system, eliminates additional overhead caused by software synchronization and reduces synchronization delay.

Description

Technical Field

[0001] This invention belongs to the field of heterogeneous computing chip synchronization control technology, and more specifically, relates to a hardware semaphore synchronization device and method for heterogeneous computing architectures such as Neural Processing Unit (NPU) and General-Purpose Graphics Processing Unit (GPGPU). Background Technology

[0002] In parallel computing acceleration chips, a single computing unit integrates multiple programmable processor cores, Domain Specific Architecture (DSA) acceleration units, and a large-capacity on-chip cache. Different DSA acceleration units are responsible for different tasks such as memory access, matrix multiplication, and vector element-wise operations. Currently, the industry generally adopts a producer-consumer parallel model to schedule hardware resources: binding the DSA acceleration unit with the processor core thread logic, with a dedicated thread responsible for issuing control instructions, and multiple core threads relying on the on-chip cache to complete data interaction, realizing parallel computation of data streams.

[0003] At present, the mainstream synchronization schemes in the field of heterogeneous computing are divided into three categories, all of which have obvious technical defects. (1) Software semaphore mechanism: It relies on the operating system and library functions to implement the acquisition and release of semaphores based on CPU interrupts and thread scheduling. This scheme has a lot of software overhead caused by system calls and context switching, and the synchronization latency is high; at the same time, dedicated computing units such as NPU are mostly bare metal running without operating system support, and software semaphores cannot be directly adapted, making it difficult to meet the low latency synchronization requirements of high-performance heterogeneous computing. (2) Fixed hardware counter mechanism: A fixed number of hardware counters are allocated to each thread and computing unit, and the fence.wait instruction is used to wait for the counter status to change. The number of hardware counters is fixed and scarce, usually only 4 to 8, and the synchronization resources are subject to hard limitations, making it impossible to achieve flexible synchronization across threads and cores. The hardware scalability is poor and the resource overhead is high. (3) Polling / interrupt synchronization mechanism: Synchronization is achieved by combining on-chip cache atomic operations with polling and interrupts. Polling requires the processor to continuously read the synchronization flag, generating a large amount of memory access traffic, causing memory bandwidth pressure and additional power consumption; interrupt-based methods rely on the CPU to participate in synchronization coordination, introducing context switching delay, crowding out normal computing resources, and significantly reducing the overall computing efficiency of the system.

[0004] In summary, existing synchronization solutions suffer from problems such as operating system dependence, high synchronization latency, limited synchronization resources, high CPU overhead, and lack of native hardware asynchronous synchronization capabilities, making them unsuitable for large-scale, low-latency, bare-metal operation heterogeneous computing application scenarios. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a hardware semaphore synchronization device and method for heterogeneous computing architectures. It implements semaphore synchronization logic entirely in hardware, is independent of the operating system, and can run directly in a bare-metal environment. It overcomes the limitations of traditional fixed hardware counters, enabling dynamic allocation of synchronization resources. Leveraging native hardware capabilities, it achieves asynchronous task instruction linkage and automatic thread wake-up, eliminating the overhead of CPU participation in synchronization, reducing synchronization latency, and meeting the cross-core and cross-acceleration unit synchronization requirements of the producer-consumer parallel model in heterogeneous computing architectures. It can be applied to heterogeneous computing chips integrating multi-processor cores, Direct Memory Access (DMA) data transfer units, matrix multiplication accelerators, and other heterogeneous computing scenarios such as AI inference and parallel numerical computation.

[0006] To achieve the above objectives, according to one aspect of the present invention, a hardware semaphore synchronization device for heterogeneous computing architecture is provided, comprising multiple computing cores, an instruction dispatch unit, an asynchronous processing unit group, a shared module, a high-speed local memory, and an interrupt controller. Each computing core has an embedded instruction control unit and instruction unit enhancement module; the instruction control unit is used to schedule and issue various instructions; the instruction unit enhancement module is used to parse semaphore-specific instructions and manage the running status of the core's threads; The instruction distribution unit is used to identify the instruction type and forward the instructions to the asynchronous processing unit group and the shared module according to the type; The asynchronous processing unit group is used to execute asynchronous tasks after receiving instructions; The shared module includes a semaphore storage management module and a semaphore processing module; the semaphore storage management module is used to allocate semaphore storage space in high-speed local memory after receiving an instruction; the semaphore processing module is used to perform semaphore synchronization control and generate interrupt requests after receiving an instruction. The interrupt controller is used to send interrupt requests to the target computing core and wake up blocked threads within the target computing core.

[0007] In some implementations, the multiple computing cores include producer cores and multiple consumer cores; the instruction unit enhancement module of the consumer core is used to block the current thread after receiving a wait instruction from the instruction control unit of its core, and to manage the running state of the thread of this core according to the semaphore state.

[0008] In some implementations, the instruction distribution unit is used to receive instructions issued by the producer core, forward asynchronous task instructions to the asynchronous processing unit group, and forward initialization instructions to the shared module; the shared module is used to allocate semaphore storage space in the high-speed local memory using the semaphore storage management module after receiving the initialization instructions, and to perform semaphore initialization using the semaphore processing module.

[0009] In some implementations, the instruction distribution unit is also used to receive instructions issued by the producer core and forward the submission instructions therein to the sharing module; the sharing module is used to perform semaphore synchronization control and generate an interrupt request after receiving the submission instructions using the semaphore processing module.

[0010] In some implementations, the shared module further includes an asynchronous execution unit tracking module, and the submission instruction has modes 0 and 1; the semaphore processing module is used to immediately execute semaphore synchronization control and generate an interrupt request after receiving the submission instruction of mode 0; the asynchronous execution unit tracking module is used to establish and maintain the binding relationship between the asynchronous task instruction and the submission instruction after receiving the submission instruction of mode 1, and trigger the semaphore processing module to execute semaphore synchronization control and generate an interrupt request after the asynchronous task instruction is executed.

[0011] In some implementations, semaphore initialization includes: Create a semaphore that includes the following fields: Post Count, Expected Value, Wait Flag, and Wait Core ID. Post Count records the cumulative number of times the commit command has been executed, Expected Value is the synchronization wake-up threshold, Wait Flag indicates whether there is a waiting thread for the current semaphore, and Wait Core ID records the computation core number where the waiting thread is located. Set Post Count, Wait Flag, and Wait Core ID to 0, and configure Expected Value to the specified threshold.

[0012] In some implementations, performing semaphore synchronization control and generating an interrupt request includes: Perform an atomic increment on PostCount; Verify that the synchronous wake-up condition Post Count == Expected Value and Wait Flag = 1; When the synchronous wake-up condition is met, a directed interrupt request carrying the Wait Core ID is generated.

[0013] In some implementations, the interrupt controller is used to resolve the Wait Core ID and route the interrupt request to the target consumer core based on the Wait Core ID; the instruction enhancement module of the target consumer core is used to clear the wait state after receiving the interrupt request, so that the thread can resume execution from the blocked point.

[0014] According to another aspect of the present invention, a hardware semaphore synchronization method for heterogeneous computing architectures is provided, comprising: Receive the initialization command issued by the producer core, allocate semaphore storage space in high-speed local memory, and perform semaphore initialization; Receive asynchronous task instructions from the producer core and execute asynchronous tasks; Receive the commit command issued by the producer core; Upon receiving a commit instruction in mode 0, immediately execute semaphore synchronization control and generate an interrupt request; upon receiving a commit instruction in mode 1, establish and maintain the binding relationship between the asynchronous task instruction and the commit instruction, and execute semaphore synchronization control and generate an interrupt request after the asynchronous task instruction has been executed. An interrupt request is sent to the target consumer core, waking up the blocked threads within the target consumer core.

[0015] In some implementations, semaphore initialization includes: Create a semaphore that includes the following fields: Post Count, Expected Value, Wait Flag, and Wait Core ID. Post Count records the cumulative number of times the commit command has been executed, Expected Value is the synchronization wake-up threshold, Wait Flag indicates whether there is a waiting thread for the current semaphore, and Wait Core ID records the computation core number where the waiting thread is located. Set Post Count, Wait Flag, and Wait Core ID to 0, and configure Expected Value to the specified threshold; Performing semaphore synchronization control and generating interrupt requests includes: Perform an atomic increment on PostCount; Verify that the synchronous wake-up condition Post Count == Expected Value and Wait Flag = 1; When the synchronous wake-up condition is met, a directed interrupt request carrying the Wait Core ID is generated.

[0016] In summary, the technical solutions conceived by this invention have the following beneficial effects compared with the prior art: (1) Independent of operating system and adapted to bare-metal operation scenarios. The entire synchronization logic is implemented in hardware, without the need for an operating system, system calls and thread scheduling. It can run directly in heterogeneous computing units without an operating system, such as NPU and DSA, and has a wider range of applications; (2) Synchronization resources can be flexibly expanded. Semaphores are stored in high-speed local memory (Tightly Coupled Memory, TCM), and the number of deployments is only limited by the memory capacity. It can support thousands of synchronization resources, which is hundreds of times more flexible than the traditional fixed hardware counters of only 4 to 8. The 64-bit lightweight data structure greatly reduces the hardware area overhead and the hardware cost is reduced by more than 90%. (3) Extremely low synchronization latency and significantly improved computing performance. Hardware atomic operations take less than 15 clock cycles, and the overall latency of thread hardware wake-up is low. Compared with traditional software semaphores, the synchronization performance is improved by 10 to 100 times, which fully meets the low latency requirements of high-performance computing; (4) Eliminate CPU overhead and improve system throughput. Support hardware linkage of asynchronous task instructions. After the asynchronous task is completed, the hardware automatically completes semaphore synchronization and thread wake-up, eliminating the need for CPU to perform polling, interrupt handling and other operations, effectively releasing computing resources and improving the overall computing power of the system; (5) Strong cross-core synchronization capability. Relying on Wait Core ID to achieve precise cross-core thread wake-up, it can support complex multi-threaded, multi-core producer-consumer models and adapt to large-scale parallel computing scenarios; (6) Flexible functions and good compatibility. It supports dual working modes of immediate execution and asynchronous binding to adapt to different synchronization scenarios; the instructions support batch operations, and the synchronization efficiency is improved by 30%~50% in batch scenarios; it can work in conjunction with the existing fence counting mechanism to take into account both fine-grained instruction synchronization and coarse-grained task synchronization, and build a complete multi-level synchronization system; (7) Low power consumption design. The thread enters a sleep state during the waiting phase, and the hardware only generates an interrupt when there is a waiting thread, reducing invalid memory access and hardware actions, effectively reducing system power consumption and memory bandwidth usage. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the hardware semaphore synchronization device for heterogeneous computing architecture according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall process of a hardware semaphore synchronization method for heterogeneous computing architecture according to an embodiment of the present invention. Figure 3 These are schematic flowcharts of the hardware semaphore synchronization method for heterogeneous computing architectures according to embodiments of the present invention, wherein (a) is the synchronization process from the producer's perspective; and (b) is the synchronization process from the consumer's perspective. Figure 4 This is a flowchart illustrating the DMA data transmission and semaphore synchronization method under asynchronous binding mode according to an embodiment of the present invention. Figure 5 This is a flowchart illustrating the matrix calculation (MMA) and semaphore synchronization method in the asynchronous binding mode of this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered exemplary in nature and not restrictive.

[0019] This invention relies on a 64-bit hardware semaphore data structure, three dedicated hardware atomic instructions, and four major hardware functional modules to achieve synchronous control. The entire logic runs within a heterogeneous computing chip, and semaphores are uniformly stored in high-speed local memory (TCM). The overall synchronous execution process is divided into four stages: semaphore initialization, producer task execution, asynchronous task and semaphore submission, and consumer thread wake-up and execution.

[0020] (1) Hardware semaphore data structure design.

[0021] The embodiments of the present invention adopt a 64-bit compact semaphore structure with a total bit width of 8 bytes. The 64-bit hardware semaphores are uniformly stored in the chip TCM. The hardware forces all semaphore addresses to be aligned to 128-byte boundaries, avoiding the performance loss caused by unaligned access and optimizing the efficiency of hardware access and address decoding.

[0022] The definitions and functions of each field are as follows: Post Count (63:48, 16-bit unsigned integer): Records the cumulative number of times the Sem.commit operation is executed, with a value range of 0 to 65535 and a maximum count value of 65535, which can meet the synchronization requirements of tens of thousands of asynchronous tasks. Each time Sem.commit is executed, an atomic increment operation is performed. After the count value reaches the upper limit, the hardware automatically stops incrementing to prevent numerical overflow and wrap-around errors. The initial value is uniformly set to 0 by the Sem.init instruction.

[0023] Expected Value (47:32, 16-bit unsigned integer): Synchronization wake-up threshold, ranging from 0 to 65535, which can be flexibly configured according to the number of parallel subtasks; for example, when the computation task is split into 4 subtasks, the threshold can be configured to 4 to implement the synchronization logic of "waiting for all subtasks to complete"; when the value of Post Count is equal to the threshold, the hardware automatically triggers the thread wake-up action, which can be reconfigured at runtime through the Sem.init instruction.

[0024] Wait Flag (31, 1-bit flag): Used to mark whether there is a waiting thread in the current semaphore. 1 indicates that there is a waiting thread, and 0 indicates that there is no waiting thread. The hardware will only generate a wake-up interrupt when this bit is 1, reducing unnecessary hardware operations. This bit is set to 1 when a thread enters the waiting state, and is automatically cleared to 0 after the thread is woken up.

[0025] Wait Core ID (30:15, 16-bit unsigned integer): Records the computing core number where the waiting thread is located. The lower 6 bits of the field are valid, and a maximum of 64 computing cores can be synchronized. The higher bits are reserved so that the hardware structure does not need to be modified when expanding to a multi-core architecture in the future, and it can be adapted to hardware architecture expansion with more cores in the future.

[0026] Reserved (14:0, 15-bit reserved fields): Used to ensure byte alignment of 64-bit data structures. It is forcibly initialized to 0 in all scenarios. Additional functions such as timeout waiting, synchronization priority, and multi-core broadcast wake-up can be added later.

[0027] (2) Dedicated hardware atomic instructions.

[0028] This invention provides three corresponding hardware atomic instructions, all of which are guaranteed to be atomic by hardware, forming a complete semaphore operation logic: Sem.init initialization directive: Command syntax: Sem.init [addr], <Expected Value>, num.

[0029] Its main function is to initialize multiple consecutive semaphores in batches, completing state reset and synchronization threshold configuration. During execution, the input address `addr` is first aligned to 128 bytes, then the atomic initialization of `num+1` consecutive semaphores is completed in a loop. The Post Count, Wait Flag, and Wait Core ID are uniformly set to 0, and the Expected Value is configured to the specified threshold. The entire initialization process is hardware-locked to prevent concurrent modification and ensure data consistency (i.e., the initialization process cannot be interrupted by other instructions). It returns immediately after execution, without blocking subsequent instructions or affecting program execution, and supports repeated calls during runtime to dynamically modify semaphore parameters.

[0030] Application example: Sem.init sem_base, 4, 3 means that 4 consecutive semaphores are initialized starting from sem_base, and the synchronization threshold of all semaphores is set to 4.

[0031] Sem.commit commit instructions: Command syntax: Sem.commit [addr], num, [mode].

[0032] Its main functions are to atomically increment a semaphore counter, determine synchronization conditions, and trigger thread wake-ups. It operates in two modes: Mode 0 (immediate execution mode, software-initiated synchronization) and Mode 1 (asynchronous binding mode, hardware event synchronization). The count increment operation of this instruction is atomic, ensuring no data loss during multi-core concurrent operations; and it only triggers a wake-up when the Post Count first reaches the threshold, without repeating the response in subsequent operations.

[0033] Mode 0 is the immediate execution mode, not bound to any asynchronous task instructions. Upon receiving an instruction, it immediately increments the count, suitable for software-triggered synchronization scenarios. After address alignment, the hardware sequentially performs an atomic increment operation (count increment) on the PostCount of n+1 semaphores. After the batch count update is complete, the synchronization condition is checked for each semaphore: whether the PostCount equals the Expected Value (condition check); if so, it further checks whether the Wait Flag is 1. If so, an interrupt request carrying the Wait Core ID is generated, which is routed by the interrupt controller to the target core to wake up the thread and simultaneously clear the Wait Flag (thread wake-up). The entire instruction uses a non-blocking execution mode, and the producer thread does not need to wait and can directly continue executing subsequent code. Submit instructions for multiple semaphores are executed in index order.

[0034] Application Example 1: Sem.commit sem_base, 0, 0, immediately increments the count of the sem_base[0] semaphore; Application Example 2: Sem.commit sem_base, 3, 0, immediately increments the count of the sem_base[0~3] semaphores in batches.

[0035] Mode 1 is an asynchronous binding mode. The submission command is bound to asynchronous task commands such as pre-processing DMA and matrix operations. It follows the asynchronous task commands into the instruction queue of the corresponding asynchronous execution unit. The asynchronous execution unit's tracking module records the instruction dependencies, binding relationships, and execution order. After all asynchronous tasks (including data transfer and matrix operations) are completed, the asynchronous execution unit triggers the semaphore processing module. The semaphore processing module sequentially performs operations such as counter incrementing, conditional judgment, and thread wake-up. The execution logic is consistent with the immediate execution mode, and the entire process requires no software intervention. The submission command operation strictly follows the issuance order of the asynchronous task commands to ensure semantic correctness.

[0036] Application Example 1: Synchronizing Asynchronous Data Transmission DMA.LD dram_addr, local_addr, size initiates asynchronous data loading. `Sem.commit sem_base, 0, 1` binds to DMA operations and automatically increments upon completion. Application Example 2: Synchronizing Asynchronous Matrix Operations COMPUTE.MMA matrix_a, matrix_b, result Initiates asynchronous matrix multiplication. `Sem.commit sem_base, 0, 1` binds to a computation task and automatically increments upon completion. The Sem.commit commit command has the following characteristics: (1) Atomicity: The increment operation of Post Count is atomic, and the update will not be lost even if multiple cores commit at the same time; (2) One-time wake-up: It is only woken up when Post Count first reaches Expected Value, and subsequent commits will not trigger wake-up; (3) Interrupt optimization: An interrupt is only sent when Wait Flag is 1 to avoid invalid interrupt overhead; (4) Flexible dual mode: It can choose immediate execution or asynchronous binding according to the mode parameter to adapt to different scenarios.

[0037] Sem.wait waits for a command: Command syntax: Sem.wait [addr].

[0038] The main function of the Wait instruction is to block the current thread, waiting for the semaphore to meet the synchronization wake-up conditions. The execution flow is as follows: the hardware first completes address alignment, reads the complete semaphore field and compares the Post Count with the Expected Value; when the count value reaches the threshold, the Post Count and Wait Flag are automatically cleared, and the thread continues to execute subsequent instructions; when the count value has not reached the threshold, the Wait Flag is set to 1, the current core number is written to the Wait Core ID, the instruction issuance path is closed and the thread is suspended, putting it into a low-power sleep state, waiting for a hardware interrupt to wake it up.

[0039] The wait instruction has usage constraints: it must be used in conjunction with the corresponding Sem.init initialization instruction and Sem.commit commit instruction; by default, a single semaphore only supports one core to execute Sem.wait; if multiple cores need to wait simultaneously, the reserved field must be used to extend the broadcast mechanism; during code writing, it is necessary to ensure that there are a sufficient number of Sem.commit operations to avoid thread deadlock.

[0040] Typical application example: Sem.wait sem_base, after execution, the current thread enters a blocked state, waiting for the sem_base[0] semaphore to meet the synchronization condition.

[0041] (3) Four major hardware functional modules.

[0042] Semaphore Storage Management Module: Responsible for the dynamic allocation and management of TCM storage space. The number of semaphores deployed is limited only by the TCM storage capacity, completely breaking through the bottleneck of the number of traditional fixed hardware counters.

[0043] Instruction Unit Enhancement Module: Extends the chip's original instruction decoder to identify and parse three dedicated semaphore instructions. The producer core primarily executes the `Sem.init` and `Sem.commit` instructions, while the consumer core additionally executes the `Sem.wait` instruction, managing local thread blocking and waking based on the semaphore state.

[0044] Asynchronous Execution Unit Tracking Module: Tracks the execution progress of various asynchronous task instructions such as data transmission and matrix calculation in real time, maintains instruction dependencies, and ensures that the Sem.commi instruction is automatically triggered after the asynchronous task is completed.

[0045] Semaphore processing module: responsible for performing semaphore initialization, atomic read and write, counter value increment and decrement, numerical comparison, synchronization condition judgment, and generating interrupt signals.

[0046] The four major hardware functional modules work together with the instruction control unit, instruction dispatch unit, asynchronous processing unit, interrupt controller, and TCM.

[0047] like Figure 1 As shown, the hardware semaphore synchronization device for heterogeneous computing architectures according to an embodiment of the present invention includes: multiple computing cores, an instruction dispatch unit, an asynchronous processing unit group, a shared module, a high-speed local memory, and an interrupt controller. Each hardware unit works collaboratively through instruction streams, data streams, and interrupt signals to achieve hardware semaphore synchronization.

[0048] The multiple computing cores are designated as Computing Core 0 (Producer Core) and Computing Core 1 (Consumer Core). Internally, the Producer Core and Consumer Core each integrate an instruction control unit and an instruction unit enhancement module. The hardware structures of the Producer Core and Consumer Core are completely identical; both are fundamental computing cores within a heterogeneous computing architecture.

[0049] The instruction control unit is an inherent fundamental unit of the computing core, responsible for instruction scheduling, instruction issuance, and instruction flow control. It is the initiator of all semaphore-specific instructions and asynchronous task instructions. The instruction unit enhancement module, one of the four major hardware functional modules, is an instruction decoding extension unit used to parse semaphore-specific instructions such as `Sem.init`, `Sem.commit`, and `Sem.wait`. It also manages the suspended, waiting, and awakened states of threads within this core. The producer core sends instructions downstream through the instruction output port, while the consumer core receives externally transmitted interrupt instructions through the interrupt signal port, thus waking up threads.

[0050] The instruction distribution unit is a chip common routing unit that receives instructions from all computing cores and forwards them in a distributed manner: asynchronous task instructions are forwarded to downstream asynchronous processing unit groups, and semaphore-specific instructions are forwarded to shared modules, thereby realizing instruction classification and scheduling.

[0051] The asynchronous processing unit group comprises multiple asynchronous processing units, primarily DMA asynchronous processing units, MMA asynchronous processing units, and other asynchronous acceleration units, used to execute background asynchronous tasks such as data transfer and matrix calculations. After completing its task, the asynchronous processing unit group sends a task completion signal to the shared module, triggering subsequent semaphore synchronization logic.

[0052] The shared module comprises a semaphore storage management module, an asynchronous execution unit tracking module, and a semaphore processing module. These are the other three major hardware functional modules, belonging to chip-level globally shared hardware, deployed independently of the computing core, and shared by all hardware units across the chip. The semaphore storage management module is responsible for the dynamic allocation and resource management of the high-speed local memory (TCM), ensuring the normal storage and access of semaphore data. The asynchronous execution unit tracking module tracks the task execution status of the asynchronous processing unit group in real time, maintains the binding relationship and execution dependency relationship between asynchronous task instructions and the Sem.commit instruction, and triggers the semaphore processing module to work upon completion of the asynchronous task. The semaphore processing module is responsible for performing semaphore initialization, atomic read / write, counter increment / decrement, numerical comparison, synchronization condition judgment, and generating interrupt request signals.

[0053] The high-speed local memory (TCM) is specifically used to store the 64-bit hardware semaphore data of this invention. It features low latency and high bandwidth, and is accessed by a globally shared module cluster. The number of semaphores stored is limited only by the capacity of the TCM.

[0054] The interrupt controller is a global interrupt management unit that receives interrupt requests generated by the semaphore processing module, performs directional routing based on the Wait Core ID field within the semaphore, and accurately sends the interrupt request signal to the target consumer core to complete thread wake-up.

[0055] It should be understood that the hardware architecture of each computing core is consistent. The producer core and consumer core are logical roles formed by software task allocation, not inherent hardware attributes. The same computing core can switch roles in different business scenarios. It can act as a producer core to issue asynchronous task instructions and semaphore commit instructions, and it can also act as a consumer core to execute semaphore wait instructions and be woken up by the interrupt controller.

[0056] (4) Overall synchronous execution process.

[0057] The overall synchronous execution process is divided into four stages, adapting to the producer-consumer parallel model under heterogeneous computing architectures. For example... Figure 2 As shown, the hardware semaphore synchronization method for heterogeneous computing architectures according to an embodiment of the present invention includes the following steps: Step S201: Semaphore initialization.

[0058] The computing core sends the Sem.init instruction to the shared module through the instruction control unit. The semaphore storage management module allocates storage space in the high-speed local memory (TCM). The semaphore processing module completes the hardware semaphore creation, synchronization wake-up threshold configuration and reset of all status bits, and the semaphore enters the ready state.

[0059] Step S203: Producer task execution and instruction issuance.

[0060] The instruction control unit within the producer core schedules and issues instructions, which are then parsed by the instruction unit enhancement module and output to the instruction distribution unit. The instruction distribution unit classifies and distributes the instructions: asynchronous task instructions such as DMA and matrix multiplication (MMA) are routed to the corresponding asynchronous processing units to execute background tasks such as data transmission and matrix operations; at the same time, the Sem.commit instruction is distributed to the shared module.

[0061] The Sem.wait instruction is issued independently by the consumer core to implement thread blocking and waiting; the Sem.init instruction is executed only once during the initialization phase and is not issued again during the task execution phase.

[0062] The Sem.commit command can be configured to mode 0 and mode 1. Mode 0 is the independent execution mode, which executes the command immediately after it is received. Mode 1 is the binding mode, in which the asynchronous execution unit tracking module establishes the binding relationship between the Sem.commit command and the asynchronous task command.

[0063] Step S205: Asynchronous task and semaphore commit.

[0064] The asynchronous processing unit receives and executes asynchronous tasks in the background. When configured in mode 1, the asynchronous task instruction is bound to the corresponding Sem.commit instruction. The asynchronous execution unit tracking module continuously monitors the task status and records the execution order and dependency relationship (binding relationship) between the two, providing a basis for automatically triggering semaphore operations after the asynchronous task is completed.

[0065] The asynchronous processing unit sends out a completion signal and triggers the bound Sem.commit instruction after the task is completed; the semaphore processing module performs atomic increment on Post Count and checks the synchronous wake-up condition (Post Count == ExpectedValue and Wait Flag=1). If the condition is met, it generates a directed interrupt request (InterruptRequest, IRQ) carrying the Wait Core ID.

[0066] Step S207: Consumer thread wake-up and execution.

[0067] The interrupt controller parses the Wait Core ID and routes the interrupt request to the consumer core based on the Wait Core ID. The consumer core receives the interrupt request, and the internal instruction unit enhancement module releases the thread from the waiting state, resets the flag bit, and the thread resumes execution from the blocked position, completing a full synchronization process.

[0068] Furthermore, such as Figure 3 As shown, from the producer's perspective, the semaphore synchronization process includes: waiting for the semaphore initialization to complete, the producer issuing an asynchronous task, executing the asynchronous task, committing the semaphore, and continuing with subsequent tasks; from the consumer's perspective, the semaphore synchronization process includes: waiting for the semaphore initialization to complete, issuing a semaphore waiting instruction, the semaphore waiting succeeding, and continuing with subsequent tasks.

[0069] like Figure 4 As shown, the DMA data transmission and semaphore synchronization method in the asynchronous binding mode of this invention includes: Step S301: The producer core issues an asynchronous data transfer instruction for DMA.LD; Step S303: The instruction dispatch unit routes to the DMA asynchronous processing unit; Step S305: The producer core issues the Sem.commit instruction in mode 1, which is bound to the DMA instruction; Step S307: The asynchronous execution unit tracing module records instruction dependencies and binding relationships; Step S309: The DMA asynchronous processing unit executes the data transfer task in the background; Step S311: After the task is completed, a completion signal is sent out, triggering the execution of Sem.commit; Step S313: The semaphore processing module atomically increments PostCount and verifies the synchronization condition; Step S315: If the condition is met, generate a directed interrupt request; Step S317: The interrupt controller routes the interrupt to the consumer core based on the Wait Core ID; Step S319: The instruction unit enhancement module clears the wait state, and the thread resumes execution from Sem.wait.

[0070] like Figure 5 As shown, the matrix computation (MMA) and semaphore synchronization method in the asynchronous binding mode of this invention includes: Step S401: The producer core issues the COMPUTE.MMA matrix multiplication asynchronous task instruction; Step S403: The instruction dispatch unit routes to the MMA asynchronous processing unit; Step S405: The producer core issues the Sem.commit command of mode 1, which is bound to the matrix calculation command; Step S407: The asynchronous execution unit tracing module records instruction dependencies and binding relationships; Step S409: The MMA asynchronous processing unit executes the matrix multiplication calculation task in the background; Step S411: After the task is completed, a completion signal is sent out, triggering the execution of Sem.commit; Step S413: The semaphore processing module atomically increments PostCount and verifies the synchronization condition; Step S415: Condition met, generate a directed interrupt request; Step S417: The interrupt controller routes the interrupt to the consumer core based on the Wait Core ID; Step S419: The instruction unit enhancement module clears the wait state, and the thread resumes execution from Sem.wait.

[0071] The thread wake-up mechanism of this invention has the following characteristics: (1) Interrupt generation: The semaphore processing module generates an IRQ interrupt only when Post Count == Expected Value and Wait Flag = 1 (i.e., the synchronization wake-up condition is met), and attaches the target core number (Wait Core ID) to prevent invalid interrupts; (2) Interrupt routing: The interrupt adopts a dedicated direct path, bypassing the conventional interrupt handling process and shortening the routing delay; (3) Core response: After the target core receives the interrupt, it only clears the local wait flag, without complex context switching, and the thread quickly resumes execution; (4) Optimization strategy: When multiple semaphores meet the synchronization condition at the same time, the hardware supports interrupt aggregation and merges them into a single interrupt signal to further reduce overhead; the interrupt supports priority configuration to ensure that high-priority synchronization tasks respond first. The end-to-end delay from interrupt generation to thread resumption of execution in this invention embodiment is less than 50 clock cycles.

[0072] For multi-parallel task scenarios, the batch capabilities of Sem.init and Sem.commit are used to operate multiple consecutive semaphores with a single instruction. For example, executing Sem.commit sem_base, 3, 0 can update four semaphores sem_base[0]~sem_base[3] at once, reducing the overhead of instruction issuance and address decoding, and improving the synchronization efficiency by 30%~50% in batch scenarios.

[0073] In heterogeneous computing pipelines, a hierarchical synchronization strategy is adopted: fine-grained instruction-level synchronization and single-unit intra-synchronization use the traditional Fence Count mechanism; coarse-grained cross-core, cross-DSA unit, and producer-consumer task synchronization use this method. The two work together to build a multi-level synchronization system that balances synchronization accuracy, operational performance, and resource utilization.

[0074] In a pure thread synchronous scenario without hardware asynchronous tasks, the producer continuously calls the Sem.commit instruction in mode 0 to actively increment the semaphore counter, and the consumer blocks and waits using the Sem.wait instruction. The instruction model is unified and requires no additional programming adaptation.

[0075] It is important to note that semaphores must be initialized using the `Sem.init` command before use. Uninitialized semaphores have random states, which can easily lead to synchronization exceptions. The Expected Value threshold should be configured appropriately to ensure that the number of `Sem.commit` executions reaches the threshold, preventing permanent thread deadlocks. While the hardware has automatic address alignment capabilities, it is recommended to follow the 128-byte alignment rule during programming to further improve access efficiency. The Post Count will stop incrementing after reaching 65535; for very large-scale tasks, they can be split into groups and managed collaboratively using multiple semaphores.

[0076] This invention also supports pure software synchronization scenarios, where producers can actively trigger synchronization using the Sem.commit command in mode 0; all three core commands support batch operations, allowing a single command to process multiple consecutive semaphores, thus improving synchronization efficiency; it can also be used in conjunction with the existing Fence Count mechanism to build a multi-level synchronization system.

[0077] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0078] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0079] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more (two or more) executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0080] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0081] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0082] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A hardware semaphore synchronization device for heterogeneous computing architectures, characterized in that, It includes multiple computing cores, instruction dispatch units, asynchronous processing unit groups, shared modules, high-speed local memory, and interrupt controllers; Each computing core embeds an instruction control unit and an instruction unit enhancement module; the instruction control unit is used to schedule and issue various instructions; the instruction unit enhancement module is used to parse semaphore-specific instructions and manage the running status of the core thread; The instruction distribution unit is used to identify the instruction type and forward the instructions to the asynchronous processing unit group and the shared module according to the type; The asynchronous processing unit group is used to execute asynchronous tasks after receiving instructions; The shared module includes a semaphore storage management module and a semaphore processing module; the semaphore storage management module is used to allocate semaphore storage space in the high-speed local memory after receiving an instruction; the semaphore processing module is used to perform semaphore synchronization control and generate an interrupt request after receiving an instruction. The interrupt controller is used to send interrupt requests to the target computing core and wake up the blocked threads within the target computing core.

2. The hardware semaphore synchronization device as described in claim 1, characterized in that, The multiple computing cores include producer cores and multiple consumer cores; the instruction unit enhancement module of the consumer core is used to block the current thread after receiving a wait instruction issued by the instruction control unit of its core, and to manage the running state of the thread of this core according to the semaphore state.

3. The hardware semaphore synchronization device as described in claim 2, characterized in that, The instruction distribution unit is used to receive instructions issued by the producer core, forward asynchronous task instructions to the asynchronous processing unit group, and forward initialization instructions to the shared module; the shared module is used to allocate semaphore storage space in the high-speed local memory using the semaphore storage management module after receiving the initialization instructions, and to perform semaphore initialization using the semaphore processing module.

4. The hardware semaphore synchronization device as described in claim 3, characterized in that, The instruction distribution unit is also used to receive instructions issued by the producer core and forward the submission instructions therein to the shared module; the shared module is used to perform semaphore synchronization control and generate an interrupt request using the semaphore processing module after receiving the submission instructions.

5. The hardware semaphore synchronization device as described in claim 4, characterized in that, The shared module also includes an asynchronous execution unit tracking module, and the submission instruction has modes 0 and 1; the semaphore processing module is used to immediately execute semaphore synchronization control and generate an interrupt request after receiving a submission instruction of mode 0; the asynchronous execution unit tracking module is used to establish and maintain the binding relationship between the asynchronous task instruction and the submission instruction after receiving a submission instruction of mode 1, and trigger the semaphore processing module to execute semaphore synchronization control and generate an interrupt request after the asynchronous task instruction is executed.

6. The hardware semaphore synchronization device as described in claim 5, characterized in that, Semaphore initialization includes: Create a semaphore that includes the following fields: Post Count, Expected Value, Wait Flag, and WaitCore ID. Post Count records the cumulative number of times the commit command has been executed, Expected Value is the synchronization wake-up threshold, Wait Flag indicates whether there is a waiting thread for the current semaphore, and Wait Core ID records the computation core number where the waiting thread is located. Set Post Count, Wait Flag, and Wait Core ID to 0, and configure Expected Value to the specified threshold.

7. The hardware semaphore synchronization device as described in claim 6, characterized in that, Performing semaphore synchronization control and generating interrupt requests includes: Perform an atomic increment on PostCount; Verify that the synchronous wake-up condition Post Count == Expected Value and Wait Flag = 1; When the synchronous wake-up condition is met, a directed interrupt request carrying the Wait Core ID is generated.

8. The hardware semaphore synchronization device as described in claim 7, characterized in that, The interrupt controller is used to parse the Wait Core ID and route the interrupt request to the target consumer core based on the Wait Core ID; the instruction enhancement module of the target consumer core is used to clear the waiting state after receiving the interrupt request, so that the thread can resume execution from the blocked point.

9. A hardware semaphore synchronization method for heterogeneous computing architectures, characterized in that, include: Receive the initialization command issued by the producer core, allocate semaphore storage space in high-speed local memory, and perform semaphore initialization; Receive asynchronous task instructions from the producer core and execute asynchronous tasks; Receive the commit command issued by the producer core; Upon receiving a commit instruction in mode 0, immediately execute semaphore synchronization control and generate an interrupt request; upon receiving a commit instruction in mode 1, establish and maintain the binding relationship between the asynchronous task instruction and the commit instruction, and execute semaphore synchronization control and generate an interrupt request after the asynchronous task instruction has been executed. An interrupt request is sent to the target consumer core, waking up the blocked threads within the target consumer core.

10. The hardware semaphore synchronization method as described in claim 9, characterized in that, Semaphore initialization includes: Create a semaphore that includes the following fields: Post Count, Expected Value, Wait Flag, and WaitCore ID. Post Count records the cumulative number of times the commit command has been executed, Expected Value is the synchronization wake-up threshold, Wait Flag indicates whether there is a waiting thread for the current semaphore, and Wait Core ID records the computation core number where the waiting thread is located. Set Post Count, Wait Flag, and Wait Core ID to 0, and configure Expected Value to the specified threshold; Performing semaphore synchronization control and generating interrupt requests includes: Perform an atomic increment on PostCount; Verify that the synchronous wake-up condition Post Count == Expected Value and Wait Flag = 1; When the synchronous wake-up condition is met, a directed interrupt request carrying the Wait Core ID is generated.

Images