Data processing device, chip, board card and data processing method

Abstract

The application discloses a data processing device, a chip, a board card and a data processing method. The data processing device can be included in a combined processing device as a computing device, and the combined processing device can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected with the computing device and the other processing devices respectively, and used for storing data of the computing device and the other processing devices. The scheme of the application improves the processing efficiency of the data processing device through instruction optimization.

Description

Technical Field

[0001] This application generally relates to the field of data processing technology. More specifically, this application relates to a data processing apparatus, chip, circuit board, and data processing method. Background Technology

[0002] With advancements in semiconductor technology and the evolution of processor architecture, the computing power and memory access bandwidth of modern processors have continuously improved, resulting in a constant reduction in instruction execution time (i.e., computation time) and actual data access time. However, memory access latency, limited by the physical characteristics of the storage medium (such as the capacitor charging and discharging mechanism of DRAM) and the physical distance of data transmission paths, has not decreased in tandem; instead, it has shown an increasing trend due to the expansion of chip size and the growth of interconnect paths. This increasing prominence of latency has become a key bottleneck restricting the overall performance improvement of processors. For example, a memory access request issued by the processor needs to go through multiple stages, including address transfer, memory activation, column access, and data return. The read latency can be as high as hundreds to thousands of nanoseconds, far exceeding the execution time of a single computation operation.

[0003] When a processor executes a task, the exposure of latency directly leads to pipeline cavitation, reducing instruction-level parallelism efficiency. Specifically, latency can occur in various scenarios: for example, the first memory access at pipeline startup, synchronization waits between instruction streams, and instruction issue blocking caused by data dependencies (such as read-after-write and write-after-read). Existing technologies typically employ pipeline scheduling or hardware prefetching to mitigate the impact of latency, but these methods still struggle to completely hide latency in the face of increasing memory access latency and complex dependencies.

[0004] In view of this, there is an urgent need to provide a data processing solution that can effectively hide latency during instruction stream execution, reduce pipeline cavitation, and improve the overall efficiency of the processor. Summary of the Invention

[0005] In order to at least solve one or more of the technical problems mentioned above, this application proposes data processing apparatus, chip, board and data processing method solutions in several aspects.

[0006] In a first aspect, this application provides a data processing apparatus, including a processing core, the processing core including an instruction decoding unit and a synchronization control unit, wherein the instruction decoding unit is configured to, in response to a synchronization instruction, determine the production end and consumer end to be synchronized as indicated by the synchronization instruction; the synchronization control unit is configured to, in response to the producer in the production end and the consumer in the consumer end belonging to the same instruction stream, release the synchronization dependency between the producer and the consumer within the same instruction stream; and in response to the producer in the production end and the consumer in the consumer end belonging to different instruction streams, maintain the synchronization dependency between the different instruction streams.

[0007] In a second aspect, this application provides a chip including the data processing apparatus described in the first aspect of this application.

[0008] In a third aspect, this application provides a board including the chip described in the second aspect of this application.

[0009] In a fourth aspect, this application provides a data processing method implemented by a data processing apparatus, wherein the data processing apparatus includes a processing core, and the data processing method includes: in response to a synchronization instruction, determining a production end and a consumer end that need to be synchronized as indicated by the synchronization instruction; in response to a producer in the production end and a consumer in the consumer end belonging to the same instruction stream, releasing the synchronization dependency between the producer and the consumer within the same instruction stream; and in response to a producer in the production end and a consumer in the consumer end belonging to different instruction streams, maintaining the synchronization dependency between the different instruction streams.

[0010] By using the data processing apparatus provided above, the embodiments of this application can selectively remove or maintain the synchronization dependency between the producer and consumer according to whether they belong to the same instruction stream as indicated by the synchronization instruction. This can eliminate unnecessary synchronization waits within the same instruction stream, reduce pipeline cavitation, improve instruction processing efficiency, and reduce processing latency. Attached Figure Description

[0011] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein:

[0012] Figure 1 This is a schematic diagram illustrating the structure of a circuit board according to an embodiment of this application; Figure 2 This is a structural diagram illustrating the combined processing apparatus according to an embodiment of this application; Figure 3This diagram shows the internal structure of a single-core computing device. Figure 4 A schematic diagram of the internal structure of a multi-core computing device is shown. Figure 5 This example illustrates how CDMA works. Figure 6 An exemplary structural diagram of a data processing apparatus according to an embodiment of this application is shown; Figure 7a A schematic diagram illustrating the execution of traditional synchronous instructions under computational bottlenecks is shown; Figure 7b A schematic diagram of the synchronization instruction control scheme of this application embodiment under computing bottleneck is shown; Figure 8a A schematic diagram illustrating the execution of traditional synchronous instructions under memory access bottlenecks is shown; Figure 8b A schematic diagram of the synchronous instruction control scheme of this application embodiment under memory access bottleneck is shown; Figure 9 Exemplary structural diagrams of a data processing apparatus according to other embodiments of this application are shown; Figure 10 An example diagram illustrating the determination of computation instruction dependencies based on the micro-operation particle size of this application is shown in an embodiment of this application; Figure 11 This illustration shows a schematic diagram of the synchronization instructions in a multi-time-slice pipeline according to an embodiment of this application; Figure 12 Exemplary structural diagrams of a data processing apparatus according to further embodiments of this application are shown; Figure 13a This diagram illustrates a pipeline with two first memory regions under a computational bottleneck. Figure 13b A schematic diagram of the pipeline for the three first memory regions under the computational bottleneck is shown; Figure 14 An exemplary structural diagram of a data processing apparatus including at least one cluster, according to some embodiments of this application, is shown; Figure 15a A schematic diagram illustrating the synchronization effect of multiple processing cores under the barrier synchronization mechanism is shown. Figure 15b A schematic diagram illustrating the synchronization effect of multiple processing cores under the semaphore mechanism in an embodiment of this application is shown. Figure 16 A schematic diagram of a four-stage pipeline with three second storage areas is shown in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0015] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0016] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0017] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0018] Exemplary hardware architecture Figure 1 This diagram illustrates the structure of a board 10 according to an embodiment of this application. Figure 1As shown, board 10 includes chip 101, which is a system-on-chip (SoC) integrating one or more combined processing units. These combined processing units are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in cloud intelligence. A significant characteristic of cloud intelligence applications is the large volume of input data, placing high demands on the platform's storage and computing capabilities. Board 10 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.

[0019] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as a PCIe interface.

[0020] The board 10 also includes a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 is configured to regulate the state of the chip 101. Therefore, in one application scenario, the controller 106 may include a microcontroller (MCU).

[0021] Figure 2 This is a structural diagram illustrating the combined processing device in chip 101 of this embodiment. (As shown) Figure 2 As shown, the combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203, and a DRAM 204.

[0022] The computing device 201 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0023] Interface device 202 is used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from storage device of computing device 201 and transmit it to processing device 203.

[0024] The processing device 203, as a general-purpose processing device, performs basic controls including but not limited to data transfer and starting / stopping the computing device 201. Depending on the implementation, the processing device 203 may be one or more types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose and / or special-purpose processors. These processors include, but are not limited to, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing device 201 of this application can be considered as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0025] DRAM 204 is used to store data to be processed. It can be DDR (Double Data Rate Synchronous Dynamic Random Access Memory) memory, typically 16G or larger, used to store data of computing device 201 and / or processing device 203.

[0026] Figure 3 The diagram shows the internal structure of the single-core computing device 201. The single-core computing device 301 is used to process input data such as computer vision, speech, natural language, and data mining. The single-core computing device 301 includes three main modules: a control module 31, a processing module 32, and a storage module 33.

[0027] The control module 31 coordinates and controls the operation of the computation module 32 and the storage module 33 to complete the deep learning task. It includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 fetches instructions from the processing device 203, and the instruction decode unit 312 decodes the fetched instructions and sends the decoding result as control information to the computation module 32 and the storage module 33.

[0028] The computation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 322 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0029] Storage module 33 is used to store or move relevant data, including neuron RAM (NRAM) 331, weight RAM (WRAM) 332, and direct memory access (DMA) module 333. NRAM 331 is used to store input neurons, output neurons, and intermediate results after computation; WRAM 332 is used to store the convolution kernels of the deep learning network, i.e., the weights; DMA 333 is connected to DRAM 204 through bus 34 and is responsible for data transfer between single-core computing device 301 and DRAM 204.

[0030] In some implementations, storage module 33 may include multi-level storage units, such as multi-level caches, to support data-level pipelines in embodiments of this disclosure.

[0031] Figure 4 A schematic diagram of the internal structure of the multi-core computing device 201 is shown. The multi-core computing device 400 adopts a hierarchical design. As a system-on-a-chip, the multi-core computing device 400 includes at least one cluster, and each cluster includes multiple processor cores. In other words, the multi-core computing device 400 is a system-on-a-chip... Cluster The processor cores are structured in layers.

[0032] From the perspective of system-on-a-chip hierarchy, such as Figure 4 As shown, the multi-core computing device 400 includes an external storage controller 41, a peripheral communication module 42, an on-chip interconnect module 43, a synchronization module 44, and multiple clusters 45.

[0033] There can be multiple external storage controllers 41; two are shown as an example in the figure. These controllers are used to access external storage devices, such as those issued by the processor core, in response to access requests from the processor core. Figure 2 The DRAM 204 in the chip allows data to be read from or written to external devices. The peripheral communication module 42 receives control signals from the processing device 203 via the interface device 202, initiating the computing device 201 to execute tasks. The on-chip interconnect module 43 connects the external storage controller 41, the peripheral communication module 42, and multiple clusters 45 to transmit data and control signals between modules. The synchronization module 44 is a global barrier controller (GBC) used to coordinate the working progress of each cluster and ensure information synchronization. The multiple clusters 45 are the computing cores of the multi-core computing device 400. Four are shown exemplary in the figure; however, with hardware development, the multi-core computing device 400 of this application may also include 8, 16, 64, or even more clusters 45. The clusters 45 are used to efficiently execute deep learning algorithms.

[0034] From the perspective of cluster hierarchy, such as Figure 4 As shown in the upper right corner, each cluster 45 includes a processing unit 402 and a memory core (MEM core) 404. The processing unit 402 performs various computational tasks. In some implementations, the processing unit can be a multi-core architecture, such as including multiple processing cores (IPU cores) 411-1 to 411-n, to perform tasks such as large-scale vector computation. This application does not limit the number of processing cores 411.

[0035] The internal architecture of the processing core 411 is as follows Figure 4 As shown below. Each processing core 411 may have multiple computing modules 424-1 to 424-m for performing computing tasks, and a local storage module 423 required for performing computing tasks. It should be noted that the local storage module 423 may include various communication modules for exchanging data with external storage units. For example, the local storage module 423 may include a communication module 421 for communicating with the shared storage module 415 in the storage core 404. The communication module 421 may be, for example, a move direct memory access (MVDMA) module. The local storage module 423 may also include a communication module 422 for exchanging data with off-chip memory, such as DRAM 408. The communication module 422 may be, for example, an input / output direct memory access (IODMA) module. The IODMA 422 controls the NRAM / WRAM in the local storage module 423. Figure 4 Not shown, see Figure 3The MVDMA 421 controls the access of NRAM / WRAM in the local storage module 423 and the shared storage module 415.

[0036] continue Figure 4 In the upper right view, storage core 404 is mainly used for storage and communication, namely storing shared data or intermediate results between processing cores 411, and performing communication between cluster 45 and DRAM 408, communication between clusters 45, and communication between processing cores 411. In other embodiments, storage core 404 has scalar operation capabilities to perform scalar operations to implement computational tasks in data communication.

[0037] The storage core 404 includes a large shared storage module (such as static random access memory, SRAM) 415, a broadcast bus 414, a cluster direct memory access (CDMA) module 418, a global direct memory access (GDMA) module 416, and a communication-time computation module 417. The SRAM 415 acts as a high-performance data relay station. Data multiplexed between different processing cores 411 within the same cluster 45 does not need to be obtained from the DRAM 408 by each processing core 411 individually. Instead, it is relayed between processing cores 411 via the SRAM 415. The storage core 404 only needs to quickly distribute the multiplexed data from the SRAM 415 to multiple processing cores 411, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip input / output access.

[0038] Broadcast bus 414, CDMA 418, and GDMA 416 are used to perform communication between processing cores 411, communication between clusters 45, and data transfer between cluster 45 and DRAM 408, respectively. These will be explained below.

[0039] The broadcast bus 414 is used to complete high-speed communication between the processing cores 411 within the cluster 45. In this embodiment, the broadcast bus 414 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point (e.g., data transmission from one processing core to another) data transmission. Multicast is a communication method that transmits a copy of data from SRAM 415 to several specific processing cores 411. Broadcast is a communication method that transmits a copy of data from SRAM 415 to all processing cores 411, and is a special case of multicast.

[0040] CDMA 418 is used to control SRAM 415 access between different clusters 45 within the same computing device 201.

[0041] GDMA 416 works in conjunction with external memory controller 41 to control memory access from SRAM 415 to DRAM 408 in cluster 45, or to read data from DRAM 408 into SRAM 415. As described above, communication between DRAM 408 and NRAM / WRAM in local storage module 423 can be achieved through two channels. The first channel is a direct connection between DRAM 408 and local storage module 423 via IODMA 422; the second channel involves first transmitting data between DRAM 408 and SRAM 415 via GDMA 416, and then transmitting data between SRAM 415 and local storage module 423 via MVDMA 421. Although the second channel appears to require more components and has a longer data flow, in some embodiments, the bandwidth of the second channel is significantly greater than that of the first channel. Therefore, communication between DRAM 408 and local storage module 423 may be more efficient via the second channel. Embodiments of this application can select the data transmission channel based on their hardware capabilities.

[0042] In some embodiments, storage core 404 can serve as a cache layer within cluster 45, significantly expanding communication bandwidth. Furthermore, storage core 404 can also facilitate communication with other clusters 45. For example, storage core 404 can implement communication functions such as broadcast, scatter, gather, reduce, and all-reduce between clusters 45. Broadcast refers to distributing the same data to all clusters; scatter refers to distributing different data to different clusters; gather refers to aggregating data from multiple clusters; reduce refers to processing data from multiple clusters according to a specified mapping function to obtain the final result and sending it to a specific cluster; and the difference between all-reduce and scatter is that the latter only sends the final result to one cluster, while all-reduce sends it to all clusters.

[0043] The communication-time computing module 417 can be used to complete computational tasks in communication processes, such as those involving the aforementioned protocols and full protocols, without relying on the processing unit 402, thereby improving communication efficiency and achieving the effect of "in-memory computing". Depending on different hardware implementations, the communication-time computing module 417 and the shared storage module 415 can be integrated in the same or different components. This disclosure embodiment is not limited in this respect, as long as the functions implemented and the technical effects achieved are similar to those disclosed herein, they are all within the protection scope of this disclosure.

[0044] Figure 5This diagram illustrates the operation of CDMA 418 when one processor core attempts to write data to another processor core in a different cluster. In this application scenario, the same computing device comprises multiple clusters. For simplicity, only clusters 0 and 1 are shown in the diagram. Both clusters 0 and 1 contain multiple processor cores; similarly, for ease of explanation, only processor core 0 is shown in cluster 0, and only processor core 1 is shown in cluster 1. Processor core 0 intends to write data to processor core 1.

[0045] First, processor core 0 sends a unicast write request to write data into its local SRAM 0. CDMA 0 acts as the master and CDMA 1 acts as the slave. The master pushes the write request to the slave, that is, the master sends the write address AW and the write data W to transmit the data to SRAM 1 of cluster 1. Then the slave sends a write response B as a reply. Finally, processor core 1 of cluster 1 sends a unicast read request to read the data from SRAM 1.

[0046] Based on the aforementioned hardware environment, this application provides a data processing scheme that supports instruction-level pipelines and data-level pipelines. More specifically, this scheme is a latency hiding technique that integrates instruction-level parallelism and data-level pipelines, thereby effectively reducing pipeline cavitation, improving instruction processing efficiency, and reducing processing latency.

[0047] Intra-core synchronization instruction optimization scheme The terms used in this article include: Synchronization dependency refers to the execution order constraint established by synchronization instructions (within the kernel or between kernels), which is used to coordinate the execution progress of different instructions or different processing kernels. Even if there is no direct data address association between instructions, they must wait for the synchronization condition to be met before they can continue to execute.

[0048] Data dependency refers to the hard constraints arising from data read / write associations or address associations between instructions. For example, a later instruction may need to read data written by a previous instruction (read after write), or write to an address that the previous instruction is about to read (write after read), or a later instruction may need to occupy space occupied by a previous instruction. This dependency is determined by the range of addresses accessed by the instructions.

[0049] An instruction stream can be understood as an instruction queue, which includes multiple instructions.

[0050] The same instruction stream refers to the same instruction queue, such as an instruction stream of the same type (IO stream or SIMD stream), or one of multiple instruction streams of the same type (e.g., one of IO0 stream or IO1 stream).

[0051] Different instruction streams refer to different instruction queues, such as instruction streams of different types, or instruction streams formed by different instruction queues of the same type.

[0052] Figure 6 An exemplary structural diagram of a data processing apparatus according to an embodiment of this application is shown. Figure 6 As shown, the data processing device 600 includes a processing core 610, which includes an instruction decoding unit 611 and a synchronization control unit 612. The instruction decoding unit 611 is configured to determine the production end and consumer end that need to be synchronized as indicated by the synchronization instruction in response to a synchronization instruction. The synchronization control unit 612 is configured to release the synchronization dependency between the producer and consumer in the same instruction stream in response to the producer in the production end and the consumer in the consumer end belonging to the same instruction stream; and to maintain the synchronization dependency between the different instruction streams in response to the producer in the production end and the consumer in the consumer end belonging to different instruction streams.

[0053] Processing core 610 is a hardware unit capable of independently executing instruction streams. In some embodiments, processing core 610 may be a single-core processing unit, for example, it may be... Figure 3 The single-core computing device 301 shown Figure 4 The processing core 411 shown is an example. The data processing device 600 may include one or more processing cores 610. The processing core 610 in this embodiment includes an instruction decoding unit 611 and a synchronization control unit 612, which work together to optimize the synchronization mechanism between instruction streams within the processing core 610.

[0054] When executing a sequence of instructions within a single processing core, instructions are typically cached into different instruction queues according to their type, ready to be issued. The instruction stream in this paper can be understood as an instruction queue. Each instruction stream (i.e., each instruction queue) can include one or more instructions. Typically, instructions within each instruction queue are issued and executed sequentially. Instructions in multiple instruction queues can be issued in parallel, so overall, the instructions are issued out of order. Dependencies between different instruction queues need to be guaranteed through synchronized instructions.

[0055] In processor architecture, synchronization instructions are special instructions used to coordinate the execution order of instruction streams. They do not perform actual data operations or transfers themselves, but rather establish dependencies by specifying producer and consumer instruction streams. This ensures that the consumer instruction stream can only begin execution after the producer instruction stream has completed, thereby maintaining data consistency and program correctness. In this embodiment, synchronization instructions are used to handle instruction stream synchronization within the kernel, such as the coordination between memory access instruction streams (or I / O streams) and computation instruction streams (or SIMD streams). The instruction decoding unit responds to the synchronization instruction to determine the indicated producer and consumer ends, providing a basis for subsequent synchronization control.

[0056] The instruction decoding unit 611 can be used to parse instructions fetched from the instruction cache or memory. When the instruction decoding unit 611 receives an instruction, it can identify the type of the instruction and its operands. Specifically, for received synchronization instructions (such as the `sync` instruction), the instruction decoding unit 611 decodes them to determine the producer and consumer ends that need to be synchronized, as indicated by the synchronization instruction. Here, the producer end refers to the set of instructions that generate data or complete a specific operation, while the consumer end refers to the set of instructions that depend on the completion of the producer end before they can begin execution. The producer end may include one or more producers, and the consumer end may include one or more consumers.

[0057] The synchronization control unit 612 is coupled to the instruction decoding unit 611 and is used to schedule the execution of the instruction stream within the processing core 610 based on the decoding results. Specifically, the synchronization control unit 612 analyzes the specific composition of the producer and consumer ends determined by the instruction decoding unit 611, and determines the relationship between a specific instruction (i.e., the "producer") in the producer end and a specific instruction (i.e., the "consumer") in the consumer end. If the determination result indicates that the producer and consumer belong to the same instruction stream (e.g., the same type of instruction stream), for example, both belong to the same memory access instruction stream or the same computation instruction stream, then the synchronization control unit 612 will remove the synchronization dependency between the producer and consumer within the instruction stream. This means that within the instruction stream, subsequent consumer instructions do not need to wait for the preceding instructions of the producer to be fully executed before they can begin preparation or issuance, thereby achieving parallel execution within the instruction stream and eliminating unnecessary pipeline dangling.

[0058] Conversely, if the synchronization control unit 612 determines that the producer and consumer belong to different instruction streams—for example, the producer is a memory access instruction while the consumer is a computation instruction—then the synchronization control unit 612 will maintain the synchronization dependency between these two different instruction streams. That is, in this case, the consumer instruction stream must wait for all instructions in the producer instruction stream to complete execution before it can begin. This approach ensures data dependencies across instruction streams and the correctness of program execution.

[0059] For example, in the synchronous instruction `sync.pio.psimd.cio.csimd`, where `sync` represents the synchronous instruction, `p` represents the producer, `c` represents the consumer, `io` represents the memory access instruction, and `simd` represents the computation instruction, then the producers to be synchronized in this synchronous instruction include `pio` and `psimd`, and the consumers include `cio` and `csimd`. In traditional synchronous instruction execution mechanisms, it is necessary to wait for all prior instructions at the producers indicated by the synchronous instruction to finish executing before releasing subsequent instructions at the consumers. That is, `pio` and `psimd` must all finish executing before `cio` and `csimd` can be released. This restriction increases unnecessary waiting time. Specifically, this restriction actually creates four instruction streams: `pio→cio`, `pio→csimd`, `psimd→cio`, and `psimd→csimd`. Here, the problem of mutual waiting within the same instruction stream (such as `pio→cio` and `psimd→csimd`) arises, exposing instruction latency.

[0060] The synchronization control unit 612 according to an embodiment of this application can automatically release the synchronization dependency between pio and cio, and between psimd and csimd, during pio→cio and psimd→csimd, while only maintaining the actual synchronization dependency between pio and csimd, and between psimd and cio. Based on this, each instruction stream only needs to wait for the other dependent streams to complete before it can begin execution. Under I / O bottlenecks, it can eliminate I / O memory access latency before and after synchronization instructions; under computational bottlenecks, it can eliminate computational latency. The following will combine... Figures 7a-8b An example is provided.

[0061] Figure 7a This diagram illustrates the execution of traditional synchronous instructions under a computational bottleneck. In processor pipeline design, a computational bottleneck refers to a situation where the execution time (including actual computation time and computational latency) of a computational instruction (SIMD) within a single time slice is significantly greater than the memory access time (including actual memory access time and memory access latency) of a memory access instruction (load / store), thus limiting the overall pipeline throughput to the completion speed of the computational units. A time slice is the number of cycles required for a processing core to complete a predetermined set of operations, such as the fixed number of cycles required to complete a processing stage (e.g., load, compute, or store), and is the basic time unit for pipeline scheduling. Pipelines achieve parallel processing of different instruction streams or data blocks by overlapping the execution of multiple time slices in time.

[0062] like Figure 7aAs shown, the complete execution cycle of an instruction (such as a memory access instruction or an arithmetic instruction) is modeled as a parallelogram. The left diagonal 701 represents the instruction's initiation overhead (such as instruction decoding, address calculation, and request issuance preparation phases), the middle rectangular area 702 represents the core effective execution time of the instruction (such as the actual data transfer time on the bus or the actual computation time of the arithmetic unit), and the right diagonal 703 represents the instruction's completion overhead (such as result write-back, resource release, and memory pre-charging phases). This modeling method clearly distinguishes the time consumption components of different stages in the instruction execution process, providing an intuitive basis for analyzing pipeline efficiency, identifying latency sources, and optimizing instruction scheduling.

[0063] Taking a three-stage pipeline (load→compute→store) as an example, under the ping-pong double buffering mechanism, assuming there are at least three sets of data blocks (01, 02, 03), and using synchronization instructions to control the synchronization dependencies between pipelines, and treating the operation between two adjacent synchronization instructions as a time slice, the execution flow is as follows: On the time axis T, in the first cycle, the load01 instruction moves data block 01 from external storage to a buffer ping in on-chip storage (such as NRAM); in the second cycle, the simd01 instruction starts to perform calculations on data block 01, while the load02 instruction moves data block 02 to another buffer pong; in the third cycle, the simd02 instruction processes data block 02, the store01 instruction writes the calculation result of the previous round (such as data block 01) back to storage, releases the buffer ping, and then the load03 instruction moves data block 03 from external storage to the buffer ping; in the fourth cycle, the simd03 instruction starts to process data block 03, the store02 instruction starts to write back the result of data block 02, and releases the buffer pong.

[0064] In this process, because the computation time within a time slice is much greater than the memory access time, SIMD instructions (i.e., computation instructions) become the slowest stage in the pipeline. This causes subsequent load and store instructions, although completed, to wait for the SIMD instructions to release the computation unit before entering the next iteration. Therefore, the overall processing time is dominated by computation time rather than memory access time. Furthermore, due to the control of synchronization instructions, not only do memory access instructions and computation instructions need to wait for each other, but instructions within the same instruction stream also need to wait for each other, such as... Figure 7aIn the computation instruction stream shown, simd01, simd02, and simd03 wait for each other, thus introducing bubbles of computational latency exposure into the pipeline (i.e., gaps between adjacent computation instructions in the diagram). For example, after computation instruction simd01 is executed, simd02, although its data is ready, still needs to wait for the computation unit to become idle. This waiting time is the additional latency caused by the synchronous dependency between computation instructions under the computational bottleneck, which manifests as a lengthening of the continuous execution interval between simd stages in the pipeline.

[0065] based on Figure 7a In the computational bottleneck scenario shown, when processing N sets of data blocks (e.g., data blocks 01, 02, 03, etc.), let Trt represent read throughput time (i.e., the actual execution time of a read operation), Trl represent read latency, Tct represent actual computation time, Tcl represent computation latency, Twt represent write throughput time (i.e., the actual execution time of a write operation), and Twl represent write latency. Then, the final execution time of the system is: Trt + Trl + N × (Tct + Tcl) + Twt + Twl. In this formula, firstly, the first read operation needs to go through the complete read latency Trl (as shown by the left diagonal 701 and right diagonal 703 in the diagram) and read throughput time Trt to load the first block of data. Subsequently, during the stable operation phase of the pipeline, each data block goes through the computation phase in sequence (including computation latency Tcl and actual computation time Tct), so the total computation time for N data blocks is N × (Tct + Tcl). Finally, the last write operation also needs to complete the write throughput time Twt and write latency Twl to completely write the result back.

[0066] Figure 7b A schematic diagram of the synchronization instruction control scheme of this application embodiment under computing bottleneck is shown. For example... Figure 7b As shown, with Figure 7a The difference shown is that the synchronization control unit 612 in this embodiment can release the synchronization dependency between producers and consumers within the same instruction stream in the synchronization instruction. As illustrated, it releases the synchronization dependency between Load01 and Load02 indicated by sync01, the synchronization dependency between Simd01 and Simd02 indicated by sync02, and the synchronization dependency between Simd02 and Simd03 indicated by sync03. Since Load01 and Simd01 belong to different instruction streams, Simd01 and Store01 belong to different instruction streams, and Simd02 and Store02 belong to different instruction streams, the synchronization dependency is still maintained.

[0067] Because the synchronization dependency in the same instruction stream (referred to as the same stream) is removed, Simd02 can start transmitting after Simd01 has completed its actual computation time, and Simd03 can start transmitting after Simd02 has completed its actual computation time, thereby eliminating cavitation between computation instructions in computational bottleneck scenarios.

[0068] At this point, when processing N sets of data blocks (e.g., data blocks 01, 02, 03, etc.), the final execution time is: Trt + Trl + Tcl + N × Tct + Twt + Twl. In this formula, firstly, the first read operation needs to go through the full read latency Trl and read throughput time Trl to load the first data block; subsequently, during the stable operation phase of the pipeline, each data block goes through the computation phase in sequence (including the actual computation time Tct), but due to pipeline overlap, only the startup overhead (left diagonal) of the computation phase of the first data block and the completion overhead (right diagonal) of the last data block are exposed. These two can be combined into one computation latency, so the total computation time for N data blocks is Tcl + N × Tct; finally, the last write operation also needs to complete the write throughput time Twt and write latency Twl to completely write the result back.

[0069] Compared to Figure 7a In the example, Figure 7b The final execution time in the example shown is reduced by (N-1)Tcl, which illustrates that under the computational bottleneck, the data processing apparatus of this application embodiment can effectively eliminate computational delay, thereby improving data processing efficiency.

[0070] Figure 8a The diagram illustrates the execution of traditional synchronous instructions under a memory access bottleneck. In processor pipeline design, a memory access bottleneck refers to a situation where the memory access time (including actual memory access time and memory access latency) of a memory access instruction (read load / write store) within a single time slice is significantly greater than the execution time (including actual computation time and computation latency) of a computation instruction (SIMD), thus limiting the overall throughput of the pipeline to the speed at which data loading or storage operations are completed.

[0071] like Figure 8aAs shown in the diagram, taking a three-stage pipeline (load→compute→store) as an example, under the ping-pong double buffering mechanism, assuming there are at least three sets of data blocks (01, 02, 03), and using synchronization instructions to control the synchronization dependencies between pipelines, and treating the operation between two adjacent synchronization instructions as a time slice, the execution flow is as follows: On the time axis T, in the first cycle, the load01 instruction moves data block 01 from external storage to a buffer ping in on-chip storage (such as NRAM); in the second cycle, the simd01 instruction starts to perform calculations on data block 01, while the load02 instruction moves data block 02 to another buffer pong; in the third cycle, the simd02 instruction processes data block 02, the store01 instruction writes the calculation result of the previous round (such as data block 01) back to storage, releases the buffer ping, and then the load03 instruction moves data block 03 from external storage to the buffer ping; in the fourth cycle, the simd03 instruction starts to process data block 03, the store02 instruction starts to write back the result of data block 02, and releases the buffer pong.

[0072] In this process, because the memory access time within a time slice is greater than the computation time, Load / Store instructions (i.e., memory access instructions) become the slowest stage in the pipeline, and the overall processing time is dominated by memory access time rather than computation time. Furthermore, due to the control of synchronization instructions, not only do memory access instructions and computation instructions need to wait for each other, but instructions within the same instruction stream also need to wait for each other, such as... Figure 8a In the memory access instruction stream shown, Load01 and Load02, and Load03 and Store02, wait for each other, thus introducing bubbles of memory access latency exposure in the pipeline (i.e., gaps between adjacent memory access instructions in the diagram). For example, when Load01 is executed, although it only occupies the ping buffer and the pong buffer is free at this time, Load02 still needs to wait for Load01 to complete due to the existence of sync01. This waiting time is the additional latency caused by the synchronization dependency between memory access instructions under the memory access bottleneck, which manifests as a lengthening of the continuous execution interval between memory access stages in the pipeline.

[0073] Figure 8b A schematic diagram of the synchronous instruction control scheme of this application embodiment under memory access bottleneck is shown. For example... Figure 8b As shown, with Figure 8aThe difference shown is that the synchronization control unit 612 in this embodiment can remove the synchronization dependency between producers and consumers within the same instruction stream in the synchronization instruction. As illustrated, it removes the synchronization dependency between Load01 and Load02 indicated by sync01, and also removes the synchronization dependency between Load03 and Store02 indicated by sync03. Since Load01 and Simd01 belong to different instruction streams, Simd01 and Store01 belong to different instruction streams, and Simd02 and Store02 belong to different instruction streams, the synchronization dependency is still maintained.

[0074] Because the synchronization dependency in the same instruction stream (referred to as the same stream) is removed, Load02 can start firing before Load01 completes, and Store02 can start firing before Load03 completes, thereby eliminating at least some of the bubble between memory access instructions in memory access bottleneck scenarios. Figure 8b The bubble between Load02 and Store01 shown in the figure was not eliminated because of the data dependency between Simd01 and Store01.

[0075] Compared to Figure 8a In the example, Figure 8b In the example shown, by eliminating at least some of the bubble between memory access instructions, the final execution time can be significantly reduced under the memory access bottleneck. This illustrates that the data processing apparatus of this application embodiment can effectively mask memory access latency, thereby significantly improving data processing efficiency and processing core performance.

[0076] The above combination Figures 6-8b The synchronization mechanism of the data processing apparatus in this application embodiment is described exemplarily. Through this mechanism, the data processing apparatus 600 provided in this application embodiment can intelligently distinguish and process synchronization dependencies within the same instruction stream and between different instruction streams, while ensuring correct program execution. Within the same instruction stream, unnecessary synchronization waits can be eliminated to maximize instruction-level parallelism and improve the execution efficiency within the processing core. Between different instruction streams, dependencies are strictly maintained to preserve data consistency. This refined synchronization control strategy effectively avoids performance loss caused by coarse-grained synchronization in the prior art, and improves the overall processing speed and resource utilization of the processor.

[0077] It is understood that the above description and illustrations are exemplary and not restrictive. For example, the data processing apparatus of this application embodiment may not be limited to only including the instruction decoding unit 611 and the synchronization control unit 612 shown in the illustrations. In some embodiments, the data processing apparatus may also include a dependency determination unit. The following will combine... Figure 9An illustrative example is provided.

[0078] Figure 9 Exemplary structural diagrams of a data processing apparatus according to other embodiments of this application are shown. For example... Figure 9 As shown, the data processing device 600 may include one or more processing cores 610, wherein each processing core 610 may include an instruction decoding unit 611, a synchronization control unit 612, a dependency judgment unit 613, and an instruction issuing unit 614. The instruction decoding unit 611 and the synchronization control unit 612 have been described in detail above with reference to several accompanying figures, and will not be repeated here. The dependency judgment unit 613 and the instruction issuing unit 614 will be described below. These two units work in conjunction with the aforementioned instruction decoding unit 611 and synchronization control unit 612 to jointly constitute the instruction scheduling and execution control logic within the processing core 610. The dependency judgment unit 613 is mainly used to analyze the data dependencies between instructions, while the instruction issuing unit 614 determines whether an instruction can be issued in advance based on the dependency judgment result, thereby further exploiting instruction-level parallelism.

[0079] The dependency determination unit 613 is configured to determine whether there is a data dependency relationship between adjacent instructions based on whether their access address ranges overlap within the same instruction stream. Here, the same instruction stream can refer to the same type of instruction queue, such as a memory access instruction stream or a computation instruction stream. Adjacent instructions refer to two consecutive instructions in program sequence, which are decoded and issued sequentially in the pipeline. The access address range refers to the range of memory addresses to be read or written during instruction execution; for memory access instructions, this is the memory address, and for computation instructions, it may involve registers or local memory addresses. It should be noted that the access address range can point to on-chip memory units (such as SRAM or NRAM) or off-chip memory units (such as DRAM), depending on the memory space accessed by the instruction. The dependency determination unit 613 makes an accurate determination based on the instruction's address information, without being limited by the memory hierarchy.

[0080] If the access address ranges of two instructions overlap at least partially, it indicates that the latter instruction may depend on the data result of the former instruction. This dependency is called data dependency and must be strictly followed to ensure program correctness. Conversely, if the access address ranges do not overlap at all, there is no data dependency between the two instructions, and they can be executed in parallel or out of order. For example, consider the following two consecutive memory access instructions: the first instruction loads data from address 0x1000~0x100F, and the second instruction loads data from address 0x1010~0x101F. Since the address ranges do not overlap, the dependency judgment unit 613 can determine that there is no data dependency between the two instructions. If the address range of the second instruction is 0x1008~0x1017, which overlaps with the first instruction, then a data dependency is determined to exist.

[0081] The instruction issuing unit 614 is coupled to the dependency judgment unit 613 and is used to schedule the execution timing of instructions based on dependencies during the instruction issuing phase. Specifically, the instruction issuing unit 614 is configured to allow the subsequent instruction to be issued before the preceding instruction completes, in response to the absence of data dependencies between adjacent instructions. This means that even if the preceding instruction has not yet finished executing, as long as its execution result does not affect the correctness of the subsequent instruction, the subsequent instruction can enter the execution pipeline in advance, thereby achieving instruction-level parallelism. Taking the two memory access instructions with non-overlapping addresses as an example, after the first instruction starts loading data, the second instruction can start immediately without waiting for its completion. The two can be executed in parallel in different functional units (such as different memory access ports), thereby hiding some memory access latency. If the dependency judgment unit 613 determines that there is a data dependency, the instruction issuing unit 614 will maintain the sequential issuance, that is, the subsequent instruction must wait for the preceding instruction to complete before it can start, to ensure data correctness.

[0082] Through the aforementioned mechanism, the collaborative work of the dependency judgment unit 613 and the instruction issuing unit 614 maximizes the potential of instruction-level parallelism while ensuring the correctness of data dependencies. Compared to traditional processors that rely solely on compiler static scheduling or conservative hardware dependency judgment (for example, if two instructions involve store and load operations on the same type of memory region, such as DDR, even if their access address ranges do not overlap, they will be judged as having a dependency and forced to be executed sequentially), this embodiment dynamically identifies dependency-free instructions and allows them to be issued earlier through fine-grained address range comparison. This effectively avoids pipeline bubbling caused by overly conservative dependency judgment, significantly improving the instruction throughput and overall performance of the processing core. This optimization, combined with the fine-grained management of synchronization dependencies by the aforementioned synchronization control unit 612, constitutes a comprehensive optimization scheme from data dependencies to synchronization dependencies, significantly enhancing the execution efficiency of the processing core.

[0083] In the aforementioned embodiments, the dependency judgment unit 613, by comparing the complete access address ranges of adjacent instructions within the same instruction stream, can identify instructions without data dependencies and allow them to be issued earlier, thereby releasing the potential of instruction-level parallelism to a certain extent. Furthermore, for computation instructions, their execution process often involves multiple stages, and performing dependency judgment only at the granularity of the entire instruction may still retain some unnecessary waiting time. Therefore, this application embodiment further introduces a micro-operation-level dependency analysis mechanism to explore parallelism at a finer granularity.

[0084] Specifically, in some embodiments, when the instruction stream executed by the processing core 610 includes a computation instruction stream, and the computation instruction stream includes multiple computation instructions (e.g., a sequence of instructions consisting of multiple vector operation instructions), the instruction decoding unit 611 is further configured to break down each computation instruction into multiple micro-operations. Here, a micro-operation refers to the smallest unit of operation that the processor hardware can directly execute. A complex computation instruction can be broken down into several micro-operations, such as data read, arithmetic operation, and result write-back. Each micro-operation occupies an independent execution cycle in the pipeline. By breaking down instructions into micro-operations, the processing core can achieve finer-grained scheduling within the instruction, providing finer-grained decision-making for subsequent dependency checks.

[0085] The dependency determination unit 613 is further configured to determine whether a data dependency exists between two adjacent computation instructions based on whether the data address ranges accessed by at least the last segment of a micro-operation in the preceding computation instruction and at least the first segment of a micro-operation in the following computation instruction overlap. Here, "at least the last segment of a micro-operation" refers to one or more micro-operations near the end of execution in the preceding instruction, such as a micro-operation responsible for writing back the result, or two micro-operations responsible for arithmetic operations and writing back the result. "At least the first segment of a micro-operation" refers to one or more micro-operations near the beginning of execution in the following instruction, such as a micro-operation responsible for reading data, or two micro-operations responsible for reading data and performing arithmetic operations. If the address ranges of at least the last segment of a micro-operation in the preceding computation instruction and at least the first segment of a micro-operation in the following computation instruction do not overlap, it is determined that there is no data dependency between at least some of the micro-operations of the two adjacent computation instructions; otherwise, if there is overlap, sequential execution must be maintained.

[0086] To facilitate understanding, the following will combine... Figure 10 An example is provided. Figure 10As shown, taking two computation instructions in a computation instruction stream (such as the adjacent first computation instruction s100 and the second computation instruction s200, where the first computation instruction s100 is the preceding instruction and the second computation instruction s200 is the following instruction) as an example, assume that the first computation instruction s100 is split into three micro-operations: s101 (read input data), s102 (perform multiplication), and s103 (write back the result); and the second computation instruction s200 is split into two micro-operations: s201 (read input data) and s202 (perform addition). The dependency judgment unit 613 compares the data address range accessed by the last micro-operation s103 of the first computation instruction s100 with the first micro-operation s201 of the second computation instruction s200. If the address range written by s103 does not overlap with the address range read by s201, it is determined that there is no data dependency between these two micro-operations from different computation instructions; otherwise, if the address written by s103 is exactly the address that s201 is about to read, there is a write-after-read dependency, and s201 must wait for s103 to complete before it can start.

[0087] It is important to clarify that, within the context of micro-operation decomposition of computation instructions, the essential differences between computation instructions and memory access instructions need to be clearly distinguished. The core function of computation instructions (such as vector multiplication and addition) is to perform arithmetic or logical operations on ready data. Their operands typically come from registers or local memory (such as NRAM), and the results are written back to registers or local memory. Memory access instructions (such as load / store) are responsible for moving data between the processor and the memory hierarchy (such as DDR and SRAM). Therefore, in the above implementation where computation instructions are decomposed into micro-operations such as data reading, arithmetic operations, and result writing back, the reading and writing back refer to data interaction between the computation unit and local registers or L1 cache, rather than memory access operations with off-chip memory or shared memory, which are still handled by a dedicated memory access instruction stream.

[0088] The instruction issuing unit 614 is also configured to allow a subsequent computation instruction to be issued before the completion of the last micro-operation of the preceding computation instruction, in response to the absence of data dependency between at least the last micro-operation and at least the first micro-operation. This means that the micro-operation of the subsequent computation instruction (e.g., s201) ​​can overlap with the last micro-operation of the preceding computation instruction (e.g., s103) in the pipeline, as long as it is ensured that data that is not yet ready is not accessed. Taking the two instructions mentioned above as an example, if the addresses of s103 and s201 do not overlap, s201 can start reading data during the execution of s103, thereby hiding some execution delay.

[0089] In some preferred embodiments, when the computation time of a computation instruction exceeds 45 cycles, the aforementioned micro-operation decomposition and dependency judgment can completely hide pipeline cavitation. Here, "cycle" refers to a clock cycle, which is the smallest unit of time for processor execution, determined by the system's main clock frequency. 45 cycles is a typical hardware design threshold, representing the critical path latency in the processor pipeline from instruction issuance to result write-back. This value is derived from statistics and trade-offs regarding the typical number of execution cycles required for common computation instructions (such as integer arithmetic, floating-point multiplication and addition) under the target process technology.

[0090] When the computation time of a regular instruction (non-memory access instruction) exceeds 45 clock cycles, it means that the execution latency of the instruction is long enough that, with the cooperation of micro-operation splitting and address dependency judgment mechanisms, the subsequent instruction can completely hide the waiting time by issuing it earlier, thus not exposing any pipeline cavitation. Conversely, if the computation time is less than 45 clock cycles, the dependency window may not be fully covered due to the instruction executing too quickly. Therefore, this 45-cycle threshold reflects the balance between instruction granularity, pipeline depth, and power consumption in hardware design, ensuring that micro-operation-level dependency optimization can effectively play a role in most compute-intensive scenarios.

[0091] Through the aforementioned micro-operation-level dependency judgment and issue control mechanism, the embodiments of this application can achieve finer-grained parallel scheduling within computation instructions. Compared to dependency judgment at the whole instruction level, this scheme analyzes the address overlap between the end of the instruction and the beginning of the micro-operation, allowing the subsequent instruction to intervene and execute at an earlier time, further reducing pipeline cavitation caused by data dependencies. This optimization is particularly important for computationally intensive tasks, effectively improving the throughput of the computation instruction stream and fully utilizing the computing power of the processing core.

[0092] In the aforementioned embodiments, the synchronization control unit determines whether the producer and consumer belong to the same instruction stream, thereby de-dependent on co-current (same instruction stream) dependencies and maintaining co-current (different instruction stream) dependencies, thus eliminating unnecessary synchronization waits while ensuring program correctness. Furthermore, the inventors have discovered that in actual execution, the establishment and de-dependent on synchronization dependencies are often not limited to two consecutive instructions within the current time slice, but may span multiple instructions or even multiple processing stages. For example, in a multi-buffered pipeline, the producer instruction is issued in an earlier time slice, while the consumer instruction needs to wait in a later time slice. Therefore, this application embodiment further provides a more flexible synchronization instruction format, namely, achieving cross-time-slice synchronization control through paired matching first and second sub-instructions.

[0093] In some embodiments, the synchronization instructions may include a pair of matching first and second sub-instructions. The first sub-instruction indicates the producer that forms a synchronization dependency, and the second sub-instruction indicates the corresponding consumer. Here, pair matching means that the first sub-instruction and the corresponding second sub-instruction are associated through, for example, the same identifier (ID), and the hardware can identify the dependency between them based on this identifier. When the first sub-instruction is executed, its function is to record the producer's information in the hardware synchronization table, but it does not block the execution of the current instruction stream. When the second sub-instruction is executed, it queries the synchronization table to see if the corresponding producer has completed. If it has not completed, it blocks the current instruction stream until all producers indicated by the matching first sub-instruction have completed execution.

[0094] In other embodiments, the producer indicated by the first sub-instruction and the consumer indicated by the second sub-instruction may be separated by at least one instruction to be executed or by at least one time slice, where the time slice is the cycle required for the processing core to complete a set of predetermined operations.

[0095] To facilitate understanding, specific examples will be provided below.

[0096] Figure 11 This diagram illustrates the use of synchronization instructions in a multi-time-slice pipeline according to an embodiment of this application. Taking a three-stage pipeline as an example, a complete cycle of load (read), compute (computation), and store (write) in a three-stage pipeline can be considered as one time slice. Figure 11 As shown, load0, simd0, and store0 are the first time slices, load1, simd1, and store1 are the second time slices, load2, simd2, and store2 are the third time slices, and load3, simd3, and store3 are the fourth time slices.

[0097] Take two sets of paired synchronization instructions as an example. The first set of synchronization instructions includes the first sub-instruction "sync.p.simd0" and the second sub-instruction "sync.v.io0 0". The first sub-instruction "sync.p.simd0" is executed in the first time slice, indicating that the producer with identifier "0" is a SIMD stream (i.e., the instruction stream containing the simd0 instruction in the current time slice). After this instruction is executed, the hardware synchronization table records "SIMD stream 0" as the producer, but it does not block the execution of subsequent instructions. The second sub-instruction "sync.v.io0 0" is executed before the load3 instruction in the fourth time slice, indicating that the consumer with identifier "0" is an IO0 stream (i.e., the memory access instruction stream containing the load3 instruction). When this second sub-instruction is executed, the synchronization control unit checks whether the corresponding producer "SIMD stream 0" in the synchronization table has completed. Since simd0 in the first time slice has already completed execution after the first sub-instruction, the synchronization condition is met, and load3 can continue execution without further waiting. This set of synchronization instructions enables cross-time-slice synchronization between the SIMD stream of the first time slice and the IO0 stream of the fourth time slice, ensuring that load3 only begins after simd0 has completed.

[0098] The second set of synchronization instructions includes the first sub-instruction "sync.p.io1 1" and the second sub-instruction "sync.v.simd1". The first sub-instruction "sync.p.io1 1" executes within the first time slice and indicates that the producer with identifier "1" is the IO1 stream (i.e., the memory access instruction stream containing the store0 instruction within the first time slice). After this instruction is executed, the hardware synchronization table records "IO1 stream 1" as the producer. The second sub-instruction "sync.v.simd 1" executes before the simd3 instruction in the fourth time slice and indicates that the consumer with identifier "1" is the SIMD stream (i.e., the computation instruction stream containing the simd3 instruction). When this second sub-instruction is executed, the synchronization control unit checks whether the corresponding producer "IO1 stream 1" in the synchronization table has completed. Since store0 in the first time slice has already been completed after the first sub-instruction, the synchronization condition is met, and simd3 can continue execution. Through this set of synchronization instructions, cross-time slice synchronization between the IO1 stream in the first time slice and the SIMD stream in the fourth time slice is achieved, ensuring that simd3 only starts after store0 is completed.

[0099] It is understandable that in the two sets of examples above, the producer and consumer ends indicated by the first and second sub-instructions are separated by two time slices (the second time slice and the third time slice). However, the two sets of synchronization instructions in the illustration are exemplary and not restrictive. For example, the interval between the paired first and second sub-instructions is not limited to two time slices; it can be more or less time slices as needed. Furthermore, the interval between the paired first and second sub-instructions is not limited to the entire time slice; it can also be one or more instructions to be executed. Here, the instructions to be executed can be instructions that have not yet been executed relative to the first sub-instruction. For example, suppose the first sub-instruction "sync.p.simd 0" is still executing within the first time slice, while the second sub-instruction "sync.v.io0 0" can be executed before the load1 instruction in the second time slice. In this case, the first and second sub-instructions are separated by a store0 instruction.

[0100] The ability to space out between the first and second sub-instructions allows the establishment of synchronization dependencies to move beyond the static order of instructions and dynamically span multiple processing stages based on the actual pipeline scheduling, providing a flexible synchronization method for complex multi-buffer and multi-core collaborative scenarios.

[0101] Through the aforementioned pairwise matching synchronization instruction design, this application's embodiments achieve decoupling and cross-cycle management of synchronization dependencies. Compared to traditional single synchronization instructions (such as sync.pio.psimd.cio.csimd) which require producers and consumers to determine and wait immediately at the same time, this scheme splits the synchronization operation into two independent "declare" and "wait" phases. This allows producer instructions to issue synchronization declarations in advance, while consumer instructions wait at any subsequent time (even across multiple time slices), thus providing greater scheduling freedom for deep pipeline optimization. This mechanism is particularly effective in scenarios such as multi-buffered pipelines and multi-core task collaboration, significantly reducing pipeline congestion caused by concentrated synchronization points and further improving processor parallel efficiency and resource utilization.

[0102] Furthermore, the aforementioned pairwise matching synchronization instruction format is not limited to use alone, but can also be used in conjunction with the aforementioned synchronization instruction mechanism for disabling dependencies in the same flow, or a single synchronization instruction mechanism, to achieve fine scheduling within a time slice and cross-cycle synchronization between time slices, thereby constructing a complete multi-level synchronization control system.

[0103] For example, such as Figure 11 As further illustrated, synchronization instructions are inserted between different instruction streams within each time slice. Figure 11Taking the first time slice as an example, this time slice contains three instructions sequentially: load0, simd0, and store0, belonging to the IO0 stream, SIMD stream, and IO1 stream, respectively. To coordinate the execution order of different instruction streams within the same time slice, a synchronization instruction "sync.pio0.csimd" can be inserted between load0 and simd0. This instruction indicates that the IO0 stream is the producer and the SIMD stream is the consumer. Since they belong to different types of instruction streams, the synchronization control unit will maintain their synchronization dependency, ensuring that simd0 can only begin execution after load0 is completed. At the same time, this synchronization instruction itself can benefit from the same-stream dependency decoupling mechanism, meaning that the IO0 stream does not need to wait for other unrelated instructions, but only needs to focus on the cross-stream dependency with the SIMD stream. Similarly, a synchronization instruction "sync.psimd.cio1" can be inserted between simd0 and store0, indicating that the SIMD stream is the producer and the IO1 stream is the consumer, also maintaining the cross-stream dependency. Based on this, if it is necessary to achieve cross-time slice synchronization at the same time, it can be achieved by combining the aforementioned first sub-instruction "sync.p.simd 0" and the second sub-instruction "sync.v.io0 0".

[0104] Through this combined approach, the embodiments of this application achieve two levels of synchronization optimization: within a time slice, the cross-flow dependency of synchronization instructions ensures the correct order of instruction flows within the same time slice, while eliminating unnecessary internal waits through the same-flow dependency decoupling mechanism; between time slices, paired matching synchronization instructions are used to realize cross-cycle dependency declarations and waits, enabling the pipeline to hide memory access latency and computation latency with a deeper degree of parallelism. This multi-level synchronization control mechanism not only ensures the correct execution of the program but also maximizes the potential of instruction-level parallelism, significantly improving the overall performance of the processing core in complex scenarios such as multi-buffer and multi-core collaboration.

[0105] In-core multi-buffered storage optimization scheme Figure 12 An exemplary structural diagram of a data processing apparatus according to some embodiments of this application is shown. Figure 12 As shown, the processing core 610 of the data processing device 600 may include an instruction decoding unit 611, a synchronization control unit 612, a dependency judgment unit 613, an instruction issuing unit 614, a local storage unit 615, and a computing unit 616.

[0106] In some embodiments, the local storage unit 615 may be a high-speed temporary storage memory inside the processing core 610, such as NRAM (Neural Random Access Memory) in a neural network processor, used to store input data and intermediate calculation results that need to be frequently accessed during instruction execution; the computing unit 616 is the core computing component of the processing core 610, such as a vector operation unit and / or a tensor operation unit, used to perform arithmetic and logical operations. To effectively hide memory access latency, the local storage unit 615 is configured with multiple first storage areas, each first storage area used for time-sharing storage of input data blocks and output data blocks of the processing results of the computing unit 616.

[0107] Each first memory region is a storage area partitioned within the local memory unit of the processing core. Its core function is to reuse data over time. That is, within different time slices, the same first memory region can alternately act as an input buffer and an output buffer. Specifically, when the pipeline enters a certain time slice, the first memory region is used to store the input data block to be processed loaded from external memory units (such as DRAM), for the computing unit to read and execute. After the calculation is completed, in the same or subsequent time slices, the first memory region can then store the output data block produced by the computing unit, waiting to be written back to external memory units. Through this time-sharing multiplexing mechanism, multiple first memory regions can work together, allowing data loading, calculation, and result write-back operations to overlap in time, thereby achieving efficient pipeline parallelism with limited on-chip storage resources and effectively hiding memory access latency.

[0108] In processor microarchitecture, memory access latency is one of the key factors affecting performance. When the processing core 610 needs to read data from off-chip memory (such as DRAM), the read request must go through multiple stages, including address transfer, memory bank activation, column access, and data return. The read latency can be as high as hundreds to thousands of nanoseconds, far exceeding the execution time of a single computation operation. If a single-buffer design is adopted, that is, only one primary memory area is used to store data in each time slice, the processing core 610 must wait for all the input data of the current data block to be loaded before it can start computation, and then wait for the result to be written back after the computation is completed. This causes frequent pipeline pauses, and the overall throughput is limited by memory access latency.

[0109] By configuring multiple first storage areas, embodiments of this application achieve overlapping execution of data loading and computation. Taking two first storage areas as an example (e.g.) Figure 12As shown in the first storage areas c01 and c02, while loading data block 01 in the first storage area c02, the computing unit 616 can perform calculations on the ready data block 00 in the first storage area c01. When data block 01 is loaded and data block 00 is calculated, the two switch roles, and the computing unit 616 begins processing data block 01 in the first storage area c02, while simultaneously loading the next data block 02 into the first storage area c01. This process is repeated, resulting in a high degree of temporal overlap between memory access operations and computation operations, effectively hiding most of the pipeline latency. When the number of first storage areas increases to three or more, a deeper pipeline overlap can be achieved, for example, computing data block 02 and writing back data block 01 while loading data block 03, thereby smoothing out fluctuations in memory access latency over a longer time window. Therefore, in some preferred embodiments, the number of first storage areas N≥3.

[0110] The number of primary memory locations directly impacts latency concealment. If the number is too small (e.g., a single buffer), memory access latency is fully exposed, and processing efficiency is dominated by memory access speed. Even with a common double-buffer (ping-pong buffer) design, in some scenarios, excessive latency or insufficient computation time can lead to pipeline cavitation, failing to completely conceal latency. The core idea of ​​a double buffer is to overlap data loading with computation: while data in one buffer is being processed by a computation unit, the other buffer simultaneously loads the next batch of data. Ideally, if the loading time is exactly equal to the computation time, the pipeline can run at full capacity with no idle waiting. However, this balance is broken when memory access latency is too high or computation time is too short. In such scenarios, a double buffer alone cannot provide a sufficient overlap window; more buffers (e.g., triple buffers) are needed to increase the number of overlapping memory access operations, thereby completely covering latency. This is precisely the technical significance of determining the number of buffers using a quantitative formula in the embodiments of this application.

[0111] Therefore, determining the appropriate number of first storage areas based on specific parameters such as memory access latency and computation time has become a pressing technical problem to be solved in this field. Traditional technologies lack systematic quantitative guidance and typically rely on experience or trial and error, making it difficult to achieve an optimal balance between performance and resources in different scenarios. To address this, embodiments of this application further provide quantitative conditions for determining the number N of the first storage areas based on two different scenarios: memory access bottlenecks and computational bottlenecks.

[0112] First, in the memory access bottleneck scenario, that is, the memory access time T of a single first memory area. IO Greater than or equal to the computation time T required for the computing unit to process a single input data block of the first memory area SIMD At this point, the overall throughput of the pipeline is limited by the memory access speed. To completely hide memory access latency, the following conditions must be met: (T IO ×(N-1) - T SIMD )>L, and N is a positive integer (Formula 1); Where N represents the number of the first storage areas, T IO T represents the access time of a single first memory area. SIMD This represents the computation time required to compute a single input data block in the first memory area, where L represents the memory access latency value.

[0113] Among them, T IO The memory access time indicated may include the total time for loading data from off-chip storage to the first storage area and writing the processing result back to off-chip storage, including memory access latency and actual data transfer throughput; T SIMD The computation time represented here refers to the total time required for a computing unit to process a single block of input data in the first memory area, including computation latency (Tcl) and actual computation time (Tct). Computation latency refers to the overhead of computation instruction initiation, pipeline filling, etc., while actual computation time refers to the core time for the arithmetic unit to perform arithmetic logic operations. The memory access latency value represented by L is the time difference from issuing a memory access request to the return of the first piece of data.

[0114] In a pipeline with N first memory areas, memory access operations for all but the first N-1 first memory areas can overlap with computation. Specifically, during the stable operation phase of the pipeline, each computation unit processes data from one first memory area within each time slice, while simultaneously performing data load and write-back operations for N-1 first memory areas. Therefore, the memory access time that can be used to hide latency is (T IO ×(N-1)), minus the time T occupied by the computation itself within the current time slice. SIMD The remaining time is the margin that can be used to cover the memory access latency value L. Only when this margin is greater than L can the memory access latency be completely hidden, and the pipeline will not experience cavitation caused by waiting for data. Therefore, by determining the number N of the first memory area according to Formula 1, the effect of completely hiding the memory access latency can be achieved.

[0115] To facilitate understanding, specific numerical examples are provided below. In one exemplary implementation, the T of a single basic block (the smallest unit of data processed in a single "load-compute-store" loop) IO and T SIMDThe memory access time (T) is related to the available space of local storage units (such as NRAM) and can be quantified based on the resource requirements of specific operators. Assuming the total available space of local storage units is 896KB, the space allocation required by the operators is as follows: the load operation requires 1 storage space, the store operation requires 1 storage space, and the simd operation requires an additional 3 temporary storage spaces. That is, the size of each storage space is x = 896 × 1024 / (N × 2 + 3), where N is the number of first storage areas. Further assuming that each basic block contains 1 load instruction, 1 store instruction, and 5 simd instructions, and that the memory access bandwidth is 115.2 (B / s) and the computation bandwidth is 499.2 (B / s) based on hardware parameters, then the memory access time T for a single basic block is... IO = 2×x / 115.2, the computation time T for a single basic block SIMD = 5×x / 499.2. Substituting the above expression into the delay hiding formula under the memory access bottleneck condition (T IO ×(N-1) -T SIMD Given that N > L, where L is a typical memory access latency of 2200ns, we can calculate that N > 2.77. Since N is a positive integer, N=3 satisfies the inequality, indicating that using three first memory areas can completely hide memory access latency under this parameter configuration. Increasing the number further, for example, to N=4, while still satisfying the condition, would consume valuable on-chip storage resources, increasing hardware cost and power consumption. Therefore, this example clearly demonstrates how to determine the minimum number of first memory areas to meet latency hiding requirements through quantitative calculation based on actual resource constraints and operator characteristics, achieving the optimal balance between performance and resources.

[0116] Furthermore, from the perspective of storage space capacity, under the constraint of a fixed total RAM capacity, if the basic block size required for a dual-buffer configuration exceeds the available resources, the size of a single basic block can be reduced by increasing the number of buffers, thereby satisfying the latency hiding condition within the limited RAM space. Specifically, the minimum local storage unit capacity required to hide latency with two first storage areas is proportional to "memory access latency value L×2 / (memory access time coefficient α1)". The calculation time coefficient α2) is used, and when there are three first memory areas, the capacity is proportional to the memory access latency value L×3 / (2×memory access time coefficient α1). Calculate the time coefficient α2), since 2 / (α1) α2)>3 / (2×α1 Therefore, the three first storage areas have lower requirements for local storage unit capacity. Here, the memory access time coefficient α1 represents the memory access time per unit of data (i.e., the ratio of memory access time to data volume), and the computation time coefficient α2 represents the computation time per unit of data (i.e., the ratio of computation time to data volume). Both are determined by parameters such as hardware bandwidth and computing power. This indicates that when RAM resources are scarce, using more buffers (such as triple buffers) can achieve the same or even better latency hiding effect as double buffers with a smaller single basic block size. Furthermore, through a "trading quantity for capacity" design strategy, memory access latency can still be effectively hidden under resource-constrained conditions.

[0117] Secondly, in the computational bottleneck scenario, that is, the computation time T required for a computing unit to process a single input data block of the first storage area. SIMD The memory access time T is greater than that of a single first memory area. IO At this point, the overall throughput of the pipeline is limited by the computation speed. The hidden nature of memory access latency can be seen in [reference needed]. Figure 13a and Figure 13b .

[0118] Figure 13a A pipeline diagram of two first memory regions under computational bottleneck is shown. Figure 13a As shown, ld0 (ld is short for load), simd0, and st0 (st is short for store) process the first data block; ld1, simd1, and st1 process the second data block; and ld2, simd2, and st2 (not shown) process the third data block.

[0119] With two first storage areas configured, assuming first storage areas c01 and c02, the first data block is loaded in the first storage area c01 (i.e., ld0 operation); while the second data block is loaded in the second storage area c02 (i.e., ld1 operation), the computing unit can perform calculations on the first data block that is ready in the first storage area c01 (i.e., simd0 operation); when the second data block is loaded and the first data block is calculated, the two switch roles, and the computing unit starts processing the second data block in the first storage area c02 (i.e., simd1 operation) while storing the first data block (i.e., st0 operation), and so on.

[0120] It can be seen that in the next round, since ld2 needs to load the third data block into the first storage area c01, it can only start after the first data block in the first storage area c01 has been processed. Therefore, there is an address dependency between ld2 and simd0, and ld2 cannot start prematurely. Similarly, since simd2 needs ld2 to load the data before it can use that data for computation, there is a data dependency between ld2 and simd2, and simd2 cannot start prematurely. Because simd0 and simd2 are only separated by one T... SIMD Therefore, when the simd time of a single time slice is insufficient to cover T load When the memory access latency value is L (i.e., 2200ns in the diagram), cavitation bubbles will be generated as shown in the diagram, causing the computing unit to idle and wait.

[0121] Figure 13b A schematic diagram of a pipeline with three first memory regions under computational bottleneck is shown. Figure 13a The difference shown is that, Figure 13b There are two T intervals between simd0, which has an address dependency with ld3, and simd3, which has a data dependency with ld3. SIMD In other words, with three primary memory areas configured, two SIMD time slices can be used to mask memory access latency (i.e., 2200ns in the diagram), thereby mitigating the exposure of memory access latency.

[0122] Therefore, an appropriate number of primary storage areas need to be configured to effectively hide memory access latency. Under computational bottlenecks, the following conditions must be met to completely hide memory access latency: (T SIMD ×(N-1) -T load )>L, and N is a positive integer (Formula 2); Among them, T SIMD Similarly, T represents the computation time required to compute a single input data block in the first memory area. load This represents the loading time required to load an input data block into a single first storage area, where L represents the memory access latency value.

[0123] The principle of hiding latency is similar to that of memory access bottlenecks. Under computational bottlenecks, the object to be hidden is still the memory access latency value L. In a pipeline with N first memory areas, the computation time of the remaining N-1 first memory areas can overlap with the memory access operations of subsequent first memory areas, except for the first first memory area. Specifically, the computation time that can be used to hide memory access latency is (T SIMDThe remaining time, after subtracting the time occupied by the loading operation itself within the current time slice from the time of ×(N-1), is the margin that can be used to cover the memory access latency value L. When this margin is greater than L, it means that the loading operation can be completed while the computing unit is processing the preceding data block, and the computing unit will not be idle due to data not being ready.

[0124] It should be noted that a larger number of first memory regions is not always better. While increasing N can smooth latency over a longer time window, an excessively large N will consume more on-chip storage resources, increasing chip area and power consumption, and may introduce new performance overhead due to factors such as memory conflicts and wiring delays. Therefore, this embodiment calculates the minimum N value that can completely hide latency using the above formula, thereby achieving the best balance between performance and resources. In practical design, the N value can be adjusted according to the specific operator T. IO T SIMD T load Parameters such as L are substituted into the above formula to obtain a positive integer N that satisfies the conditions, and the minimum value among them is taken as the configuration basis. This quantitative design method avoids the latency exposure caused by insufficient first storage area and prevents the waste of resources caused by excessive buffering, providing scientific and effective guidance for high-performance processor design.

[0125] By designing multiple first storage areas as described above, the embodiments of this application can transform the originally serial data loading, computation, and write-back operations into parallel pipelined operations, significantly reducing the impact of memory access latency on processing efficiency. Simultaneously, by optimizing the number of first storage areas according to the actual bottleneck type and latency parameters, efficient resource utilization is achieved, improving the overall throughput and energy efficiency of the processing cores, and providing strong hardware support for high-performance computing scenarios.

[0126] Inter-core synchronization optimization scheme In the foregoing embodiments, the present application mainly focuses on the synchronization optimization within a single processing core. By using a synchronization control unit to eliminate unnecessary synchronization dependencies within the same instruction stream, it improves the efficiency of intra-core instruction-level parallelism. In multi-core processor architectures, multiple processing cores often need to collaborate to complete more complex tasks. For example, one processing core might be responsible for data loading, while multiple cores handle parallel computation. This cross-core data flow and task dependency also requires an efficient synchronization mechanism to ensure correctness and performance. Therefore, the present application further extends synchronization management capabilities from intra-core to inter-core by introducing cluster and inter-core synchronization units to achieve fine-grained synchronization control in multi-core scenarios. The following will combine... Figure 14 An illustrative example is provided.

[0127] Figure 14 An exemplary structural diagram of a data processing apparatus including at least one cluster, according to some embodiments of this application, is shown. Figure 14As shown, the data processing device 600 may include at least one cluster 620, each cluster 620 includes one or more processing cores 610, and each cluster 620 is configured with an inter-core synchronization unit 621. The inter-core synchronization unit 621 may be configured to independently maintain the synchronization dependency between the producer core and the consumer core indicated by the semaphore instruction in response to the semaphore instruction.

[0128] Cluster 620 is a hardware organization unit situated between a single processing core and the entire data processing device. For example, in a neural network processor, a cluster (or core cluster) can consist of multiple IPU cores (intelligent processing units) and one MPU core (main processing unit), sharing some storage resources (such as SRAM) and collaboratively completing data transfer and computation tasks. Within each cluster 620, an inter-core synchronization unit 621 is configured. This unit is independent of the synchronization control unit within the aforementioned processing core and is specifically responsible for handling the synchronization dependencies between different processing cores.

[0129] Semaphore instructions are specialized instructions used for inter-core synchronization. Examples include send instructions (such as `semaphore.send`) and wait instructions (such as `semaphore.wait`). Each semaphore instruction carries an identifier specifying the target semaphore resource. The producer core refers to the processing core that executes the semaphore send instruction, typically indicating that the core has completed a certain stage of its task (such as data loading). The consumer core refers to the processing core that executes the semaphore wait instruction, indicating that the core needs to wait for the producer core to complete before it can continue execution. The inter-core synchronization unit 621 is responsible for maintaining a count value for each semaphore resource and determining the waiting and waking-up of the consumer core based on this count value: when the producer core executes the send instruction, the semaphore count is incremented by 1; when the consumer core executes the wait instruction, if the count is greater than 0, it is decremented by 1 and the consumer core is allowed to continue execution; if the count is equal to 0, the consumer core is blocked until the count is incremented by another core.

[0130] Compared to traditional barrier synchronization mechanisms, semaphore synchronization offers significant advantages. Barrier mechanisms require all participants (typically a group of processing cores) to reach the synchronization point before allowing access. This many-to-one synchronization method easily leads to mutual waiting in multi-buffered pipelines, and the order in which barrier instructions are issued can block subsequent instructions. Semaphore mechanisms, on the other hand, achieve true one-to-one synchronization: producer and consumer cores establish a direct dependency relationship. Consumer cores only need to wait for their matched producer core, without waiting for other unrelated cores. For example, in a scenario where an MPU core collaborates with multiple IPU cores, after the MPU core loads data, it sends a semaphore command to notify all IPU cores. Each IPU core independently waits for the semaphore, and once the semaphore is available (i.e., the count is greater than 0), it can begin its own computation without waiting for others. This independently maintained synchronization method allows multiple consumer cores to retrieve data from the producer core in parallel and asynchronously, greatly improving the throughput of multi-core pipelines.

[0131] By utilizing the inter-core synchronization unit 621 to respond to and maintain semaphore instructions, this embodiment of the application achieves flexible and efficient synchronization control in multi-core scenarios. Working together with the intra-core synchronization control unit 612, it forms a complete synchronization optimization system from within a single processing core to between multiple cores within a cluster: the intra-core synchronization control unit enhances single-core instruction-level parallelism by eliminating co-flow dependencies, while the inter-core synchronization unit achieves asynchronous pipelined data flow between cores through a semaphore mechanism. The two work in concert to effectively hide cross-core memory access latency and computational wait time, providing strong hardware support for high-performance multi-core processor design.

[0132] To facilitate understanding, the following will be combined with... Figure 15a and Figure 15b An exemplary description is provided.

[0133] Figure 15a A schematic diagram illustrating the synchronization effect of multiple processing cores under the barrier synchronization mechanism is shown. Figure 15b A schematic diagram illustrating the synchronization effect of multiple processing cores under the semaphore mechanism of this application is shown. For example... Figure 15a and Figure 15b As shown, each row represents an operation within a processing core, and each color block represents a group of operations within a time slice. I / O represents memory access operations, MV represents move operations (such as moving data from shared memory to local memory within the corresponding processing core), and vertical lines indicate synchronization dependencies. For ease of display, some operations within a time slice have been omitted, such as computation operations.

[0134] like Figure 15aAs shown, under the barrier synchronization mechanism, each IPU core (IPU0, IPU1, IPU2, and IPU3) not only needs to wait for the MPU core to complete data loading, but also needs to wait for all other IPU cores to reach the synchronization point before it can be released uniformly. This many-to-one synchronization method causes a large number of cavitation bubbles to be generated in the pipeline.

[0135] like Figure 15b As shown, under the semaphore mechanism of this embodiment, each IPU core only needs to wait for the MPU core to complete data loading, without needing to wait for all other IPU cores to reach the synchronization point. This one-to-one synchronization method effectively eliminates the latency caused by inter-core waiting for each other. Meanwhile, as... Figure 15b As further shown, the IPU1 core can also execute the core synchronization instructions of this application embodiment, which removes the synchronization dependency relationship within the same stream and further improves data processing efficiency.

[0136] Inter-core multi-buffer storage optimization scheme In the aforementioned embodiments, this application implements one-to-one semaphore synchronization between multiple cores through an inter-core synchronization unit, enabling producer cores and consumer cores to work asynchronously and effectively alleviating the waiting latency caused by inter-core task dependencies. Simultaneously, in multi-core collaborative data flow scenarios, the data itself needs to flow efficiently between multiple storage layers. For example, in a typical architecture where the MPU core is responsible for loading data from off-chip storage (such as DDR) and multiple IPU cores are responsible for computation and writing back the results, data needs to pass through three layers sequentially: off-chip storage, shared storage (such as SRAM), and local storage (such as NRAM). If the shared storage unit is configured with only a single buffer, each load must wait for the previous round of data to be completely consumed before the next round of loading can begin, fully exposing the off-chip memory access latency and forming a new performance bottleneck. Therefore, this application further configures a shared storage unit within the cluster and hides the off-chip memory access latency through the design of multiple second storage areas.

[0137] return Figure 14 As further shown, each cluster 620 may also be configured with a shared memory unit 622, which is an on-chip cache that can be accessed by multiple processing cores 610, such as SRAM (Static Random Access Memory) in a neural network processor. The shared memory unit 622 is configured with multiple second memory areas (such as d01, d02, d03, etc. in the figure), each second memory area is used to temporarily store data loaded from off-chip memory units for reading by each processing core in the cluster.

[0138] Similar to the aforementioned design of multiple first storage areas for local storage units, the number M of second storage areas also needs to be quantitatively determined based on latency hiding requirements. The calculation formula is as follows: (M 1)×T load T move >L, and M is a positive integer (Formula 3); Among them, T load This represents the time required to load data from an off-chip memory location to a shared memory location in a single operation. This time includes the data transfer time on the off-chip bus and the memory access time. move This represents the time required to move data from a shared memory unit to the processing core's local memory unit in a single operation, i.e., the time required for data to be transferred from shared memory to local memory in the on-chip interconnect (including the data transfer time for the move operation); L represents the memory access latency value.

[0139] In a pipeline design with M second memory areas, the loading time of the remaining M-1 second memory areas (excluding the first) can overlap with the data movement operations of subsequent second memory areas. Specifically, the total time available to hide off-chip memory access latency is (M 1)×T load Subtract the time T occupied by the data movement within the current time slice. move Then, the remaining time is the margin that can be used to cover the off-chip memory access latency L. When this margin is greater than L, it means that data movement operations can be interspersed in the gaps between the completion of loading operations, without causing the pipeline to stop waiting for off-chip data.

[0140] Through the aforementioned design of multiple second storage areas within shared storage units, this embodiment effectively hides off-chip memory access latency at the cluster level. Similar to the aforementioned design of multiple first storage areas within local storage in the processing core, a larger number of second storage areas is not necessarily better. In practical design, the number can be determined based on the specific operator's T... load T move Parameters such as L are substituted into the formula to obtain a positive integer M that satisfies the conditions, and the minimum value among them is taken as the configuration basis. This quantitative design method avoids the exposure of latency caused by insufficient second storage area and prevents the waste of resources caused by excessive buffering.

[0141] Furthermore, based on the aforementioned design of the second storage area, for multi-level storage architectures such as four-stage or five-stage pipelines, the number N of the first storage area of ​​each processing core's local storage unit can adopt a simplified design scheme, that is, directly configured as 2 (double buffer), or the same as the number of the second storage area. This is because the latency L′ (i.e., the movement latency) of moving data from shared storage to local storage is usually much smaller than the latency L of loading data from off-chip storage to shared storage, and the data volume of the movement operation itself is small and the transmission path is short. Therefore, even if a simple double-buffer ping-pong structure is adopted, or an easily configurable number is used, the movement latency can be hidden in the busy time of the computing unit in most scenarios.

[0142] By flexibly configuring the number N of the first storage area described above, this embodiment of the application achieves latency hiding optimization at the local storage level. When N=2, it meets the needs of most scenarios with minimal hardware overhead; when N is equal to M, it helps to simplify configuration complexity and scheduling logic. This design complements the multi-second storage area design of the shared storage unit in a hierarchical manner. The combination of the two, along with inter-core semaphore synchronization and intra-core synchronization optimization, jointly constructs a full-link latency hiding system from off-chip storage to on-chip computing, significantly improving the overall performance and resource utilization efficiency of multi-core processors in complex data flow scenarios.

[0143] To facilitate understanding, the following will combine... Figure 16 An exemplary description is provided.

[0144] like Figure 16 As shown, in a four-stage pipeline design (load→move→compute→store, i.e., ld→mv→simd→st in the diagram), the stage of loading data from off-chip memory (such as DRAM) to shared memory (such as SRAM) is the main source of memory access latency. Therefore, it is necessary to hide this latency by configuring multiple second memory areas (i.e., SRAM buffers). The core dependency relationship in this stage is: with the configuration of M second memory areas, the i-th load operation depends on the completion of the iM-th move operation (i.e., the memory area can only be reused after the data loaded in the previous M operations has been moved), and the iM-th move operation in turn depends on the completion of the iM-th load operation (i.e., the move operation needs to wait for the corresponding data to be loaded and ready), thus forming a circular dependency.

[0145] Figure 16 In the four-stage pipeline design (abbreviated as ld, mv, simd, st in the diagram), three buffers are configured (represented by 0, 1, and 2 respectively). That is, when M=3, the load operation can cycle between the three secondary storage areas (i.e., ld0, ld1, ld2 in a loop), making data loading and data movement highly overlapped in time, effectively hiding the off-chip memory access latency (i.e., the memory access latency value L in Formula 3).

[0146] The move phase moves data from shared memory to local memory. Here, the latency of the load operation doesn't need to be considered; only the latency of the move or SIMD operation needs to be taken into account. For example... Figure 16As shown, if the local storage unit is configured with two first storage areas, the time from the start of mv2 to the completion of simd0 needs to be greater than the mv latency (i.e., the move latency value L′) to mask the move latency. In other words, the condition for hiding the move latency is that the difference between the start time of the move operation and the completion time of the previous round of calculation is greater than the move latency mv Latency. Meanwhile, the start time of mv2 is also affected by the load. In this example, because the move time is short, even if it is exposed, it is much less than the load time. Therefore, the number of buffers in the local storage can be decoupled from the number of buffers in the shared storage. For example, a combination of three buffers in the shared storage and two buffers in the local storage can also meet the requirements.

[0147] Of course, it is also possible to expand the number of the first storage area to the same number as the second storage area, for example... Figure 16 As mentioned above, configuring the number of each storage area uniformly at the operational level will make it easier to operate.

[0148] The processing logic of the five-stage pipeline is similar to that of the four-stage pipeline. The core optimization still focuses on the large latency of the loading stage. The minimum number of buffers that meet the latency hiding requirements can be calculated through the above quantification conditions. The latency of other stages (such as storage, computation, and movement) is relatively small and does not need to be the main optimization target.

[0149] Furthermore, in multi-buffered designs, address alignment of memory cells is a crucial implementation detail for ensuring efficient parallel access in hardware. Local memory cells (such as NRAM) and shared memory cells (such as SRAM) typically consist of multiple independent banks, each with its own address decoder, read / write driver, and data port, enabling parallel responses to access requests from different banks. When software stores data in blocks across different first or second memory areas, if the starting addresses of each memory area are aligned with the boundaries of the memory banks, the hardware can efficiently route access requests from different memory areas to different memory banks, achieving truly parallel data loading and computation. Conversely, if addresses are not aligned, multiple memory areas may map to the same memory bank, causing bank conflicts and forcing access requests to be serialized, thus diminishing the parallel advantages of multi-buffered designs.

[0150] In some embodiments, the addresses of N first memory areas are aligned to a preset byte; and / or the addresses of M second memory areas are aligned to a preset byte. Address alignment to a preset byte means that the starting address of each first or second memory area is an integer multiple of the preset alignment value. This alignment value is typically determined based on the data width and bus width of the memory bank. For example, in a processor with a 16-byte (128-bit) data bus, aligning the starting address of the memory area to 16 bytes ensures that each memory area occupies several consecutive rows of memory, avoiding access overhead across memory bank boundaries.

[0151] Taking local storage units as an example, assuming each first storage area is 64KB in size and the default alignment value is 16B, then the starting addresses of each storage area are 0x0000, 0x10000, 0x20000, and so on, all of which are integer multiples of 16. This alignment method allows the hardware to quickly determine the target storage device through the high-order bits of the address and distribute access requests from different storage areas to different storage devices in parallel, achieving conflict-free concurrent access.

[0152] Through the address alignment design described above, the embodiments of this application can significantly improve the actual parallel efficiency of the multi-buffer pipeline. When the addresses of each memory area are aligned to a preset byte, the probability of memory conflicts is greatly reduced, the overlapping execution of data loading and computation is more sufficient, and the memory access latency hiding effect is closer to the theoretical model. Conversely, if the addresses are not aligned, frequent memory conflicts may lead to actual performance lower than expected performance. Therefore, address alignment, as an auxiliary means of multi-buffer hardware implementation, combined with the quantitative design of the number of buffers, is conducive to achieving a more effective and complete latency hiding scheme, ensuring that the processor can stably obtain the expected performance improvement in actual operation.

[0153] This application also provides a chip that may include the data processing apparatus of any of the embodiments described above in conjunction with the accompanying drawings. Furthermore, this application also provides a board that may include the aforementioned chip. In some embodiments, the off-chip storage units described herein may be disposed on the chip and / or on the board.

[0154] This application also provides a data processing method implemented by a data processing device, wherein the data processing device includes a processing core, and the data processing method includes: in response to a synchronization instruction, determining the production end and consumer end that need to be synchronized as indicated by the synchronization instruction; in response to the producer in the production end and the consumer in the consumer end belonging to the same instruction stream, releasing the synchronization dependency between the producer and the consumer in the same instruction stream; and in response to the producer in the production end and the consumer in the consumer end belonging to different instruction streams, maintaining the synchronization dependency between the different instruction streams.

[0155] In some embodiments, the data processing method further includes: determining whether there is a data dependency between adjacent instructions based on whether the access address ranges of adjacent instructions within the same instruction stream overlap; and allowing the latter instruction in the adjacent instructions to start transmitting before the former instruction completes in response to the absence of a data dependency between adjacent instructions.

[0156] In other embodiments, the synchronization instruction includes a pair of matching first and second sub-instructions, the first sub-instruction being used to instruct the producer end that forms a synchronization dependency, and the second sub-instruction being used to instruct the corresponding consumer end.

[0157] In some other embodiments, the production end indicated by the first sub-instruction and the consumption end indicated by the second sub-instruction are separated by at least one instruction to be executed or at least one time slice, which is the cycle required for the processing core to complete a set of predetermined operations.

[0158] In some embodiments, the aforementioned instruction stream includes a memory access instruction stream and a computation instruction stream.

[0159] In other embodiments, the aforementioned instruction stream includes a computation instruction stream, which includes multiple computation instructions. The data processing method further includes: splitting each computation instruction into multiple micro-operations; determining whether there is a data dependency relationship between two adjacent computation instructions based on whether the data address ranges accessed by at least the last micro-operation of the preceding computation instruction and at least the first micro-operation of the following computation instruction overlap; and allowing the following computation instruction to start transmitting before the last micro-operation of the preceding computation instruction is completed if there is no data dependency relationship between at least the last micro-operation and at least the first micro-operation.

[0160] In some other embodiments, the processing core further includes a local storage unit and a computing unit, wherein the local storage unit is configured with a plurality of first storage areas, each first storage area being used to time-division store input data blocks and output data blocks of the processing results of the computing unit.

[0161] In some embodiments, when the access time of a single first storage area is greater than or equal to the computation time required by the computing unit to compute the input data block of a single first storage area, the number N of first storage areas satisfies: (T IO ×(N-1) - T SIMD T > L, and N is a positive integer; where T IO T represents the access time of a single first memory area. SIMD This represents the computation time required to compute a single input data block in the first memory area, where L represents the memory access latency value.

[0162] In other embodiments, when the computation time required for a computing unit to compute an input data block in a single first storage area is greater than the memory access time of a single first storage area, the number N of first storage areas satisfies: (T SIMD ×(N-1) - T load T > L, and N is a positive integer; where T SIMD T represents the computation time required to compute a single input data block in the first memory area. load This represents the loading time required to load an input data block into a single first storage area, where L represents the memory access latency value.

[0163] In some other embodiments, the number of first storage areas N ≥ 3.

[0164] In some embodiments, the data processing apparatus includes at least one cluster, each cluster including one or more processing cores, and the data processing method further includes: independently maintaining the synchronization dependency between the producer core and the consumer core indicated by the semaphore instruction in response to the semaphore instruction.

[0165] In other embodiments, each cluster is further configured with a shared storage unit, which is configured with multiple second storage areas, the number M of which satisfies: (M 1)×T load T move >L, and M is a positive integer; where T load T represents the time required to load data from an off-chip memory location to a shared memory location in a single operation. move L represents the time it takes to move data from a shared storage unit to a local storage unit in a single transaction, where L represents the memory access latency value.

[0166] In some other embodiments, the number N of the first storage area for each processing core is 2, or the same as the number M of the second storage area.

[0167] In some other embodiments, the data processing method further includes: aligning the addresses of N first storage areas by a preset byte; and / or aligning the addresses of M second storage areas by a preset byte.

[0168] Those skilled in the art will understand that the description of the data processing apparatus of the present application in conjunction with the accompanying drawings can also be applied to the data processing method of the present application, and therefore will not be repeated.

[0169] It should be noted that, for the sake of brevity, this application describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solution of this application is not limited to the order of the described actions. Therefore, based on the disclosure or teachings of this application, those skilled in the art will understand that some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this application can be considered as optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions of this application. In addition, depending on the solution, the description of some embodiments in this application also has different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this application can also be referred to the relevant descriptions of other embodiments.

[0170] In terms of specific implementation, based on the disclosure and teachings of this application, those skilled in the art will understand that the several embodiments disclosed in this application can also be implemented in other ways not disclosed herein. For example, regarding the various units in the device or apparatus embodiments described above, this document divides them based on logical functions, but in actual implementation, there may be other division methods. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection utilizing an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0171] In this application, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this application. Additionally, in some scenarios, multiple units in the embodiments of this application may be integrated into one unit or each unit may exist physically independently.

[0172] The foregoing can be better understood in accordance with the following terms: Clause A1. A data processing apparatus includes a processing core, the processing core including an instruction decoding unit and a synchronization control unit, wherein the instruction decoding unit is configured to, in response to a synchronization instruction, determine the production end and consumer end to be synchronized as indicated by the synchronization instruction; the synchronization control unit is configured to, in response to the producer in the production end and the consumer in the consumer end belonging to the same instruction stream, release the synchronization dependency between the producer and the consumer within the same instruction stream; and in response to the producer in the production end and the consumer in the consumer end belonging to different instruction streams, maintain the synchronization dependency between the different instruction streams.

[0173] Clause A2. The data processing apparatus according to Clause A1, wherein the processing core further includes: a dependency determination unit configured to determine whether there is a data dependency relationship between adjacent instructions based on whether the access address ranges of adjacent instructions within the same instruction stream overlap; and an instruction issuing unit configured to allow the subsequent instruction in the adjacent instructions to be issued before the previous instruction is completed in response to the absence of a data dependency relationship between the adjacent instructions.

[0174] Clause A3. The data processing apparatus according to Clause A1, wherein the synchronization instruction includes a pair of matching first sub-instructions and second sub-instructions, the first sub-instruction being used to instruct the producer end forming the synchronization dependency, and the second sub-instruction being used to instruct the corresponding consumer end.

[0175] Clause A4. The data processing apparatus according to Clause A3, wherein there is at least one instruction to be executed or at least one time slice between the production end indicated by the first sub-instruction and the consumption end indicated by the second sub-instruction, the time slice being the cycle required for the processing core to complete a predetermined set of operations.

[0176] Clause A5. A data processing apparatus according to any one of Clauses A1-A4, wherein the instruction stream includes a memory access instruction stream and a computation instruction stream.

[0177] Clause A6. The data processing apparatus according to Clause A2, wherein the instruction stream includes a computation instruction stream, the computation instruction stream including a plurality of computation instructions; the instruction decoding unit is further configured to decompose each of the computation instructions into a plurality of micro-operations; the dependency determination unit is further configured to determine whether there is a data dependency relationship between two adjacent computation instructions based on whether the data address ranges accessed by at least the last segment of the preceding computation instruction and at least the first segment of the following computation instruction overlap; the instruction issuing unit is further configured to allow the following computation instruction to begin issuing before the completion of the last segment of the preceding computation instruction in response to the absence of a data dependency relationship between the at least last segment of the preceding computation instruction and the at least first segment of the preceding computation instruction.

[0178] Clause A7. The data processing apparatus according to Clause A1, wherein the processing core further includes a local storage unit and a computing unit, wherein the local storage unit is configured with a plurality of first storage areas, each first storage area being used for time-sharing storage of input data blocks and output data blocks of the processing results of the computing unit.

[0179] Clause A8. The data processing apparatus according to Clause A7, wherein when the access time of a single first storage area is greater than or equal to the computation time required by the computing unit to compute the input data block of a single first storage area, the number N of the first storage areas satisfies: (T IO ×(N-1) - T SIMD T > L, and N is a positive integer; where T IO T represents the access time of a single first memory area. SIMD This represents the computation time required to compute a single input data block in the first memory area, where L represents the memory access latency value.

[0180] Clause A9. The data processing apparatus according to Clause A7, wherein when the computation time required by the computing unit to compute an input data block of a single first storage area is greater than the memory access time of a single first storage area, the number N of the first storage areas satisfies: (T SIMD ×(N-1) - T load T > L, and N is a positive integer; where T SIMD T represents the computation time required to compute a single input data block in the first memory area. load This represents the loading time required to load an input data block into a single first storage area, where L represents the memory access latency value.

[0181] Clause A10, a data processing apparatus according to any one of clauses A7-A9, wherein the number N of the first storage areas is greater than or equal to 3.

[0182] Clause A11. The data processing apparatus according to Clause A7 includes at least one cluster, each cluster including one or more of the processing cores, and each cluster is configured with an inter-core synchronization unit, the inter-core synchronization unit being configured to: independently maintain the synchronization dependency between the producer core and the consumer core indicated by the semaphore instruction in response to the semaphore instruction.

[0183] Clause A12. The data processing apparatus according to Clause A11, wherein each cluster is further configured with a shared storage unit, the shared storage unit being configured with a plurality of second storage areas, the number M of the second storage areas satisfying: (M 1)×T load T move >L, and M is a positive integer; where T load T represents the time required to load data from an off-chip memory location to a shared memory location in a single operation. move L represents the time it takes to move data from a shared storage unit to a local storage unit in a single transaction, where L represents the memory access latency value.

[0184] Clause A13. The data processing apparatus according to Clause A12, wherein the number N of the first storage areas of each processing core is 2, or the same as the number M of the second storage areas.

[0185] Clause A14. The data processing apparatus according to any one of Clauses A11-A13, wherein the addresses of N first storage areas are aligned to a preset byte; and / or the addresses of M second storage areas are aligned to a preset byte.

[0186] Clause A15, a chip, comprising, according to Clause A1 A14 Any of the data processing apparatuses described above.

[0187] Clause A16, a board including the chip described in Clause A15.

[0188] Clause A17. A data processing method implemented by a data processing apparatus, wherein the data processing apparatus includes a processing core, the data processing method comprising: in response to a synchronization instruction, determining a producer end and a consumer end to be synchronized as indicated by the synchronization instruction; in response to a producer in the producer end and a consumer in the consumer end belonging to the same instruction stream, releasing the synchronization dependency between the producer and the consumer within the same instruction stream; and in response to a producer in the producer end and a consumer in the consumer end belonging to different instruction streams, maintaining the synchronization dependency between the different instruction streams.

[0189] Clause A18, the data processing method according to Clause A17, further includes: determining whether there is a data dependency relationship between adjacent instructions based on whether the access address ranges of adjacent instructions within the same instruction stream overlap; and, in response to the absence of a data dependency relationship between the adjacent instructions, allowing the latter instruction in the adjacent instructions to begin transmission before the former instruction is completed.

[0190] Clause A19, the data processing method according to Clause A17, wherein the synchronization instruction includes a pair of matching first sub-instructions and second sub-instructions, the first sub-instruction being used to instruct the producer end forming the synchronization dependency, and the second sub-instruction being used to instruct the corresponding consumer end.

[0191] Clause A20, the data processing method according to Clause A19, wherein there is at least one instruction to be executed or at least one time slice between the production end indicated by the first sub-instruction and the consumption end indicated by the second sub-instruction, the time slice being the cycle required for the processing core to complete a set of predetermined operations.

[0192] Clause A21. A data processing method according to any one of Clauses A17-A20, wherein the instruction stream includes a memory access instruction stream and a computation instruction stream.

[0193] Clause A22. The data processing method according to Clause A18, wherein the instruction stream includes a computation instruction stream, the computation instruction stream includes a plurality of computation instructions, and the data processing method further includes: splitting each computation instruction into a plurality of micro-operations; determining whether there is a data dependency relationship between two adjacent computation instructions based on whether the data address ranges accessed by at least the last segment of the preceding computation instruction and at least the first segment of the following computation instruction overlap; and allowing the following computation instruction to begin transmission before the completion of the last segment of the preceding computation instruction if there is no data dependency relationship between the at least last segment of the micro-operation and the at least first segment of the micro-operation.

[0194] Clause A23. The data processing method according to Clause A17, wherein the processing core further includes a local storage unit and a computing unit, wherein the local storage unit is configured with a plurality of first storage areas, each first storage area being used for time-sharing storage of input data blocks and output data blocks of the processing results of the computing unit.

[0195] Clause A24. According to the data processing method described in Clause A23, when the access time of a single first storage area is greater than or equal to the computation time required by the computing unit to compute the input data block of a single first storage area, the number N of the first storage areas satisfies: (T IO ×(N-1) - T SIMD T > L, and N is a positive integer; where T IO T represents the access time of a single first memory area. SIMD This represents the computation time required to compute a single input data block in the first memory area, where L represents the memory access latency value.

[0196] Clause A25. According to the data processing method described in Clause A23, when the computation time required by the computing unit to compute a single input data block of a first storage area is greater than the access time of a single first storage area, the number N of the first storage areas satisfies: (T SIMD ×(N-1) - T load T > L, and N is a positive integer; where T SIMD T represents the computation time required to compute a single input data block in the first memory area. load This represents the loading time required to load an input data block into a single first storage area, where L represents the memory access latency value.

[0197] Clause A26, the data processing method according to any one of clauses A23-A25, wherein the number N of the first storage areas is ≥3.

[0198] Clause A27. The data processing method according to Clause A23, wherein the data processing apparatus includes at least one cluster, each cluster including one or more of the processing cores, and the data processing method further includes: independently maintaining the synchronization dependency between the producer core and the consumer core indicated by the semaphore instruction in response to a semaphore instruction.

[0199] Clause A28. According to the data processing method described in Clause A27, each cluster is further configured with a shared storage unit, said shared storage unit being configured with multiple second storage areas, the number M of which satisfies: (M 1)×T load T move >L, and M is a positive integer; where T loadT represents the time required to load data from an off-chip memory location to a shared memory location in a single operation. move L represents the time it takes to move data from a shared storage unit to a local storage unit in a single transaction, where L represents the memory access latency value.

[0200] Clause A29. The data processing method according to Clause A28, wherein the number N of the first storage area for each processing core is 2, or the same as the number M of the second storage area.

[0201] Clause A30, the data processing method according to any one of Clauses A27-A29, further includes: aligning the addresses of N first storage areas by preset bytes; and / or aligning the addresses of M second storage areas by preset bytes.

[0202] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A data processing apparatus, comprising a processing core, the processing core including an instruction decoding unit and a synchronization control unit, wherein, The instruction decoding unit is configured to respond to a synchronization instruction and determine the production end and consumer end that need to be synchronized as indicated by the synchronization instruction. The synchronization control unit is configured to, in response to the fact that the producer in the production end and the consumer in the consumption end belong to the same instruction stream, release the synchronization dependency between the producer and the consumer in the same instruction stream; And in response to the fact that the producer in the production end and the consumer in the consumption end belong to different instruction streams, the synchronous dependency relationship between the different instruction streams is maintained.

2. The data processing apparatus according to claim 1, wherein the processing core further comprises: The dependency determination unit is configured to determine whether there is a data dependency relationship between adjacent instructions based on whether the access address ranges of adjacent instructions within the same instruction stream overlap. The instruction issuing unit is configured to allow the subsequent instruction to be issued before the preceding instruction is completed, in response to the absence of data dependency between the adjacent instructions.

3. The data processing apparatus according to claim 1, wherein the synchronization instruction includes a first sub-instruction and a second sub-instruction that are matched in pairs, the first sub-instruction being used to instruct the production end that forms the synchronization dependency relationship, and the second sub-instruction being used to instruct the corresponding consumer end.

4. The data processing apparatus according to claim 3, wherein the production end indicated by the first sub-instruction and the consumption end indicated by the second sub-instruction are spaced apart by at least one instruction to be executed or at least one time slice, the time slice being the cycle required for the processing core to complete a set of predetermined operations.

5. The data processing apparatus according to any one of claims 1-4, wherein the instruction stream includes a memory access instruction stream and a computation instruction stream.

6. The data processing apparatus according to claim 2, wherein the instruction stream includes a calculation instruction stream, and the calculation instruction stream includes a plurality of calculation instructions; The instruction decoding unit is further configured to break down each calculation instruction into multiple micro-operations; The dependency determination unit is further configured to determine whether there is a data dependency relationship between two adjacent computation instructions based on whether the data address ranges accessed by at least the last segment of the micro-operation of the preceding computation instruction and at least the first segment of the micro-operation of the following computation instruction overlap. The instruction issuing unit is further configured to allow the subsequent calculation instruction to be issued before the completion of the last micro-operation of the preceding calculation instruction, in response to the absence of a data dependency between the at least last micro-operation and the at least first micro-operation.

7. The data processing apparatus according to claim 1, wherein the processing core further includes a local storage unit and a computing unit, wherein the local storage unit is configured with a plurality of first storage areas, each first storage area being used for time-sharing storage of input data blocks and output data blocks of the processing results of the computing unit.

8. The data processing apparatus according to claim 7, wherein when the access time of a single first storage area is greater than or equal to the computation time required by the computing unit to compute the input data block of a single first storage area, the number N of the first storage areas satisfies: (T IO ×(N-1) - T SIMD The condition is that N > L, and N is a positive integer; in, T IO T represents the access time of a single first memory area. SIMD This represents the computation time required to compute a single input data block in the first memory area, where L represents the memory access latency value.

9. The data processing apparatus according to claim 7, wherein when the computation time required by the computing unit to compute an input data block of a single first storage area is greater than the memory access time of a single first storage area, the number N of the first storage areas satisfies: (T SIMD ×(N-1) - T load The condition is that N > L, and N is a positive integer; in, T SIMD T represents the computation time required to compute a single input data block in the first memory area. load This represents the loading time required to load an input data block into a single first storage area, where L represents the memory access latency value.

10. The data processing apparatus according to any one of claims 7-9, wherein the number N of the first storage areas is greater than or equal to 3.

11. The data processing apparatus according to claim 7, comprising at least one cluster, each cluster comprising one or more of the processing cores, each cluster being configured with an inter-core synchronization unit, the inter-core synchronization unit being configured as follows: In response to semaphore instructions, the synchronization dependency between the producer core and consumer core indicated by the semaphore instructions is maintained independently.

12. The data processing apparatus according to claim 11, wherein each cluster is further configured with a shared storage unit, the shared storage unit being configured with a plurality of second storage areas, the number M of the second storage areas satisfying: (M 1)×T load T move >L, and M is a positive integer; in, T load T represents the time required to load data from an off-chip memory location to a shared memory location in a single operation. move L represents the time it takes to move data from a shared storage unit to a local storage unit in a single transaction, where L represents the memory access latency value.

13. The data processing apparatus according to claim 12, wherein the number N of the first storage areas of each processing core is 2, or the same as the number M of the second storage areas.

14. The data processing apparatus according to any one of claims 11-13, wherein, The addresses of the N first memory areas are aligned to a preset byte order; and / or The addresses of the M second storage areas are aligned to a preset byte order.

15. A chip comprising a data processing apparatus according to any one of claims 1-14.

16. A circuit board comprising the chip of claim 15.

17. A data processing method implemented by a data processing apparatus, wherein the data processing apparatus includes a processing core, the data processing method comprising: In response to a synchronization command, determine the production end and consumer end that need to be synchronized as indicated by the synchronization command; In response to the fact that the producer in the production end and the consumer in the consumption end belong to the same instruction stream, the synchronization dependency between the producer and the consumer in the same instruction stream is released; And in response to the fact that the producer in the production end and the consumer in the consumption end belong to different instruction streams, the synchronous dependency relationship between the different instruction streams is maintained.

18. The data processing method according to claim 17, further comprising: Determine whether there is a data dependency relationship between adjacent instructions based on whether the access address ranges of adjacent instructions within the same instruction stream overlap. In response to the absence of data dependencies between the adjacent instructions, the subsequent instruction in the adjacent instructions is allowed to begin transmission before the preceding instruction is completed.

19. The data processing method according to claim 17, wherein the synchronization instruction includes a pair of matching first sub-instructions and second sub-instructions, the first sub-instruction being used to instruct the producer end forming the synchronization dependency, and the second sub-instruction being used to instruct the corresponding consumer end.

20. The data processing method according to claim 19, wherein there is at least one instruction to be executed or at least one time slice between the production end indicated by the first sub-instruction and the consumption end indicated by the second sub-instruction, the time slice being the cycle required for the processing core to complete a set of predetermined operations.

21. The data processing method according to any one of claims 17-20, wherein the instruction stream includes a memory access instruction stream and a computation instruction stream.

22. The data processing method according to claim 18, wherein the instruction stream includes a computation instruction stream, the computation instruction stream includes a plurality of computation instructions, and the data processing method further includes: Each computation instruction is broken down into multiple micro-operations; Based on whether the data address ranges accessed by at least the last segment of the micro-operation of the preceding computation instruction and at least the first segment of the micro-operation of the following computation instruction overlap, it is determined whether there is a data dependency relationship between the two. In response to the absence of a data dependency between the at least last micro-operation and the at least first micro-operation, the subsequent computation instruction is allowed to begin transmission before the completion of the last micro-operation of the preceding computation instruction.

23. The data processing method according to claim 17, wherein the processing core further includes a local storage unit and a computing unit, wherein the local storage unit is configured with a plurality of first storage areas, each first storage area being used for time-sharing storage of input data blocks and output data blocks of the processing results of the computing unit.

24. The data processing method according to claim 23, wherein when the access time of a single first storage area is greater than or equal to the computation time required by the computing unit to compute the input data block of a single first storage area, the number N of the first storage areas satisfies: (T IO ×(N-1) - T SIMD The condition is that N > L, and N is a positive integer; in, T IO T represents the access time of a single first memory area. SIMD This represents the computation time required to compute a single input data block in the first memory area, where L represents the memory access latency value.

25. The data processing method according to claim 23, wherein when the computation time required by the computing unit to compute an input data block of a single first storage area is greater than the memory access time of a single first storage area, the number N of the first storage areas satisfies: (T SIMD ×(N-1) - T load The condition is that N > L, and N is a positive integer; in, T SIMD T represents the computation time required to compute a single input data block in the first memory area. load This represents the loading time required to load an input data block into a single first storage area, where L represents the memory access latency value.

26. The data processing method according to any one of claims 23-25, wherein the number N of the first storage areas is ≥3.

27. The data processing method of claim 23, wherein the data processing apparatus comprises at least one cluster, each cluster comprising one or more of the processing cores, and the data processing method further comprises: In response to semaphore instructions, the synchronization dependency between the producer core and consumer core indicated by the semaphore instructions is maintained independently.

28. The data processing method according to claim 27, wherein each cluster is further configured with a shared storage unit, the shared storage unit being configured with a plurality of second storage areas, the number M of the second storage areas satisfying: (M 1)×T load T move >L, and M is a positive integer; in, T load T represents the time required to load data from an off-chip memory location to a shared memory location in a single operation. move This represents the time required to move data from a shared storage unit to a local storage unit in a single transaction, where L represents the memory access latency value.

29. The data processing method according to claim 28, wherein the number N of the first storage area of ​​each processing core is 2, or the same as the number M of the second storage area.

30. The data processing method according to any one of claims 27-29, further comprising: Align the addresses of the N first storage areas according to a preset byte order; and / or Align the addresses of the M second storage areas according to a preset byte order.

Images