Data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a data processing method and device, electronic equipment and a storage medium. The data processing method is applied to an artificial intelligence processor and comprises the steps that a single target instruction is received, and the target instruction comprises multiple fields used for target data, a target base address and a target cursor address; and decoding the target instruction, calculating a corresponding target storage address based on the target base address and the target vernier address, and writing target data according to the target storage address. According to the data processing method, the hardware automatically calculates the storage address through the single instruction, and the data processing performance is improved.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In parallel computing and high-performance data processing, a key issue is how to efficiently and correctly aggregate intermediate results from multiple threads or processors into shared memory. For example, in implementing algorithms such as Top-k selection, the efficiency and scalability of data aggregation directly determine overall performance. To coordinate parallel writes and avoid data contention, a two-step operation of "dynamic offset allocation - conflict-free write" is typically used. However, this approach increasingly exhibits inherent performance bottlenecks when dealing with modern massively parallel hardware architectures, becoming a significant factor limiting system throughput and energy efficiency. Summary of the Invention

[0003] At least one embodiment of this disclosure provides a data processing method applied to an artificial intelligence processor, wherein the data processing method includes: receiving a single target instruction, wherein the target instruction includes multiple fields for target data, a target base address, and a target cursor address; decoding the target instruction; calculating a corresponding target storage address based on the target base address and the target cursor address; and writing the target data according to the target storage address.

[0004] In at least one embodiment of this disclosure, a data processing method is provided, wherein the artificial intelligence processor is configured to process multiple threads in parallel. Decoding the target instruction and calculating the corresponding target storage address based on the target base address and the target cursor address includes: parsing the target instruction to obtain target data, the target base address, and the target cursor address corresponding to each thread; merging access requests corresponding to threads pointing to the same target cursor address and without memory access conflicts into a first request, wherein the same target cursor address is the first cursor address; and calculating the target storage address corresponding to each thread in the first request based on the target base address and the first cursor address corresponding to each thread in the first request.

[0005] In at least one embodiment of this disclosure, a data processing method is provided, wherein calculating the target storage address corresponding to each thread in the first request based on the target base address and the first cursor address of each thread in the first request includes: reading and updating the cursor value corresponding to the first cursor address, wherein the read cursor value is the first cursor value; allocating different offsets to each thread in the first request; and calculating the target storage address corresponding to each thread in the first request based on the target base address, the first cursor value, and the offset.

[0006] In at least one embodiment of this disclosure, a data processing method is provided, wherein allocating a different offset to each thread in the first request includes: obtaining the number N of threads in the first request, where N is a positive integer; and allocating a different offset to each thread in the first request, wherein the offset is an integer between 0 and N-1.

[0007] In at least one embodiment of this disclosure, a data processing method is provided, wherein reading and updating the cursor value corresponding to the first cursor address includes: reading the cursor value corresponding to the first cursor address as the first cursor value; adding the first cursor value and the number of threads to obtain a second cursor value; and writing the second cursor value to the first cursor address.

[0008] In at least one embodiment of this disclosure, a data processing method is provided, wherein calculating the target storage address corresponding to each thread in the first request based on the target base address, the first cursor value, and the offset includes: adding the target base address, the first cursor value, and the offset corresponding to each thread to obtain the target storage address corresponding to each thread.

[0009] In at least one embodiment of this disclosure, a data processing method is provided, wherein writing the target data according to the target storage address includes: in response to the target storage address corresponding to each thread being located in different storage banks, merging and writing the target data corresponding to each thread.

[0010] In at least one embodiment of this disclosure, a data processing method is provided, wherein the plurality of fields of the target instruction include: a first field declaring the address space to which the target storage address belongs; a second field indicating the total number of target cursor address registers and temporary registers corresponding to the target cursor address; a third field indicating the target cursor address register; and at least one fourth field indicating the temporary register, wherein the temporary register includes a target base address register corresponding to the target base address and a target data register corresponding to the target data.

[0011] In at least one embodiment of this disclosure, a data processing method is provided, wherein the at least one fourth field includes: a fifth field representing the target base address register; and a sixth field representing the target data register.

[0012] At least one embodiment of this disclosure provides a data processing apparatus, including: an instruction fetching unit configured to receive a single target instruction, wherein the target instruction includes multiple fields for target data, a target base address, and a target cursor address; an instruction decoding unit configured to decode the target instruction; and a load storage unit configured to calculate a corresponding target storage address based on the target base address and the target cursor address, and write the target data according to the target storage address.

[0013] At least one embodiment of this disclosure provides an electronic device, including: at least one processor; at least one memory including one or more computer program modules; wherein the one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, and the one or more computer program modules are used to implement the data processing method provided in the at least one embodiment above.

[0014] At least one embodiment of this disclosure provides a non-transitory computer-readable storage medium storing computer-readable instructions thereon, wherein the computer-readable instructions, when executed by at least one processor, perform the data processing method provided in the at least one embodiment described above. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0016] Figure 1A This is a schematic structural diagram of a general-purpose graphics processor;

[0017] Figure 1B A schematic block diagram of a streaming multiprocessor provided for at least one embodiment of this disclosure;

[0018] Figure 2 A flowchart illustrating a data processing method provided in at least one embodiment of this disclosure;

[0019] Figure 3 A schematic diagram illustrating a data processing method provided in at least one embodiment of this disclosure;

[0020] Figure 4 A schematic block diagram of a data processing apparatus provided in at least one embodiment of this disclosure;

[0021] Figure 5 A schematic block diagram of an electronic device provided for at least one embodiment of this disclosure;

[0022] Figure 6A schematic block diagram of another electronic device provided for at least one embodiment of this disclosure; and

[0023] Figure 7 This is a schematic block diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0025] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as “connected” or “linked” are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0026] The present disclosure will now be described through several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and known components may be omitted. When any component of an embodiment of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numerals in each drawing.

[0027] In the field of artificial intelligence, attention mechanisms are widely used in deep learning models. As an example, various attention operators are extensively applied in Large Language Models (LLMs). A LLM is a deep learning model trained on massive amounts of text data, capable of understanding and generating natural language text. LLMs are typically based on the Transformer architecture and learn statistical patterns and semantic information of language from massive amounts of text data through self-supervised learning. They are commonly used for tasks such as text generation, translation, question answering, and summarization. As other examples, attention mechanisms can also be applied to deep learning models in fields such as speech processing, image processing, text processing, and video processing, without further limitation.

[0028] As model size continues to increase and application scenarios become increasingly complex, the computational overhead of attention mechanisms has become a significant factor affecting overall model performance. For example, in long sequence modeling tasks, the time and space complexity of attention operators increases quadratically with the input length, leading to decreased inference speed, reduced training efficiency, and increased resource consumption. This overhead not only limits the model's ability to train on large-scale data but also places higher demands on the hardware conditions of the deployment environment.

[0029] To overcome this limitation, some large language models have introduced fine-grained sparse attention mechanisms, aiming to improve the computational efficiency of training and inference in long contexts while maintaining the quality of model output as much as possible. The design idea of ​​this fine-grained sparse attention mechanism is to avoid fully connected attention calculations between all tokens, and instead, through intelligent selection, allow each token to interact only with a subset of key tokens in the sequence.

[0030] For example, some large language models currently achieve this goal through two key components working together: a lightweight indexer and top-k token selection. The Lightning Indexer, for instance, is the dynamic routing component in this mechanism. For each query token in the sequence, the indexer quickly compares it with historical tokens, calculating a lightweight index score. This index score assesses the potential relevance or importance between tokens, and its calculation significantly reduces computational overhead compared to traditional full attention calculations, thus enabling real-time and efficient preprocessing of long sequences. Top-k token selection, for example, is a key step in performing fine-grained sparsity. For each query token, the Lightning Indexer outputs a score I for all candidate key tokens. Then, Top-k token selection selects only the top k key tokens based on their scores I, discarding the rest. In subsequent attention calculations, the query token interacts only with these k selected key tokens to calculate the actual attention weights and context vectors. Through the above two-stage processing, the space and time complexity of attention computation can be reduced from O(L) of standard self-attention. 2 The computation time is significantly reduced to O(L⋅k), where k≪L (L is the sequence length). This provides a solution for efficiently handling extremely long contexts in large language models.

[0031] However, in practical deployments of extremely long sequences, the traditional Top-k selection algorithm itself may become a new performance bottleneck. For example, when it is necessary to select the top k values ​​from a large number of candidate scores, data handling and sorting operations can lead to huge memory access overhead, thereby limiting system bandwidth.

[0032] Currently, some computer models optimize their custom Top-k operator implementations using data aggregation methods. For example, the Top-k operator can utilize the atomic_add operation in its implementation. For instance, during the selection process, multiple parallel threads can atomically accumulate or reduce their respective candidate data (such as scores or indices) into a shared, contiguous address space. The aforementioned data aggregation method may include the following steps:

[0033] 1. Cursor initialization: Maintain a global cursor counter in shared memory to identify the currently available write position.

[0034] 2. Atomic offset acquisition: Each thread increments this counter using an atomic addition instruction (e.g., atomic_add). Thus, each thread can complete both "reading the current cursor value" and "increasing the cursor by a specified step," and the value read is the write offset exclusively for that thread.

[0035] 3. Data writing: Each thread, based on the obtained offset, stores the data (such as index, score, etc.) it has calculated or filtered into the corresponding consecutive positions in the target buffer through a storage instruction (such as store).

[0036] The inventors of this disclosure have noted that the above method has at least the following limitations: First, since multiple threads must perform atomic operations on the same cursor counter sequentially, this atomic operation will become a performance bottleneck when the concurrency density is high, limiting the overall throughput of the system; Second, the above method requires the sequential execution of two instructions, such as atomic_add and store, which not only increases instruction issuance overhead but also introduces additional operation latency, further affecting execution efficiency.

[0037] This disclosure provides at least one embodiment of a data processing method, apparatus, electronic device, and storage medium.

[0038] This data processing method is applied to an artificial intelligence processor and includes: receiving a single target instruction, wherein the target instruction includes multiple fields for target data, target base address, and target cursor address respectively; decoding the target instruction; calculating the corresponding target storage address based on the target base address and target cursor address; and writing the target data according to the target storage address.

[0039] The data processing device includes: an instruction fetching unit configured to receive a single target instruction, wherein the target instruction includes multiple fields for target data, target base address, and target cursor address; and an instruction decoding unit configured to decode the target instruction, calculate the corresponding target storage address based on the target base address and target cursor address, and write the target data according to the target storage address.

[0040] This data processing method and apparatus are implemented by extending the functionality of existing instruction set architectures. This disclosure defines a new set of target instructions, which are assigned previously unused or reserved opcodes within a hardware-supported instruction set architecture. These target instructions achieve a more efficient data processing mode by implementing the data processing method, but their execution still relies on existing computational and data path resources within the computing unit, such as arithmetic logic units, special function units, and load-memory units. This allows for improved execution efficiency of specific computational tasks while fully reusing existing hardware computational and data path resources.

[0041] This data processing method and apparatus can merge two previously necessary instructions (e.g., `atomic_add`) and two storage instructions (e.g., `store`) into a single target instruction, reducing the number of instruction issues by 50%. This lowers instruction issue overhead and pipeline pressure, as well as operational latency. Furthermore, by reducing the number of instructions and centralizing memory access, the active time of execution units and related memory units is reduced, thereby decreasing overall power consumption. By reducing the number of instructions and optimizing data scheduling paths, instruction throughput and hardware resource utilization are improved, while task processing latency is reduced.

[0042] The data processing method and apparatus provided in at least one embodiment of this disclosure can perform an atomic operation only once on the same target cursor address, thereby eliminating the serial bottleneck of multi-threaded atomic accumulation. For example, this data processing method can reduce an operation that originally required O(N) cycles (where N is the number of threads in the same group) to an operation that is close to O(1) cycles, thereby improving concurrent write performance.

[0043] For example, the data processing method provided in at least one embodiment of this disclosure can be applied to an artificial intelligence processor, which may include a graphics processing unit (GPU), a tensor processing unit (TPU), a general-purpose graphics processing unit (GPGPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), a neural network processing unit (NPU), etc. Of course, the embodiments of this disclosure are not limited to this, and the artificial intelligence processor may also be any other type of processor.

[0044] Figure 1AThis is a schematic diagram of a general-purpose graphics processing unit (GPGPU).

[0045] like Figure 1A As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1A The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0046] like Figure 1A As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 1A The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple cores (also called computational kernels), each of which includes an Arithmetic Logic Unit (ALU), a floating-point unit, etc. These cores are used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 1A The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0047] like Figure 1A As shown, each streaming processor cluster also provides a buffer for caching data across the N computing units within the streaming processor cluster.

[0048] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 1A(Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0049] In each computing unit, the thread beam scheduling / distribution module ( Figure 1A (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 1A High Bandwidth Memory (HBM) is used for read and write operations.

[0050] Figure 1B This is a schematic block diagram of a streaming multiprocessor (SM) provided for at least one embodiment of the present disclosure. For example, the streaming multiprocessor can be... Figure 1A The computational unit in the process.

[0051] Each stream multiprocessor 10 can execute multiple thread bundles simultaneously. The execution of different thread bundles can be independent or sequential. In some examples, the data processing method provided in at least one embodiment of this disclosure can be applied to, for example... Figure 1B The shown is a streaming multiprocessor 10.

[0052] like Figure 1BAs shown, the streaming multiprocessor 10 includes an instruction cache 11 for storing multiple instructions from the kernel. The streaming multiprocessor 10 also includes a warp instruction scheduler 12 for fetching a series of instructions for each warp and storing them in the instruction cache 11, retrieving instructions to be executed from the instruction cache 11 for each warp based on a program counter. For example, each warp has an independent program counter (PC) register to record the location (i.e., the instruction address) of the instruction currently being executed. Each time an instruction is fetched from the instruction cache for a warp, the corresponding program counter is incremented. The warp instruction scheduler 12 delivers instructions to the Arithmetic Logic Unit (ALU), Special Function Unit (SFU), or Load Store Unit (LSU) 15 for execution at appropriate times; these instructions are defined in the instruction set architecture (ISA) of the specific computing system.

[0053] The arithmetic logic unit 13 can perform various operations, such as addition and multiplication of integers and floating-point numbers, comparison operations, Boolean operations, bit shifting, etc. During execution, the arithmetic logic unit 13 can read data from a specified location (also known as the source address) in the general-purpose register 16 (GPR) and write back the execution result to a specified location (also known as the destination address) in the general-purpose register 16. The special function unit 14 is used to execute transcendental function instructions, which can implement various complex mathematical operations (such as finding reciprocals, square roots, trigonometric functions, etc.). The load-store unit 15 can execute load and store instructions, and is responsible for handling memory load and store operations.

[0054] The streaming multiprocessor 10 also includes a barrier register 17, which allows software to synchronize execution between different thread bundles, and a resource-per-warp register 18, which allows software to dynamically configure the space range of general-purpose registers 16 available to each thread bundle during execution. It should be noted that... Figure 1BThe streaming multiprocessor shown is merely an example, and those skilled in the art will understand that a streaming multiprocessor may include more or fewer components. For example, a streaming multiprocessor may also include shared memory, a Level 1 cache, and other components.

[0055] Figure 2 This is a flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0056] For example, such as Figure 2 As shown, the data processing method provided in at least one embodiment of this disclosure may include the following steps S110 to S120.

[0057] Step S110: Receive a single target instruction, wherein the target instruction includes multiple fields for target data, target base address and target cursor address respectively.

[0058] For example, in step S110, the Instruction Fetch Unit (IFU) can fetch a single target instruction from the instruction cache. The IFU can determine and fetch the instruction to be executed based on the address indicated by the program counter. The IFU is located in the processor; in some examples, the IFU may be located in... Figure 1B Unified Fetch-Decode Pipeline in Streaming Multiprocessors 10 Figure 1B In (not shown), as part of the shared instruction processing front end, it is... Figure 1B The thread bundle instruction scheduler 12 shown provides instructions.

[0059] For example, in step S110, the single target instruction is a machine command, which is an operation command represented by binary encoding and directly recognized and executed by the processor hardware. Each machine instruction typically includes an opcode and operands, used to instruct the processor to perform specific computation, data transfer, or control flow operations. A single target instruction refers to an independent and complete machine instruction, capable of being parsed and executed by the processor, and able to achieve a predefined function. This target instruction can serve as the basic unit of program execution, representing the smallest logical unit of a program that can be executed at the hardware level, and also the smallest granularity of operation processed by the processor within a certain execution cycle.

[0060] For example, in step S110, the target instruction may include an opcode and operands. For example, the opcode identifies the type of the target instruction, instructing the processor to perform a specific operation. For example, the operands may include multiple fields, representing, for example, the target cursor address, the target base address, and the target data. For example, these fields can be indirectly referenced through register numbers. For example, the target cursor address and the target base address can be stored in corresponding registers, allowing indirect addressing to access the corresponding memory data; the target data can be stored in the corresponding data register as the source operand. For example, the target cursor address may represent the memory address of a shared counter. The memory address of this shared counter may correspond to an integer variable shared by multiple threads, for example, located in shared memory or global memory. The value of this shared counter can be used as the object of an atomic increment operation, whereby its value can be atomically incremented by a step size, for example, 1, each time it is executed. For example, the value before the increment can be used by each thread as an offset to calculate the target memory address.

[0061] For example, the target base address can represent the starting memory address of the target storage region. The target storage region can be the address space to which the target storage address belongs, such as shared memory or global memory. The target base address is a reference address; combining this reference address with the offset obtained from the target cursor address allows calculation of the final physical address where the data is written, i.e., the target storage address, by adding the offset to the base address. The target base address can define a contiguous or structured storage space, such as an array or buffer. Furthermore, the target base address ensures that each thread writes the target data to different locations within this storage space in an orderly and conflict-free manner, based on its own obtained offset.

[0062] For example, target data can represent the actual data to be written to storage. For example, target data can originate from registers, such as calculation results or intermediate values ​​stored in a general-purpose or special-purpose register. For another example, target data can be immediate values, such as constants inherent in the instruction code. For yet another example, target data can be indices, keys, and attention scores that each thread needs to save.

[0063] Step S120: Decode the target instruction, calculate the corresponding target memory address based on the target base address and the target cursor address, and write the target data according to the target memory address.

[0064] For example, in step S120, the instruction decoder can decode the single target instruction obtained in step S110. The instruction decoder can parse the binary code of the target instruction and convert it into a set of control signals to drive the corresponding functional unit to perform the specified operation. The instruction decoder is located in the processor; in some examples, it may be located in... Figure 1B Unified Fetch-Decode Pipeline in Streaming Multiprocessors 10 Figure 1B In (not shown), as part of the shared instruction processing front end, it is... Figure 1B The thread bundle instruction scheduler 12 shown provides instructions. For example, a set of control signals obtained after decoding a single target instruction is used to drive a functional unit to perform the following combination of operations: calculating the corresponding target memory address based on the target base address and the target cursor address, and writing target data according to the target memory address. For example, the above functional unit can be... Figure 1B Loading storage unit 15 in the middle.

[0065] For example, the AI ​​processor can be configured to process multiple threads in parallel. For example, the AI ​​processor can be a graphics processing unit (GPU), where the unit of hardware scheduling and execution is a thread bundle. For example, a thread bundle can include 32 threads executing in parallel, physically bound together and synchronously executed in lockstep mode within the same clock cycle by the same streaming multiprocessor. For example, the thread bundle enables each thread in the bundle to execute operations by sharing the aforementioned single target instruction.

[0066] For example, in at least one embodiment of this disclosure, step S120 may include steps 1-3 as follows.

[0067] Step 1: Parse the target instructions to obtain the target data, target base address, and target cursor address for each thread.

[0068] For example, the target instruction may include the target data, target base address, and target cursor address for each thread in a thread bundle (e.g., 32 threads).

[0069] Step 2: Merge the access requests of threads that point to the same target cursor address and have no memory access conflicts into a first request, where the same target cursor address is the first cursor address.

[0070] For example, when each thread in a thread bundle executes the single target instruction, multiple access requests can be generated; for instance, each thread can correspond to one access request. For example, a functional unit (e.g.) Figure 1BThe loading memory unit (in the memory) can merge access requests from threads that point to the same target cursor address and have no memory access conflicts into a first request.

[0071] For example, a bank is a physical memory module divided into several independently operable sub-banks. For instance, a GPU's shared memory can be divided into 32 or 64 banks; if 32 threads in a thread bundle each correspond to different banks, they can complete 32 read / write operations within the same clock cycle. For example, a bank access conflict refers to a forced serialization operation caused when multiple concurrent memory access requests are too close in timing and their access addresses, after address mapping, point to different active rows of the same bank. For example, a bank access conflict can occur when two or more threads in the same thread bundle of a GPU attempt to access different addresses within the same bank within the same clock cycle.

[0072] For example, in step 2, it can be determined whether there is a memory access conflict in the access requests of each thread by reading the cursor value corresponding to the target cursor address and the target base address. For example, taking the GPU as an AI processor, in shared memory, the address-to-memory mapping formula can be: bank_index = (byte_address ÷ 4) % 32, where the byte address byte_address can be the result of adding the cursor value corresponding to the target base address and the target cursor address. For example, the target base address can represent the starting byte address of this frame of data in shared memory, and the cursor value corresponding to the target cursor address can represent the offset of the byte data written by the current thread relative to the target base address. Adding the two gives the absolute byte address where the byte data falls in shared memory, and substituting it into the above formula can calculate the corresponding memory bank number bank_index. For example, the memory bank numbers of 32 threads in the same thread bundle can be compared. If any two threads have the same bank_index but different thread numbers, it can be determined that the two threads will have a memory access conflict.

[0073] For example, each thread in the first request may have an access request pointing to the same target cursor address, which is then used as the first cursor address for subsequent calculations. For example, the first request can be single or multiple. For example, when there are multiple first requests, it supports processing multiple independent first requests simultaneously, each with its own first cursor address, and the processing of these first requests can be parallel.

[0074] The data processing method provided in at least one embodiment of this disclosure can eliminate the serial bottleneck of multi-threaded atomic accumulation by merging access requests corresponding to threads that point to the same target cursor address and have no memory access conflicts into a first request, thereby performing an atomic operation only once on the same target cursor address.

[0075] Step 3: Based on the target base address and the first cursor address corresponding to each thread in the first request, calculate the target storage address corresponding to each thread in the first request.

[0076] For example, in step 3, the offset of each thread can be calculated, and then the target storage address corresponding to each thread in the first request can be calculated based on the offset and the target base address.

[0077] For example, in at least one embodiment of this disclosure, step 3 may include the following steps: reading and updating the cursor value corresponding to the first cursor address, wherein the read cursor value is the first cursor value and each thread in the first request is assigned a different offset; and calculating the target storage address corresponding to each thread in the first request based on the target base address, the first cursor value and the offset.

[0078] For example, the offset can be an integer index in units of data items, or it can represent a thread's unique logical number within its group.

[0079] For example, in at least one embodiment of this disclosure, assigning a different offset to each thread in the first request may include: obtaining the number of threads N in the first request, where N is a positive integer; and assigning a different offset to each thread in the first request, where the offset is an integer between 0 and N-1.

[0080] For example, the number of threads N refers to the total number of threads contained in the current first request. For example, when merging the first request, the number of threads can be passed as a parameter and directly obtained. After obtaining the number of threads N, an offset can be assigned to each thread according to the number of threads. For example, assuming there are N threads in the first request, numbered from thread 0 to thread N-1, thread 0 can be assigned 0, thread 1 can be assigned 1, and so on, with thread N-1 assigned N-1, thus serving as a different offset for each thread. It should be noted that this disclosure does not restrict the specific method of assigning offsets; any deterministic rule that can generate a unique offset for each thread in the first request (such as linear mapping, hash transformation, or any user-defined mapping function) is applicable.

[0081] For example, in at least one embodiment of this disclosure, reading and updating the cursor value corresponding to the first cursor address may include: reading the cursor value corresponding to the first cursor address as the first cursor value; adding the first cursor value and the number of threads to obtain a second cursor value; and writing the second cursor value to the first cursor address.

[0082] For example, reading and updating the cursor value corresponding to the first cursor address can be an atomic operation. For instance, the first cursor address can be obtained from the corresponding register (such as the cursor address register), and the corresponding cursor value can be read from the memory location pointed to by that first cursor address as the first cursor value C. Based on the number of threads N in the first request, the second cursor value C' = C + N is calculated. The second cursor value C' is then atomically written to the memory location pointed to by the first cursor address A to update the value of the shared counter mentioned above.

[0083] For example, in at least one embodiment of this disclosure, calculating the target storage address corresponding to each thread in the first request based on the target base address, the first cursor value, and the offset may include: adding the target base address, the first cursor value, and the offset corresponding to each thread to obtain the target storage address corresponding to each thread.

[0084] For example, addition is pointer (address) arithmetic operations on bytes. For example, each thread can calculate the corresponding target memory address in parallel. For example, the target memory address for each thread in the first request can be calculated using the following formula:

[0085] Final_Addr = Base_Addr + (C + offset)

[0086] Where Final_Addr represents the target storage address corresponding to each thread, Base_Addr represents the target base address corresponding to each thread, C represents the first cursor value read from the first cursor address (i.e., the cursor value before the update), and offset represents the offset corresponding to each thread.

[0087] For example, in at least one embodiment of this disclosure, writing target data according to the target storage address may include: in response to the target storage address corresponding to each thread being located in different storage banks, merging and writing the target data corresponding to each thread.

[0088] For example, when write requests from different threads need to be written to different storage devices, these write requests can be merged into a more efficient memory transaction to improve write efficiency.

[0089] For example, in at least one embodiment of this disclosure, multiple fields of the target instruction may include: a first field declaring the address space to which the target storage address belongs; a second field indicating the total number of the target cursor address register and the temporary register; a third field indicating the target cursor address register; and at least one fourth field indicating the temporary register, wherein the temporary register includes the target base address register corresponding to the target base address and the target data register corresponding to the target data.

[0090] For example, the first field can indicate whether the target memory address belongs to shared memory or global memory. For instance, taking a GPU as an example of an AI processor, global memory is the largest main memory in the GPU, accessible to all threads. However, its access latency is extremely high, and although its bandwidth is high, it needs to be fully utilized through merged accesses. That is, threads within the same thread block should access contiguous, aligned memory addresses so that the hardware can merge multiple accesses into a few wide transactions. Shared memory, on the other hand, is a very small, low-latency on-chip programmable cache located within each streaming multiprocessor, shared only between threads within the same thread block.

[0091] For example, the second field could represent the total number of the target cursor address register and the temporary registers corresponding to the target cursor address. For example, the temporary registers are physical general-purpose registers that can be dynamically assigned by the instruction scheduler window or register allocator and can be overridden by any subsequent instruction. For example, the third field could represent the target cursor address register corresponding to the target cursor address. For example, at least one fourth field could represent temporary registers, where the temporary registers include the target base address register corresponding to the target base address and the target data register corresponding to the target data.

[0092] The following is a specific example of the target instructions provided in at least one embodiment of the present disclosure.

[0093] For example, in the instruction set, the target instruction can be named as the atappend instruction, as shown in the following example:

[0094] atappend .* .mt .gcnt, g0, gi

[0095] For example, the symbol .* above refers to other related instruction fields; only the most relevant instruction fields are listed here.

[0096] .mt: {gsm|glm} memory type, also known as the first field, indicates which address space the target storage address belongs to, such as shared memory (gsm) or global memory (glm).

[0097] .gcnt: staging count, also known as the second field, indicates the total number of target cursor address registers and temporary registers corresponding to the target cursor address. For example, gcnt can be 3, indicating that it includes one target cursor address register (e.g., g0) and two temporary registers (e.g., g1 and g2). For example, the temporary registers include the target base address register g1, which represents the target base address, and the target data register g2, which represents the target data.

[0098] g0: address register, also known as the third field, represents the target cursor address register corresponding to the target cursor address.

[0099] gi: This refers to at least one fourth field, representing a temporary register. For example, a temporary register might include a target base address register g1 corresponding to the target base address and a target data register g2 corresponding to the target data.

[0100] The following is a specific example of a data processing method provided by at least one embodiment of the present disclosure.

[0101] Step 1: Request Reception and Parsing. The loading memory unit receives the atappend instruction and parses out the target data register g2, target base address register g1, target cursor address register g0, etc., corresponding to each thread. For example, the target cursor address register g0, target base address register g1, and target data register g2 can be read to obtain the target cursor address, target base address, and target data corresponding to each thread.

[0102] Step 2: Cursor Request Merging and Atomic Update. When each thread in the thread bundle executes the atappend instruction, it can generate multiple access requests, for example, one access request per thread. The loading memory unit can include a merging unit, which can divide access requests pointing to the same target cursor address from all threads and access requests without bank conflicts into a first request. This same target cursor address is referred to as the first cursor address. For each first request, atomically execute: read the current cursor value C of the first cursor address, record C as the first cursor value, calculate the new cursor value C' = C + N (N is the number of threads falling at the same cursor address), C' is the second cursor value, and write the second cursor value C' back to the memory unit pointed to by the first cursor address to update the value of the shared counter. At the same time, allocate a unique offset from 0 to N-1 to each thread accessing the same target cursor address, that is, each thread in the first request, where N is a positive integer.

[0103] Step 3: Parallel computation of the final address. Each thread in each first request computes its final target memory address in parallel, for example, using the following formula:

[0104] Final_Addr = Base_Addr(g1) + (C + offset).

[0105] Where Final_Addr is the target storage address corresponding to each thread, Base_Addr(g1) is the target base address, C is the first cursor value, and offset is the offset.

[0106] Step 4: Data Merging and Writing. The loaded storage unit can merge the target data based on the calculated target storage address. If multiple threads have target storage addresses located in different storage banks, they can be merged into a wider memory transaction (e.g., 128B).

[0107] For example, suppose there are three threads, numbered thread 0, thread 1, and thread 2, executing the above atappend instruction. The target cursor address for thread 0, thread 1, and thread 2 is A, and the target base address for thread 2 is B. Furthermore, the access requests for threads 0, 1, and 2 do not involve memory access conflicts.

[0108] Initially, the cursor value corresponding to the target cursor address A is 10. The load memory unit merges the three access requests corresponding to these three threads into the first request. At this time, A is the first cursor address, and 10 is the first cursor value. The number of threads is 3, and the new first cursor value, which is also the second cursor value, is calculated to be 13. The load memory unit atomically writes the second cursor value 13 back to the memory unit pointed to by the first cursor address A. At the same time, the load memory unit allocates the offsets corresponding to each thread as follows: thread 0 gets 0, thread 1 gets 1, and thread 2 gets 2. Therefore, based on the above first cursor value, target base address, and offsets, the target memory addresses B+10, B+11, and B+12 corresponding to threads 0, 1, and 2 can be calculated respectively. Finally, the data of threads 0, 1, and 2 are stored in the target memory addresses B+10, B+11, and B+12 respectively.

[0109] Figure 3 This is a schematic diagram of a data processing method provided in at least one embodiment of the present disclosure.

[0110] Figure 3 The above indicates the memory bank to which the target memory address for each thread belongs. For example... Figure 3As shown, for example, the target memory address corresponding to thread 0 belongs to memory bank 1, the target memory address corresponding to thread 1 belongs to memory bank 1, the target memory address corresponding to thread 2 belongs to memory bank 1, the target memory address corresponding to thread 3 belongs to memory bank 9, the target memory address corresponding to thread 4 belongs to memory bank 1, the target memory address corresponding to thread 5 belongs to memory bank 1, the target memory address corresponding to thread 6 belongs to memory bank 1, the target memory address corresponding to thread 7 belongs to memory bank 9, the target memory address corresponding to thread 8 belongs to memory bank 9, the target memory address corresponding to thread 9 belongs to memory bank 9, the target memory address corresponding to thread 10 belongs to memory bank 1, the target memory address corresponding to thread 11 belongs to memory bank 7, the target memory address corresponding to thread 12 belongs to memory bank 1, and the target memory address corresponding to thread 13 belongs to memory bank 7.

[0111] like Figure 3 As shown, threads with the same grayscale level represent threads accessing the same target cursor address. For example, threads 0, 4, 5, 10, and 12 access the same target cursor address; threads 1, 2, and 6 access the same target cursor address; threads 3, 7, 8, and 9 access the same target cursor address; and threads 11 and 13 access the same target cursor address.

[0112] like Figure 3 As shown, access requests from threads that all point to the same target cursor address and have no memory access conflicts can be merged into a first request. For example, access requests from threads 0, 4, 5, 10, and 12 can be merged into one first request; access requests from threads 1, 2, and 6 can be merged into one first request; access requests from threads 3, 7, 8, and 9 can be merged into one first request; and access requests from threads 11 and 13 can be merged into one first request.

[0113] For example, different threads accessing the same target cursor address can be assigned incremental offsets. For instance, threads 0, 4, 5, 10, and 12 can be assigned offsets 0, 1, 2, 3, and 4, respectively.

[0114] For example, if the same storage device experiences a conflict, it needs to be split into different requests for access. Figure 3 As shown, the access requests of threads 0, 4, 5, 10 and 12 are merged into a first request, which accesses memory 1 in cycle 1; while the access requests of threads 1, 2 and 6 are merged into another first request, which accesses memory 1 in cycle 2.

[0115] It should also be noted that the execution order of the various steps of the data processing method in the various embodiments of this disclosure is not limited. Although the execution process of each step has been described in a specific order above, this does not constitute a limitation on the embodiments of this disclosure. The various steps in the data processing method can be executed serially or in parallel, which can be determined according to actual needs.

[0116] Multiple threads pointing to the same target cursor address can cause atomic operation conflicts, resulting in their requests being split into multiple serialized atomic add operations, creating a performance bottleneck. However, in the data processing method provided in at least one embodiment of this disclosure, multiple threads within the same group targeting the same cursor address only collaboratively execute an atomic accumulation operation once, jointly obtaining a continuous address range, and then performing conflict-free writes based on their independent offsets. Thus, regardless of the number of threads N in the first request, the overhead of atomic operations on the same target cursor address is reduced from the original O(N) cycles to nearly O(1) cycles, significantly improving data write throughput in high-concurrency scenarios. Simultaneously, this method still supports merging requests accessing different storage banks into wide-bit memory transactions, fully utilizing the parallel bandwidth of the memory system.

[0117] For example, compared to the above description, the data processing method provided in at least one embodiment of this disclosure may include more or fewer steps, and the embodiments of this disclosure do not limit this.

[0118] Figure 4 This is a schematic block diagram of a data processing apparatus provided for at least one embodiment of the present disclosure.

[0119] The data processing device may be, for example, an artificial intelligence processor or an artificial intelligence processor. The artificial intelligence processor may include a graphics processing unit (GPU), a tensor processing unit (TPU), a general-purpose graphics processing unit (GPGPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), a neural network processing unit (NPU), etc. Of course, the embodiments disclosed herein are not limited to this, and the artificial intelligence processor may also be any other type of processor.

[0120] For example, such as Figure 4As shown, the data processing apparatus provided in at least one embodiment of this disclosure may include an instruction fetching unit 401, an instruction decoding unit 402, and a loading storage unit 403.

[0121] For example, the instruction fetch unit 401 is configured to receive a single target instruction, wherein the target instruction includes multiple fields for target data, target base address, and target cursor address, respectively.

[0122] For example, instruction decoding unit 402 is configured to decode target instructions.

[0123] For example, the load storage unit 403 is configured to calculate the corresponding target storage address based on the target base address and the target cursor address, and write the target data according to the target storage address.

[0124] For example, in at least one embodiment of this disclosure, the artificial intelligence processor is configured to process multiple threads in parallel, and the loading storage unit is further configured to parse the target instruction to obtain the target data, target base address, and target cursor address corresponding to each thread; merge the access requests corresponding to threads that point to the same target cursor address and have no memory access conflict into a first request, wherein the same target cursor address is the first cursor address; and calculate the target storage address corresponding to each thread in the first request based on the target base address and the first cursor address corresponding to each thread in the first request.

[0125] For example, in at least one embodiment of this disclosure, the loading storage unit is further configured to parse the target instruction to obtain the target data, target base address, and target cursor address corresponding to each thread; merge the access requests corresponding to threads that point to the same target cursor address and have no memory access conflict into a first request, wherein the same target cursor address is the first cursor address; and calculate the target storage address corresponding to each thread in the first request based on the target base address and the first cursor address corresponding to each thread in the first request.

[0126] For example, in at least one embodiment of this disclosure, the loading storage unit is further configured to read and update the cursor value corresponding to the first cursor address, wherein the read cursor value is the first cursor value; allocate a different offset to each thread in the first request; and calculate the target storage address corresponding to each thread in the first request based on the target base address, the first cursor value, and the offset.

[0127] For example, in at least one embodiment of this disclosure, the loading storage unit is further configured to obtain the number of threads N in the first request, where N is a positive integer; and to assign a different offset to each thread in the first request, where the offset is an integer between 0 and N-1.

[0128] For example, in at least one embodiment of this disclosure, the loading storage unit is further configured to read the cursor value corresponding to the first cursor address as the first cursor value; add the first cursor value and the number of threads to obtain the second cursor value; and write the second cursor value to the first cursor address.

[0129] For example, in at least one embodiment of this disclosure, the loading storage unit is further configured to add the target base address, the first cursor value, and the offset corresponding to each thread to obtain the target storage address corresponding to each thread.

[0130] For example, in at least one embodiment of this disclosure, the loading storage unit is further configured to merge and write the target data corresponding to each thread in response to the target storage address being located in a different storage bank.

[0131] For example, in at least one embodiment of this disclosure, the target instruction includes multiple fields: a first field declaring the address space to which the target storage address belongs; a second field indicating the total number of target cursor address registers and temporary registers corresponding to the target cursor address; a third field indicating the target cursor address register; and at least one fourth field indicating a temporary register, wherein the temporary register includes a target base address register corresponding to the target base address and a target data register corresponding to the target data.

[0132] For example, in at least one embodiment of this disclosure, at least one fourth field includes: a fifth field representing a target base address register; and a sixth field representing a target data register.

[0133] It should be noted that the various modules and units described above can be implemented by software, hardware, firmware, or any combination thereof. For example, the instruction fetch unit, instruction decoding unit, and load memory unit can be implemented as an instruction fetch circuit, a decoding circuit, and a load memory circuit, respectively. The embodiments of this disclosure do not limit their specific implementation methods. For example, in some examples, the load memory unit can be the load memory unit 15 shown in FIG1.

[0134] It should be understood that the data processing apparatus 400 provided in at least one embodiment of this disclosure can be used to implement the aforementioned data processing method and can also achieve similar technical effects as the aforementioned data processing method, which will not be elaborated here.

[0135] It should be noted that in the embodiments of this disclosure, the data processing device 400 may include more or fewer modules or units, and the connection relationship between the various modules or units is not limited and can be determined according to actual needs. The specific configuration of each module or unit is not limited; it can be constructed from analog devices according to circuit principles, or from digital chips, or in other applicable ways.

[0136] Figure 5 This is a schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure; for example, the electronic device can be used to implement an artificial intelligence processor of the data processing apparatus provided for at least one embodiment of the present disclosure.

[0137] For example, such as Figure 5 As shown, the electronic device 500 includes at least one processor 501 and at least one memory 502. For example, the at least one memory 502 includes one or more computer program modules. For example, the one or more computer program modules are stored in the memory 502 and configured to be executed by the at least one processor 501. The one or more computer program modules include instructions for performing the data processing method described above. When executed by the at least one processor 501, they can perform one or more steps of the data processing method provided in at least one embodiment of this disclosure. The memory 502 and the processor 501 can be interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0138] For example, processor 501 may be an artificial intelligence processor of the data processing apparatus provided in at least one embodiment of this disclosure. For example, it may be a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processor (TPU), an artificial intelligence (AI) accelerator, or other forms of processing units with data processing capabilities and / or program execution capabilities, such as a field-programmable gate array (FPGA). For example, the central processing unit (CPU) may be an x86, ARM, or RISC-V architecture.

[0139] For example, processor 501 can be a general-purpose processor or a special-purpose processor, which can control other components in electronic device 500 to perform desired functions.

[0140] For example, memory 502 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc.

[0141] Figure 6 This is a schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0142] The electronic devices in at least one embodiment of this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0143] The electronic device includes at least one processor and a memory. The processor may be referred to as processing device 601 as described below, and the memory may include at least one of read-only memory (ROM), random access memory (RAM), and storage device 608 as described below. The memory is used to store programs for performing the methods described in the various method embodiments above; the processor is configured to execute the programs stored in the memory. The processor may include a central processing unit (CPU) or other processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0144] like Figure 6 As shown, the electronic device 600 may include a processing device 601 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device 608 into a random access memory (RAM), such as an artificial intelligence processor according to any embodiment of this disclosure. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, ROM 602, and RAM 603 are interconnected via a bus 604. Input / output (I / O) interfaces are also connected to the bus 604.

[0145] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, displays, speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0146] In particular, according to at least one embodiment of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, at least one embodiment of this disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by a processing device 601, it performs the functions defined in the methods of at least one embodiment of this disclosure.

[0147] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In at least one embodiment of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In at least one embodiment of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.

[0148] The aforementioned computer-readable medium may be included in the aforementioned electronic device 600; or it may exist independently and not assembled into the electronic device 600.

[0149] Figure 7 This is a schematic block diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure.

[0150] For example, such as Figure 7 As shown, a non-transitory computer-readable storage medium 700 stores computer-readable instructions 701, which, when executed by at least one processor, perform one or more steps of the data processing method described above.

[0151] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard drive for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media. For example, the readable storage medium may also be... Figure 7 The memory 702 in the memory is described in the foregoing content and will not be repeated here.

[0152] Although the present disclosure has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to the embodiments of the present disclosure, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present disclosure are within the scope of protection claimed by the present disclosure.

[0153] The following points should be noted regarding this disclosure:

[0154] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0155] (2) For clarity, the thickness of layers or regions in the drawings used to describe embodiments of the present disclosure is enlarged or reduced, i.e., these drawings are not drawn to actual scale.

[0156] (3) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0157] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. A data processing method applied to an artificial intelligence processor, characterized in that, The data processing method includes: Receive a single target instruction, wherein the target instruction includes multiple fields for target data, target base address, and target cursor address, respectively; The target instruction is decoded, and the corresponding target storage address is calculated based on the target base address and the target cursor address. The target data is then written according to the target storage address.

2. The method according to claim 1, characterized in that, The AI ​​processor is configured to process multiple threads in parallel. Decoding the target instruction, based on the target base address and the target cursor address, and calculating the corresponding target memory address, includes: Parse the target instructions to obtain the target data, target base address, and target cursor address corresponding to each thread; Access requests from threads pointing to the same target cursor address and without memory access conflicts are merged into a first request, wherein the same target cursor address is the first cursor address; and Based on the target base address and the first cursor address corresponding to each thread in the first request, calculate the target storage address corresponding to each thread in the first request.

3. The method according to claim 2, characterized in that, The step of calculating the target storage address corresponding to each thread in the first request based on the target base address and the first cursor address includes: Read and update the cursor value corresponding to the first cursor address, wherein the read cursor value is the first cursor value; Assign a different offset to each of the threads in the first request; and calculate the target storage address corresponding to each of the threads in the first request based on the target base address, the first cursor value, and the offset.

4. The method according to claim 3, characterized in that, Assigning different offsets to each thread in the first request includes: Obtain the number of threads N in the first request, where N is a positive integer; and Each thread in the first request is assigned a different offset, wherein the offset is an integer between 0 and N-1.

5. The method according to claim 4, characterized in that, The step of reading and updating the cursor value corresponding to the first cursor address includes: Read the cursor value corresponding to the first cursor address as the first cursor value; The second cursor value is obtained by adding the first cursor value and the number of threads; and Write the second cursor value to the first cursor address.

6. The method according to claim 3, characterized in that, The step of calculating the target storage address corresponding to each thread in the first request based on the target base address, the first cursor value, and the offset includes: The target base address, the first cursor value, and the offset corresponding to each thread are added together to obtain the target storage address corresponding to each thread.

7. The method according to claim 2, characterized in that, The step of writing the target data according to the target storage address includes: Since the target storage address corresponding to each thread is located in a different storage bank, the target data corresponding to each thread is merged and written.

8. The method according to any one of claims 1-7, characterized in that, The plurality of fields of the target instruction include: The first field declares the address space to which the target storage address belongs; The second field indicates the total number of target cursor address registers and temporary registers corresponding to the target cursor address; The third field represents the target cursor address register; and At least one fourth field represents the temporary register, wherein the temporary register includes a target base address register corresponding to the target base address and a target data register corresponding to the target data.

9. The method according to claim 8, characterized in that, The at least one fourth field includes: The fifth field represents the target base address register; and The sixth field represents the target data register.

10. A data processing apparatus, characterized in that, The data processing device includes: The instruction fetch unit is configured to receive a single target instruction, wherein the target instruction includes multiple fields for target data, target base address, and target cursor address, respectively; An instruction decoding unit is configured to decode the target instruction; and The loading storage unit is configured to calculate the corresponding target storage address based on the target base address and the target cursor address, and write the target data according to the target storage address.

11. An electronic device, characterized in that, The electronic device includes: At least one processor; At least one memory, including one or more computer program modules; The one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, and the one or more computer program modules are used to implement the data processing method according to any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by at least one processor, the data processing method according to any one of claims 1-9 is performed.

Images