Memory-based data interaction method, processor, device, and medium

Abstract

Embodiments of the present application provide a memory-based data interaction method, a processor, a device, and a medium. The method comprises: obtaining a first operation instruction from a first instruction queue, obtaining pre-stored data indicated by the first operation instruction from a first memory, and obtaining a second operation instruction from a second instruction queue; executing a first-type operation corresponding to the first operation instruction on the pre-stored data to generate first operation data, and updating the data state of the first operation data on the basis of the writing progress of the first operation data in a second memory; when the data state of the first operation data is updated to a preparation state, obtaining first operation data indicated by the second operation instruction from the second memory; and executing a second-type operation corresponding to the second operation instruction on the first operation data to generate second operation data, and writing the second operation data into a third memory. By using the embodiments of the present application, synchronization overhead between different operations can be reduced.

Description

Memory-based data interaction method, processor, device, and medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 28, 2023, with application number 202311089901.8 and application name “Memory-based data interaction method, processor, device and medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a memory-based data interaction method, processor, device, and medium. Background Art

[0003] AI (Artificial Intelligence) processors are specialized chips that combine AI technology and machine learning to make mobile devices intelligent enough to mimic the human brain. They are used to optimize deep learning AI tasks and are systems that utilize multiple processors with specific functions. AI processors involve operations such as data manipulation, vector operations, and matrix operations. These operations may have dependencies, such as vector operations that rely on the resulting data from data manipulation operations.

[0004] In current AI processors, the waiting relationship between different operations is described to ensure that the dependencies between these operations are met. For example, a vector operation can wait for the completion of a data transfer operation before starting to execute. When describing the synchronization relationship between different operations in an AI processor, a barrier or wait type instruction can be inserted between two operations with a waiting relationship. Barrier or wait type instructions can be executed in an instruction control circuit or an instruction queue. Such instructions can block subsequent instructions from being sent to the instruction queue until the waiting condition is met. Due to the delay between the time an instruction is sent to the execution unit and the time the execution unit initiates data transmission, the synchronization overhead between different operations is too large.

[0005] Summary of the Invention

[0006] Embodiments of the present application provide a memory-based data interaction method, processor, device, and medium, which can reduce the synchronization overhead between different operations.

[0007] An embodiment of the present application provides a memory-based data interaction method, including:

[0008] Obtaining a first operation instruction from a first instruction queue, obtaining pre-stored data indicated by the first operation instruction from a first memory, and obtaining a second operation instruction from a second instruction queue;

[0009] Performing a first type of operation corresponding to a first operation instruction on pre-stored data to generate first operation data;

[0010] Writing the first operation data into the second memory, and updating the data state of the first operation data according to the writing progress of the first operation data in the second memory;

[0011] When the data state of the first operation data is updated to the ready state, acquiring the first operation data indicated by the second operation instruction in the second memory;

[0012] A second type of operation corresponding to the second operation instruction is performed on the first operation data to generate second operation data, and the second operation data is written into the third memory.

[0013] In one aspect, an embodiment of the present application provides a processor, comprising N execution units and N instruction queues, wherein one execution unit corresponds to one instruction queue, the N execution units include a first execution unit and a second execution unit, the N instruction queues include a first instruction queue corresponding to the first execution unit, and a second instruction queue corresponding to the second execution unit, where N is an integer greater than 1;

[0014] The first execution unit is used to obtain a first operation instruction from the first instruction queue and obtain pre-stored data indicated by the first operation instruction from the first memory;

[0015] The second execution unit is used to obtain a second operation instruction from the second instruction queue;

[0016] The first execution unit is further configured to execute a first type of operation corresponding to a first operation instruction on pre-stored data to generate first operation data, write the first operation data into the second memory, and update a data state of the first operation data according to a writing progress of the first operation data into the second memory;

[0017] The second execution unit is configured to obtain the first operation data indicated by the second operation instruction from the second memory when the data state of the first operation data is a ready state;

[0018] The second execution unit is further configured to execute a second type of operation corresponding to a second operation instruction on the first operation data, generate second operation data, and write the second operation data into the third memory.

[0019] The processor further includes an instruction control unit;

[0020] The instruction control unit is used to obtain a first initial instruction and a second initial instruction, decode the first initial instruction and the second initial instruction, and obtain a first operation instruction corresponding to the first initial instruction and a second operation instruction corresponding to the second initial instruction;

[0021] The instruction control unit is further configured to transmit the first operation instruction to the first instruction queue and transmit the second operation instruction to the second instruction queue.

[0022] In one aspect, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory is connected to the processor, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method provided in the above aspect of the embodiment of the present application.

[0023] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded and executed by a processor, so that a computer device with a processor executes the method provided in the above aspect of the embodiment of the present application.

[0024] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in the above aspect.

[0025] In an embodiment of the present application, a first operation instruction is obtained from a first instruction queue, pre-stored data indicated by the first operation instruction is obtained from a first memory, and a second operation instruction is obtained from a second instruction queue; a first type of operation corresponding to the first operation instruction is performed on the pre-stored data to generate first operation data, and the first operation data is written to a second memory; when the data state of the first operation data in the second memory is a ready state, the first operation data indicated by the second operation instruction is obtained from the second memory; a second type of operation corresponding to the second operation instruction is performed on the first operation data to generate second operation data, and the second operation data is written to a third memory. Wherein, the first operation instruction and the second operation instruction need to read data from the memory, complete a specific operation, and write the result back to the memory. The first operation instruction and the second operation instruction need to be executed in the correct order. The data state of the first operation data is updated according to the writing progress of the first operation data in the second memory. When the data state is a ready state, the operation corresponding to the second operation instruction is immediately executed. This can reduce the synchronization overhead between different operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] FIG1 is a schematic diagram of an architecture of a processor provided in an embodiment of the present application;

[0027] FIG2 is a flowchart of a memory-based data interaction method according to an embodiment of the present application;

[0028] FIG3 is a schematic diagram of an embodiment of the present application providing a method of using a data tag to indicate a data status;

[0029] FIG4 is a first schematic diagram of an execution flow between different operations in a processor provided by an embodiment of the present application;

[0030] FIG5 is a first schematic diagram of operation splitting between different operations in a processor provided by an embodiment of the present application;

[0031] FIG6 is a second flow chart of a memory-based data interaction method provided in an embodiment of the present application;

[0032] FIG7 is a schematic diagram of an embodiment of the present application providing a method of using an address storage space to indicate a data state;

[0033] FIG8 is a second schematic diagram of operation splitting between different operations in a processor provided by an embodiment of the present application;

[0034] FIG9 is a second schematic diagram of the architecture of a processor provided in an embodiment of the present application;

[0035] FIG10 is a second schematic diagram of an execution flow between different operations in a processor provided in an embodiment of the present application;

[0036] FIG11 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0038] Please refer to Figure 1, which is a schematic diagram of the architecture of a processor provided in an embodiment of the present application. The processor may be an artificial intelligence (AI) processor. An AI processor is a dedicated processor for running AI algorithms. The difference from a traditional processor lies in the emphasis on specificity or versatility. The AI ​​processor can be understood as sacrificing a certain degree of versatility in exchange for efficiency improvements in specific businesses. The three major supports of AI are hardware, algorithms, and data, where hardware refers to the chip that runs the AI ​​algorithm and the corresponding computing platform; the AI ​​processor may include a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), etc., which is not limited in this application. For ease of understanding, the embodiment of this application takes the AI ​​processor as an example to describe the processor architecture.

[0039] Among them, the AI ​​processor architecture may include an instruction control unit, N instruction queues, N execution units, and S memories (memories may also be referred to as storage units), etc. One instruction queue may correspond to one execution unit, one execution unit may correspond to one or two memories, and of course, one memory may correspond to one or more execution units, and both N and S may be integers greater than 1. For example, when the data that an execution unit needs to read in and the data that is ultimately written are stored in the same memory, then one execution unit at this time corresponds to one memory; when the data that an execution unit needs to read in and the data that is ultimately written are stored in different memories, then one execution unit at this time corresponds to two memories. Among them, N instruction queues may refer to instruction queues in the AI ​​processor for receiving different operation instructions; N execution units may refer to execution units in the AI ​​processor for performing different operations; and S memories may refer to memories for data interaction with each execution unit in the AI ​​processor. It should be understood that the embodiments of the present application do not limit the number of instruction queues, execution units, and memories in the AI ​​processor architecture.

[0040] As shown in Figure 1, the N instruction queues in the AI ​​processor architecture may specifically include: DMA (Direct Memory Access, direct memory access, DMA is a memory access technology) instruction queue, vector instruction queue and matrix instruction, etc. The DMA instruction queue may include: DMA0 instruction queue, DMA1 instruction queue, DMA2 instruction queue and DMA3 instruction queue, etc. For the convenience of description, they can be referred to as DMA0~3 instruction queues. Here, DMA0~3 can be understood as the priority of DMA operation instructions, such as the priority of DMA0 operation instruction is higher than the priority of DMA1 operation instruction, and the priority of DMA1 operation instruction is higher than the priority of DMA2 operation instruction, etc.

[0041] The N execution units in the AI ​​processor architecture may specifically include: DMA0 execution unit, vector execution unit, DMA1 execution unit, DMA2 execution unit, matrix execution unit, and DMA3 execution unit. The DMA0 execution unit is used to execute DMA0 operation instructions in the DMA0 instruction queue, the vector execution unit is used to execute vector operation instructions in the vector instruction queue, the DMA1 execution unit is used to execute DMA1 operation instructions in the DMA1 instruction queue, the DMA2 execution unit is used to execute DMA2 operation instructions in the DMA2 instruction queue, the matrix execution unit is used to execute matrix operation instructions in the matrix instruction queue, and the DMA3 execution unit is used to execute DMA3 operation instructions in the DMA3 instruction queue.

[0042] The S memories in the AI ​​processor architecture may specifically include: memory A, memory B, memory C, memory D, and memory E. Memory A can exchange data with the DMA0 execution unit and the DMA1 execution unit; memory B can exchange data with the DMA0 execution unit, the vector execution unit, and the DMA1 execution unit; memory C can exchange data with the DMA2 execution unit and the DMA3 execution unit; memory D can exchange data with the DMA2 execution unit and the matrix execution unit; and memory E can exchange data with the matrix execution unit and the DMA3 execution unit.

[0043] The instruction control unit in the AI ​​processor is responsible for instruction fetching and decoding. The DMA0-3 instruction queues, vector instruction queues, and matrix instruction queues in the AI ​​processor are responsible for receiving the operation instructions decoded by the instruction control unit and sending them to the corresponding execution units. The DMA0-3 execution units, vector execution units, and matrix execution units are responsible for receiving the operation instructions sent by the corresponding instruction queues and executing the corresponding operations. Different execution units can be used to perform different operations.

[0044] As shown in Figure 1, the six execution units in the AI ​​processor architecture can perform the following six operations. The details of these six operations are as follows:

[0045] DMA0 operation: Executed by the DMA0 execution unit in the AI ​​processor architecture. During the DMA0 operation, the DMA0 execution unit can read data 0 from memory A, then perform a DMA0 operation on the data 0 to generate data 1, and then write data 1 back to memory B.

[0046] Vector operations are performed by the vector execution unit in the AI ​​processor architecture. During the execution of vector operations, the vector execution unit can read data 1 from memory B, perform vector operations on data 1, generate data 2, and write data 2 back to memory B.

[0047] DMA1 operation: Executed by the DMA1 execution unit in the AI ​​processor architecture. During the DMA1 operation, the DMA1 execution unit can read data 2 from memory B, then perform a DMA1 operation on the data 2 to generate data 3, and then write data 3 back to memory A.

[0048] DMA2 operation: Executed by the DMA2 execution unit in the AI ​​processor architecture. During the DMA2 operation, the DMA2 execution unit can read data 4 from memory C, perform a DMA2 operation on the data 4, generate data 5, and write data 5 back to memory D.

[0049] Matrix operation: Executed by the matrix execution unit in the AI ​​processor architecture. During the matrix operation, the matrix execution unit can read data 5 from memory D, perform matrix operations on the data 5, generate data 6, and write data 6 back to memory E.

[0050] DMA3 operation: Executed by the DMA3 execution unit in the AI ​​processor architecture. During the DMA3 operation, the DMA3 execution unit can read data 6 from memory E, perform a DMA3 operation on the data 6, generate data 7, and write data 7 back to memory C.

[0051] DMA0-3 operations may include, but are not limited to, linear copy, data transposition, and data bordering. Vector operations may include, but are not limited to, mathematical operations such as addition, subtraction, multiplication, and division between vectors, scientific operations such as exponentials and logarithms, shifts, and sorting. Matrix operations may include, but are not limited to, matrix multiplication and matrix multiplication-accumulation. Data 0 to Data 7 may be data blocks stored in the memory. A data block may be one or more groups of sequentially arranged records, serving as the unit of data transferred between the memory and an input device, output device, or external register.

[0052] It is understandable that the above six operations have the following four dependency relationships, which can be expressed as follows:

[0053] Dependency 1: The vector operation depends on the result data generated by the DMA0 operation.

[0054] Dependency 2: The DMA1 operation depends on the result data generated by the vector operation.

[0055] Dependency 3: Matrix operations depend on result data generated by DMA2 operations.

[0056] Dependency 4: DMA3 operations depend on the result data generated by matrix operations.

[0057] Among them, there are multiple hardware execution units inside the AI ​​processor (for example, the DMA0~3 execution units, vector execution units and matrix execution units shown in Figure 1, etc.), and DMA0~3 operations are used to be responsible for data transfer between different memories. Operations such as matrix operations and vector operations are used to read data from the memory, complete specific operations, and write the results back to the memory. The data transfer and computing operations of different hardware execution units need to be synchronized according to the production and consumer relationship, memory capacity limitations and other requirements to ensure that the operations of different hardware execution units are executed in the correct order. The embodiment of the present application proposes a distributed synchronous hardware architecture to ensure the correctness of the execution order of different operations, such as the synchronous control of the processor data transfer operation when the hardware execution unit interacts with the memory. The "dependency" involved in the embodiment of the present application can be understood as the data actually processed by one operation is the result data generated by another operation; for example, the data actually processed by the vector operation in dependency 1 is the result data generated by the DMA0 operation, so the vector operation can only be executed after the DMA0 operation is completed.

[0058] It should be noted that the AI ​​processor architecture shown in Figure 1 can be configured in a server or in a terminal device, and this application does not limit this. Among them, terminal devices may include smart phones, tablet computers, laptops, PDAs, mobile internet devices (MIDs), wearable devices (such as smart watches, smart bracelets, etc.), intelligent voice interaction devices, smart home appliances (such as smart TVs, etc.), vehicle-mounted devices, aircraft and other electronic devices. This application does not limit the type of terminal device. The server can be an independent physical server or a server cluster or distributed system composed of multiple physical servers. This application does not limit the type of server.

[0059] Please refer to Figure 2, which is a flowchart of a memory-based data interaction method provided in an embodiment of the present application. It is understandable that the memory-based data interaction method can be executed by a processor. The present embodiment of the application uses an AI processor as an example for illustration. As shown in Figure 2, the memory-based data interaction method can include steps S101 to S105:

[0060] Step S101: obtain a first operation instruction from a first instruction queue, obtain pre-stored data indicated by the first operation instruction from a first memory, and obtain a second operation instruction from a second instruction queue.

[0061] In an embodiment of the present application, the AI ​​processor may include an instruction control unit, N execution units, and N instruction queues, with one execution unit corresponding to one instruction queue. For ease of understanding, the following description is taken as an example the first execution unit and the second execution unit among the N execution units. Here, the operation performed by the second execution unit depends on the result data generated by the operation performed by the first execution unit. The instruction queue corresponding to the first execution unit among the N instruction queues is called the first instruction queue, and the instruction queue corresponding to the second execution unit among the N instruction queues is called the second instruction queue.

[0062] The instruction control unit in the AI ​​processor can obtain the first initial instruction and the second initial instruction, decode the first initial instruction and the second initial instruction, and obtain a first operation instruction corresponding to the first initial instruction and a second operation instruction corresponding to the second initial instruction; transmit the first operation instruction to the first instruction queue and the second operation instruction to the second instruction queue through the instruction control unit.

[0063] Among them, during the instruction execution process, subsequent instructions can be pre-fetched in advance, and multiple instructions can be read into the instruction cache at one time. The instruction cache here may refer to a storage area in the AI ​​processor for caching multiple instructions. Multiple instructions can enter the instruction cache in the AI ​​processor from the system memory through the bus interface. All instructions in the instruction cache can wait for subsequent hardware (for example, each execution unit in the AI ​​processor) to be quickly and automatically decoded or calculated. The instruction control unit in the AI ​​processor can read the first initial instruction and the second initial instruction from the instruction cache. The first initial instruction and the second initial instruction may be the original instructions in the instruction cache, and the above-mentioned decoding process may refer to the instruction control unit in the AI ​​processor parsing the original instructions in the instruction cache into operation instructions that can be recognized by the execution unit in the AI ​​processor.

[0064] It can be understood that the first operation instruction in the embodiment of the present application can be an operation instruction that is dependent on any one of the dependency relationships in the embodiment corresponding to Figure 1, such as the first operation instruction can be the DMA0 operation instruction in dependency relationship 1, or can be the vector operation instruction in dependency relationship 2, or can be the DMA2 operation instruction in dependency relationship 3, or can be the matrix operation instruction in dependency relationship 4, etc.

[0065] It should be noted that the second operation instruction in the embodiment of the present application depends on the result data generated by the first operation instruction, that is, the data actually processed by the second type operation corresponding to the second operation instruction is the result data after the first type operation corresponding to the first operation instruction is completed, that is, the second type operation depends on the first operation data generated by the first type operation. For example, when the first operation instruction is a DMA0 operation instruction, the first instruction queue can be a DMA0 instruction queue, the second operation instruction can be a vector operation instruction, and the second instruction queue can be a vector instruction queue. When the first operation instruction is a vector operation instruction, the first instruction queue can be a vector instruction queue, the second operation instruction can be a DMA1 operation instruction, and the second instruction queue can be a DMA1 instruction queue. When the first operation instruction is a DMA2 operation instruction, the first instruction queue can be a DMA2 instruction queue, the second operation instruction can be a matrix operation instruction, and the second instruction queue can be a matrix instruction queue. When the first operation instruction is a matrix operation instruction, the first instruction queue can be a matrix instruction queue, the second operation instruction can be a DMA3 operation instruction, and the second instruction queue can be a DMA3 instruction queue, etc. The embodiment of the present application does not limit this.

[0066] It should be understood that each instruction queue involved in the embodiments of the present application can refer to a special linear list, in which the front end (front) of the linear list performs deletion operations, and the rear end (rear) of the linear list performs insertion operations. The queue can be considered as a linear list with restricted operations; wherein, the end where the insertion operation is performed can be called the tail of the queue, and the end where the deletion operation is performed can be called the head of the queue.

[0067] Furthermore, the first operation instruction in the first instruction queue can be sent to the first execution unit according to the order in which the elements in the first instruction queue are read, and the first operation instruction in the first instruction queue can be cleared; the first operation instruction is parsed and processed by the first execution unit to obtain instruction parsing information corresponding to the first operation instruction, and pre-stored data is obtained in the first memory according to the instruction parsing information. Among them, the first execution unit may refer to a hardware execution unit for executing the operation instruction in the first instruction queue, that is, the first execution unit can be used to execute the above-mentioned first operation instruction. The instruction parsing information corresponding to the first operation instruction can be used to instruct the first execution unit to read pre-stored data from the first memory, and the pre-stored data can refer to the data pre-stored in the first memory that the first operation instruction actually needs to process. The first operation instruction may include but is not limited to: the instruction type corresponding to the first operation instruction, the data that the first operation instruction needs to operate (or the storage location of the data that the first operation instruction needs to operate), etc.

[0068] Similarly, the second operation instruction in the second instruction queue can be sent to the second execution unit according to the order in which the elements in the second instruction queue are read, and the second operation instruction in the second instruction queue can be cleared at this time. After receiving the second operation instruction sent by the second instruction queue, the second execution unit can parse and process the second operation instruction to obtain the instruction parsing information corresponding to the second operation instruction. It should be understood that the second operation instruction at this time is temporarily unable to read data into the corresponding second memory, but needs to wait at the read interface associated with the second memory until the result data generated after the execution of the first operation instruction is completed is written to the second memory. Among them, the second execution unit can refer to a hardware execution unit for executing the operation instruction in the second instruction queue, that is, the second execution unit can be used to execute the above-mentioned second operation instruction. The instruction parsing information corresponding to the second operation instruction can be used to instruct the second execution unit to read the result data generated after the execution of the first operation instruction from the second memory.

[0069] For example, as shown in Figure 1, when the first operation instruction is a DMA0 operation instruction, the first execution unit at this time can be a DMA0 execution unit, the first memory can be memory A, the pre-stored data can be data 0 in memory A, and the result data generated after the DMA0 operation instruction is executed can be data 1 in memory B (memory B at this time can be used as a second memory); the second operation instruction can be a vector operation instruction, and the second execution unit can be a vector execution unit.

[0070] In an embodiment of the present application, the time when the first execution unit receives the first operation instruction and the time when the second execution unit receives the second operation instruction may be the same or different. For example, the time when the first execution unit receives the first operation instruction may be earlier than the time when the second execution unit receives the second operation instruction, or the time when the first execution unit receives the first operation instruction may be later than the time when the second execution unit receives the second operation instruction. The present application does not limit the timing relationship of each execution unit receiving the operation instruction. Regardless of whether the time when the first execution unit receives the first operation instruction is the same as the time when the second execution unit receives the second operation instruction, the second execution unit needs to wait for the first execution unit to complete the first type of operation corresponding to the first operation instruction before it can execute the second type of operation corresponding to the second operation instruction. In other words, the first type of operation may refer to a dependent operation in any dependency relationship in the embodiment corresponding to Figure 1, and the second type of operation may refer to a dependent operation in any dependency relationship in the embodiment corresponding to Figure 1, and the first type of operation and the second type of operation are two operations in the same dependency relationship. The data state of the first operation data in the second memory may be configured as an unprepared state. When the write progress of the first operation data in the second memory meets a specific condition, the data state corresponding to the first operation data may be updated from an unprepared state to a ready state. The specific condition may be that the write progress is 100% (i.e., the first execution unit completes the write data operation on the first operation data), or the write progress indicates that the first address segment (multiple address segments may be used to store the first operation data) used to store the first operation data in the second memory completes the write data operation (the first execution unit may continue to perform write data operations on the second address segment, the third address segment, etc.). The specific implementation process will be described in subsequent steps.

[0071] Among them, the data written into the memory can correspond to a data state. For ease of understanding, the data state corresponding to the first operation data is described below by taking the first operation data as an example. The data state corresponding to the second data can be represented by a status bit. The status bit here does not need to occupy the storage space in the second memory, but is represented by a bit in the register on the side of the second memory; for example, when the bit in the register is 0, it indicates that the corresponding data state is unprepared; when the bit in the register is 1, it indicates that the corresponding data state is ready. The register is a component of the central processing unit (CPU). The register is a high-speed storage component with limited storage capacity that can be used to temporarily store instructions, data and addresses. By using registers to temporarily store the data state corresponding to the data in the memory within the AI ​​processor, the storage space of the memory can be saved.

[0072] In a feasible embodiment, the first execution unit can configure a data tag for the first operation data in the second memory, and characterize the data state of the first operation data in the second memory by the tag state of the data tag. For example, in the process of the first execution unit writing the first operation data to the second memory, the writing progress of the first operation data in the second memory can be obtained, and a data tag can be configured for the first operation data in the second memory. At this time, the data tag can be bound to the data state corresponding to the first operation data; when the writing progress of the first operation data in the second memory indicates that the writing of the first operation data is complete (for example, the writing progress is 100%), the tag state corresponding to the data tag can be updated to a ready state. When the tag state is a ready state, it means that the data state of the first operation data in the second memory is a ready state.

[0073] It can be understood that a data tag in a prepared state can be used to indicate that the data state of the first operation data in the second memory is in a prepared state, i.e., the first operation data has been written to the second memory, and the data tag in a prepared state is used to indicate that the second execution unit can start reading (or reading) the first operation data; a data tag in an unprepared state indicates that the data state of the first operation data in the second memory is in an unprepared state, i.e., the first operation data has not been written to the second memory, and the data tag in an unprepared state indicates that the second execution unit needs to wait and is temporarily unable to read the first operation data from the second memory. The data tag corresponding to the first operation data can be stored in a register on the side of the second memory, and the data tag corresponding to the first operation data can occupy a bit in the register, where the bit can serve as a status bit corresponding to the data tag, and the status bit can be used to indicate the tag state of the data tag corresponding to the first operation data; when the status bit is 0, it indicates that the tag state corresponding to the data tag of the first operation data is in an unprepared state (it can also be understood that the data state corresponding to the first operation data is in an unprepared state); when the status bit is 1, it indicates that the tag state corresponding to the data tag of the first operation data is in a prepared state (it can also be understood that the data state corresponding to the first operation data is in a prepared state).

[0074] Please refer to Figure 3, which is a schematic diagram of an embodiment of the present application that uses a data label to indicate the data status. As shown in Figure 3, after the first execution unit in the AI ​​processor receives the first operation instruction issued by the first instruction queue, it can read the pre-stored data indicated by the first operation instruction into the first memory, perform the first type of operation corresponding to the first operation instruction on the pre-stored data, and generate result data (first operation data). The first execution unit can write the first operation data to the second memory and configure a data label for the first operation data in the second memory. The data label can occupy a status bit in the register on the side of the second memory (a status bit can correspond to a bit in a register).

[0075] During the process of the first execution unit writing the first operation data to the second memory, the status bit of the data tag corresponding to the first operation data in the register may be "0". Here, "0" indicates that the first operation data has not been completely written to the second memory. It can also be understood that the first execution unit has not completed the write operation of the first operation data, that is, the data status of the first operation data in the second memory is not ready. At this time, the second execution unit cannot read the first operation data into the second memory temporarily and can only wait to read it in. When the first operation data is written to the second memory, the status bit of the data tag corresponding to the first operation data in the register can be updated to "1". Here, "1" indicates that the first execution unit has completed the write operation of the first operation data, that is, the data status of the first operation data in the second memory is ready. At this time, the second execution unit can start to read the first operation data into the second memory.

[0076] Optionally, the embodiment of the present application may also use the address storage space in the memory to indicate the data state of the first operation data in the second memory. At this time, the data state may be bound to the address storage space. The specific implementation process will be described in subsequent embodiments.

[0077] Step S104 : when the data state of the first operation data is updated to the ready state, obtaining the first operation data indicated by the second operation instruction in the second memory.

[0078] Specifically, when the data state of the first operation data in the second memory is updated to the ready state (for example, the tag state of the data tag corresponding to the first operation data is the ready state, that is, the state bit in the register is "1"), the second execution unit that has received the second operation instruction issued by the second instruction queue can read the first operation data indicated by the second operation instruction from the second memory. The process of the second execution unit reading the first operation data is the same as the process of the first execution unit reading the pre-stored data, and will not be repeated here.

[0079] Step S105 , performing a second type of operation corresponding to a second operation instruction on the first operation data to generate second operation data, and writing the second operation data into a third memory.

[0080] Specifically, the second execution unit in the AI ​​processor can perform a second type of operation corresponding to a second operation instruction on the first operation data to generate new data. In this case, the new data can be understood as the result data generated by the second type of operation. In the embodiment of the present application, the result data generated by the second type of operation can be referred to as second operation data, and the second operation data can be written to the third memory. The process of writing the second operation data to the third memory can refer to the process of writing the first operation data to the second memory, and will not be repeated here. The second memory and the third memory can be the same memory or different memories, which is related to the first operation instruction and the second operation instruction.

[0081] Optionally, when the first type of operation includes a first data transfer operation (the aforementioned DMA0 operation), the second type of operation includes a vector operation, the first memory and the second memory are different memories, and the second memory and the third memory are the same memory; when the first type of operation includes a vector operation, the second type of operation includes a second data transfer operation (the aforementioned DMA1 operation), the first memory and the second memory are the same memory, and the second memory and the third memory are different memories; when the first type of operation includes a third data transfer operation (the aforementioned DMA2 operation), the second type of operation includes a matrix operation, and the first memory, the second memory and the third memory are all different memories; when the first type of operation includes a matrix operation, the second type of operation includes a fourth data transfer operation (the aforementioned DMA3 operation), and the first memory, the second memory and the third memory are all different memories.

[0082] In one or more embodiments, for ease of understanding, dependency relationship 1 in the embodiment corresponding to FIG. 1 is used as an example for specific description. In this case, the first type of operation may be a DMA0 operation, and the second type of operation may be a vector operation; in other words, the first execution unit is a DMA0 execution unit, and the second execution unit is a vector execution unit. Please refer to FIG. 4 , which is a schematic diagram of the execution flow between different operations in a processor provided in an embodiment of the present application. As shown in FIG. 4 , dependency relationship 1 means that the vector operation depends on the result data generated by the DMA0 operation (the aforementioned first operation data, which may be referred to as data 1 here).

[0083] As shown in FIG4 , the data state of the completed data 1 can be configured as ready in the DMA0 operation. A corresponding configuration can be added to the vector operation. This configuration can be expressed as “checking whether the data state of data 1 is ready.” For example, the status bit of the data tag corresponding to data 1 in the side register of memory B can be checked to see if it is “1.” Specifically, the DMA0 execution unit begins writing data 1 to memory B (the second memory). When the DMA0 execution unit completes writing data 1, the data state of data 1 can be updated to ready, for example, by updating the status bit of the data tag corresponding to data 1 in the side register of memory B from “0” to “1.”

[0084] During the process of the vector execution unit executing a vector operation, the vector execution unit can check whether the data status of data 1 in memory B is in the ready state. If the data status of data 1 is in the ready state, the vector execution unit can begin to read data 1 from memory B. If the data status of data 1 is not in the ready state (for example, the status bit in the side register of memory B is "0"), the vector execution unit needs to wait until the data status of data 1 is updated to the ready state. The vector execution unit then performs a data read operation, i.e., begins to read data 1 from memory B. In other words, in the AI ​​processor, when a data label is used to indicate the data status of data 1, for DMA0 operations and vector operations that require synchronization, the DMA0 operation writing data 1 to memory B and the vector operation reading data 1 from memory B can be performed continuously; among which, writing data 1 to memory B can be understood as writing memory B, and reading data 1 from memory B can be understood as reading memory B.

[0085] Please also refer to Figure 5, which is a schematic diagram of the first operation breakdown between different operations in a processor provided by an embodiment of the present application. In the coordinate diagram shown in Figure 5, the horizontal axis represents time, and the vertical axis represents operation. Taking dependency relationship 1 in the embodiment corresponding to Figure 1 as an example, the steps of a DMA0 operation can be broken down into the instruction control unit sending an instruction, the DMA0 instruction queue sending an instruction, the DMA0 execution unit parsing the instruction, the DMA0 execution unit reading data 0 from memory A, and the DMA0 execution unit writing data 1 from memory B. The instructions here can refer to DMA0 operation instructions.

[0086] The steps of vector operation can be divided into instruction control unit synchronous control and sending instructions, vector instruction queue sending instructions, vector execution unit parsing instructions, vector execution unit reading data 1 from memory B, and vector execution unit writing data 2 from memory B; the instructions here may refer to vector operation instructions.

[0087] It's understandable that when using data tags to indicate data status, data tags must be configured for both the data written by DMA0 and the data read by vector operations, ensuring that the same data tag is configured for the same data. When using data tags to configure data status, a vector operation reading data 1 must wait until DMA0 has completely finished writing data 1 before it can begin. As shown in Figure 5, the time at which DMA0 completes writing data 1 from memory B coincides with the time at which the vector execution unit begins reading data 1 from memory B. In other words, when using data tags to indicate data status, the two synchronized memory read and write operations can proceed continuously.

[0088] In an embodiment of the present application, the first type of operation and the second type of operation that depends on the first operation data generated by the first type of operation need to be executed in the correct order, that is, the first type of operation needs to be executed before the second type of operation. After the first execution unit reads the pre-stored data from the first memory, it can perform the first type of operation on the pre-stored data to generate the first operation data; the first execution unit can write the first operation data into the second memory, and update the data state corresponding to the first operation data according to the writing progress of the first operation data in the second memory (such as a data tag can be used to indicate the data state corresponding to the first operation data). When the data state corresponding to the first operation data is in the ready state, the second execution unit can immediately read the first operation data from the second memory and perform the second type of operation on the first operation data. This can reduce the read and write time interval between the two synchronized operations, thereby reducing the time interval between different operations. The synchronization overhead is reduced, thereby improving the data processing efficiency of the AI ​​processor. In addition, the synchronization control between different operations in the AI ​​processor can be performed when the hardware execution unit (for example, the first execution unit and the second execution unit) interacts with the memory. This can avoid inserting a barrier or wait-type instruction between two operations with a dependency to block subsequent instructions from being sent from the instruction control unit to the instruction queue, or blocking subsequent instructions from being sent from the instruction queue to the corresponding execution unit. In other words, it can avoid synchronization control between different operations on the instruction control path in the AI ​​processor, thereby reducing the instruction complexity in the AI ​​processor.

[0089] Please refer to Figure 6, which is a second flow diagram of a memory-based data interaction method provided in an embodiment of the present application. It is understandable that the memory-based data interaction method can be executed by a processor. The present embodiment of the application uses an AI processor as an example for illustration. As shown in Figure 6, the memory-based data interaction method can include steps S201 to S208:

[0090] Step S201: obtain a first operation instruction from a first instruction queue, obtain pre-stored data indicated by the first operation instruction from a first memory, and obtain a second operation instruction from a second instruction queue.

[0091] Step S202 : performing a first type of operation corresponding to a first operation instruction on pre-stored data to generate first operation data.

[0092] The specific implementation process of step S201 and step S202 can refer to the relevant description of step S101 to step S102 in the embodiment corresponding to FIG2 , and will not be repeated here.

[0093] Step S203: allocate an address storage space for the first operation data in the second memory, write the first operation data into the address segment included in the address storage space, and obtain the writing progress of the first operation data in the second memory.

[0094] Specifically, the first execution unit can allocate address storage space in the second memory for the first operation data, and represent the data state of the first operation data in the second memory by the storage state corresponding to the address segment in the address storage space. At this time, the storage state corresponding to the address segment in the address storage space can be bound to the data state of the first operation data in the second memory. The process of the first execution unit in the AI ​​processor allocating address storage space in the second memory for the first operation data may include but is not limited to: the first execution unit in the AI ​​processor can obtain the data capacity corresponding to the first operation data, and obtain the hardware rule information corresponding to the second memory; then, based on the hardware rule information, a unit address space range can be determined in the second memory, where the unit address space range is used to represent the address space range of an address segment; and based on the data capacity and the unit address space range, the address storage space of the first operation data in the second memory is determined.

[0095] In which, the address storage space of the first operation data in the second memory may include one or more address segments, and the number of address segments contained in the address storage space can be determined by the data capacity of the first operation data and the unit address space range. Each address segment contained in the address storage space can be used to store part of the data information in the first operation data, such as each address segment can be used to store a data information fragment in the first operation data.

[0096] It can be understood that, in the process of writing the first operation data into the second memory, the first operation data can be split into multiple data information segments, and one data information segment can be written into an address segment; each address segment can have its corresponding storage status, and the storage status of an address segment can be used to indicate whether the address segment stores a data information segment, such as the storage status can include a stored state and an unstored state. When an address segment stores a data information segment, it indicates that the storage status corresponding to the address segment is a stored state; when an address segment is empty, that is, no data information segment has been stored, it indicates that the storage status corresponding to the address segment is an unstored state. Among them, the address storage space of the first operation data in the second memory can include one or more address segments, and the storage status corresponding to each address segment can occupy a status bit in the register on the side of the second memory, and the status bit here can be a bit in the register; that is, each address segment in the address storage space can maintain a status bit.

[0097] For example, assuming that the address storage space of the first operation data in the second memory includes M address segments, then M status bits (e.g., M bits) can be occupied in the register on the side of the second memory to represent the data state of the first operation data in the second memory; M can be a positive integer, such as M can be 1, 2, etc. When the first execution unit has not yet written the first operation data to the second memory, or the first execution unit has not yet completed the write operation of the first address segment in the address storage space, the M status bits of the M address segments in the address storage space in the register on the side of the second memory are all "0"; that is, the storage state of the M address segments in the address storage space is all unstored. When the first execution unit completes the write operation of the first address segment in the address storage space, the status bit of the first address segment in the register on the side of the second memory is updated from "0" to "1", and the status bits of the remaining address segments of the M address segments except the first address segment in the register on the side of the second memory remain "0"; that is, the storage state of the first address segment in the address storage space is stored, and the storage state of the remaining address segments is unstored. It should be understood that when the storage status corresponding to the first address segment in the address storage space is updated to the stored state, it can indicate that the data status corresponding to the first operation data in the second memory is the ready state. At this time, the second execution unit can start to read the first operation data from the second memory, specifically, read the data information fragment stored in the first address segment from the second memory.

[0098] It is understandable that, according to different hardware specification information, the unit address space range in the memory corresponding to a status bit can be flexibly defined, that is, the unit address space range contained in each address segment in the address storage space can be flexibly defined. For example, 1 status bit is maintained for every 512Byte (byte, one byte equals 8 bits), and an address segment at this time contains 512Byte; 1 status bit is maintained for every 1KB (kilobyte, one kilobyte equals 1024 bytes), and an address segment at this time includes 1KB, etc., and the embodiments of the present application do not limit this. Every time the DMA0 operation completes the write operation of an address segment in the address storage space, the storage status of the address segment can be updated. Starting from the storage status of the first address segment in the address storage space being updated to the stored state, the second execution unit can start to read the first operation data from the second memory.

[0099] Step S204 , when the writing progress of the first operation data indicates that the writing of the i-th address segment in the address storage space is completed, the storage state corresponding to the i-th address segment in the address storage space is updated to a stored state; i is a positive integer less than or equal to M.

[0100] Specifically, in the process of the first execution unit writing the first operation data to the second memory, the data can be written in the order of the M address segments in the address storage space. When the writing of any address segment in the address storage space (for example, the i-th address segment, i is a positive integer less than or equal to M) is completed, the storage state corresponding to the i-th address segment in the address storage space can be updated to the stored state, that is, the state bit of the i-th address segment in the side register of the second memory can be updated to "1". For example, when the writing of the first address segment in the address storage space is completed, the storage state corresponding to the first address segment can be updated to the stored state; the first execution unit can continue to write the first operation data to the second memory, and when the writing of the second address segment in the address storage space is completed, the storage state corresponding to the second address segment can be updated to the stored state, until the writing of the last address segment in the address storage space is completed, that is, the complete first operation data is written to the second memory, and the storage states corresponding to the M address segments in the address storage space can all be the stored state.

[0101] Step S205 : When the storage state of the i-th address segment in the address storage space is updated to the stored state, it is determined that the data state of the first operation data in the second memory is the ready state.

[0102] Step S206 : obtaining data of the i-th address segment in the address storage space through the second execution unit, and determining the data of the i-th address segment as the first operation data indicated by the second operation instruction.

[0103] Specifically, when the second execution unit in the AI ​​processor is waiting for the data state of the first operation data in the second memory to be updated to the ready state, when the storage state of the first address segment in the address storage space of the first operation data in the second memory is updated to the stored state, it can immediately start reading the first operation data from the second memory, specifically reading the data stored in the first address segment (i.e., the aforementioned data information fragment) from the second memory. When the storage state of any address segment (for example, the i-th address segment) in the address storage space of the first operation data in the second memory is the stored state, it can be determined that the data state of the first operation data in the second memory is the ready state.

[0104] For example, when the storage status of the first address segment in the address storage space is the stored state, the second execution unit can start to read the data stored in the first address segment from the second memory. If the second execution unit completes the read operation of the first address segment, the storage status of the second address segment in the address storage space has been updated to the stored state, then the second execution unit can continue to read the data in the second address segment; if the second execution unit completes the read operation of the first address segment, the storage status of the second address segment in the address storage space has not been updated to the stored state, then the second execution unit needs to wait for the storage status of the second address segment to be updated to the stored state before continuing to read the data in the second address segment; and so on, until the second execution unit completes the write operation of the last address segment in the address storage space.

[0105] It should be noted that, after the second execution unit finishes reading the data in the first address segment in the address storage space, it can release the first address segment in the address storage space, and update the storage status corresponding to the first address segment from the stored status to the unstored status, such as updating the status bit of the first address segment in the second memory side register from "1" to "0"; and then, when the storage status of the second address segment in the address storage space is updated to the stored status, the second execution unit continues to read the data of the second address segment in the address storage space. After the read operation is completed, the second address segment in the address storage space can be released, and the storage status corresponding to the first address segment can be updated from the stored status to the unstored status, such as updating the status bit of the second address segment in the second memory side register from "1" to "0"; that is, each time the second execution unit completes the read operation of an address segment, it can release an address segment in the address storage space, and so on, releasing all address segments in the address storage space used to store the first operation data. Optionally, when the second execution unit has finished reading the data in all address segments in the address storage space, that is, when the complete first operation data has been read, the address storage space of the first operation data in the second memory can be released at one time, and the storage states corresponding to all address segments in the address storage space can be updated from a stored state to a non-stored state. For example, the status bits of all address segments in the address storage space in the side register of the second memory can be updated from "1" to "0", and this embodiment of the present application does not limit this.

[0106] Please refer to Figure 7, which is a schematic diagram of an embodiment of the present application that uses an address storage space to indicate the data status. As shown in Figure 7, after the first execution unit in the AI ​​processor receives the first operation instruction issued by the first instruction queue, it can read the pre-stored data indicated by the first operation instruction into the first memory, perform the first type of operation corresponding to the first operation instruction on the pre-stored data, and generate the first operation data. The first execution unit can write the first operation data to the second memory and allocate an address storage space in the second memory for the first operation data; the address storage space can include M (M is a positive integer) address segments, such as address segment 1, address segment 2, address segment 3... in sequence, each address segment can be used to store part of the data information (data information fragment) in the first operation data, and the M address segments contained in the address storage space can be used to store the first operation data written by the first execution unit to the second memory. Each address segment in the address storage space can have a storage state, and the storage state corresponding to each address segment can be represented by a status bit in the register on the second memory side, and a status bit can be a bit in the register; that is, the storage state corresponding to each address segment can occupy a bit in the register on the second memory side, such as address segment 1, address segment 2, address segment 3, etc. can all correspond to a status bit in the register.

[0107] During the process of the first execution unit starting to write the first operation data to the second memory, the status bits of each address segment contained in the address storage space in the register can all be "0". The "0" here means that the storage status corresponding to each address segment in the address storage space is all unstored, or it can be understood that the write operation of address segment 1 in the address storage space has not been completed, that is, the data status of the first operation data in the second memory is unprepared. At this time, the second execution unit cannot temporarily read the first operation data into the second memory and can only wait for reading.

[0108] It can be understood that when the first execution unit writes the first operation data to the second memory, the first operation data is first written to the address segment 1 in the address storage space in the second memory. After the writing of address segment 1 is completed, the status bit of the address segment in the register can be updated from "0" to "1". The "1" here indicates that the first execution unit has completed the write data operation on address segment 1. At this time, the storage status corresponding to address segment 1 is the stored state, and the storage status corresponding to the remaining address segments in the address storage space except address segment 1 are all unstored states. It can be determined that the data status of the first operation data in the second memory is the ready state. At this time, the second execution unit can start to read the first operation data into the second memory, specifically, it can point to the address segment 1 in the second memory to read the first operation data.

[0109] After the second execution unit completes the read operation on address segment 1, it detects that the status bit of address segment 2 in the register has been updated to "1". Then, the second execution unit can continue to read the first operation data from address segment 2 in the second memory. If the status bit of address segment 2 in the register has not yet been updated to "1", the second execution unit can wait for the status bit of address segment 2 in the register to be updated to "1" before continuing to write the first operation data. This process continues in this way until the second execution unit completes the write operation on all address segments contained in the address storage space, that is, it has read the complete first operation data in the second memory.

[0110] For ease of understanding, the following is a specific explanation using dependency relationship 1 in the embodiment corresponding to Figure 1 as an example. At this time, the first type of operation can be a DMA0 operation, and the second type of operation can be a vector operation; in other words, the first execution unit is a DMA0 execution unit, and the second execution unit is a vector execution unit. Please refer to Figure 8, which is a second schematic diagram of the operation splitting between different operations in an AI processor provided in an embodiment of the present application; the horizontal axis in the coordinate diagram shown in Figure 8 represents time, and the vertical axis represents operation; the steps of the DMA0 operation can be divided into the instruction control unit sending instructions, the DMA0 instruction queue sending instructions, the DMA0 execution unit parsing instructions, the DMA0 execution unit reading data 0 from memory A, and the DMA0 execution unit writing data 1 from memory B; the instructions here can refer to DMA0 operation instructions. The steps of the vector operation can be divided into the instruction control unit synchronously controlling and sending instructions, the vector instruction queue sending instructions, the vector execution unit parsing instructions, the vector execution unit reading data 1 from memory B, and the vector execution unit writing data 2 from memory B; the instructions here can refer to vector operation instructions.

[0111] It is understood that when using the address storage space to indicate the data status, the DMA0 execution unit is required to update the data status corresponding to Data 1. This data status is reflected by the storage status corresponding to the address segment in the address storage space, or it can be understood as being reflected by the status bit of the address segment in the memory side register. When using the address storage space mode to configure the data status of data, the vector operation to read Data 1 can begin without waiting for DMA0 to fully complete the write of Data 1. As shown in Figure 8, after the DMA0 operation completes the write operation of the first address segment in the address storage space from memory B, the vector execution unit can begin to read Data 1 from memory B. That is, the time point at which the vector execution unit begins to read Data 1 from memory B is the same as the time point at which the DMA0 operation completes the write operation of the first address segment in the address storage space from memory B. In other words, in the AI ​​processor, when the storage status corresponding to the address segment in the address storage space is used to indicate the data status corresponding to Data 1, for DMA0 operations and vector operations that require synchronization, the DMA0 operation to write Data 1 to memory B and the vector operation to read Data 1 from memory B can be executed in parallel.

[0112] Step S207 : when the first execution unit generates third operation data, and the third operation data and the first operation data have the same address storage space in the second memory, obtaining a read status of the first operation data in the second memory.

[0113] Step S208 : When the read status of the first operation data indicates that the reading is completed, the third operation data is written into the address storage space of the second memory, and the same data tag as the first operation data is configured for the third operation data in the second memory.

[0114] It is understandable that in actual AI processor business, each execution unit can process multiple data, and different data may be stored in the same address storage space of the memory. For example, when the first execution unit generates new data in the process of executing subsequent operations, the new data can become the third operation data; and when the address storage space configured for the third operation data in the second memory is the same as the address storage space of the first operation data, the read status of the first operation data in the second memory can be obtained. Among them, the read status of the first operation data can be used to indicate whether the first operation data has been read, and the read status can be read completed or read incomplete; read completion can indicate that the first operation data stored in the second memory has been read, and then the address storage space corresponding to the first operation data can be released in the second memory, that is, the first operation data in the second memory is cleared; read incomplete can indicate that the first operation data stored in the second memory has not been read, and the address storage space of the first operation data in the second memory cannot be released temporarily.

[0115] Furthermore, when the read status of the first operation data in the second memory indicates read completion, the third operation data is written into the address storage space of the second memory, and the third operation data is assigned the same data tag in the second memory as the first operation data. It should be understood that the third operation data can only be written into the same address storage space after the first operation data is read and its corresponding address storage space is released.

[0116] Optionally, the first operation data stored in the second memory can be read by one or more operations (or can be understood as execution units). If the first operation data stored in the second memory only needs to be read by the second execution unit, then when the second execution unit completes reading the first operation data in the second memory, it can be determined that the read status of the first operation data in the second memory indicates that the reading is complete, and then the third operation data can be written into the address storage space of the second memory. If the first operation data stored in the second memory needs to be read by multiple execution units, then when the second execution unit completes reading the first operation data in the second memory, the read status of the first operation data in the second memory cannot indicate that the reading is complete, but it is necessary to wait until all relevant execution units have finished reading the first operation data in the second memory before it can be determined that the read status of the first operation data in the second memory indicates that the reading is complete, and then the third operation data can be written into the address storage space of the second memory.

[0117] Please refer to Figure 9, which is a second architectural diagram of a processor provided in an embodiment of the present application. In actual AI processor business, each execution unit can process multiple data, and different data may be stored in the same address location (address storage space) in the memory. As shown in Figure 9, the hardware structure of the AI ​​processor can refer to the relevant description in the embodiment corresponding to Figure 1, and will not be repeated here; when data 1.1 and data 1 are stored in the same address storage space of memory B, the DMA0 execution unit needs to wait for the vector execution unit to read data 1 before writing data 1.1 to memory B. When achieving this synchronization, data 1 and data 1.1 can use the same data label. Similarly, when data 2.1 and data 2 are stored in the same address storage space of memory B, the vector execution unit needs to wait for the DMA1 execution unit to read data 2 before writing data 2.1 to memory B. When achieving this synchronization, data 2 and data 2.1 can use the same data label, etc.

[0118] Please also refer to Figure 10, which is a second schematic diagram of the execution flow between different operations in an AI processor provided in an embodiment of the present application; as shown in Figure 10, the embodiment of the present application is illustrated using dependency relationship 1 in the embodiment corresponding to Figure 1 as an example, where dependency relationship 1 means that the vector operation depends on the result data generated by the DMA0 operation (the aforementioned first operation data, which can be recorded here as data 1).

[0119] As shown in Figure 10, the data status of the completed data 1 can be configured as ready in the DMA0 operation, and a corresponding configuration can be added to the vector operation. The configuration in this case can be "check whether the data status of data 1 is ready." Specifically, the DMA0 execution unit begins writing data 1 to memory B (the second memory). When the DMA0 execution unit completes writing data 1, the data status of data 1 can be updated to ready. During the process of the vector execution unit executing the vector operation, the vector execution unit can check whether the data status of data 1 in memory B is ready. If the data status of data 1 is ready, the vector execution unit can then begin reading data 1 from memory B. When the vector execution unit completes the operation of reading data 1, the data status can be updated to no data. For example, the data 1 in memory B can be cleared to release the address storage space in memory B. Alternatively, it can be understood as clearing the data tag or status bit (the storage status of the address segment) corresponding to data 1. If the data status of data 1 is not ready, the vector execution unit needs to wait on its corresponding read interface until the data status of data 1 is updated to ready, and then the vector execution unit can perform the data read operation again.

[0120] When the DMA0 execution unit writes data 1.1, it can check whether data 1 has been read and whether the ready status flag (data tag or status bit) configured for the first operation data has been cleared. If the ready status flag corresponding to the first operation data has not been cleared, the DMA0 execution unit waits on the write interface of the DMA0 execution unit until the ready status flag is cleared, and then the DMA0 execution unit proceeds with the write operation of data 1.1. If the ready status flag corresponding to the first operation data has been cleared, the DMA0 execution unit can begin writing data 1.1 to memory B.

[0121] In an embodiment of the present application, when a data tag is used to indicate the data status, the operations of the two synchronized read and write memories can be performed continuously; that is, the write operation starts immediately after the read operation it is waiting for is completed, and the read operation starts immediately after the write operation it is waiting for. When the address storage space is used for identification, the two synchronized read and write memory operations can be executed in parallel; for example, the read operation needs to read 10 address segments, the write operation needs to write 10 address segments, and the write operation needs to wait for the read operation to be completed. Then, after reading the first address segment, the write operation can start execution. Similarly, the write operation needs to write 10 address segments, the read operation needs to read 10 address segments, and the read operation needs to wait for the write operation to be completed. Then, after writing the first address segment, the read operation can start execution. The synchronization overhead between multiple execution units of the AI ​​processor can be reduced.

[0122] In an embodiment of the present application, the first type of operation and the second type of operation that depends on the first operation data generated by the first type of operation need to be executed in the correct order, that is, the first type of operation needs to be executed before the second type of operation. After the first execution unit reads the pre-stored data from the first memory, it can perform the first type of operation on the pre-stored data to generate the first operation data; the first execution unit can write the first operation data into the second memory, and update the data state corresponding to the first operation data according to the writing progress of the first operation data in the second memory (such as the address storage space of the data can be used to indicate the data state corresponding to the first operation data). When the data state corresponding to the first operation data is in the ready state, the second execution unit can immediately read the first operation data from the second memory and perform the second type of operation on the first operation data. This can reduce the read and write time interval between the two synchronized operations, thereby reducing the time interval between different operations. In addition, the synchronization control between different operations in the AI ​​processor can be performed when the hardware execution unit (for example, the first execution unit and the second execution unit) interacts with the memory for data. This can avoid inserting a barrier or wait-type instruction between two operations with a dependency to block subsequent instructions from being sent from the instruction control unit to the instruction queue, or blocking subsequent instructions from being sent from the instruction queue to the corresponding execution unit. In other words, it can avoid performing synchronization control between different operations on the instruction control path in the AI ​​processor, thereby reducing the instruction complexity in the AI ​​processor.

[0123] Please refer to Figure 11, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 11, the computer device 1000 can be a server or a terminal device, and will not be limited here. For ease of understanding, the embodiment of the present application takes the terminal device as an example. The computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and optionally, the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in FIG. 11 , the memory 1005 , which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0124] Among them, in the computer device 1000 shown in Figure 11, the network interface 1004 can also provide network communication functions, while the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005.

[0125] Optionally, the processor 1001 in the computer device 1000 shown in FIG11 may be the processor shown in FIG1 , and the processor 1001 may include N execution units and N instruction queues, where one execution unit corresponds to one instruction queue, the N execution units include a first execution unit and a second execution unit, and the N instruction queues include a first instruction queue corresponding to the first execution unit and a second instruction queue corresponding to the second execution unit, where N is an integer greater than 1;

[0126] The first execution unit is used to obtain a first operation instruction from the first instruction queue and obtain pre-stored data indicated by the first operation instruction from the first memory;

[0127] The second execution unit is used to obtain a second operation instruction from the second instruction queue;

[0128] The first execution unit is further configured to execute a first type of operation corresponding to a first operation instruction on pre-stored data to generate first operation data, write the first operation data into the second memory, and update a data state of the first operation data according to a writing progress of the first operation data into the second memory;

[0129] The second execution unit is configured to obtain the first operation data indicated by the second operation instruction from the second memory when the data state of the first operation data is updated to the ready state;

[0130] The second execution unit is further configured to execute a second type of operation corresponding to a second operation instruction on the first operation data, generate second operation data, and write the second operation data into the third memory.

[0131] Optionally, the processor 1001 further includes an instruction control unit;

[0132] The instruction control unit is used to obtain a first initial instruction and a second initial instruction, decode the first initial instruction and the second initial instruction, and obtain a first operation instruction corresponding to the first initial instruction and a second operation instruction corresponding to the second initial instruction;

[0133] The instruction control unit is further configured to transmit the first operation instruction to the first instruction queue and transmit the second operation instruction to the second instruction queue.

[0134] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the description of the memory-based data interaction method in any of the embodiments corresponding to Figures 2 and 6 above, which will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.

[0135] In addition, it should be noted that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the aforementioned processor, and the computer program includes computer instructions. When the processor executes the computer instructions, it can execute the description of the memory-based data interaction method in the embodiment corresponding to any one of Figures 2 and 6 above, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the computer instructions can be deployed on a computer device for execution, or executed on multiple computer devices located at one location, or executed on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network can constitute a blockchain system.

[0136] In addition, it should be noted that: the embodiment of the present application also provides a computer program product or computer program, which may include computer instructions, and the computer instructions may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor can execute the computer instructions so that the computer device executes the description of the memory-based data interaction method in the embodiment corresponding to any one of Figures 2 and 6 above, and therefore, will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0137] It should be noted that for the aforementioned various method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0138] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0139] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0140] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data interaction method based on a memory, characterized in that: The method is executed by a processor, and the method includes: Acquire a first operation instruction from a first instruction queue, acquire pre-stored data indicated by the first operation instruction in a first memory, and acquire a second operation instruction from a second instruction queue; Performing a first type of operation corresponding to the first operation instruction on the pre-stored data to generate first operation data; Writing the first operation data into a second memory, and updating a data state of the first operation data according to a writing progress of the first operation data in the second memory; When the data state of the first operation data is updated to a ready state, acquiring the first operation data indicated by the second operation instruction in the second memory; Perform a second type of operation corresponding to the second operation instruction on the first operation data to generate second operation data, and write the second operation data into a third memory.

2. The method according to claim 1, characterized in that: The method further comprises: Acquire a first initial instruction and a second initial instruction through an instruction control unit, decode the first initial instruction and the second initial instruction, and obtain a first operation instruction corresponding to the first initial instruction and a second operation instruction corresponding to the second initial instruction; The first operation instruction is transmitted to the first instruction queue and the second operation instruction is transmitted to the second instruction queue through the instruction control unit.

3. The method according to any one of claims 1 to 2, characterized in that: The obtaining of the first operation instruction from the first instruction queue and obtaining the pre-stored data indicated by the first operation instruction in the first memory include: According to the order of reading elements in the first instruction queue, sending the first operation instruction in the first instruction queue to the first execution unit, and clearing the first operation instruction in the first instruction queue; The first operation instruction is parsed and processed by the first execution unit to obtain instruction parsing information corresponding to the first operation instruction, and pre-stored data is acquired in the first memory according to the instruction parsing information.

4. The method according to any one of claims 1 to 3, characterized in that: The performing the first type of operation corresponding to the first operation instruction on the pre-stored data to generate first operation data includes: If the first type of operation includes a data transfer operation, the first execution unit executes the data transfer operation corresponding to the first operation instruction on the pre-stored data to copy the pre-stored data into the first operation data; If the first type of operation includes a vector operation, performing the vector operation corresponding to the first operation instruction on the pre-stored data through the first execution unit to obtain a vector operation result corresponding to the pre-stored data, and determining the vector operation result as the first operation data; If the first type of operation includes a matrix operation, the first execution unit executes the matrix operation corresponding to the first operation instruction on the pre-stored data to obtain a matrix operation result corresponding to the pre-stored data, and determines the matrix operation result as the first operation data.

5. The method according to any one of claims 1 to 4, characterized in that: The step of writing the first operation data into the second memory and updating the data state of the first operation data according to the writing progress of the first operation data in the second memory includes: Writing the first operation data into a second memory through a first execution unit, obtaining a writing progress of the first operation data in the second memory, and configuring a data tag for the first operation data in the second memory; When the writing progress of the first operation data indicates that the writing is completed, the tag state corresponding to the data tag is updated to a ready state; the tag state of the data tag is used to represent the data state of the first operation data in the second memory.

6. The method according to any one of claims 1 to 4, characterized in that: The step of writing the first operation data into the second memory and updating the data state of the first operation data according to the writing progress of the first operation data in the second memory includes: Allocating, by the first execution unit, an address storage space in the second memory for the first operation data, writing the first operation data into an address segment included in the address storage space, and acquiring a writing progress of the first operation data in the second memory; According to the writing progress of the first operation data, the storage state corresponding to the address segment contained in the address storage space is updated; the storage state is used to represent the data state of the first operation data in the second memory.

7. The method according to claim 6, characterized in that The allocating, by the first execution unit, an address storage space in the second memory for the first operation data comprises: Acquire the data capacity corresponding to the first operation data, and acquire the hardware rule information corresponding to the second memory; According to the hardware rule information, a unit address space range is determined in the second memory; the unit address space range is used to represent an address space range of an address segment; An address storage space of the first operation data in the second memory is determined according to the data capacity and the unit address space range.

8. The method according to claim 6, characterized in that The number of address segments contained in the address storage space is M, where M is a positive integer; The updating, according to the writing progress of the first operation data, of the storage state corresponding to the address segment contained in the address storage space includes: When the writing progress of the first operation data indicates that the writing of the i-th address segment in the address storage space is completed, the storage state corresponding to the i-th address segment in the address storage space is updated to a stored state; i is a positive integer less than or equal to M; Then when the data state of the first operation data is a ready state, acquiring the first operation data indicated by the second operation instruction in the second memory includes: When the storage state of the i-th address segment in the address storage space is updated to a stored state, determining that the data state of the first operation data in the second memory is a ready state; The data of the i-th address segment in the address storage space is acquired through the second execution unit, and the data of the i-th address segment is determined as the first operation data indicated by the second operation instruction.

9. The method according to any one of claims 6 to 8, characterized in that: The method further comprises: When the second execution unit obtains the first operation data from the second memory, the first operation data in the address storage space of the second memory is cleared, and the storage state corresponding to the address segment contained in the address storage space is updated from a stored state to a non-stored state.

10. The method according to claim 5, characterized in that The method further comprises: When the first execution unit generates third operation data, and the third operation data and the first operation data have the same address storage space in the second memory, obtaining a read status of the first operation data in the second memory; When the read status of the first operation data indicates that the reading is completed, the third operation data is written into the address storage space of the second memory, and the third operation data in the second memory is configured with the same data tag as the first operation data.

11. The method according to any one of claims 1 to 10, characterized in that: When the first type of operation includes a first data transfer operation, the second type of operation includes a vector operation, the first memory and the second memory are different memories, and the second memory and the third memory are the same memory; When the first type of operation includes a vector operation, the second type of operation includes a second data transfer operation, the first memory and the second memory are the same memory, and the second memory and the third memory are different memories; When the first type of operation includes a third data handling operation, the second type of operation includes a matrix operation, and the first memory, the second memory, and the third memory are all different memories; When the first type of operation includes a matrix operation, the second type of operation includes a fourth data transfer operation, and the first memory, the second memory, and the third memory are all different memories.

12. A processor, characterized in that: The processor includes N execution units and N instruction queues, one execution unit corresponds to one instruction queue, the N execution units include a first execution unit and a second execution unit, the N instruction queues include a first instruction queue corresponding to the first execution unit, and a second instruction queue corresponding to the second execution unit, and N is an integer greater than 1; The first execution unit is used to obtain a first operation instruction from the first instruction queue, and obtain pre-stored data indicated by the first operation instruction in the first memory; The second execution unit is used for obtaining a second operation instruction from the second instruction queue; The first execution unit is further configured to execute a first type of operation corresponding to the first operation instruction on the pre-stored data to generate first operation data, write the first operation data into the second memory, and, according to a writing progress of the first operation data in the second memory, Updating the data state of the first operation data; The second execution unit is further configured to obtain the first operation data indicated by the second operation instruction in the second memory when the data state of the first operation data is updated to a ready state; The second execution unit is further configured to execute a second type of operation corresponding to the second operation instruction on the first operation data, generate second operation data, and write the second operation data into a third memory.

13. The processor according to claim 12, characterized in that The processor also includes an instruction control unit; The instruction control unit is used to obtain a first initial instruction and a second initial instruction, and decode the first initial instruction and the second initial instruction to obtain a first operation instruction corresponding to the first initial instruction and a second operation instruction corresponding to the second initial instruction; The instruction control unit is further configured to transmit the first operation instruction to the first instruction queue, and transmit the second operation instruction to the second instruction queue.

14. A computer device, characterized in that: including memory and processor; The memory is connected to the processor, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 11.

16. A computer program product, characterized in that The method comprises computer instructions, which implement the method according to any one of claims 1 to 11 when executed by a processor.

Images