Processing device, method and system for performing data processing on multiple channels

Abstract

Disclosed are a processing device, method, and system for performing data processing on multiple channels. The processing device for performing data processing on multiple channels includes: a channel information acquisition unit configured to acquire channel information for the multiple channels; a storage unit including multiple storage areas corresponding to the multiple channels, wherein the storage areas are configured to store data information for the multiple channels; a data read control unit configured to read target data information corresponding to the channel information from a target storage area among the multiple storage areas of the storage unit based on the channel information; and a cache unit configured to pre-store the target data information read by the data read control unit from the target storage area of ​​the storage unit, pending use for data processing. The processing device, method, and system for performing data processing on multiple channels provided in accordance with embodiments of the present disclosure can reduce or eliminate delays in data processing operations and improve data processing efficiency.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a processing device, method, and computer system for performing data processing on multiple channels. Background Art

[0002] In computer systems, data often needs to be exchanged between I / O memory and between memory. Direct Memory Access (DMA) technology is a high-speed data transfer method that allows direct data transfer between external devices and memory, between memory and memory, and so on. The DMA process is primarily implemented by hardware. In this process, data exchange between external devices and memory is not controlled by the Central Processing Unit (CPU). Instead, it uses the system bus, with the DMA hardware controller controlling the bus to directly complete the data exchange between the external device and memory. Summary of the Invention

[0003] According to a first aspect of the present disclosure, a processing device for performing data processing on multiple channels is proposed, comprising: a channel information acquisition unit configured to acquire channel information of the multiple channels; a storage unit comprising multiple storage areas corresponding to the multiple channels, wherein the storage areas are configured to store data information for the multiple channels; a data read control unit configured to read target data information corresponding to the channel information from a target storage area among the multiple storage areas of the storage unit according to the channel information; and a cache unit configured to pre-store the target data information read by the data read control unit from the target storage area of ​​the storage unit to wait for use in the data processing.

[0004] In at least one embodiment, the cache unit includes a first-in-first-out queue based on combinational logic, and the first-in-first-out queue is configured to enable a request signal for the first-in-first-out queue and data information associated with the request signal to appear in the same clock cycle.

[0005] In at least one embodiment, the cache unit is configured to pre-store at least two items of data information used for the data processing.

[0006] In at least one embodiment, the cache unit includes a first-in-first-out queue based on sequential logic, and the cache unit is configured to pre-store at least three items of data information used for the data processing.

[0007] In at least one embodiment, the channel information acquiring unit is configured to query different channels in at least two adjacent pollings.

[0008] In at least one embodiment, the processing device is a direct memory access device (DMA), and the data processing includes a data transfer operation.

[0009] In at least one embodiment, the data transfer operation includes the transfer of the data information or the transfer of target data indicated by the data information, the target data is the data required by the neural network processor to perform neural network operations, the neural network processor includes at least one processing unit array and multiple global buffer clusters, the data information indicates the address information of the target data, the address information is obtained based on the mapping relationship between the target data and at least one processing unit in the processing unit array, and each of the global buffer clusters is used to receive the transferred data information or the transferred target data on one communication information among the multiple channel information.

[0010] In at least one embodiment, the processing device further includes: an interface control unit configured to obtain the target data information from the cache unit, generate a data transfer request according to the target data information, and perform the data transfer operation according to the data transfer request.

[0011] In at least one embodiment, the channel information determining unit includes: a polling scheduling unit configured to poll and determine a plurality of channel information on which the data processing is to be performed.

[0012] In at least one embodiment, the polling scheduling unit is further configured to: in response to completing the data transfer operation on the current channel information and the data information in the storage unit is not empty, perform the next polling to determine the next channel information on which to perform the data transfer operation.

[0013] According to a second aspect of the present disclosure, a computer system is proposed, comprising: any processing device according to the first aspect of the present disclosure; and a neural network processor, wherein the neural network processor is electrically coupled to the processing device, the neural network processor comprises at least one processing unit array and a plurality of global buffer clusters, the data information indicates address information of the target data, the address information is obtained based on a mapping relationship between the target data and at least one processing unit in the processing unit array, and each of the global buffer clusters is used to receive the transported data information or the transported target data indicated by the data information on one communication information among the multiple channel information.

[0014] In at least one embodiment, the computer system further includes: an interface decoder configured to receive an initial instruction and decode the initial instruction to obtain the data information, and provide the decoded data information to the processing device.

[0015] In at least one embodiment, the computer system further comprises: an auxiliary processor configured to provide the initial instruction to the interface decoder.

[0016] In at least one embodiment, the auxiliary processor is coupled to the interface decoder via a first data transmission protocol, and the neural network processor is coupled to the direct memory access device via a second data transmission protocol, and the first data transmission protocol and the second data transmission protocol are different.

[0017] According to a third aspect of the present disclosure, a scheduling method for performing data processing on multiple channels is proposed, including: obtaining channel information of the multiple channels; reading target data information corresponding to the channel information from a target storage area among multiple storage areas corresponding to the multiple channels according to the channel information; and pre-storing the read target data information corresponding to the channel information into a cache unit to wait for use in the data processing.

[0018] In at least one embodiment, obtaining the channel information of the multiple channels includes: querying different channels in at least two adjacent pollings.

[0019] In at least one embodiment, the scheduling method is used in a direct memory access device, and the data processing includes a data moving operation.

[0020] In at least one embodiment, the data transfer operation includes the transfer of the data information or the transfer of target data indicated by the data information, the target data is the data required by the neural network processor to perform neural network operations, the neural network processor includes at least one processing unit array and multiple global buffer clusters, the data information indicates the address information of the target data, the address information is obtained based on the mapping relationship between the target data and at least one processing unit in the processing unit array, and each of the global buffer clusters is used to receive the transferred data information or the transferred target data on one communication information among the multiple channel information.

[0021] In at least one embodiment, the scheduling method further includes: acquiring the target data information from the cache unit, generating a data transfer request according to the target data information, and executing the data transfer operation according to the data transfer request.

[0022] In at least one embodiment, obtaining the channel information of the plurality of channels includes: polling to determine the plurality of channel information on which the data processing is to be performed.

[0023] In at least one embodiment, the polling to determine the multiple channel information on which the data processing is to be performed includes: in response to completing the data transfer operation on the current channel information and the data information in the storage area is not empty, performing the next polling to determine the next channel information on which the data transfer operation is to be performed.

[0024] The processing device, method, and computer system for performing data processing on multiple channels provided in accordance with the embodiments of the present disclosure can reduce or eliminate data processing operation delays and improve data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. A person skilled in the art can derive other drawings based on these drawings without inventive effort. The following drawings are not intentionally scaled to actual size; their focus is on illustrating the main points of the present invention.

[0026] Figure 1 An example of a pipeline processing instruction is shown;

[0027] Figure 2 An example architecture of a coprocessor-based DMA for a fixed-row NPU is shown;

[0028] Figure 3 An example of a working timing diagram for a timing chip is shown;

[0029] Figure 4A and Figure 4B right Figure 2 The working mechanism of the polling scheduling module in the DMA is described;

[0030] Figure 5 The delay of the polling scheduling module when switching between channels in the embodiment of the present disclosure is described;

[0031] Figure 6A A schematic structural diagram of a processing device for performing data processing on multiple channels according to at least one embodiment of the present disclosure is shown;

[0032] Figure 6B An example architecture of a coprocessor-based DMA according to at least one embodiment of the present disclosure is shown;

[0033] Figure 7 An example of a DMA operation timing diagram is shown in the embodiment of the present disclosure when the preset FIFO does not pre-fetch instructions in advance;

[0034] Figure 8 An example of a preset FIFO in the present disclosure is described;

[0035] Figure 9 An example of a DMA operation timing diagram is shown when instruction data of three different channels are pre-stored in a preset FIFO according to the present disclosure;

[0036] Figure 10 The present invention shows a preset FIFO pre-stored instruction mechanism using at least one embodiment of the present disclosure. Figure 6B An example of the DMA working timing diagram in single channel switching mode;

[0037] Figure 11A A schematic structural diagram of a processing device for performing data processing on multiple channel information according to at least one embodiment of the present disclosure is shown;

[0038] Figure 11B An example architecture of a coprocessor-based DMA according to at least one embodiment of the present disclosure is shown;

[0039] Figure 12 Shown Figure 11B An example of the DMA working timing diagram in single-channel switching mode;

[0040] Figure 13A right Figure 11B The risk of DMA reading multiple instruction data is explained;

[0041] Figure 13B A schematic structural diagram of a device for accessing data according to at least one embodiment of the present disclosure is shown;

[0042] Figure 14 An example of a working timing diagram showing control of writing of instruction data according to the falling edge of a full signal of a preset FIFO is shown;

[0043] Figure 15 The risk of writing invalid data caused by only detecting the falling edge of the full signal to control data writing is explained;

[0044] Figure 16 FIG2 shows an operation timing diagram of controlling data writing according to an example of a flag signal disclosed herein;

[0045] Figure 17An example of an operation timing diagram for controlling data writing using an Almost_Full signal (abbreviated as afull signal) according to the present disclosure is shown. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts also fall within the scope of protection of the present invention.

[0047] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0048] Although the present application makes various references to certain modules in the system according to embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0049] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0050] Pipeline technology refers to breaking down a repetitive process into several sub-processes. Each sub-process is implemented by a dedicated functional component. Multiple processing processes are staggered in time and pass through each functional segment in sequence. Each sub-process can be carried out in parallel with other sub-processes.

[0051] Figure 1 An example of a pipeline processing instruction is shown. Figure 1As shown, when a computer executes an instruction, it needs to go through five steps: instruction fetch, decode, fetch data, execute, and write back, requiring five CPU cycles to complete. Pipelining technology allows each step of each instruction to be executed in an overlapping manner, rather than waiting for one instruction to complete before starting the next. That is, while the first instruction is being fetched and decoded, the second instruction is fetched; while the first instruction is fetching data, the second instruction is decoded, the third instruction is fetched, and so on. With pipelining technology, although each instruction still needs to go through the above five steps and require the same number of CPU cycles, when viewed as a whole, although each instruction is still executed within one instruction cycle, the number of instructions completed per unit time increases, thereby improving the CPU's IPC (instruction per clock) and enhancing instruction processing efficiency.

[0052] For example, in a neural network processing unit (NPU) system with complex row-stationary data flow, direct memory access (DMA) needs to move data based on data mapping relationships. Using algorithms, the input feature map or filter data is sliced ​​and diced. The coprocessor outputs instructions to the DMA, which decodes the instructions and moves data to the NPU based on the decoded results.

[0053] Figure 2 An example architecture of a coprocessor-based DMA for a fixed-row NPU is shown. A description of the DMA can be found in, for example, Chinese patent application number 202111659158.6, entitled “Method for Transferring Data, Direct Memory Access Device, and Computer System,” and is further described below.

[0054] like Figure 2 As shown, the SCIE decoder is in the coprocessor clock domain, and the coprocessor outputs instructions through the SCIE bus interface. It should be recognized that the instructions output by the coprocessor can be R-type instructions in the RISC-V instruction set. The domain of the R-type instruction is 32 bits, including 7 bits of opcode, 5 bits of rd, 3 bits of func3, 5 bits of rs1, 5 bits of rs2 and 7 bits of func7, where opcode indicates the operation code, rd indicates the number of the destination register, func3 is an extensible operation code / function code, rs1 indicates the number of the first source register, rs2 indicates the number of the second source register, and func7 is an extensible operation code / function code.

[0055] The func3 in the R-type instruction is used to indicate different functions of the R-type instruction, including address, logic, operation, etc., wherein the func3 of the R-type instruction indicating the address is a fixed value

[000] . The instruction whose function indicated by func3 is an address can be a data handling instruction. Some bits (for example, three bits) in the func7 of the R-type instruction can be used to encode the type of the data handling instruction. In one example, the three bits in the func7 of the R-type instruction can be used to encode the following five types of data handling instructions:

[0056] ①: Request for uncompressed data of input feature map;

[0057] ②: Request for uncompressed data of the filter;

[0058] ③: Request for compressed data of the filter;

[0059] ④: Read the valid number of requests for input feature map data;

[0060] ⑤: Read the valid number of requests for filter data.

[0061] Exemplarily, the five types of encodings for the above data handling instructions are as follows:

[0062] Types of data movement instructions ① ② ③ ④ ⑤ coding 100 110 111 010 011

[0063] The 5-bit rs1, rs2, and rd in the data transfer instruction respectively indicate the addresses of the first source register, the second source register, and the destination register. In the present disclosure, at least one of the first source register, the second source register, and the destination register can use at least part of the bits to encode information associated with the data transfer performed by the DMA according to the mapping relationship of the input feature map or the filter. The 32-bit first source register can be used to encode the address information of the data to be transferred in the data transfer, and the address information of the data to be transferred may include base address information and offset address information. At least part of the bits of the second source register can be used to encode the identification information (ID) of the DMA that executes the data transfer instruction. In an NPU system with a fixed-row data flow, multiple DMAs can be used to perform data transfer tasks. For example, the first task of data transfer can be performed by the first DMA, the second task of data transfer can be performed by the second DMA, and so on. At least part of the bits of the second source register can be used to encode the channel information in the DMA that executes the data transfer instruction. Eyeriss V2 is a neural network processor with fixed-row data flow. Eyeriss V2 includes multiple global buffer (GLB) clusters. Each GLB cluster is used to store data or instructions moved by DMA by executing data moving instructions. When DMA executes data moving instructions, the moved data or instructions can be stored in different GLB clusters. When DMA executes data moving instructions, the moved data or instructions can be stored in different GLB clusters, and the data channels in the corresponding DMA can also be different. For example, the moved data or instructions can be stored in 8 different GLB clusters, and DMA can include 8 data channels corresponding to these 8 GLB clusters. The data channels here can be the channels of the DMA described above in this disclosure. At least part of the bits of the destination register can be used to encode the valid number of requests for data of the input feature map or the valid number of requests for data of the filter. The valid number of requests for data can be the number of requests for data. For example, if the valid number is 5, it means that there are 5 requests for data.

[0064] Exemplarily, the instruction output by the coprocessor may be a first type instruction, which is an R-type instruction (indicated by the opcode being a fixed value [0110011]) and a data transfer instruction (indicated by the func3 being a fixed value

[000] ), and the corresponding data transfer instruction type is ① (indicated by the encoding of func7 in the present disclosure). At least some of the bits in the first source register of the instruction indicate the offset information of the data to be transferred, and at least some of the bits in the second source register in the instruction indicate the length information of the data to be transferred. The first type instruction may also include at least one of the encoding of the identification information of the DMA described above in the present disclosure and the encoding of the channel information in the DMA, and at least some of the bits in the second source register of the first type instruction may indicate the identification information of the DMA and / or the channel information in the DMA.

[0065] Exemplarily, the instruction output by the coprocessor may also be a third type of instruction, which is an R-type instruction (indicated by the opcode being a fixed value [0110011]) and a data handling instruction (indicated by the func3 being a fixed value

[000] ), and the corresponding data handling instruction type is ④ (indicated by the encoding of func7 in the present disclosure). At least some of the bits in the destination register in the instruction indicate the valid number described above in the present disclosure, and at least some of the bits in the second source register in the instruction indicate the DMA ID described above in the present disclosure. The number of requests for data stored in the corresponding DMA can be read through this instruction. For the convenience of description, the present disclosure hereinafter refers to the first type of instruction as instruction 1 and the third type of instruction as instruction 3.

[0066] The SCIE decoder includes a first instruction decoder, and instruction 1 or instruction 3 output by the coprocessor is transmitted to the first instruction decoder. The first instruction decoder is used to perform a first-level decoding on instruction 1 or instruction 3 to determine whether the instruction output by the coprocessor is of a preset type and whether the output instruction is a read instruction or a write instruction. Exemplarily, the first instruction decoder can be used to determine whether the instruction output by the coprocessor is an R-type data handling instruction. In this example, the first instruction decoder decodes the opcode field and the func3 field in instruction 1 or instruction 3 to obtain that instruction 1 or instruction 3 is an R-type data handling instruction. The first instruction decoder can also be used to determine whether the instruction is a read instruction or a write instruction. If the decoding result of the first instruction decoder is that the instruction output by the coprocessor is a data handling instruction of the ①, ②, or ③ type described above in this disclosure, the instruction is a write instruction; if the decoding result of the first instruction decoder is that the instruction output by the coprocessor is a data handling instruction of the ④ or ⑤ type described above in this disclosure, the instruction is a read instruction. In this example, the first instruction decoder decodes the func7 field in instruction 1 or instruction 3 to obtain that instruction 1 is a write instruction and instruction 3 is a read instruction.

[0067] The SCIE decoder may also include an instruction first-in-first-out queue (FIFO) and a valid number decoder. If the decoding result of the first instruction decoder is that the instruction output by the coprocessor is of a preset type and the output instruction is a write instruction, the decoded instruction is written into the instruction FIFO; if the decoding result of the first instruction decoder is that the instruction output by the coprocessor is of a preset type and the output instruction is a read instruction, the value of the valid number counter in the DMA is read according to the decoded read instruction. For example, after the first instruction decoder decodes the func7 field in instruction 1, the decoded instruction 1 can be written into the instruction FIFO; after the first instruction decoder decodes the func7 field in instruction 3, the SCIE decoder can read the value of the valid number counter in the DMA according to the decoded instruction 3, and decode the read valid number through the valid number decoder, thereby obtaining the number of write instructions that can be written in one or more channels in the DMA (i.e., the number of data requests in the above disclosure).

[0068] In the coprocessor clock domain, each time a write instruction is written into the instruction FIFO, the valid number counter in the DMA can be controlled to decrease by 1. For example, when the decoded instruction 1 is written into the instruction FIFO, the valid number counter can be controlled to decrease by 1.

[0069] The SCIE decoder also includes a second instruction decoder, to which the instruction FIFO transfers decoded instruction 1. The second instruction decoder performs a second-level decoding on the instruction to determine the DMA to which the instruction is transferred. In this example, the second instruction decoder decodes the DMA ID field in instruction 1 to determine the DMA to which instruction 1 will be written. Figure 2 The DMA in is determined according to the decoded DMA ID. After the second instruction decoder decodes the instruction 1, the decoded instruction 1 can be transmitted to the third instruction decoder.

[0070] Figure 2 The DMA in the SCIE decoder is in the NPU clock domain, and the third instruction decoder (for example, which may correspond to the second decoding unit in the present disclosure) is located in the DMA. Since the DMA and the coprocessor are in different clock domains, the instruction FIFO in the SCIE decoder can use an asynchronous FIFO for instruction synchronization. The third instruction decoder can perform a third-level decoding on the instruction to determine which channel in the DMA the instruction is written to. In this example, the third instruction decoder decodes the field indicating the channel ID in instruction 1 to obtain the channel in the DMA to which instruction 1 will be written.

[0071] DMA also includes an instruction write control module and an instruction virtual channel storage unit. The instruction virtual channel storage unit includes a storage area corresponding to each channel. Figure 2 As shown, DMA includes eight channels, namely channel 1, channel 2, channel 3, channel 4, channel 5, channel 6, channel 7, and channel 8, each of which corresponds to the GLB described above in this disclosure. DMA is used to transfer data to the GLB on the corresponding channels respectively, and the instruction virtual channel storage unit includes eight storage areas, each of which is used to store write instructions on the corresponding channel. For example, if DMA is used to transfer data to GLB#1 on channel 1, storage area #1 stores the write instructions on channel 1; if DMA is used to transfer data to GLB#2 on channel 2, storage area #2 stores the write instructions on channel 2, and so on. The instruction write control module is used to write the decoded instructions into the storage area of ​​the corresponding channel in the instruction virtual channel storage unit according to the decoding result of the third instruction decoder. For example, if the decoding result of the third instruction decoder is channel 1, the instruction write control module can write the decoded instruction 1 into the storage area #1 corresponding to channel 1 in the instruction virtual channel storage unit.

[0072] In an optional example, the instruction write control module can control the write address. The starting address and ending address of each storage area in the instruction virtual channel storage unit can be configured by the configuration unit in the DMA. In this example, the configuration unit is in the Advanced Peripheral Bus (APB) clock domain, which interacts through the APB interface. If the instruction write control module successfully writes an instruction from a write address in a storage area, the instruction write control module can control the write address to be added by 1 so that the instruction can be written from the next write address of the storage area next time. When the write address reaches the ending address of the storage area, the instruction write control module can control the write address to be flipped so that the instruction can be written from the starting address of the storage area next time.

[0073] In an optional example, the instruction write control module can also determine whether the storage area of ​​the corresponding channel in the instruction virtual channel storage unit is full. If the storage area of ​​the corresponding channel is full (that is, there is no address space to store the write instruction again), a signal indicating that the corresponding channel is full is output. Optionally, if the storage area of ​​the corresponding channel is full and there is still a write request, an error signal is output. If the storage area of ​​the corresponding channel is not full (that is, there is address space to store the write instruction again), a write enable signal (denoted as wr_en) is output, and the instruction virtual channel storage unit can write the decoded instruction to the storage area of ​​the corresponding channel according to the write enable signal. For example, if the decoding result of the third instruction decoder is channel 1 and the instruction write control module outputs a write enable signal, the instruction write control module can write the decoded instruction 1 into the storage area #1 corresponding to channel 1 in the instruction virtual channel storage unit. At this point, DMA has completed the writing of instruction 1, and the data flow of instruction 1 written to the instruction virtual channel storage unit is as follows. Figure 2 α is shown by the dotted line in .

[0074] Next, continue to combine Figure 2 The reading process of instruction 1 is described.

[0075] The DMA also includes a polling scheduling module and an instruction read control module. The polling scheduling module is used to determine which channel in the instruction virtual channel unit to read the write instruction from. The polling scheduling module can generate a channel tag signal (ch_tag) and transmit the channel tag signal to the instruction read control module. The instruction read control module reads the write instruction for the corresponding channel in the instruction virtual channel storage unit based on the channel tag. For example, if instruction 1 is written to storage area #1 corresponding to channel 1 in the instruction virtual channel storage unit, if the signal generated by the polling scheduling module is the channel 1 tag signal, the instruction read control module can read instruction 1 from storage area #1.

[0076] The instruction read control module controls the read address. The starting and ending addresses of each storage area in the instruction virtual channel storage unit can be configured using the configuration unit in the DMA. If the instruction read control module successfully reads an instruction from a read address in a storage area, it increments the read address by 1, allowing the next instruction to be read from the next read address in the same storage area. When the read address reaches the end address of the storage area, the instruction read control module controls the read address to flip, allowing the next instruction to be read from the starting address of the storage area.

[0077] In an optional example, the instruction read control module can also judge whether the storage area of ​​the corresponding channel in the instruction virtual channel storage unit is empty based on the channel label signal. If there is no readable instruction in the storage area of ​​the corresponding channel, a signal indicating that the instruction is empty is returned. Optionally, if the storage area of ​​the corresponding channel is empty and there is still a read request, an error signal is output; if there is a readable instruction in the storage area of ​​the corresponding channel, a read enable signal (denoted as rd_en) is returned, and the polling scheduling module can select the channel to read the instruction based on the read enable signal.

[0078] Each time the instruction read control module successfully reads an instruction from the instruction virtual channel storage unit, the effective number counter can be controlled to increase by 1. For example, each time the instruction read control module successfully reads an instruction, a valid number increment signal (denoted as credit_add) can be generated, and then the credit_add signal is synchronized to the effective number counter through the synchronizer to increase the value of the effective number counter by 1. Figure 2 In the example, by dynamically adjusting the value of the valid number counter, the available space size of the storage area in the instruction virtual channel storage unit can be reflected in real time, reducing the error rate of instruction writing and improving the performance of the NPU system.

[0079] DMA also includes an AXI interface control module, and the polling scheduling module is controlled by the handshake signal of the subsequent module (such as GLB) and the AXI handshake signal fed back by the AXI interface control module. These handshake signals are used to indicate the status of the channel and the switching timing. For example, if data is currently being transferred on channel 1, which means that the current channel 1 is not in an idle state, no signal for handshake (such as a valid AXI request) will be generated on channel 1; if the data transfer task has been completed on the current channel 1 or the current channel 1 is in an idle state, channel 1 can generate a signal for handshake. For example, if the current AXI interface control module is processing data (described in detail below), no signal for handshake (such as a valid AXI request) will be generated; if the current AXI interface control module is in an idle state, a signal for handshake can be generated.

[0080] The instruction read control module can read instructions from the instruction virtual channel storage unit and send them to the AXI interface control module. After receiving the instruction read from the instruction virtual channel storage unit, the AXI interface control module performs a fourth-layer decoding on the instruction to extract the data content required by the AXI interface control module's subsequent modules (e.g., GLB) and perform conversion operations based on the extracted data content, simultaneously generating an AXI request corresponding to the extracted data content. In this example, after receiving instruction 1, the AXI interface control module decodes the fields indicating the offset and length information in instruction 1 to obtain the address information of the data to be transferred. The AXI interface control module then performs burst length control, cross-4K address checking, and other operations based on the address information of the data to be transferred, while simultaneously generating a corresponding AXI request. At this point, the DMA has completed reading instruction 1 and converting it into an AXI request recognizable by the NPU system. In another example, the data content required by the AXI interface control module's subsequent modules (e.g., GLB) can be instructions. In this case, after the AXI interface control module performs layer 4 decoding and conversion on the instruction, the instruction can be sent to a subsequent module of the AXI interface control module without generating an AXI request.

[0081] DMA can transmit AXI requests to the on-chip network. Based on the AXI request, the on-chip network can read the data to be transferred from SRAM and transfer the data, or read the data to be transferred from DRAM and transfer the data. The data flow of transferring data from SRAM or DRAM to the GLB of the corresponding channel is as follows: Figure 2 γ is shown by the dotted line in .

[0082] The DMA also includes an interrupt control module in the APB clock domain. Once all data and / or instructions are transferred, the AXI interface control module generates a transfer completion signal (denoted as trans_done) to indicate task completion. The interrupt control module generates and outputs an interrupt signal based on the received transfer completion signal. The DMA also includes a performance monitor in the APB clock domain for testing DMA performance.

[0083] Typically, computer memory units used to store data, such as registers, memory, and counters, are composed of timing chips. In computer systems, the passage of time can be represented by a master clock, which provides a continuous sequence of alternating signals that alternate between two signal values: 0 and 1 (low level and high level). The time interval between two consecutive rising edges is called a clock cycle. This alternating signal is transmitted to the timing chip using hardware circuits. Figure 3 An example of a timing diagram for a timing chip is shown. Figure 3 As shown, for a sequential chip, its output at time t depends on the input at time t-1, that is:

[0084] out[t]=function(in[t-1])

[0085] because Figure 2 The instruction virtual channel storage unit in the DMA uses the above-mentioned timing chip. Therefore, there will be a one-beat delay between the read request and the read data of the instruction virtual storage unit, which will cause the round robin scheduling module to introduce delay when switching between channels.

[0086] Data transmission can be achieved through handshaking signals. Handshaking means that the source and destination each send a control signal to notify the other party of their status. For example, when the receiver (destination) is able to receive data, it can actively send a request signal to the sender (source) to notify the sender that it can send data. After receiving the request signal, the sender sends the data to the data bus and sends a response signal to notify the receiver that the data is ready. After the receiver takes the data, it cancels the request signal and the sender also cancels the response signal. For ease of description, in this disclosure, when the receiver needs data, the request signal input to the sender is called the ready signal, and the signal sent by the sender to notify the receiver that the data is ready is called the valid signal. When the ready signal and the valid signal are valid, the sender sends the data to the receiver.

[0087] Figure 2 The DMA in the NPU system can, for example, move data or instructions to eight global buffer (GLB) clusters in the NPU system on eight channels, such as moving data or instructions to GLB cluster 1 on channel 1 and to GLB cluster 2 on channel 2. In this example, the GLB cluster can send a ready signal to the DMA as a receiver, and the DMA, as a data sender, can send a valid signal to the GLB cluster when the data or instruction is ready. The ready signal sent by the GLB cluster can be recorded as GLB_instr_ready, and the valid signal sent by the DMA can be recorded as Ar_valid.

[0088] Figure 2The DMA in the coprocessor can convert the instructions output by the coprocessor into, for example, AXI requests that comply with the Advanced eXtensible Interface (AXI) bus protocol, and send the AXI requests to the network on chip (NoC). The NoC then moves data from the storage device (such as dynamic random access memory (DRAM) or static random access memory (SRAM)) to the GLB cluster based on the AXI request. In this example, the NoC can send a ready signal to the DMA as the receiver, and the DMA can send a valid signal when the AXI request is ready. The ready signal provided by the NoC to the DMA through the AXI interface can be recorded as Ar_ready, and the valid signal sent by the DMA through the AXI interface can be recorded as Ar_valid.

[0089] Figure 2 The instruction virtual channel storage unit of the DMA may include eight storage areas, each storage area is used to store instruction data on a channel; the polling scheduling module can output a channel label, and the instruction reading control module can read instructions from the storage area of ​​the corresponding channel according to the channel label. Figure 4A and Figure 4B right Figure 2 The working mechanism of the polling scheduling module in the DMA is illustrated.

[0090] like Figure 4A As shown, the polling scheduling module includes a channel tag pointer tag_pointer, which is used to switch the polling channel priority. The default value is 1, which points to channel 1 (CH_1). The input signals of the polling scheduling module may include the CH_done signal and the CH_vld signal. The CH_done signal is used to indicate the channel status. When the CH_done signal is valid, it indicates that the DMA has successfully sent data or instructions on the channel. The CH_vld signal is used to indicate whether the channel is valid. When the CH_vld signal is valid, it indicates that the channel is valid. When the CH_vld signal is invalid, it indicates that the channel is invalid. The CH_vld signal can be determined according to the GLB_instr_ready signal described above. For example, when GLB_instr_ready is valid (that is, the GLB cluster needs data) and there is instruction data, the CH_vld signal is valid; when GLB_instr_ready is invalid (that is, the GLB cluster does not need data) or there is no instruction data, the CH_vld signal is invalid. In this example, when the Ar_ready signal and the Ar_valid signal described above are both valid, the CH_done signal may be valid, otherwise the CH_done signal is invalid. The output of the polling scheduling module is the channel tag CH_tag.

[0091] like Figure 4B As shown in the figure, in case a, the tag_pointer indicates the current channel CH_3 (i.e., channel 3). When the CH_done signal is valid, the tag_pointer switches (for example, in one beat) to the next channel (regardless of whether the CH_vld signal of that channel is valid). At this time, the tag_pointer indicates channel CH_4 (i.e., channel 4). In case b, if the CH_done signal is discontinuous, the tag_pointer will point to the current valid channel when the CH_done signal is invalid. Since channels CH_4 (i.e., channel 4) and CH_5 (i.e., channel 5) are invalid channels, the tag_pointer will jump to the next valid channel CH_6 (i.e., channel 6) and output the label of channel 6. In case c, if the CH_done signal is continuous, the tag_pointer will switch to the next channel CH_7 (i.e., channel 7) when the CH_done signal is valid.

[0092] The following combination Figure 5 The delay of the polling scheduling module in the present disclosure when switching between channels is explained.

[0093] like Figure 5 As shown, GLB_instr_ready is Figure 2 The ready signal of the DMA's subsequent module in the DMA will back pressure the DMA when it is pulled low; Ar_ready and Ar_valid are Figure 2 The handshake signal between DMA and on-chip network in the chip, when both are high, it means that the current request transmission is completed; CH_done is Figure 2 The DMA polling scheduling module in the channel switching signal, if CH_done is detected to be high at the rising edge of the main clock, the channel switching is executed; CH_tag is Figure 2 The output signal of the DMA polling scheduling module in the DMA_ represents which channel is currently selected; SRAM_ read data is Figure 2 The instruction read control module in the instruction virtual channel storage unit reads the data, which can be approximately represented when the data is read into the AXI interface control module. Figure 2 The DMA in the SRAM has prepared the instruction data, and the Ar_valid signal is pulled high; the SRAM_ read request is for Figure 2 The DMA instruction in the virtual channel stores a read request.

[0094] like Figure 2 The DMA backend modules and the on-chip network are always in a ready state (i.e. Figure 5GLB_instr_ready and Ar_ready in the master clock are always high after the second beat of the master clock. In the second beat, Ar_valid is high, which means Figure 2 The DMA AXI interface control module has prepared data 1, and Ar_ready is high, so the current channel has completed a data transmission in the second beat. At this time, CH_done is pulled high to indicate that the current channel has completed a data transmission and needs to switch to the next channel, and outputs the channel label Ch_1 in the second beat. On the rising edge of the third beat, CH_done is detected to be high, and the polling scheduling module executes the channel switching, that is, switching from channel CH_1 to channel CH_2. At the same time, in the third beat, Figure 2 The DMA instruction read control module receives the channel label Ch_1, SRAM_read request is high, and starts to read from Figure 2 The DMA instruction in the virtual channel storage unit reads data 2 from channel 1 and reads data 2 into the next beat. Figure 2 The DMA AXI interface control module in the DMA controller. Figure 2 The DMA instruction virtual channel storage unit is read, so Figure 2 The DMA in the DMA is not ready for data yet, so Ar_valid is pulled low. Since the data transmission in the current channel CH_2 is not completed in the third beat, CH_done is pulled low, that is, when Ar_valid is pulled low, CH_done is also pulled low, resulting in a one-beat delay in the current channel CH_2. In the fourth beat, the data is sent from Figure 2 The DMA instruction virtual channel storage unit is successfully read out, Ar_valid is pulled high, and Ar_ready is high. Therefore, the current channel CH_2 completes a request transmission in the fourth beat. At this time, CH_done is pulled high, indicating that the current channel CH_2 has completed a data transmission and needs to switch to the next channel CH_3. Figure 2 The processing of DMA in the fifth beat and thereafter is similar to that from the second beat to the fourth beat, and will not be repeated here.

[0095] By combining the above Figure 5 From the description, we can see that Figure 2 The DMA in the system has a delay between the read signal and the read data of the instruction virtual channel storage unit, which causes the DMA polling scheduling module to generate a delay when switching between channels, resulting in the inability to pipeline processing and reduced efficiency.

[0096] In order to solve the above technical problems, the present invention proposes a method for Figure 2 The DMA optimization scheme in DMA is used to realize pipeline processing when switching channels.

[0097] Figure 6AA schematic structural diagram of a processing device for performing data processing on multiple channels according to at least one embodiment of the present disclosure is shown.

[0098] like Figure 6A As shown, the processing device for performing data processing on multiple channels includes a channel information acquisition unit, which is configured to acquire channel information of multiple channels; a storage unit, which includes multiple storage areas corresponding to multiple channels, wherein the storage areas are configured to store data information for multiple channels; a data read control unit, which is configured to read target data information corresponding to the channel information from a target storage area in the multiple storage areas of the storage unit according to the channel information; and a cache unit, which is configured to pre-store the target data information read by the data read control unit from the target storage area of ​​the storage unit to wait for the data processing. In this example, the data information can be the R-type instruction in the above text of the present disclosure, the storage unit can be the instruction virtual channel storage unit in the context of the present disclosure, and the data read control unit can be the instruction read control module or instruction read control unit in the context of the present disclosure. In this example, the channel information can be the data channel corresponding to the GLB cluster in the context of the present disclosure. In this example, the interface control unit can be the AXI interface control module or AXI interface control unit in the context of the present disclosure. It should be recognized that although Figure 6A The processing device for performing data processing on multiple channels shown in FIG includes an interface control unit. However, if the subsequent module of the processing device needs data information, the cache unit can directly output the data information to the subsequent module of the processing device without passing through the interface control unit. In other words, Figure 6A The interface control unit shown in FIG. 1 is not an essential functional module of the processing device.

[0099] Figure 6B An example architecture of a coprocessor-based DMA is shown in accordance with at least one embodiment of the present disclosure.

[0100] like Figure 6B As shown, Figure 2 Compared to the DMA shown in the previous example, the DMA architecture in this embodiment inserts a pre-set first-in-first-out queue (FIFO) between the instruction virtual channel storage unit and the AXI interface control module. This pre-set FIFO is used to pre-store channel instructions. This pre-set FIFO allows instructions from different channels to be pre-fetched, thus reducing the delay caused by the instruction virtual channel storage unit reading data.

[0101] According to an example of this embodiment, the preset FIFO can be a FIFO using the above-mentioned timing chip, that is, there is a one-beat delay between the read request and the read data of the preset FIFO. Figure 7 and Figure 8An example of an operation timing diagram of the DMA architecture in the present disclosure is described.

[0102] Figure 7 An example of a DMA operation timing diagram is shown in the embodiment when the preset FIFO does not pre-fetch instructions in advance.

[0103] like Figure 7 As shown, if Figure 6B The DMA backend modules and the on-chip network are always in a ready state (i.e. Figure 7 GLB_instr_ready and Ar_ready in the DMA are always high). In the first beat, since DMA has prepared data 1, Ar_valid is high. Since Ar_valid and Ar_ready are both high in the first beat, CH_done is high. The polling scheduling module in DMA outputs the channel label CH_1 and switches the channel to channel CH_2. In the second beat, the instruction read control module in DMA receives the channel label CH_1 sent by the polling scheduling module, so the SRAM_read request signal is pulled high in the second beat. Since data 2 has not yet been read into the preset FIFO in the second beat (corresponding to Figure 7In the third beat, according to the preset FIFO_write request, data 2 is read from channel 1 in the instruction virtual channel storage unit of DMA into the preset FIFO; because in the third beat, data 2 has not been read from the preset FIFO into the AXI interface control module of DMA, that is, DMA has not prepared data in the third beat, Ar_valid continues to be low; because in the third beat, DMA has not successfully sent data on the channel, CH_done continues to be low. In the fourth beat, according to the preset FIFO_read request, data 2 is read from the preset FIFO into the AXI interface control module of DMA and is successfully read into the AXI interface control module of DMA in the next beat. Similarly, Ar_valid and CH_done continue to be low. In the fifth beat, the DMA has already prepared data 2, so Ar_valid goes high. Since both Ar_valid and Ar_ready are high in the fifth beat, CH_done goes high, and the polling scheduler in the DMA outputs channel tag CH_2 and switches the channel to channel CH_3. In the sixth beat, the instruction read control module in the DMA receives channel tag CH_2 sent by the polling scheduler, so SRAM_read_request goes high in the sixth beat. Since data 3 has not yet been read into the preset FIFO in the sixth beat, the DMA is not ready for data, so Ar_valid goes low. Since the DMA has not successfully sent data on the channel in the sixth beat, CH_done goes low. In the seventh beat, according to the preset FIFO_write_request, data 3 is read from channel 2 in the DMA's instruction virtual channel storage unit into the preset FIFO. Since data 3 has not yet been read from the preset FIFO to the DMA's AXI interface control module in the seventh beat, the DMA is still not ready for data in the seventh beat, so Ar_valid remains low. Since the DMA has not successfully sent data on the channel in the seventh beat, CH_done remains low. In the eighth beat, according to the preset FIFO_read request, data 3 is read from the preset FIFO into the AXI interface control module of the DMA and successfully read into the AXI interface control module of the DMA in the next beat. Similarly, Ar_valid and CH_done continue to be low, and so on.

[0104] Combine Figure 7 As can be seen from the description, in the case where the preset FIFO in the present disclosure adopts the above-mentioned timing chip, Figure 6B The DMA shown introduces at least three beats of latency when switching between different channels.

[0105] The preset FIFO in the present disclosure can also be a memory using combinational logic, which can realize that the read request and the read data appear at the same time, thereby saving a delay of one beat. Figure 8 An example of a preset FIFO in the present disclosure is described.

[0106] like Figure 8 As shown, the preset FIFO includes a multiplexer (MUX) and N storage addresses. Each row represents a storage address. Data 1, data 2, ..., data N are stored in the corresponding storage addresses respectively. Each storage address is connected to the MUX. The selection end of the MUX is the pointer of the preset FIFO. The default value is to point to the address where data 1 is located. When a read request is received, the pointer is valid, the MUX outputs data 1, and the pointer is increased by 1. If a read request is received again, the pointer is valid again, the MUX outputs data 2, and the pointer is increased by 1, and so on. It should be recognized that if Figure 6B The DMA in this disclosure is combined with Figure 8 The preset FIFO described above will introduce at least two beats of delay when the DMA switches between different channels. Under this premise, the present disclosure proposes to eliminate the delay of combining the instruction data of at least one channel by pre-fetching the instruction data of at least one channel in the preset FIFO. Figure 7 Some or all of the delays described in .

[0107] Figure 9 An example of a DMA operation timing diagram is shown when instruction data of three different channels are pre-stored in a preset FIFO according to the present disclosure. In this example, the preset FIFO is a memory using combinational logic.

[0108] like Figure 9 As shown in the figure, data 1, data 2 and data 3 are the data of three different channels pre-fetched in advance in the preset FIFO. Figure 6B The DMA backend modules and the on-chip network are always in a ready state (i.e. Figure 9(GLB_instr_ready and Ar_ready are always high in the first beat to the third beat). Since the instruction data of three different channels are pre-stored in the preset FIFO of the DMA and the read request and read data of the preset FIFO appear in the same beat, Ar_valid is always high in the first beat to the third beat. Since Ar_valid and Ar_ready are both high in the first beat to the third beat, CH_done is always high. The polling scheduling module in the DMA implements pipeline switching, and it outputs channel labels CH_1, CH_2, and CH_3 in the first beat, second beat, and third beat respectively. In the second beat, third beat, and fourth beat, the instruction read control module in the DMA receives the channel labels CH_1, CH_2, and CH_3 sent by the polling scheduling module in sequence. Therefore, the SRAM_read request signal is always high in the second beat to the fourth beat. In the third beat, fourth beat, and fifth beat, according to the preset FIFO_write request, data 4, data 5, and data 6 (data 5 and data 6 are not shown) are read into the preset FIFO from channel 1, channel 2, and channel 3 in the instruction virtual channel storage unit of the DMA in sequence. In the fourth, fifth and sixth beats, according to the preset FIFO_read request, data 4, data 5 and data 6 are read into the AXI interface control module of DMA in sequence from the preset FIFO and data 4, data 5 and data 6 are successfully read into the AXI interface control module of DMA in the same beat, that is, DMA has prepared data in the fourth, fifth and sixth beats, so Ar_valid continues to be high from the fourth to the sixth beat, thereby realizing the pipelined output instruction of DMA and eliminating the delay caused by switching between different channels.

[0109] The DMA disclosed in this invention adds a preset FIFO pre-stored instruction mechanism to Figure 2 When the AXI interface control module of the DMA is busy, the instruction data is sent from Figure 2 The DMA instruction is taken out from the virtual channel storage unit, saving the delay of its read operation.

[0110] Combined with the preset FIFO pre-stored instruction mechanism proposed in this disclosure, Figure 6B The DMA in the instruction saves the delay of the instruction virtual channel storage unit read operation when switching between different channels. On the other hand, for the single-channel switching mode, there will still be a delay of at least two beats.

[0111] Figure 10 The present invention shows a preset FIFO pre-stored instruction mechanism using at least one embodiment of the present disclosure. Figure 6B An example of the DMA working timing diagram in single-channel switching mode.

[0112] like Figure 10As shown, in the first beat, since the DMA has prepared data 1, Ar_valid is high; since Ar_valid and Ar_ready are both high in the first beat, CH_done is high, and the polling scheduling module in the DMA outputs the channel label CH_1 and switches the channel to channel CH_1. In the second beat, the instruction read control module in the DMA receives the channel label CH_1 sent by the polling scheduling module, so the SRAM_read request signal is pulled high in the second beat. Since the current channel is still channel CH_1 (i.e., channel 1) from the second beat to the fifth beat, according to the polling scheduling mechanism, it is necessary to wait for a data transmission to be completed on channel 1 before switching. In other words, it is necessary to wait for the data 2 of channel 1 in the instruction virtual channel storage unit of the DMA to be successfully written to the DMA's AXI interface control module according to the channel label CH_1 output in the first beat before switching. Therefore, from the second beat to the fourth beat, the DMA cannot output instruction data (the DMA successfully outputs data 2 in the fifth beat), thereby introducing a delay of at least three beats. Figure 10 The working timing of the signals SRAM_read request, SRAM_read data, preset FIFO_write request, preset FIFO_read request, preset FIFO_read data, etc. is consistent with the present disclosure. Figure 7 In addition, even if the preset FIFO adopts the memory of the combination logic disclosed in the present invention, Figure 6B The DMA in the system still has a two-beat delay in single-channel switching mode.

[0113] Based on the above embodiments, at least one embodiment of the present disclosure proposes Figure 2 Another optimization scheme for DMA in , to achieve pipeline processing when switching channels.

[0114] Figure 11A A schematic structural diagram of a processing device for performing data processing on multiple channel information according to at least one embodiment of the present disclosure is shown.

[0115] like Figure 11AAs shown, the processing device for performing data processing on multiple channel information includes a channel information acquisition module, a storage module, a reading module, and a pre-storage module, wherein: the channel information acquisition module is configured to acquire target channel information from the multiple channel information; the storage module includes multiple storage areas corresponding to the multiple channel information, wherein each of the multiple storage areas is configured to store data information for any channel information in the multiple channel information; the reading module is configured to pre-storage the data information from the multiple storage areas of the storage module to the pre-storage module in sequence; and the pre-storage module is configured to pre-storage the data information from the multiple storage areas of the storage module to wait for the data processing, and output the pre-stored target data information corresponding to the target channel information based on the acquired target channel information for the data processing. In this example, the data information can be the R-type instruction mentioned above in the present disclosure, the storage module can be the instruction virtual channel storage unit mentioned in the context of the present disclosure, the reading module can include the instruction read control module or instruction read control unit mentioned in the context of the present disclosure, and the reading module can also include the first polling scheduling module or unit mentioned below in the present disclosure. In this example, the channel information may be a data channel corresponding to a GLB cluster in the context of this disclosure, and the channel information acquisition module may be a second polling scheduling module or unit in the following text of this disclosure. In this example, the interface control unit may be an AXI interface control module or AXI interface control unit in the context of this disclosure. It should be recognized that although Figure 11A The processing device for performing data processing on multiple channels shown in FIG includes an interface control unit. However, if the subsequent module of the processing device needs data information, the pre-stored module can directly output the data information to the subsequent module of the processing device without passing through the interface control unit. In other words, Figure 11A The interface control unit shown in FIG. 1 is not an essential functional module of the processing device.

[0116] Figure 11B An example architecture of a coprocessor-based DMA according to at least one embodiment of the present disclosure is shown. Figure 11B As shown, Figure 2 Compared with the DMA shown in the figure, the DMA architecture in the present disclosure inserts multiple preset FIFOs between the instruction virtual channel storage unit and the AXI interface control module, and introduces a dual polling scheduling mechanism to separate the operation of fetching instructions from the instruction virtual channel storage unit from the DMA's subsequent modules and the on-chip network, eliminating or alleviating the back pressure of the DMA's subsequent modules and the on-chip network on the instruction fetch operation, thereby eliminating or reducing delays and improving DMA efficiency.

[0117] The preset FIFO in this embodiment is used to pre-store channel instructions. According to one example of this embodiment, the preset FIFO can be a FIFO using the aforementioned timing chip, meaning that there is a one-beat delay between the read request and the read data of the preset FIFO. According to another example of the present disclosure, the preset FIFO can be a memory using combinational logic, meaning that the read request and the read data of the preset FIFO appear in the same beat.

[0118] Continue to refer Figure 11B , each preset FIFO can be used to store instruction data of at least one channel. In one example, Figure 11B For example, the DMA in the embodiment has eight preset FIFOs, each of which is used to store instruction data of a channel, that is, preset FIFO #1 stores instruction data of channel 1, preset FIFO #2 stores instruction data of channel 2, and so on.

[0119] In another example, Figure 11B For example, the DMA in the embodiment has four preset FIFOs, each of which is used to store instruction data of two channels, that is, preset FIFO#1 stores instruction data of channel 1 and channel 2, preset FIFO#2 stores instruction data of channel 3 and channel 4, and so on.

[0120] The following is an example of each preset FIFO storing the instruction data of one channel. Figure 11B As shown, each time the preset FIFO completes writing of a command data, it can transmit a signal for handshake to the first polling scheduling module. For example, it can transmit a CH_done signal to the first polling scheduling module to indicate that a data write has been successfully completed on the channel corresponding to the preset FIFO. After receiving the CH_done signal transmitted by the preset FIFO, the first polling scheduling module can switch channels (for example, once) and output the switched channel tag CH_tag. The instruction read control module reads data from the corresponding channel in the instruction virtual channel storage unit into the corresponding preset FIFO according to the channel tag CH_tag, thereby realizing the early removal of the instruction data of each channel into the preset FIFO.

[0121] In one example, before prefetching instruction data into a preset FIFO, the instruction read control module can determine whether the storage area of ​​the corresponding channel in the instruction virtual channel storage unit is empty and whether the corresponding preset FIFO is full. If the storage area of ​​the corresponding channel in the instruction virtual channel storage unit is not empty and the corresponding preset FIFO is not full, the instruction data is prefetched into the preset FIFO. In this example, when fetching instructions from the instruction virtual channel storage unit, the first polling scheduling module polls and transfers instruction data on eight channels to the eight preset FIFOs, thereby completing the prefetch operation on the instruction data on each channel.

[0122] Continue to refer Figure 11B The AXI interface control module includes a second polling and scheduling module. This module is subject to backpressure from the DMA's downstream modules and the on-chip network. This backpressure is applied when the GLB_instr_ready or Ar_ready signals are pulled low. The second polling and scheduling module polls the data transfer status of the eight channels and switches to the next channel after completing a data transfer on the current channel. The second polling and scheduling module also outputs a channel tag, CH_tag.

[0123] A multiplexer is further included between the preset FIFOs and the AXI interface control module. In the above example, the outputs of the eight preset FIFOs can be connected to the inputs of the multiplexer, and the channel tag CH_tag output by the second polling scheduling module can serve as the selection terminal of the multiplexer. For example, when the channel tag output by the second polling scheduling module is CH_1 (i.e., channel 1), the multiplexer can select preset FIFO #1 (i.e., the preset FIFO for prefetching instruction data on channel 1), thereby reading the instruction data on channel 1 into the AXI interface control module.

[0124] Figure 12 Shown Figure 11B An example of the DMA working timing diagram in single channel switching mode. Figure 12 As shown, the depth of the preset FIFO is 6, which can store six instructions, namely instruction 1, instruction 2, instruction 3, instruction 4, instruction 5, and instruction 6. The preset FIFO is used to store instruction data on a single channel, for example, Figure 11B DMA instruction data on channel 1. Assume Figure 11B The DMA downstream modules and the on-chip network are always in the ready state, that is, GLB_instr_ready and Ar_ready are always high. At the same time, it is assumed that the maximum processing speed of the AXI interface control module is to process one instruction data per beat.

[0125] At the beginning of the first beat (e.g., the rising edge of the first beat), the AXI interface control module is in an idle state. During the first beat, the preset FIFO_read request signal is high. The preset FIFO can utilize the memory of the combinational logic described above in the present disclosure. Since the preset FIFO read request and the read data occur in the same beat, instruction 1 read in accordance with the preset FIFO_read request during the first beat is read from the preset FIFO into the AXI interface control module, which then processes instruction 1 during the first beat.

[0126] In the second beat, the preset FIFO_read request signal continues to be high, and the AXI interface control module has already processed instruction 1. Therefore, in the second beat, it continues to read instruction 2 to the AXI interface control module according to the preset FIFO_read request, and the AXI interface control module processes instruction 2 in the second beat. Since the empty / full status of the FIFO can only be detected in the next beat of the read or write operation, in the first beat of reading instruction 1 from the preset FIFO to the AXI interface control module, the first polling scheduling module has not yet detected that the preset FIFO is not full, so SRAM_read request is low (for example, 0). In the second beat, the first polling scheduling module detects that the preset FIFO is not full and can continue to write instruction data to the preset FIFO. Therefore, SRAM_read request is high (for example, 1) and the instruction data (for example, instruction 7) is read from the instruction virtual channel storage unit in the next beat. In the second beat, the instruction data (for example, instruction 7) has not yet been read from the instruction virtual channel storage unit, so the preset FIFO_write request is low (for example, 0).

[0127] In the third beat, the preset FIFO_read request signal remains high, and the AXI interface control module has already completed processing instruction 2. Therefore, in the third beat, instruction 3 is read out to the AXI interface control module according to the preset FIFO_read request. The AXI interface control module processes instruction 3 in the third beat. In the third beat, according to the SRAM_read request in the second beat, the instruction data (for example, instruction 7) is successfully written to the write port of the preset FIFO. At this time, the preset FIFO_write request is high (for example, 1). Since the preset FIFO is still not full in the third beat, the SRAM_read request continues to be high (for example, 1) and the instruction data (for example, instruction 8) is read out from the instruction virtual channel storage unit in the next beat.

[0128] In the fourth beat, the preset FIFO_read request signal remains high, and the AXI interface control module has already completed processing instruction 3. Therefore, in the fourth beat, the module continues to read instruction 4 to the AXI interface control module according to the preset FIFO_read request signal. The AXI interface control module processes instruction 4 in the fourth beat. While the AXI interface control module is processing instruction 4, the new instruction data (for example, instruction 7) has already been successfully written into the preset FIFO.

[0129] Similarly, in the fifth beat, when the AXI interface control module processes instruction 5, the new instruction data (such as instruction 8) has also been successfully written into the preset FIFO, and so on, thereby realizing DMA pipeline processing of instructions and improving DMA efficiency.

[0130] The DMA in at least some embodiments of the present disclosure introduces a dual polling scheduling mechanism to separate the DMA's subsequent modules and the on-chip network from the logic for instruction fetching at the instruction virtual channel storage unit. The instruction fetch at the instruction virtual channel storage unit is no longer subject to back pressure from the DMA's subsequent modules and the on-chip network, thereby eliminating or reducing not only the delay caused by switching between different channels but also the delay caused by single-channel switching mode, thereby improving DMA efficiency.

[0131] In some embodiments of the present disclosure, the DMA can not only eliminate or reduce the delay caused by switching between different channels, but also eliminate the delay caused by the single-channel switching mode by introducing a dual polling scheduling mechanism. Figure 11B When the DMA in the system writes instructions to the preset FIFO, there is a risk of reading more than one instruction data. Figure 13A right Figure 11B The risk of multiple DMA read instruction data is described in detail.

[0132] like Figure 13A As shown, it is assumed that the depth of the single-channel preset FIFO in the present disclosure is 2 (that is, the preset FIFO can only write two instruction data). In the second and third beats, SRAM_read request is high, and the data can be read from the instruction virtual channel storage unit in the DMA and placed in the write port of the preset FIFO in the next beat, that is, data 1 and data 2 are read out and placed in the write port of the preset FIFO in the third and fourth beats respectively. In the third and fourth beats, preset FIFO_write request is high, so data 1 and data 2 are written to the preset FIFO in the third and fourth beats respectively.

[0133] Since the preset FIFO is already full of instruction data in the fourth beat, the full signal of the preset FIFO is pulled high on the rising edge of the fifth beat, that is, the preset FIFO_full is high in the fifth beat. Since the empty / full status of the preset FIFO can only be sensed in the next beat of the read / write operation, the instruction read control module in the DMA has not yet sensed that the preset FIFO is full in the fourth beat. If there is still instruction data in the instruction virtual channel storage unit in the fourth beat DMA, the SRAM_read request continues to be high. In the fifth beat, according to the SRAM_read request of the previous beat, data 3 is read out from the instruction virtual channel storage unit and placed in the write port of the preset FIFO. Since in the fifth beat, the preset FIFO_full is high, thereby counter-pressuring the preset FIFO_write request, the preset FIFO_write request is pulled low in the fifth beat, thereby causing the risk of the over-read data 3 being blocked in the preset FIFO write port and unable to be written.

[0134] Based on the above embodiments, at least one embodiment of the present disclosure proposes a method for controlling data writing, thereby avoiding Figure 11BThe risk of multiple DMA read instructions.

[0135] Figure 13B FIG. 1 shows a schematic diagram of a device for accessing data according to at least one embodiment of the present disclosure. Figure 13B As shown, the device for accessing data includes a storage module, a read module, a first-in-first-out queue, and a fetch module, wherein: the storage module is configured to store the data received by the device; the read module is configured to read the data from the storage module and write it into the first-in-first-out queue according to a read request signal; the first-in-first-out queue is configured to write the data according to a control signal; and the fetch module is configured to fetch the data from the first-in-first-out queue, wherein the read request signal is controlled by a first status signal of the first-in-first-out queue and a second status signal of the storage module, the first status signal indicating whether the first-in-first-out queue is full or not full, and the second status signal indicating whether the storage module is empty or not empty. In this example, the data can be the R-type instruction mentioned above in the present disclosure, and the storage module can be the instruction virtual channel storage unit mentioned in the context of the present disclosure. It should be noted that in this example, the data can also be any other type of instruction data or any non-instruction data, for example, the data can be instruction data used to perform a specific operation, or various types of data using information as a carrier. In this example, the read module can obtain a first status signal from the first-in-first-out queue and a second status signal from the storage module. If the first-in-first-out queue is full of data (i.e., there is no address space to store the next piece of data), the first status signal can indicate the full state of the first-in-first-out queue; otherwise, the first status signal can indicate the non-full state of the first-in-first-out queue. If the storage module has no data (e.g., the read module has read all the data therein into the first-in-first-out queue), the second status signal can indicate the empty state of the storage module; otherwise, the second status signal can indicate the non-empty state of the storage module. As an example of a read request signal being controlled by the first status signal and the second status signal, if the first status signal indicates the non-full state of the first-in-first-out queue and the second status signal indicates the non-empty state of the storage module, the read request signal is high (i.e., indicating that the read request is valid); otherwise, the read request signal is low (i.e., indicating that the read request is invalid). In one example, the control signal can be controlled by the first status signal. In another example, the control signal can be controlled by the read request signal and the first status signal. In yet another example, the control signal may be controlled by a third state signal indicating a nearly full / not nearly full state of the FIFO queue. Specific embodiments of the control signal will be described in detail below.

[0136] According to an example of the present disclosure, the writing of data can be controlled according to the falling edge of the preset FIFO full signal. In this example, when the full signal of the preset FIFO is pulled low, the multi-read instruction data is written by detecting the falling edge of the full signal.

[0137] Figure 14 An example of an operation timing diagram for controlling the writing of instruction data according to the falling edge of a full signal of a preset FIFO is shown.

[0138] exist Figure 14 The working sequence diagram of the first to fifth beats is the same as Figure 13A The same as shown will not be repeated here. Figure 14 As shown, in the fifth beat, the extra data 3 read out is blocked in the preset FIFO write port and cannot be written. Figure 11B The DMA AXI interface control module in the DMA takes out data 1 from the preset FIFO, thereby freeing up a storage address in the preset FIFO. Therefore, the preset FIFO is not full in the sixth beat and can be sensed in the next beat. In the seventh beat, Figure 11B The DMA AXI interface control module in the DMA takes data 2 from the preset FIFO, thereby freeing up a storage address in the preset FIFO. At the same time, the preset FIFO_full is pulled low in the seventh beat. The non-full state of the preset FIFO is sensed in the seventh beat, so the SRAM_read request is pulled high in the seventh beat and data 4 is taken from the DMA in the next beat. Figure 11B The DMA instruction virtual channel storage unit is taken out. In this example, Figure 14 As shown in α, the falling edge of the full signal "preset FIFO_full" of the preset FIFO is detected. If the falling edge of the full signal is detected, the preset FIFO_write request is pulled high, so that the more read data 3 is written to the next beat (i.e., when the preset FIFO is not full) Figure 14 The seventh beat in the process) is written into the preset FIFO, thereby avoiding the risk that the multi-read instruction data is blocked in the preset FIFO write port and cannot be written.

[0139] Although at least some embodiments of the present disclosure provide a solution for controlling the writing of data by detecting the falling edge of a preset FIFO full signal, in extreme cases, Figure 11B When the DMA in the system stores instructions in the preset FIFO, there is a situation where more than one instruction data is not read. Under this premise, controlling the writing of data only by detecting the falling edge of the preset FIFO full signal will lead to the risk of writing invalid data.

[0140] Figure 15 The risk of writing invalid data caused by only detecting the falling edge of the full signal to control data writing is explained. Figure 15 As shown, from the second to the fourth beat, Figure 11B The status of the preset FIFO perceived by the DMA instruction read control module is not full, so from the second to the fourth beat, the SRAM_read request will not be affected by the back pressure of the preset FIFO_full signal. However, if in the fourth beat, Figure 11B If there is no readable instruction data in the instruction virtual channel storage unit of the DMA (that is, the storage area of ​​the corresponding channel in the instruction virtual channel storage unit is empty), then the SRAM_read request will be back-pressured by the empty state of the instruction virtual channel storage unit in the fourth beat, so the SRAM_read request is pulled low in the fourth beat, and the preset FIFO is just in a full state at this time, so there is no risk of reading more instruction data. When the falling edge of the preset FIFO full signal is detected, the preset FIFO_write request is pulled high (that is, Figure 15 In the seventh beat, invalid data will be written into the preset FIFO, thereby introducing risks.

[0141] Based on the above embodiments, the present disclosure provides a solution for controlling the writing of data by using a flag signal, thereby avoiding the risk of multiple reads of instruction data while avoiding the writing of invalid data.

[0142] According to an example of the present disclosure, a flag signal may be determined according to a full signal of a preset FIFO and an SRAM_read request signal. Figure 11B The instruction virtual channel storage unit in the DMA shakes hands with the preset FIFO through a flag signal. When the flag signal is valid, instruction data can be written from the instruction virtual channel storage unit to the preset FIFO. When the flag signal is invalid, writing instruction data from the instruction virtual channel storage unit to the preset FIFO is prohibited.

[0143] Figure 16 FIG1 shows an example of a flag signal according to the present disclosure to control the working timing diagram of writing data. In this example, the flag signal is obtained by performing a logical AND operation on the falling edge of the SRAM_read request signal and the full signal of the preset FIFO. Figure 16 As shown, the falling edge of the SRAM_read request is detected in the third beat (high), and the FIFO full signal is preset to low at the falling edge of the third beat. The two can be ANDed together to obtain a low flag signal. When the flag signal is low, the FIFO_write request signal can be preset in the seventh beat, so that data is not written to the preset FIFO in the seventh beat, avoiding the risk of writing invalid data into the preset FIFO. Similarly, Figure 14As shown, if the falling edge of the SRAM_read request is detected in the fourth beat (high), and the full signal of the preset FIFO is high at the falling edge of the fourth beat, the two can be ANDed together to obtain a flag signal that is high. When the flag signal is high, the preset FIFO_write request signal in the seventh beat can be enabled, thereby writing the multi-read instruction data into the preset FIFO in the seventh beat, avoiding the risk of the multi-read instruction data being blocked in the preset FIFO write port and unable to be written.

[0144] At least some embodiments of the present disclosure propose to avoid the risk of multiple readings of instruction data by controlling the writing of data through flag signals. On the other hand, when this solution is applied to switch between multiple different channels, there may be a risk of writing the multiple readings of instruction data on the current channel into the preset FIFOs on other channels, thereby causing erroneous instruction writing.

[0145] Based on the above embodiments, the present disclosure proposes controlling data writing by utilizing the Almost_Full mechanism in the FIFO.

[0146] Figure 17 FIG. 2 shows an example of a timing diagram for controlling data writing using an Almost_Full signal (abbreviated as afull signal) according to the present disclosure. Figure 17 As shown, in the second and third beats, SRAM_read request is high, according to the SRAM_read request signal Figure 11B The DMA in the program writes data 1 and data 2 into the preset FIFO in the third and fourth beats respectively. Since the depth of the preset FIFO is 2, when the first data is stored in the preset FIFO (i.e., the third beat), the prevention FIFO is almost full and the afull signal indicating the almost full state is pulled high in the next beat. Therefore, the preset FIFO_afull (i.e., the afull signal of the preset FIFO) is high in the fourth beat. In the fourth beat, the preset FIFO_afull signal after being pulled high can back-pressure the SRAM_read request signal, thereby avoiding Figure 11B The instruction read control module of the DMA continues to read data from the instruction virtual channel unit.

[0147] At least some embodiments of the present disclosure propose to avoid the risk of multiple read instruction data in single-channel switching mode and multi-channel switching mode by controlling the writing of data using the Almost_Full mechanism in the FIFO. On the other hand, if Figure 11B The depth of the preset FIFO in the DMA is 2, which will introduce a delay of at least two beats. Based on this, preferably, in an example of the present disclosure, the depth of the preset FIFO is selected to be 4, and the Almost_Full signal is selected to be pulled high when 3 data are stored, thereby solving the two-beat delay introduced when the preset FIFO depth is 2, and achieving Figure 11BThe DMA pipeline processing of instruction data improves the efficiency of DMA.

[0148] This application uses specific terms to describe the embodiments of this application. For example, "first / second embodiment", "one embodiment", and / or "some embodiments" refer to a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different places in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application may be appropriately combined.

[0149] In addition, it will be understood by those skilled in the art that various aspects of the present application can be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application can be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0150] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or highly formal sense, unless expressly defined as such herein.

[0151] The above is an illustration of the present invention and should not be considered as limiting thereof. Although several exemplary embodiments of the present invention have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined by the claims. It should be understood that the above is an illustration of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.

Claims

1. A processing device for performing data processing on a plurality of channels, comprising: a channel information acquiring unit, configured to acquire channel information of the plurality of channels; a storage unit comprising a plurality of storage areas corresponding one-to-one to the plurality of channels, wherein the plurality of storage areas are configured to store data information for the plurality of channels, the data information including data handling instructions, and each storage area is configured to store the data handling instructions for the corresponding channel; a data read control unit configured to read target data information corresponding to the channel information from a target storage area among the plurality of storage areas of the storage unit according to the channel information; and A cache unit is configured to pre-store the target data information read by the data read control unit from the target storage area of ​​the storage unit to wait for the data processing, and the cache unit includes a first-in-first-out queue based on combinational logic or sequential logic.

2. The processing device according to claim 1, wherein The cache unit includes a first-in-first-out queue based on combinational logic, and The first-in-first-out queue based on combinatorial logic is configured to enable a request signal for the first-in-first-out queue based on combinatorial logic and data information associated with the request signal to appear in the same clock cycle.

3. The processing device according to claim 2, wherein The cache unit is configured to pre-store at least two items of data information used for the data processing.

4. The processing device according to claim 1, wherein The cache unit includes a first-in-first-out queue based on sequential logic, and The cache unit is configured to pre-store at least three items of data information used for the data processing.

5. The processing device according to claim 1 or 3, wherein The channel information acquiring unit is configured to query different channels in at least two adjacent polling rounds.

6. The processing device according to claim 1, wherein The processing device is a direct memory access device (DMA), and the data processing includes data transfer operations.

7. The processing device according to claim 6, wherein: The data transfer operation includes transferring the data information or transferring the target data indicated by the data information. The target data is the data required by the neural network processor to perform neural network operations. The neural network processor includes at least one processing unit array and multiple global buffer clusters. The data information indicates the address information of the target data. The address information is obtained based on the mapping relationship between the target data and at least one processing unit in the processing unit array. Each of the global buffer clusters is used to receive the data information being transported or the target data being transported on one communication information among the multiple channel information.

8. The processing device according to claim 6, further comprising: An interface control unit is configured to obtain the target data information from the cache unit, generate a data transfer request according to the target data information, and execute the data transfer operation according to the data transfer request.

9. The processing device according to claim 6, wherein: The channel information determining unit includes: A polling scheduling unit is configured to poll and determine a plurality of channel information on which the data processing is to be performed.

10. The processing device according to claim 9, wherein The polling scheduling unit is further configured to: In response to completing the data transfer operation on the current channel information and the data information in the storage unit is not empty, performing a next polling to determine the next channel information on which the data transfer operation is to be performed.

11. A computer system comprising: The processing device according to any one of claims 1-6, 8-10; as well as A neural network processor, wherein the neural network processor is electrically coupled to the processing device, the neural network processor includes at least one processing unit array and multiple global buffer clusters, the data information indicates address information of the target data, the address information is obtained based on a mapping relationship between the target data and at least one processing unit in the processing unit array, and each of the global buffer clusters is used to receive the transported data information or the transported target data indicated by the data information on one communication information among the multiple channel information.

12. The computer system of claim 11, further comprising: The interface decoder is configured to receive an initial instruction, decode the initial instruction to obtain the data information, and provide the decoded data information to the processing device.

13. The computer system of claim 12, further comprising: An auxiliary processor is configured to provide the initial instruction to the interface decoder.

14. The computer system according to claim 13, wherein: The auxiliary processor is coupled to the interface decoder via a second data transmission protocol, The neural network processor is coupled to the processing device via a first data transmission protocol, The first data transmission protocol and the second data transmission protocol are different.

15. A scheduling method for performing data processing on a plurality of channels, comprising: Obtaining channel information of the multiple channels; Reading target data information corresponding to the channel information from a target storage area in a plurality of storage areas corresponding one-to-one to the plurality of channels according to the channel information, the target data information including a data transfer instruction, each storage area being used to store the data transfer instruction on the corresponding channel; The read target data information corresponding to the channel information is pre-stored in a cache unit to wait for the data processing, wherein the cache unit includes a first-in-first-out queue based on combinational logic or sequential logic.

16. The scheduling method according to claim 15, wherein: The acquiring channel information of the plurality of channels includes: Query different channels in at least two adjacent polling cycles.

17. The scheduling method according to claim 15, wherein: The scheduling method is used for a direct memory access device, and the data processing includes a data moving operation.

18. The scheduling method according to claim 17, wherein: The data transfer operation includes transferring the data information or transferring the target data indicated by the data information. The target data is the data required by the neural network processor to perform neural network operations. The neural network processor includes at least one processing unit array and multiple global buffer clusters. The data information indicates the address information of the target data. The address information is obtained based on the mapping relationship between the target data and at least one processing unit in the processing unit array. Each of the global buffer clusters is used to receive the data information being transported or the target data being transported on one communication information among the multiple channel information.

19. The scheduling method according to claim 17, further comprising: The target data information is obtained from the cache unit, a data transfer request is generated according to the target data information, and the data transfer operation is performed according to the data transfer request.

20. The scheduling method according to claim 17, wherein: The acquiring channel information of the plurality of channels includes: The plurality of channel information on which the data processing is to be performed is determined by polling.

21. The scheduling method according to claim 20, wherein: The polling to determine the plurality of channel information on which the data processing is to be performed includes: In response to the data transfer operation being completed on the current channel information and the data information in the storage area being non-empty, a next poll is performed to determine the next channel information on which the data transfer operation is to be performed.

Images