A streaming vector processing apparatus supporting efficient processing of sparse matrices

Abstract

The application provides a streaming vector processing device, comprising: a scalar processing core for executing general scalar instructions, vector extension instructions, inner product type streaming processing instructions corresponding to at least one sparse compression format data and a streaming descriptor corresponding to an inner product task of the sparse compression format data; the vector processing core is used for triggering execution of the inner product task corresponding to the streaming descriptor in response to the inner product type streaming processing instruction, so as to perform streaming loading, streaming operation, streaming reduction and streaming storage of inner product related data according to the guidance of the streaming descriptor through the internal hardware of the vector processing core; full hardware automatic processing of the sparse matrix inner product operation is realized, the instruction emission and branch prediction overhead of the scalar processing core in processing the sparse format is significantly reduced; the streaming loading and reduction mechanism is used, the storage access delay can be effectively covered, and the memory access throughput and the calculation parallelism between the sparse data and the vector data are improved.

Description

Technical Field

[0001] This invention relates to the field of processor architecture technology, more specifically to the field of on-chip storage hierarchy optimization technology, and more specifically to a streaming vector processing device that supports efficient processing of sparse matrices. Background Technology

[0002] In artificial intelligence operators, such as sparse matrix and vector multiplication, sparse convolution, attention sparsification, and graph neural network message aggregation, the computing end prefers an explicit data layout characterized by dense blocks and graph tiles to match the alignment requirements of single instruction multiple data, single instruction multiple thread, and vector ports.

[0003] However, existing implementations primarily rely on general-purpose processors (scalar processing cores) to perform a series of non-numerical computational tasks such as index decoding, pointer and cursor advancement, address calculation, and zero-value filling. This leads to a rapid increase in instruction entries, making it difficult to efficiently merge memory access requests on the bus. The computational units are often occupied by structured access control logic, further weakening the predictability of scheduling for real-time systems. Especially in typical applications such as robotics and embodied intelligence, tasks like simultaneous localization and mapping (SLAM), sparse factor graph optimization, point cloud sparse convolution, multi-sensor fusion, contact dynamics solving, and graph search and path planning widely exhibit high sparsity, strict time constraints, and high energy efficiency sensitivity. For inner product operations related to sparse formats, relying on general-purpose processors and their instruction sets for memory access and layout results in significant instruction issuance and branch prediction overhead, leading to high memory access latency. When using vector processing cores to co-process sparse data obtained from general-purpose processing, the mismatch between storage and computation layout introduces structural overhead, affecting memory access throughput and computational parallelism.

[0004] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0005] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a streaming vector processing device that supports efficient processing of sparse matrices.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] According to a first aspect of the present invention, a streaming vector processing apparatus is provided, comprising: a scalar processing core, configured to execute general scalar instructions, vector expansion instructions, inner product-type streaming processing instructions corresponding to at least one sparse compressed format data, and a stream descriptor corresponding to an inner product task configuring the sparse compressed format data; and a vector processing core, configured to, in response to the inner product-type streaming processing instructions, trigger the execution of an inner product task corresponding to the stream descriptor, so as to perform streaming loading, streaming operation, streaming reduction, and streaming storage of inner product-related data through the internal hardware of the vector processing core according to the guidance of the stream descriptor. This solution achieves at least the following beneficial technical effects: the solution designs the scalar processing core's responsibility to execute inner product-type streaming processing instructions to trigger the vector processing core to perform streaming loading, streaming operation, streaming reduction, and streaming storage of inner product-related data according to the guidance of the stream descriptor, without requiring the scalar processing core to complete the related work through a large number of instructions. It achieves fully automated hardware processing of sparse matrix inner product operations, significantly reducing the instruction issuance and branch prediction overhead of scalar processing cores when processing sparse formats; by utilizing streaming loading and reduction mechanisms, it can effectively mask memory access latency and improve memory access throughput and computational parallelism between sparse data and vector data.

[0008] Optionally, the vector processing core includes a streaming engine sub-core: a descriptor management module, used to store at least one stream descriptor configured by the scalar processing core, each stream descriptor describing the compressed storage area of ​​the sparse matrix, the storage area of ​​the input vector, and the storage area of ​​the output vector; a flow control module, used to select a target stream descriptor from the descriptor management unit according to the stream descriptor number carried by the instruction when the scalar processing core issues an inner product-type streaming processing instruction, and distribute the sparse matrix data stream, vector data stream, and result write-back stream task identifier to the downstream module; and a streaming loading module, used under the control of the flow control module, to generate memory addresses pointing to the row pointer array, index array, and non-zero element array according to the metadata location of the compressed sparse row format or other sparse compressed format given in the stream descriptor, request to read the corresponding metadata and non-zero data row by row from the storage subsystem, and generate memory addresses pointing to the input vector elements according to the index information, read the vector element corresponding to each non-zero element from the storage subsystem, and write the obtained non-zero matrix element data into the first loading module. A first-in-first-out (FIFO) buffer queue is used to write the obtained vector element data into a second loading FIFO buffer queue. A streaming computation module is used to read non-zero matrix elements and corresponding vector elements in pairs from the first and second loading FIFO buffer queues, perform multiplication operations on each pair of data, and accumulate the multiplication into a row-level reduction register. When all non-zero elements corresponding to the current row have been processed, the accumulated value is read from the row-level reduction register to form the row result of the output vector element of that row. A streaming write module is used to write the row result output by the streaming computation module into a result buffer queue, and generate a write-back address according to the output vector layout information given in the stream descriptor. The row result is written to the corresponding output vector position in the storage subsystem through the on-chip interconnect and storage interface. The inner product-type streaming processing instruction is triggered by the stream control module to coordinate the above-mentioned streaming loading module, streaming computation module, and streaming write module, thereby completing the inner product calculation between the sparse compressed matrix rows and the input vector and the result write-back within the hardware. This scheme can achieve at least the following beneficial technical effects: when the inner product class streaming processing instruction is executed, the streaming control module triggers the above-mentioned streaming loading module, streaming computing module and streaming storage and writing module to work together to realize the streaming loading and reduction mechanism, which can effectively mask the storage access latency and improve the memory access throughput and computing parallelism between sparse data and vector data.

[0009] Optionally, the stream descriptor includes at least: a base address field and a length field for the array of non-zero elements of the sparse matrix; a base address field and a length field for the array of indexes of the sparse matrix; a base address field and a length field for the array of row pointers; a base address field and a length field for the input vector; a base address field and a length field for the output vector; and control fields for identifying the sparse format type, element data width, alignment, and stepping mode. When generating memory access addresses, the streaming loading module only uses the fields in the stream descriptor to implement the memory access task covering the entire large loop body line by line, without requiring the scalar processing core to explicitly calculate the address inside the row loop. This scheme can achieve at least the following beneficial technical effects: by predefining complete sparse format metadata and layout information in the stream descriptor, it realizes hardware self-driven access to ultra-long loop bodies with "one command for multiple lines," eliminating the computational burden of the scalar core frequently calculating addresses inside the row loop, and significantly improving the efficiency and accuracy of address generation.

[0010] Optionally, the vector processing core further includes a conventional vector execution subcore, which contains logic for performing vector loading, vector storage, vector arithmetic operations, and vector reduction operations. It can complete data transfer between the vector register and the storage subsystem through explicit vector loading and vector storage instructions, so as to support vector extension instructions based on instruction set standards through the conventional vector execution subcore.

[0011] This solution achieves at least the following beneficial technical effects: it realizes the deep integration of conventional vector instruction sets and descriptor-driven streaming architecture, which not only retains the flexibility of standard vector processing cores in processing general tasks, but also enhances their dedicated acceleration capabilities for processing large-scale sparse data and complex memory access patterns, so that streaming processing instructions and the vector extension instructions can coexist in the same vector processing core architecture, taking into account both versatility and high performance.

[0012] Optionally, a row-level reduction register is set in the streaming computation module. At the beginning of each row, it is cleared to zero under the control logic. During the processing of that row, only the local multiply-accumulate path performs read, modify, and write operations. When the streaming loading module issues a flag indicating the end of processing for the current row, the streaming computation module reads the accumulated value in the row-level reduction register, writes it to the result buffer queue, and triggers the row-level reduction register to be cleared again before starting processing the next row. This scheme achieves at least the following beneficial technical effects: through the hardware-controlled row-level reduction register and its automatic clearing mechanism, frequent access to the register file or memory for intermediate accumulated results is avoided, reducing the power consumption of the data path and ensuring logical isolation and result consistency of reduction operations between rows.

[0013] Optionally, both the first and second loading FIFO buffer queues adopt a ring storage structure and are equipped with independent read pointers, write pointers, and storage status registers. When the streaming computing module reads data from the two buffer queues, it only initiates a paired read operation if the storage status registers of both queues indicate the presence of valid data. Reading is paused when either queue is empty, ensuring a one-to-one correspondence between non-zero matrix elements and their corresponding vector elements in the time domain. This scheme achieves at least the following beneficial technical effects: by utilizing a dual-ring FIFO buffer structure and synchronous read control logic, precise alignment of non-zero elements of the sparse matrix with their corresponding vector elements in the time domain is achieved, effectively mitigating pipeline bubbles caused by irregular sparse memory access and enhancing the system's fault tolerance to latency fluctuations in the storage subsystem.

[0014] According to a second aspect of the present invention, a memory access control method based on stream descriptors is provided. This method is implemented in a streaming vector processing apparatus as described in the first aspect. The method includes: Step S1, determining the layout information of each data structure in the target computation task within the storage subsystem on the scalar processing core, encoding the base address, length, element data width, and step information of the compressed storage area of ​​the sparse matrix, the storage area of ​​the input vector, and the storage area of ​​the output vector into at least one stream descriptor, and writing it into the descriptor management module via a configuration instruction; Step S2, issuing an inner product-type streaming processing instruction corresponding to the stream descriptor when the scalar processing core executes the main program. This streaming processing instruction carries a descriptor number and a task identifier, which is identified and received by the flow control module inside the vector processing core; Step S3, the flow control module reads the corresponding stream descriptor field from the descriptor management module according to the descriptor number, and distributes the task identifiers of the matrix data stream, the vector data stream, and the result write-back stream to the streaming loading module, the streaming computation module, and the streaming storage module, respectively; Step S4 4. The streaming loading module automatically generates memory access addresses pointing to the sparse matrix compressed storage area based on the layout information of the row pointer array, index array, and non-zero element array given in the stream descriptor. It loads non-zero matrix elements row by row from the storage subsystem, and simultaneously generates memory access addresses pointing to input vector elements based on the index information, forming two aligned loading data streams. Step S5. The streaming computing module reads non-zero matrix elements and corresponding vector elements in pairs from the two loading data streams corresponding to the first and second loading first-in-first-out buffer queues. It performs inner product operations on each pair of non-zero matrix elements and corresponding vector elements and performs row-level accumulation in the row-level reduction register. When it is detected that the non-zero elements corresponding to a certain row have been processed, it outputs the row result of the inner product operation to the result buffer queue. Step S6. The streaming write module maps each row result in the result buffer queue to the corresponding storage location of the output vector according to the row number based on the layout information of the output vector given in the stream descriptor, and writes it back to the storage subsystem through the storage interface, completing the overall processing of a large loop memory access task. This solution can achieve at least the following beneficial technical effects: it provides a full-process streaming control method from task configuration to result write-back, which greatly simplifies the complexity of software programming by sinking the complex sparse memory access and operation logic to the hardware streaming engine, realizes a high degree of overlap between computation and memory access, and significantly improves the overall execution efficiency of the system.

[0015] Optionally, the process of generating memory access addresses by the streaming loading module includes: calculating the start and end positions of the non-zero elements in the current row in the index array and the non-zero element array based on two adjacent pointer values ​​in the row pointer array; for each non-zero element in the current row, calculating the element address of the input vector based on its column index value in the index array, the base address of the input vector, and the element's step information; sequentially initiating read requests to the non-zero element array and the input vector array according to the order from the start position to the end position, and writing the returned non-zero matrix element data and the corresponding input vector element data into the first and second load first-in-first-out buffer queues, respectively. This scheme can achieve at least the following beneficial technical effects: by automatically resolving row pointers and column indices in hardware, it accelerates the processing of indirect memory access mode, effectively solves the performance bottleneck of "index-load" operations under sparse format, and optimizes the utilization of storage bandwidth.

[0016] Optionally, the streaming computation module confirms the end of processing for the current row when all of the following conditions are met: First condition: The streaming loading module detects that all read requests for the index and non-zero data corresponding to the current row have been initiated and returned, and indicates to the streaming computation module via a control signal that the last data pair of the current row has been dequeued; Second condition: The read / write control logic of the first and second loading first-in-first-out buffer queues detects that all data pairs corresponding to the current row have been dequeued and there is no longer any valid data for that row in the buffer. This scheme can achieve at least the following beneficial technical effects: When the first and second conditions are met simultaneously, the streaming computation module determines that the processing for the current row has ended, outputs the accumulated value in the row-level reduction register as the row result to the result buffer queue, and triggers the row-level reduction register to be cleared. Thus, through the row end detection mechanism triggered by multiple conditions, the accuracy of boundary processing during streaming processing is ensured, mutual interference between different rows of data is prevented, and the correctness and robustness of the calculation results under large-scale streaming data processing are guaranteed.

[0017] Optionally, after issuing a streaming processing instruction, the scalar processing core obtains the completion status of the streaming processing task by polling the task status register in the vector processing core or by receiving an interrupt signal. When the flow control module inside the vector processing core detects that the streaming loading module, streaming computing module, and streaming write module have all reported that all data rows corresponding to the current task have been processed, it updates the task status register to the completion status and sends an interrupt request to the scalar processing core or modifies the task status register to allow the scalar processing core to read the task completion flag indicating the completion status of the corresponding streaming processing task in the next polling. This scheme can achieve at least the following beneficial technical effects: asynchronous cooperation between the scalar core and the vector core is realized through the task status register and the interrupt mechanism, allowing the scalar processing core to process other tasks during the execution of the streaming task, thereby improving the overall multi-task parallel processing capability and resource utilization of the system. Attached Figure Description

[0018] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0019] Figure 1 This is a schematic diagram of a streaming engine sub-core in a vector processing kernel according to an embodiment of the present invention;

[0020] Figure 2 This is a flowchart illustrating a memory access control method based on a stream descriptor according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0022] As mentioned in the background section, for inner product operations related to sparse data, relying on general-purpose processors and their instruction sets for memory access and layout leads to significant instruction issuance and branch prediction overhead, resulting in high memory access latency. When using vector processing cores to co-process sparse data obtained from general-purpose processing, the mismatch between storage and computation layouts introduces structural overhead, affecting memory throughput and computational parallelism. How to efficiently process sparse structures on the memory access side while presenting data in an explicit, dense form to vector computation units on the execution side to eliminate the structural overhead caused by the mismatch between storage and computation layouts has become a key technical bottleneck restricting the overall energy efficiency and real-time performance of such systems. To address this, this invention designs the scalar processing core's role to execute inner product-type streaming processing instructions to trigger the vector processing core to perform streaming loading, streaming computation, streaming reduction, and streaming storage of inner product-related data according to the guidance of the stream descriptor, without requiring the scalar processing core to complete the related work through a large number of instructions. It achieves fully automated hardware processing of sparse matrix inner product operations, significantly reducing the instruction issuance and branch prediction overhead of scalar processing cores when processing sparse formats; by utilizing streaming loading and reduction mechanisms, it can effectively mask memory access latency and improve memory access throughput and computational parallelism between sparse data and vector data.

[0023] According to an embodiment of the present invention, a streaming vector processing apparatus is provided, comprising: a scalar processing core, configured to execute general scalar instructions, vector expansion instructions, inner product-type streaming processing instructions corresponding to at least one sparse compressed format data, and a stream descriptor corresponding to an inner product task configuring the sparse compressed format data; and a vector processing core, configured to trigger the execution of an inner product task corresponding to the stream descriptor in response to the inner product-type streaming processing instructions, so as to perform streaming loading, streaming operation, streaming reduction, and streaming storage of inner product-related data through the internal hardware of the vector processing core according to the guidance of the stream descriptor.

[0024] According to one embodiment of the present invention, the scalar processing core is a general-purpose processor. The vector processing core includes a conventional vector execution subcore and / or a streaming engine subcore. The conventional vector execution subcore contains logic for performing vector loading, vector storage, vector arithmetic operations, and vector reduction operations, and is capable of data transfer between the vector register and storage subsystem via explicit vector loading and vector storage instructions, thereby supporting vector extension instructions based on instruction set standards through the conventional vector execution subcore. See also... Figure 1 The streaming engine sub-core includes a descriptor management module, a flow control module, a streaming loading module, a streaming computation module, and a streaming write module. When inner product-type streaming processing instructions are executed, the flow control module triggers the aforementioned streaming loading module, streaming computation module, and streaming write module to work together, thereby completing the inner product calculation between the rows of the sparsely compressed matrix and the input vector, and writing the result back within the hardware. Each module is illustrated below.

[0025] According to one embodiment of the present invention, a descriptor management module, specifically a descriptor generation and management unit, is used to store at least one stream descriptor configured by the scalar processing core. Each stream descriptor describes a compressed storage area for a sparse matrix, a storage area for input vectors, and a storage area for output vectors. The descriptor management module establishes a unified structured configuration carrier between the data paths within the scalar processing core and the vector processing core. The descriptor management module includes a descriptor register group and its management logic. The descriptor register group receives configuration fields written by the scalar processing core through configuration instructions or storage mapping, registering the base address and length of the non-zero element array of the sparse matrix, the base address and length of the index array, the base address and length of the row pointer array, and the base address and length of the input and output vectors in different register entries. The management logic internally assigns descriptor numbers and version tags to each register entry and establishes a mapping relationship between stream descriptor numbers and task identifiers. When the vector processing core prepares to execute a sparse matrix and vector operation task, the descriptor register group, under the control of the management logic, provides various address fields, length fields, and step fields associated with the task as requested.

[0026] According to one embodiment of the present invention, the flow control module and task distribution unit are used to select a target flow descriptor from the descriptor management unit based on the flow descriptor number carried by the scalar processing core when the scalar processing core issues an inner product-type streaming processing instruction, and distribute the sparse matrix data stream, vector data stream, and task identifier of the result write-back stream to the downstream module. The flow control module is used to complete the reception, parsing, and task-level control signal distribution of custom streaming processing instructions within the vector processing core. The flow control module includes an instruction distribution unit and task status management logic. The instruction distribution unit receives the streaming processing instruction and its carried descriptor number and task identifier from the scalar processing core through the coprocessing request channel, internally parses the instruction type and confirms that the operation corresponding to the instruction is a sparse matrix and vector operation or other long loop task that requires streaming processing; the task status management logic accesses the descriptor register group according to the descriptor number, extracts the configuration information of the sparse matrix storage area, index metadata area, input vector area, and output vector area corresponding to the current task, and records the current stage of the task in the internal task status register. The instruction distribution unit distributes the task identifier and corresponding configuration to the address generation unit, vector data loading unit, arithmetic logic unit, and write-back address generation unit via the control path, so that these units always work together around the same task identifier during subsequent operation.

[0027] According to one embodiment of the present invention, the sparse format-aware address generation and streaming loading unit of the streaming loading module, under the control of the flow control module, generates memory access addresses pointing to row pointer arrays, index arrays, and non-zero element arrays based on the metadata locations of compressed sparse row formats or other sparse compression formats given in the stream descriptor. It requests to read the corresponding metadata and non-zero data line by line from the storage subsystem, generates memory access addresses pointing to input vector elements based on the index information, reads the vector element corresponding to each non-zero element from the storage subsystem, writes the obtained non-zero matrix element data into a first load FIFO buffer queue, and writes the obtained vector element data into a second load FIFO buffer queue. The streaming loading module is used to construct a two-way aligned loading data stream from the sparse matrix compression format and the input vector under a given stream descriptor configuration. The streaming loading module includes an address generation unit, a vector data loading unit, and a load FIFO buffer queue. The address generation unit constructs a row pointer register file internally based on the row pointer array address and length provided by the descriptor register set, registers the start and end offsets of each row, and establishes a row-level status record for each row. Guided by the row-level status record, the address generation unit generates access addresses for the column index array and the non-zero element array, sends read requests pointing to non-zero elements to the external memory controller through the on-chip interconnect and bus interface circuit, and processes the returned data through the vector data loading unit before writing it into the matrix data loading first-in-first-out buffer queue (corresponding to the first loading first-in-first-out buffer queue), so that the buffer queue carries the non-zero matrix element data stream expanded row by row; simultaneously, the address generation... When parsing the column index array, the unit combines each column index value with the base address of the input vector and the element step information to generate the address of the input vector element. It then initiates a read request for the input vector and processes the returned data through the vector data loading unit before writing it into the vector data loading first-in-first-out buffer queue (corresponding to the second loading first-in-first-out buffer queue). This buffer queue carries the vector element data stream that corresponds one-to-one with the aforementioned non-zero matrix elements. The two loading first-in-first-out buffer queues adopt a circular storage structure, with independent read / write pointers and empty / full flag registers. The read / write rhythm is coordinated by shared row-level control logic to ensure that the data provided to the arithmetic logic unit is always output in pairs of "non-zero matrix elements and corresponding vector elements".

[0028] According to one embodiment of the present invention, a streaming computing module is used to read non-zero matrix elements and corresponding vector elements in pairs from a first loading FIFO buffer queue and a second loading FIFO buffer queue. It performs a multiplication operation on each pair of data and accumulates the multiplication into a row-level reduction register. When all non-zero elements corresponding to the current row have been processed, the accumulated value is read from the row-level reduction register to form the row result of the output vector element for that row. The streaming computing module is used to complete the inner product operation between sparse matrix rows and input vectors without explicitly exposing complex memory access details. The streaming computing module includes an arithmetic logic unit and a row-level reduction register. The arithmetic logic unit is connected to the matrix data loading FIFO buffer queue and the vector data loading FIFO buffer queue through two input ports respectively. When both buffer queues are detected to be non-empty, it simultaneously reads one element from each buffer queue and sends both elements into the multiplication operation path to generate an intra-row partial product. The row-level reduction register is cleared to zero at the beginning of each row under the action of the control logic. During row processing, each time a new partial product is received, a read-modify-write operation is performed, adding the current register value to the new partial product and writing it back to the register, thereby accumulating the sum of all products of the current row in the local register structure. When the address generation unit and the control logic of the load buffer queue jointly indicate that all pairs of data in a certain row have been completely consumed, the arithmetic logic unit reads the final accumulated value of the row from the row-level reduction register, writes it to the result first-in-first-out buffer queue through the result output channel, and at the same time triggers the row-level reduction register to be cleared to prepare for the next row operation.

[0029] According to one embodiment of the present invention, the streaming write module's streaming storage and write-back unit is used to write the row results output by the streaming computing module into a result buffer queue, generate a write-back address based on the output vector layout information given in the stream descriptor, and write the row results into the corresponding output vector position in the storage subsystem through on-chip interconnects and storage interfaces. The streaming write module is used to complete continuous write-back from the row-level results to the output vector storage area after the results are generated. The streaming write module includes a write-back address generation unit, a result buffer queue, and a vector data storage unit. The result buffer queue is a first-in-first-out (FIFO) result buffer queue. The first-in-first-out (FIFO) buffer queue is connected to the write-back address generation unit via its output port. When there is a row result to be written back in the queue, the write-back address generation unit generates the target address of the row result in the output vector based on the output vector base address and element step information provided by the descriptor register set, combined with the current row number. The write-back address generation unit then passes the row result and target address to the vector data storage unit for processing. The vector data storage unit constructs a write transaction to the external memory controller through on-chip interconnect and bus interface circuits, writing the row result to the corresponding position of the output vector array in the external memory. During the continuous write-back of multiple rows, the write-back address generation unit maintains a row number counter and a queue empty / full flag. On the one hand, it controls the write-back rhythm based on the valid results in the queue, and on the other hand, it adjusts the timing of initiating new write transactions based on the external bus handshake signal, so that the FIFO buffer queue plays the role of intermediate buffer and rhythm coordination on the write-back path.

[0030] According to one embodiment of the present invention, see again Figure 1 The streaming vector processing device may also include a status monitoring module. This module establishes a unified status observation and control interface between the functional modules within the scalar and vector processing cores and external software. The status monitoring module receives task identifiers and task status change information from the instruction dispatch unit via the control path. Simultaneously, it collects operational information such as the current line number, buffer queue full / empty status, number of memory access requests, operation progress status, and write-back completion status from the address generation unit, vector data loading unit, arithmetic logic unit, write-back address generation unit, and vector data storage unit. Internally, the status monitoring module establishes task status registers and statistics registers, classifying and organizing multi-source status information according to task identifiers and module identifiers. This information is exposed to the scalar processing core through reserved register address space, enabling the scalar processing core to learn about task initiation, progress, and completion status by reading the status registers, and to learn about the operational status and resource usage of each module by reading the statistics registers. In addition, the status monitoring module can also send an interrupt request or other notification signal to the scalar processing core when it detects a task completion event or an abnormal event, so that the upper-layer software can obtain the necessary task-level feedback information in a timely manner without having to actively poll all internal states.

[0031] According to one embodiment of the present invention, a streaming vector processing device includes a scalar processing core compatible with a general-purpose instruction set, a vector processing core (adopting a vector processing core architecture) connected to the scalar processing core via a coprocessor interface, and a streaming engine subcore tightly coupled within the vector processing core. The sparse matrix is ​​stored in external memory in a compressed row-oriented storage format. The array of non-zero elements, the column index array, and the row pointer array occupy different address ranges. The input vector and output vector also reside in a reserved area of ​​external memory as linear arrays. Each storage area is connected to the vector processing core via on-chip interconnects and peripheral bus interface circuits, providing a unified storage channel for subsequent data streaming loading and writing back.

[0032] On the software side, programmers first execute initialization code on the scalar processing core to register the storage layout of sparse matrices and vectors. For example, the software obtains the base addresses and lengths of the non-zero element array, the column index array, the row pointer array, and the input and output vectors from the compile-time or runtime environment, and encodes this information into a set of descriptor fields according to a predefined format. These fields are used to record stream descriptors. The descriptor fields are written to the descriptor register set exposed by the streaming engine sub-core via configuration instructions or memory mapping registers, or to the descriptor table periodically scanned by the streaming engine, thus forming a complete configuration state for a sparse matrix-vector multiplication task on the hardware side. The stream descriptor defines the mathematical model for memory access address generation, and its computational logic is expressed as follows:

[0033]

[0034]

[0035] in, This represents a set of variables that represent the generated memory access addresses. Indicates the base address. Represents the total dimension. Indicates the first Index variables of each dimension, This indicates the step size of the record in the step field. This represents a set of indexed variables as input. Represents the first index variable to Elements of each dimension express The starting index, express The span.

[0036] When the scalar processing core encounters a streaming instruction while executing the main program, the instruction dispatch logic on the scalar side identifies it as an operation that needs to be handed over to the vector processing core for execution. The instruction dispatch logic packages the instruction's opcode, target descriptor number, and task identifier, and sends it to the task entry port of the vector processing core via the coprocessing request channel. Simultaneously, the scalar processing core can choose to enter a polling loop, idle, or continue executing other instruction sequences in parallel while waiting for the task to complete, so that the task's completion status can be obtained later through a status register or interrupt mechanism.

[0037] After receiving a request from the scalar side, the instruction dispatch unit inside the vector processing core decodes the custom streaming instructions, parses out that the task is a sparse matrix and vector multiplication type, and reads the descriptor fields corresponding to the task identifier from the local descriptor register set. These fields include the starting address, number of elements, and single-element storage step of the array of row pointers, column indices, and non-zero elements, as well as the base address and length of the input and output vectors. In the direction of loading sparse matrix data, the address generation unit inside the vector processing core first initiates several read requests to the external memory controller through the on-chip interconnect and memory control interface circuitry, based on the address and length of the row pointer array in the descriptor, to read the row pointer array entirely or in blocks into the local buffer. The address generation unit calculates the number of non-zero elements in each row by comparing adjacent row pointer values ​​and internally establishes a row-level state record for each row for subsequent row-by-row traversal and data loading processes.

[0038] For each row, the address generation unit calculates the valid range to be accessed in the address space of the column index array and the non-zero element array based on the start and end offsets given by the row pointer. For example, the address generation unit can use the same element index counter for the column index array and the non-zero element array, continuously incrementing the current element index, and combining the current index with the array base address and the element byte width to form the access address to the external memory. The read request is sent to the external memory controller via the on-chip interconnect and peripheral bus interface module. The returned data is first aligned to the internal data bus width by the width adaptation circuit, then sent to the vector data loading unit for protocol conversion and alignment, and finally written into the first load first-in-first-out buffer queue, which is used to cache the non-zero matrix elements expanded row by row.

[0039] In the loading direction of the input vector data, the address generation unit performs address transformation for each column index value while parsing the column index array. For example, by combining the column index value with the byte width of the vector element and adding the base address of the input vector, the byte address of the corresponding input vector element in external memory can be obtained. For each column index value, the address generation unit initiates a read request again through the on-chip interconnect and memory control interface. The returned data is also processed by the width adaptation circuit and the vector data loading unit, aligned to the internal standard width, and then written to the second load FIFO buffer queue. This buffer queue is specifically used to cache vector elements that logically pair with the aforementioned non-zero matrix elements. The write pointers and read pointers of the two load FIFO buffer queues are coordinated by shared row-level control logic, ensuring that each pair of "non-zero matrix element values ​​and corresponding vector element values" maintains a temporal correspondence.

[0040] The internal implementation of the two loading first-in-first-out (FIFO) buffer queues can adopt a ring storage structure, with independent read / write pointers and empty / full flag registers. When the vector data loading unit writes data to the matrix data buffer queue or the vector data buffer queue, the write pointer increments by the element width, and the queue's internal logic generates a "not empty" flag. When the arithmetic logic unit is about to read data from the queue, if it detects that the queue is not empty, it retrieves an element from the position pointed to by the current read pointer and moves the read pointer forward. When either queue enters an "empty" state, the arithmetic logic unit suspends its read operation on that queue until the address generation unit and the loading unit replenish new data. In this way, the loading buffer queues internally coordinate the rhythm of data production and data consumption.

[0041] On the computation path, the arithmetic logic unit inside the vector processing core is connected to the matrix data loading buffer queue and the vector data loading buffer queue through two independent input ports respectively. When both buffer queues are detected to be non-empty, the arithmetic logic unit reads one element from each queue at the same time, and sends the value from the non-zero matrix data buffer queue and the value from the vector data buffer queue into the multiplication operation path to obtain an in-row partial product. The output of the multiplication operation is accumulated through the accumulation path composed of the adder and the row-level reduction register, and the product of all non-zero elements in the current row and the corresponding vector elements is continuously accumulated in the row-level reduction register.

[0042] The row-level reduction register is cleared by dedicated control logic before each row begins processing. This control logic determines the start event of the current row based on the row pointer state and the row number counter. During the processing of the entire row, whenever the multiply-accumulate unit generates a new partial product, the control logic triggers a read-modify-write operation of the row-level reduction register: first, the current register value is read out and added to the new product, and then the result is written back to the register. This process remains transparent to external memory, and all intermediate accumulation results only circulate in the local register structure and are not output to the outside on the data path. By setting row start flag, row middle flag, and row end flag outside the row-level reduction register, the arithmetic logic unit can identify which stage of row processing is currently in, thereby starting or ending the accumulation process at the appropriate time.

[0043] When the address generation unit determines, based on the start and end offsets of the current row, that all non-zero elements of the row have been requested and sent to the arithmetic logic unit via the load buffer queue, and simultaneously the control logic of the two load buffer queues detects that all paired elements of the current row have been completely consumed, the address generation unit sends a row end signal to the arithmetic logic unit. Upon receiving this signal, the arithmetic logic unit reads the current accumulated value from the row-level reduction register and writes it as the complete result (row result) of the row's output vector elements into the result output channel. For example, this result output channel is connected to a dedicated result first-in-first-out buffer queue, which can simultaneously store multiple row results to support relatively independent progress between the write-back path and the computation path.

[0044] On the write-back path, the result first-in-first-out buffer queue is connected to the write-back address generation unit and the vector data storage unit through its output terminal. The write-back address generation unit maintains a write-back index counter related to the row number. Based on the base address of the output vector in the descriptor and the single-element step information, it calculates the corresponding storage address for each row of results. When the result buffer queue is detected to be non-empty, the write-back address generation unit reads a row of results from the queue, combines it with the address corresponding to the current row number, and encapsulates it into a write request to the external memory controller via the vector data storage unit. The vector data storage unit is responsible for converting the result value and write-back address on the internal data bus into a write transaction that conforms to the on-chip interconnect and peripheral bus interface protocol, and sending it to the external memory controller so that the row of results is written to the corresponding position of the output vector array in the external memory.

[0045] During multi-line continuous processing, the row number counter of the write-back address generation unit increments by one after each row is written back, keeping consistent with the current row number maintained by the front-end address generation unit. During row result write-back, the result first-in-first-out buffer queue may be temporarily "empty" or "full". The write-back address generation unit monitors the queue empty / full flag and the bus interface handshake signal to decide whether to continue reading new results or postpone the initiation of new write transactions. When there is a difference between the speed of the results generated by the front-end arithmetic logic unit and the processing capacity of the back-end write-back path, the result buffer queue in this embodiment plays the role of intermediate buffer and rhythm balancing, so that the result data has a certain turnover space during the transmission from the arithmetic unit to the storage system.

[0046] During the entire task execution, control information is transmitted through independent control paths between the instruction dispatch unit, descriptor register group, address generation unit, arithmetic logic unit, and write-back unit within the scalar processing core and vector processing core. The scalar processing core can learn about the stage of task execution by reading the status register or waiting for interrupt signals, such as task start, start of processing a certain row, end of processing a certain row, and completion of processing all rows. The sub-modules within the vector processing core work together through task identifiers, row-level identifiers, and internal handshake signals. The numerical data strictly circulates along the path of "external memory — on-chip interconnect and peripheral bus interface circuit — vector data loading unit — loading first-in-first-out buffer queue — arithmetic logic unit and row-level reduction register — result first-in-first-out buffer queue — vector data storage unit — on-chip interconnect and peripheral bus interface circuit — external memory". In this specific embodiment, a complete data transmission method and module connection relationship from sparse matrix and input vector to output vector are formed.

[0047] According to one embodiment of the present invention, see Figure 2This paper provides a memory access control method based on stream descriptors, implemented in the streaming vector processing device of the aforementioned embodiment. The method includes: Step S1: Determining the layout information of each data structure in the target operation task in the storage subsystem on the scalar processing core, encoding the base address, length, element data width, and step information of the compressed storage area of ​​the sparse matrix, the storage area of ​​the input vector, and the storage area of ​​the output vector into at least one stream descriptor, and writing it into the descriptor management module through configuration instructions; Step S2: The scalar processing core issues an inner product class streaming processing instruction corresponding to the stream descriptor when executing the main program. The streaming processing instruction carries the descriptor number and task identifier, which is identified and received by the flow control module inside the vector processing core; Step S3: The flow control module reads the corresponding stream descriptor field from the descriptor management module according to the descriptor number, and distributes the task identifiers of the matrix data stream, vector data stream, and result write-back stream to the streaming loading module, the streaming computation module, and the streaming storage and write module, respectively; Step S4: The streaming loading module... The loading module automatically generates memory access addresses pointing to the sparse matrix compressed storage area based on the layout information of the row pointer array, index array, and non-zero element array given in the stream descriptor. It loads non-zero matrix elements row by row from the storage subsystem, and simultaneously generates memory access addresses pointing to input vector elements based on the index information, forming two aligned loading data streams. Step S5: The streaming computing module reads non-zero matrix elements and corresponding vector elements in pairs from the two loading data streams corresponding to the first and second loading first-in-first-out buffer queues. It performs inner product operations on each pair of non-zero matrix elements and corresponding vector elements and performs row-level accumulation in the row-level reduction register. When it detects that the non-zero elements corresponding to a certain row have been processed, it outputs the row result of the inner product operation to the result buffer queue. Step S6: The streaming write module maps each row result in the result buffer queue to the corresponding storage location of the output vector according to the row number based on the layout information of the output vector given in the stream descriptor, and writes it back to the storage subsystem through the storage interface, completing the overall processing of a large loop memory access task.

[0048] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A streaming vector processing device, characterized in that, include: A scalar processing kernel is used to execute general scalar instructions, vector extension instructions, inner product-type streaming instructions corresponding to at least one sparse compressed format data, and stream descriptors corresponding to inner product tasks that configure sparse compressed format data. The vector processing kernel is used to respond to inner product-type streaming processing instructions, triggering the execution of the inner product task corresponding to the stream descriptor. The internal hardware of the vector processing kernel performs streaming loading, streaming operation, streaming reduction, and streaming storage of inner product-related data according to the guidance of the stream descriptor.

2. The apparatus according to claim 1, characterized in that, The vector processing kernel includes the streaming engine sub-kernel: The descriptor management module is used to store at least one stream descriptor configured by the scalar processing core. Each stream descriptor is used to describe the compressed storage area of ​​the sparse matrix, the storage area of ​​the input vector, and the storage area of ​​the output vector. The flow control module is used to select the target flow descriptor from the descriptor management unit according to the flow descriptor number carried by the scalar processing core when it issues an inner product-type streaming processing instruction, and to distribute the sparse matrix data stream, vector data stream and result write-back stream task identifiers to the downstream modules. The streaming loading module, under the control of the flow control module, generates memory access addresses pointing to the row pointer array, index array, and non-zero element array based on the metadata location of the compressed sparse row format or other sparse compressed format given in the stream descriptor. It requests to read the corresponding metadata and non-zero data line by line from the storage subsystem, and generates memory access addresses pointing to the input vector elements based on the index information. It reads the vector element corresponding to each non-zero element from the storage subsystem, writes the obtained non-zero matrix element data into the first load first-in-first-out buffer queue, and writes the obtained vector element data into the second load first-in-first-out buffer queue. The streaming computing module is used to read non-zero matrix elements and corresponding vector elements in pairs from the first loading first-in-first-out buffer queue and the second loading first-in-first-out buffer queue, perform multiplication operation on each pair of data, and accumulate the multiplication into the row-level reduction register. When all non-zero elements corresponding to the current row have been processed, the accumulated value is read from the row-level reduction register to form the row result of the output vector element of that row. The streaming write module is used to write the row results output by the streaming computing module into the result buffer queue, generate a write-back address according to the output vector layout information given in the stream descriptor, and write the row results into the corresponding output vector position in the storage subsystem through the on-chip interconnect and storage interface. When the inner product-type streaming processing instruction is executed, the streaming control module will schedule the streaming loading module, streaming computation module, and streaming write module to work together to complete the inner product calculation between the sparse matrix rows in sparse compressed format and the input vector and write back the result inside the hardware.

3. The apparatus according to claim 2, characterized in that, The stream descriptor includes at least: The base address field and length field of the array of non-zero elements in a sparse matrix; The base address field and length field of the sparse matrix index array; The base address field and length field of the row pointer array; The base address field and length field of the input vector; The base address field and length field of the output vector; and Control fields used to identify sparse format type, element data width, alignment, and step mode; When generating memory access addresses, the streaming loading module only uses each field in the stream descriptor to implement a complete large loop memory access task line by line, without requiring the scalar processing core to explicitly calculate the address inside the line loop.

4. The apparatus according to claim 2 or 3, characterized in that, The vector processing core also includes a conventional vector execution subcore, which contains logic for performing vector loading, vector storage, vector arithmetic operations, and vector reduction operations. It can complete data transfer between the vector register and the storage subsystem through explicit vector loading and vector storage instructions, so as to support vector extension instructions based on the instruction set standard through the conventional vector execution subcore.

5. The apparatus according to claim 2 or 3, characterized in that, The streaming computing module sets up a row-level reduction register, which is cleared to zero at the beginning of each row under the action of control logic. During the processing of that row, only the local multiply-accumulate path performs read, modify and write operations. When the streaming loading module issues the end-of-current-line processing flag, the streaming computing module reads the accumulated value in the row-level reduction register, writes it into the result buffer queue, and triggers the row-level reduction register to be cleared again before starting to process the next line.

6. The apparatus according to claim 2 or 3, characterized in that, Both the first and second loading first-in-first-out buffer queues adopt a circular storage structure and are equipped with independent read pointers, write pointers, and storage status registers; The streaming computing module initiates a paired read operation only when both queues' storage status registers indicate that valid data exists in the queues each time it reads data from the two buffer queues. Reading is paused when either queue is empty, in order to ensure that the non-zero matrix element data and the corresponding vector element data maintain a one-to-one correspondence in the time domain.

7. A memory access control method based on stream descriptors, implemented in a streaming vector processing apparatus as described in any one of claims 1-6, the method comprising: Step S1: Determine the layout information of each data structure in the storage subsystem of the target operation task on the scalar processing core, and encode the base address, length, element data width and step information of the compressed storage area of ​​the sparse matrix, the storage area of ​​the input vector and the storage area of ​​the output vector into at least one stream descriptor, and write it into the descriptor management module through configuration instructions. Step S2: When the scalar processing core executes the main program, it issues an inner product-type streaming instruction corresponding to the stream descriptor. This streaming instruction carries the descriptor number and task identifier, which is identified and received by the flow control module inside the vector processing core. Step S3: The flow control module reads the corresponding flow descriptor field from the descriptor management module according to the descriptor number, and distributes the task identifiers of the matrix data stream, vector data stream and result write-back stream to the streaming loading module, streaming computing module and streaming storage and writing module respectively; Step S4: The streaming loading module automatically generates memory access addresses pointing to the sparse matrix compressed storage area based on the layout information of the row pointer array, index array and non-zero element array given in the stream descriptor. It loads non-zero matrix elements row by row from the storage subsystem, and at the same time generates memory access addresses pointing to the input vector elements based on the index information, forming two aligned loading data streams. Step S5: The streaming computing module reads non-zero matrix elements and corresponding vector elements in pairs from the two loading data streams corresponding to the first and second loading first-in-first-out buffer queues. It performs inner product operations on each pair of non-zero matrix elements and corresponding vector elements and completes the row-level accumulation in the row-level reduction register. When it is detected that the non-zero elements corresponding to a certain row have been processed, the row result of the inner product operation of that row is output to the result buffer queue. Step S6: The streaming write module maps each row of results in the result buffer queue to the corresponding storage location of the output vector according to the row number based on the layout information of the output vector given in the stream descriptor, and writes it back to the storage subsystem through the storage interface to complete the overall processing of a large loop memory access task.

8. The method according to claim 7, characterized in that, The process of generating memory access addresses for streaming modules includes: Calculate the start and end positions of the non-zero element in the current row in the index array and the non-zero element array based on two adjacent pointer values ​​in the row pointer array; For each non-zero element in the current row, calculate the element address of the input vector based on its column index in the index array, the base address of the input vector, and the element's step information; Read requests for the non-zero element array and the input vector array are initiated sequentially from the start position to the end position. The returned non-zero matrix element data and the corresponding input vector element data are written into the first and second loading first-in-first-out buffer queues, respectively.

9. The method according to claim 7, characterized in that, The streaming computation module confirms the end of processing for the current row when all of the following conditions are met: First condition: The streaming loading module detects that all read requests have been initiated and returned for the index and non-zero data corresponding to the current row, and indicates to the streaming computing module through control signals that the last data pair of the current row has been dequeued; Second condition: The read / write control logic of the first and second loading first-in-first-out buffer queues detects that the data pair corresponding to the current row has been completely dequeued and there is no longer any valid data for that row in the buffer.

10. The method according to claim 7, characterized in that, After issuing streaming processing instructions, the scalar processing core obtains the completion status of the streaming processing task by polling the task status register in the vector processing core or by receiving interrupt signals. When the flow control module inside the vector processing core detects that the streaming loading module, streaming computing module, and streaming write module have all reported that all data rows corresponding to the current task have been processed, it updates the task status register to the completion status and sends an interrupt request to the scalar processing core or modifies the task status register so that the scalar processing core can read the task completion flag indicating the completion status of the corresponding streaming processing task in the next polling.

Images