Serialization memory access and calculation collaboration method, system and equipment and medium

Abstract

The invention provides a serialized memory access and computing collaboration method, system, equipment and medium, which parses the same external command in parallel through two command parsers with the same structure to collaboratively generate address information for memory access and an operation instruction for computing control, and the address information and the operation instruction are complementary in function and synchronous in time sequence. Compared with a traditional scheme depending on an OST buffer and matched control logic thereof, the architecture does not need to additionally cache control information or an intermediate state, so that the OST buffer and related read-write logic are effectively omitted, and the chip area overhead of the AI accelerator is remarkably reduced; meanwhile, dynamic read-write power consumption and static leakage power consumption are eliminated, and the energy efficiency ratio of the system is remarkably increased. Besides, the triggering of the memory access operation does not depend on the capacity of a buffer area, so that the number of memory accesses supported by a single task is not limited by the scale of the OST any more, and the flexibility and the parallel capability of task processing are greatly enhanced.

Description

Technical Field

[0001] This invention relates to the field of AI accelerator technology, and in particular to a method, system, device and medium for serialized memory access and computational collaboration. Background Technology

[0002] Current AI accelerators, when accessing memory, need to record relevant information returned by the data, such as usage method, data type, path status, and output address, when issuing a request. This information is stored in on-chip SRAM, and the size of the SRAM limits the number of requests that can be issued, thus affecting memory access capabilities. Because of the large amount of information recorded and the need for large Outstanding Transactions (OTS), the SRAM depth and area increase, negatively impacting power consumption and area, ultimately reducing the energy efficiency and area efficiency of the AI ​​accelerator. Therefore, an effective method for coordinating serialized memory access and computation is urgently needed to solve these problems. Summary of the Invention

[0003] In view of the above problems, the present invention is proposed to provide a serialized memory access and computation coordination method, system, device and medium that overcomes or at least partially solves the above problems.

[0004] To achieve the above and other related objectives, this invention provides a serialized memory access and computation coordination method, and an AI accelerator communicatively connected to main memory. The AI ​​accelerator includes a request decoder, a response decoder, and an arithmetic logic unit. The method includes: The system receives external commands from the master controller, parses the external commands through a request decoder, and generates an address request signal for accessing the main memory; at the same time, it parses the external commands through a response decoder and generates an operation control signal for controlling the arithmetic logic unit to perform specific operations. Based on the preset serialization access rules, the target data is read sequentially from the main storage according to the address request signal to complete the serialized data reading operation; Based on the collaborative control logic between memory access and computation, after the target data is ready, the arithmetic logic unit is driven to perform corresponding logical operations on the target data according to the operation control signal to generate the calculation result; According to the preset serialization access rules, the calculation results are written back to the main storage in sequence, completing the serialized data write-back operation.

[0005] Optionally, the step of parsing the external command through a request decoder to generate an address request signal for accessing main storage includes: The request decoder parses the external command to extract the base address register encoding, immediate offset, and read / write attributes contained therein; According to the address width standard specified by the main memory, the immediate value offset is subjected to sign extension or zero extension processing, and the extended immediate value is added to the register value pointed to by the base address register encoding to generate the target physical address; The target physical address and read / write attributes are encapsulated into an address request signal and sent to the main storage to trigger the corresponding data read operation.

[0006] Optionally, the step of parsing the external command through a response decoder to generate an operation control signal for controlling the arithmetic logic unit to perform a specific operation includes: The external command is parsed by the response decoder to extract the combination of opcodes and function fields contained therein in order to determine the target operation type; According to the target operation type, the opcode is decoded into the corresponding ALU operation signal, source operand selection signal and destination write enable signal; The ALU operation signal, the source operand selection signal, and the destination write enable signal are integrated, formatted, and encapsulated into an operation control signal, which is then sent to the data path to drive the arithmetic logic unit to perform the specified operation.

[0007] Optionally, the step of sequentially reading target data from main storage according to the address request signal based on preset serialization access rules to complete the serialized data reading operation includes: Based on the address request signal, the target physical address, read / write attributes and valid enable bits contained therein are parsed, and combined with the preset serialization access rules, data dependency analysis and access priority sorting are performed on multiple target physical addresses involved in the current task to generate an ordered address access sequence with a strict sequential execution order. According to the ordered address access sequence, data read requests are initiated to the main storage in sequence; wherein, the next address read request is triggered only when the previous address read request has received a valid response from the main storage, or when the target data cached in the current data path has been completely consumed by the arithmetic logic unit, so as to avoid data path congestion. Receive each target data returned by the main storage and cache and align it according to the execution order of the ordered address access sequence to ensure that the data receiving order is completely consistent with the request initiation order; Once all data at all target physical addresses has been read and verified to be correct, a data read ready signal is output to indicate that the serialized data read operation is complete.

[0008] Optionally, the collaborative control logic based on memory access and computation, after the target data is ready, drives the arithmetic logic unit to perform corresponding logical operations on the target data according to the operation control signal to generate a calculation result, including: Once the target data is detected to be loaded and the operation control signal is successfully decoded, the collaborative operation process of memory access and computation is triggered, and the operation execution of the arithmetic logic unit is started. Based on the source operand selection signal obtained by decoding the operation control signal, the target data corresponding to the current operation is selected from the input data path of the arithmetic logic unit and used as the source operand of the arithmetic logic unit. Based on the ALU operation mode signal decoded from the operation control signal, the operation mode of the arithmetic logic unit is dynamically configured, and it is driven to perform specified operations on the loaded source operands to generate calculation results. Based on the target write enable signal obtained by decoding the operation control signal, the calculation result output by the arithmetic logic unit is written into the temporary result cache unit configured inside it. The hardware logic automatically associates and records the result sequence identifier corresponding to the current operation step during the writing process to ensure the sequential consistency of the data stream in the subsequent serialization write-back process.

[0009] Optionally, the step of writing the calculation results back to the main storage sequentially according to the preset serialization access rules to complete the serialized data write-back operation includes: When it is detected that the calculation result stored in the temporary result cache unit of the arithmetic logic unit has completed the operation exception verification, and the main storage interface is in an idle and writable state, the serialization write-back process is triggered; at the same time, the legality verification of the preset serialization access rules is performed. After the verification is passed, the calculation results stored in the temporary result cache unit are format converted and aligned according to the data storage format requirements of the main storage, and the processed calculation results that conform to the main storage interface protocol are generated. Based on the write address sequence generated by the preset serialization access rules, the processed calculation results are written back to the target physical address corresponding to the main storage in sequence, thus completing the serialized data write-back operation.

[0010] Optionally, after the step of writing the calculation results back to the main storage sequentially according to the preset serialization access rules to complete the serialized data write-back operation, the method further includes: After confirming that all calculation results to be written back have been written to the corresponding target physical address in the main storage in order according to the write-back address, a task completion signal is generated and fed back to the main controller, and an integrity check is performed on the data written back to the main storage. If the integrity check passes, the temporary resources occupied by the current task are released to restore the context state of the AI ​​accelerator. According to the preset scheduling strategy, the external commands or data of the next task are preloaded into the designated cache unit and put into standby state to support the efficient pipelined processing of continuous tasks.

[0011] Secondly, the present invention also provides a serialized memory access and computation co-processing system applied to an AI accelerator communicatively connected to main memory. The AI ​​accelerator includes a request decoder, a response decoder, and an arithmetic logic unit. The system includes: The parsing module is used to receive external commands issued by the main controller, parse the external commands through the request decoder, and generate an address request signal for accessing the main memory; at the same time, it parses the external commands through the response decoder and generates an operation control signal for controlling the arithmetic logic unit to perform specific operations. The memory access module is used to read target data sequentially from the main memory according to the address request signal based on the preset serialization access rules, and complete the serialized data reading operation. The arithmetic module is used to drive the arithmetic logic unit to perform corresponding logical operations on the target data according to the arithmetic control signal after the target data is ready, based on the coordinated control logic between memory access and computation, and generate the calculation result. The write-back module is used to write the calculation results back to the main storage sequentially according to the preset serialization access rules, thereby completing the serialized data write-back operation.

[0012] Thirdly, the present invention provides an electronic device comprising: a memory and a processor; the memory for storing a computer program; and the processor for executing the computer program stored in the memory to cause the electronic device to perform the steps of the serialized memory access and computation co-processing method as described above.

[0013] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an electronic device, implements the steps of the serialized memory access and computational co-processing method as described above.

[0014] The above-described one or more technical solutions provided by this invention can have the following advantages or at least achieve the following technical effects: This invention uses two identical command parsers to parse the same external command in parallel, collaboratively generating address information for memory access and operation instructions for computational control. These two functions are complementary and time-synchronized. Compared to traditional solutions that rely on OST buffers and their associated control logic, this architecture eliminates the need for additional cached control information or intermediate states, effectively saving the OST buffer and related read / write logic, significantly reducing the chip area overhead of the AI ​​accelerator; it also eliminates dynamic read / write power consumption and static leakage power consumption, significantly improving system energy efficiency. Furthermore, memory access operations are not triggered by the buffer capacity, meaning the number of memory accesses supported by a single task is no longer limited by the OST size, greatly enhancing the flexibility and parallelism of task processing. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the internal workflow of a conventional AI accelerator in one embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of the internal workflow of an AI accelerator without an operation buffer in one embodiment of the present invention.

[0017] Figure 3 The diagram shows a flowchart of a serialization memory access and computation coordination method in one embodiment of the present invention.

[0018] Figure 4 The diagram shows a functional module schematic of a serialized memory access and computation co-processing system in one embodiment of the present invention.

[0019] Figure 5 The diagram shown is a schematic representation of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0020] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0021] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0022] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0023] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0024] Unless otherwise stated, the term "multiple" means two or more.

[0025] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0026] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0027] The technical solutions of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] Please see Figure 1 This diagram illustrates the internal workflow of a traditional AI accelerator. Its core steps are as follows: the external command (cmd) first enters the command decoder (cmd decode), which generates request information (req info); subsequently, the request information (req info) enters the operation buffer (ots buffer), where it is buffered and processed to generate operation information; then, the operation information, along with the externally input read data (rd data), enters the arithmetic logic unit (ALU) for computation, ultimately outputting write data (wr data). Furthermore, the diagram shows that the command decoder outputs a request signal (req), and the entire process is completed within the AI ​​accelerator. This approach is limited by the operation buffer (ots buffer), mainly in the following aspects: (1) The capacity and read / write speed of the OTS buffer directly affect the throughput of the data stream. If the buffer is too small or the processing speed is slow, the ALU will wait for data, thereby reducing the overall computing efficiency.

[0029] (2) The processing of the OTS buffer introduces additional latency, especially in high-concurrency or large-data-volume scenarios, which may become a bottleneck for the response time of AI accelerators.

[0030] (3) To ensure performance, it may be necessary to increase the capacity of the OTS buffer, which will consume more hardware resources (such as memory or registers) and increase the chip area overhead and power consumption of the AI ​​accelerator.

[0031] Please see Figure 2 The core features and advantages of the internal workflow of the AI ​​accelerator without an operation buffer in this application are as follows: Once an external command (cmd) is received, it is distributed to two functionally independent but structurally identical decoding units. The request decoder (req cmd decode) is responsible for parsing and generating memory access request information (req), indicating the storage address of the required data and driving external storage access. The response decoder (resp cmd decode) is responsible for parsing and generating response processing information, specifying the processing method (such as addition, multiplication, shift, etc.) of the arithmetic logic unit (ALU) for the data to be read.

[0032] Data read from main memory (rd data) is directly input to the ALU. After receiving the operation control signal generated by resp cmd decode, the ALU performs real-time calculations on the rd data and finally outputs the calculation results through the write data bus (wrdata).

[0033] The AI ​​accelerator proposed in this application achieves parallel decoding and generation of memory access addresses and data processing methods through the collaborative operation of dual decoders. While sacrificing some data buffer flexibility, this architecture significantly optimizes energy efficiency and area cost, making it an ideal solution for power- and area-sensitive scenarios with strong memory access regularity, such as edge inference.

[0034] Please see Figure 3 An embodiment of the present invention provides a serialization memory access and computation coordination method, applied to an AI accelerator communicatively connected to main memory. The AI ​​accelerator includes a request decoder, a response decoder, and an arithmetic logic unit. The method may include the following steps S10-S40: Step S10: Receive an external command issued by the master controller, parse the external command through a request decoder, and generate an address request signal for accessing the main memory; at the same time, parse the external command through a response decoder and generate an operation control signal for controlling the arithmetic logic unit to perform specific operations.

[0035] External commands are instructions sent by the master controller to the AI ​​accelerator to instruct it to perform memory access (read / write) or computation operations.

[0036] The address request signal is used to represent the address request generated after the request decoder parses the external command. It covers core address information such as the parameter address of the computing task and the read / write address of the memory access operation. It is the request identifier for address interaction between the AI ​​accelerator and the main controller / main storage, and provides the basis for address location and permissions of subsequent memory access operations.

[0037] Operation control signals are used to represent the computation operation control instructions generated by the response decoder after parsing external commands. Specifically, they can include operation type selection instructions (such as addition, multiplication, activation function calculation, etc.) and data processing full-process control instructions (such as data loading, operation execution, result temporary storage, etc.). They are the core control basis for driving the arithmetic logic unit (ALU) to complete the specified operation and related data processing actions.

[0038] In its implementation, after receiving external commands from the master controller, the AI ​​accelerator can parse these commands using a request decoder to generate an address request signal for accessing main memory. Simultaneously, it can parse the external commands using a response decoder to generate operation control signals for controlling the arithmetic logic unit (ALU) to perform specific operations. This process is essentially a mapping relationship of "instruction decoding → control signal generation," which can be implemented using hardware connection logic or microprogram control. The parallel operation of the response decoder and request decoder achieves decoupling between computation and memory access, effectively improving pipeline processing efficiency.

[0039] Step S20: Based on the preset serialization access rules, the target data is read sequentially from the main storage according to the address request signal to complete the serialized data reading operation.

[0040] Among them, the preset serialization access rules are used to represent the rules that the AI ​​accelerator predefines before interacting with the main storage (read / write back) to ensure the consistency of data access timing and sequence. The core purpose is to avoid data conflicts across tasks / computation steps and ensure that the data flow strictly matches the original operation / memory access request sequence.

[0041] Target data represents the raw computation data stored in main memory that needs to be located via address request signals. Its core components include: parameter data for the computation task (such as weight matrices and bias vectors in neural networks), input feature maps during the computation process, intermediate activation values, and pooling results (such as the Q / K / V matrix in a Transformer). Essentially, it is the core input data that drives the Arithmetic Logic Unit (ALU) to complete preset operations (such as addition, multiplication, and activation function calculations). Its reading order must strictly follow serialization access rules to ensure matching with the ALU's computation timing.

[0042] In practical implementation, the target data required by the Arithmetic Logic Unit (ALU) can be read sequentially from the main memory according to the preset serialization access rules and the address request signal generated by the request decoder, thus completing the serialized data read operation. During the read process, the memory access sequence corresponding to the address request signal is strictly followed to ensure that the reading sequence of the target data matches the execution sequence of the ALU operation logic, providing a data flow guarantee for the orderly execution of subsequent operations.

[0043] Step S30: Based on the collaborative control logic between memory access and computation, after the target data is ready, the arithmetic logic unit is driven to perform corresponding logical operations on the target data according to the operation control signal to generate the calculation result.

[0044] The collaborative control logic between memory access and computation represents the hardware scheduling and timing control mechanism in the AI ​​accelerator that connects the two core components of "data memory access" and "computation execution". Its core objective is to ensure the timeliness of data memory access, the continuity of computation execution, and the timing matching between the two through unified rule parsing, status monitoring, and instruction distribution. Ultimately, it achieves efficient pipeline collaboration of "computing as soon as data is ready and storing data back as soon as computation is completed", avoiding idle computation due to unread data or result blocking due to uneven storage workload.

[0045] The calculation result represents the core data output by the Arithmetic Logic Unit (ALU) after performing operations (such as addition, multiplication, matrix multiplication and addition, activation function calculation, etc.) on the target data as input, according to the operation type specified by the operation control signal. It can include matrix multiplication and addition results from neural network operations, activation function output values, feature map convolution results, Transformer attention calculation results, etc., and is the core output of the AI ​​accelerator in completing the preset computational task. This calculation result is first stored in a temporary result cache unit within the ALU for computational anomaly verification. After successful verification, it is written back to main storage according to preset serialization access rules, ultimately serving as input data for subsequent operations or the final task result.

[0046] In practical implementation, the readiness status of target data can be monitored in real time based on the collaborative control logic between memory access and computation. Once it is confirmed that all target data is ready, the arithmetic logic unit (ALU) is driven by the operation control signal to perform corresponding logical operations (such as addition, subtraction, matrix multiplication, activation function calculation, etc.) on the target data. During the operation, data overflow detection, carry / misalignment logic processing, and preliminary judgment of operation anomalies are completed simultaneously. Finally, the corresponding calculation result is generated and stored in the temporary result cache unit configured by the ALU. This achieves efficient collaboration of the "computation-equation" pipeline and significantly reduces the impact of storage.

[0047] Step S40: According to the preset serialization access rules, the calculation results are written back to the main storage in sequence to complete the serialized data write-back operation.

[0048] In the specific implementation, after the arithmetic logic unit (ALU) completes the operation and stores the calculation result in its configured temporary result cache unit, the calculation result stored in the temporary result cache unit can be written back to the main storage in an orderly manner according to the preset serialization access rules, thus completing the serialized data write-back operation; thereby ensuring the consistency of the data write-back order and the correctness of the calculation result.

[0049] Furthermore, in one embodiment, step S10 may include the following sub-steps A10~A30: Sub-step A10 involves the request decoder parsing the external command to extract the base address register encoding, immediate offset, and read / write attributes contained therein. Sub-step A20: According to the address width standard specified by the main memory, perform sign extension or zero extension on the immediate value offset, and add the extended immediate value to the register value pointed to by the base address register encoding to generate the target physical address; Sub-step A30: Encapsulate the target physical address along with the read / write attributes into an address request signal and send it to the main storage to trigger the corresponding data read operation.

[0050] In practical implementation, the AI ​​accelerator's built-in request decoder can parse external commands, extracting the base address register code (identifying a specific register in the AI ​​accelerator's internal general-purpose register set), immediate offset (an offset directly embedded in the external command relative to the address pointed to by the base address register), and read / write attributes (identifiers specifying data operation type as read / write). Then, according to the address width standard specified by main memory, sign extension or zero extension is performed on the immediate offset, and the extended immediate value is added to the register value pointed to by the base address register code to generate the target physical address. Subsequently, the target physical address and read / write attributes can be encapsulated into an address request signal (i.e., Figure 2 The `req` signal is sent to the main storage to trigger the corresponding data read operation; after the main storage completes the read, the target data will be transmitted through the data path (i.e., the `req` signal). Figure 2 The rd data signal is transmitted to the arithmetic logic unit (ALU) of the AI ​​accelerator. This enables direct connection between memory access and computation, significantly reducing data transfer latency and bandwidth bottlenecks, and is highly adaptable to the core requirements of AI-intensive computing tasks for high throughput and low latency response.

[0051] Furthermore, in one embodiment, step S10 may further include the following sub-steps B10 to B30: Sub-step B10 involves the response decoder parsing the external command and extracting the combination of opcodes and function fields contained therein to determine the target operation type; Sub-step B20: According to the target operation type, decode the opcode into the corresponding ALU operation signal, source operand selection signal and destination write enable signal; Sub-step B30 integrates and formats the ALU operation signal, the source operand selection signal, and the destination write enable signal into an operation control signal, and sends it to the data path to drive the arithmetic logic unit to perform the specified operation.

[0052] In practical implementation, the AI ​​accelerator's built-in response decoder parses external commands, extracting the opcode (including ALU operation signals, source operand selection signals, and destination write enable signals) and function field combinations to determine the target operation type. Based on this, the opcode is decoded into corresponding ALU operation signals (such as addition, multiplication, and shift instructions), source operand selection signals (controlling the multiplexer and selecting the ALU's input data source, such as a base address register, immediate value, or main memory read), and destination write enable signals (controlling the write-back target of the operation result, such as a register file or main memory address). Subsequently, the ALU operation signals, source operand selection signals, and destination write enable signals are integrated, formatted, and encapsulated into operation control signals, which are then sent to the ALU and its associated data paths to drive the ALU to execute the specified operation. This allows data paths (such as multiplexers and buses) to accurately select and transmit target data based on the operation control signals, effectively avoiding path congestion caused by data contention and data overwriting or misreading due to address errors.

[0053] In this embodiment, two identical command parsers perform parallel parsing of the same external command, collaboratively generating address information for memory access and operation instructions for computational control. These two functions are complementary and synchronized in timing. Compared to traditional solutions relying on OST buffers and their associated control logic, this architecture eliminates the need for additional caching of control information or intermediate states, effectively saving the OST buffer and related read / write logic, significantly reducing the chip area overhead of the AI ​​accelerator; simultaneously, it eliminates dynamic read / write power consumption and static leakage power consumption, significantly improving the system's energy efficiency. Furthermore, the triggering of memory access operations is independent of buffer capacity, meaning the number of memory accesses supported by a single task is no longer limited by the OST size, greatly enhancing the flexibility and parallelism of task processing.

[0054] Based on the foregoing embodiments, a second embodiment of the serialized memory access and computation coordination method of the present invention is proposed. In this embodiment, step S20 may include the following sub-steps S201~S204: Sub-step S201: Based on the address request signal, parse the target physical address, read / write attributes and valid enable bit contained therein, and combine it with the preset serialization access rules to perform data dependency analysis and access priority sorting on multiple target physical addresses involved in the current task, generating an ordered address access sequence with a strict sequential execution order.

[0055] Among them, the ordered address access sequence is used to represent the strictly time-sequential address sequence generated after the AI ​​accelerator summarizes the target physical address based on serialization rules and task data characteristics, and performs priority, data dependency and access conflict analysis. It is used to ensure memory access order, data consistency and arithmetic logic unit (ALU) input determinism, and is the core mechanism for achieving deterministic memory access scheduling.

[0056] In practical implementation, the address request signal can be parsed to obtain the target physical address, read / write attributes, and valid enable bits contained within it. Then, combined with preset serialization access rules (such as address increment / decrement order, data batch number order, and instruction priority order), data dependency analysis and access priority sorting are performed on multiple target physical addresses designed for the current task, generating an ordered address access sequence with a strict sequential order. Therefore, performing serialized memory access operations based on this ordered address access sequence can effectively eliminate data races caused by out-of-order access, avoid pipeline blockage, and reduce additional cache consistency overhead, significantly improving the determinism and energy efficiency of AI accelerators when performing critical tasks such as matrix operations, sparse computation, and attention mechanisms.

[0057] Sub-step S202: Based on the ordered address access sequence, data read requests are sequentially initiated to the main storage; wherein, the next address read request is only triggered when the previous address read request has received a valid response from the main storage, or when the target data cached in the current data path has been completely consumed by the arithmetic logic unit, in order to avoid data path congestion.

[0058] In a specific implementation, data read requests can be initiated to the main storage sequentially according to the aforementioned ordered address access sequence to obtain the corresponding target data. The next address read request is only triggered when the previous address read request has received a valid response from the main storage, or when the target data cached in the current data path has been completely consumed by the arithmetic logic unit (ALU). Thus, through this request triggering mechanism, data congestion and resource contention in the data path can be effectively avoided, ensuring the determinism and stability of memory access operations.

[0059] Sub-step S203: Receive each target data returned by the main storage, and cache and align it according to the execution order of the ordered address access sequence to ensure that the data receiving order is completely consistent with the request initiation order.

[0060] In its implementation, the system can receive the target data returned by the main storage for each read request. Then, following the execution order of the ordered address access sequence, the received target data is cached and aligned to ensure that the data reception order strictly matches the order in which the read requests are initiated. This mechanism supports deterministic processing in subsequent pipelines, effectively guaranteeing the sequential integrity of serialized memory access operations and the predictability of the data flow.

[0061] In sub-step S204, after all data at the target physical addresses has been read and verified to be correct, a data reading ready signal is output to indicate that the serialized data reading operation is complete.

[0062] In its implementation, the system can detect the completion status of data reading for all target physical addresses corresponding to the current task and perform data integrity checks on all received target data. If any incomplete items exist, the reading process returns to continue reading the incomplete addresses (i.e., the aforementioned sub-steps S202 to S203). If all data is complete and the checks pass, a data reading ready signal is output, indicating that the serialized memory access is complete. This mechanism ensures the integrity of the input data, providing reliable data support for the subsequent deterministic operations of the arithmetic logic unit (ALU).

[0063] In this embodiment, by combining preset serialization access rules and address request signals, and through a collaborative design involving data dependency analysis, access priority sorting, dual-trigger request control, data cache alignment, and integrity verification, the serialization read operation of the target data is efficiently completed from the main storage. This process effectively avoids data path congestion and out-of-order data reception, ensuring the accuracy and sequential consistency of the read data. It provides a reliable ordered data flow for the arithmetic logic unit of the AI ​​accelerator, significantly improving the efficiency and stability of memory access and computation collaboration.

[0064] Based on the foregoing embodiments, a third embodiment of the serialized memory access and computation co-operation method of the present invention is proposed. In this embodiment, step S30 may include the following sub-steps S301~S304: Sub-step S301: When it is detected that the target data has been loaded and the operation control signal has been successfully decoded, the collaborative operation process of memory access and computation is triggered, and the linkage mechanism of data loading and arithmetic processing is started.

[0065] The operation control signals are parsed by the response decoder (resp cmd decoder) to generate a set of sub-control signals used to drive the arithmetic logic unit (ALU) to perform specific operations. When all key fields in this set (including the ALU operation mode signal, destination write enable signal, and source operand selection signal) meet the preset logic valid state, it indicates that the current operation instruction has been correctly parsed and the relevant data path configuration is ready.

[0066] The valid status of this signal set conveys a confirmation message to the ALU that "configuration is complete and the path is available." Together with the "data ready signal" sensed by the ALU's input control logic, it constitutes the dual triggering conditions for the "memory access-computation" collaborative mechanism. The ALU is only triggered to initiate computation when both the aforementioned set of operation control signals and the data ready signal are valid simultaneously. This mechanism effectively prevents invalid computations and logical conflicts caused by data hazards or control hazards, avoiding the waste of computing resources and thus significantly improving the reliability and energy efficiency of the AI ​​accelerator.

[0067] In its implementation, once the target data loading is complete and the operation control signal is successfully decoded by the response decoder, a collaborative operation process involving memory access and computation can be triggered, initiating the execution of the Arithmetic Logic Unit (ALU). This dual-verification mechanism uses data readiness and valid control configuration as criteria, only initiating ALU operations when both are simultaneously valid, thus achieving pipelined linkage between data loading and arithmetic processing. This mechanism effectively eliminates invalid computations, logical conflicts, and operational anomalies caused by incomplete data or invalid instructions (invalid operation control signals), significantly reducing wasted computing power and hardware resource consumption, and improving the operational reliability and energy efficiency of the AI ​​accelerator.

[0068] Sub-step S302: Based on the source operand selection signal obtained by decoding the operation control signal, select the target data corresponding to the current operation from the input data path of the arithmetic logic unit, and use it as the source operand of the arithmetic logic unit.

[0069] In practical implementation, the target data corresponding to the current operation can be selected from the ALU's input data path based on the source operand selection signal obtained by decoding the aforementioned operation control signal through the response decoder, and used as the ALU's source operand input. Specifically, during the loading process, data is returned strictly in the original memory access sequence initiated by the address request signal, ensuring that the timing of the source operand supply strictly matches the execution order of the operation logic. This mechanism guarantees the serialization consistency and sequential traceability of the data stream at the hardware level, effectively avoiding logical errors caused by out-of-order data.

[0070] Sub-step S303: Based on the ALU operation mode signal obtained by decoding the operation control signal, dynamically configure the operation mode of the arithmetic logic unit and drive it to perform the specified operation on the loaded source operand to generate the calculation result; and simultaneously complete the data overflow detection, carry and borrow logic processing during the operation, and generate the corresponding status flag for external query or reporting.

[0071] In practical implementation, the operation mode of the arithmetic logic unit (ALU) can be dynamically configured based on the ALU operation mode signal in the operation control signal (such as basic arithmetic operations, logical operations, data comparison and judgment operations, and AI-specific vectorized operations). The ALU is then driven to perform specified types of operations on the loaded source operands to generate calculation results. During the operation, data overflow detection, carry and borrow logic processing are completed synchronously, and preliminary judgment and reporting of operation anomalies are performed to achieve real-time judgment and response to operation anomalies, thereby ensuring the reliability and diagnosability of the calculation process.

[0072] Sub-step S304: Based on the target write enable signal obtained by decoding the operation control signal, the calculation result output by the arithmetic logic unit is written to its configured temporary result cache unit, and the result sequence identifier corresponding to the current operation step is automatically associated with the record through hardware logic during writing, so as to ensure the sequential consistency of the data flow in the subsequent serialization write-back process.

[0073] In practical implementation, the calculation result output by the Arithmetic Logic Unit (ALU) can be latched into a temporary result buffer unit (i.e., the data latch or pipeline register at the ALU output) based on the valid state of the write enable signal in the destination register of the arithmetic control signal. During the write process, the result sequence identifier (i.e., a unique binary code identifier bound one-to-one with the calculation result) corresponding to the current operation step and the source operand sequence is associated and recorded. This mechanism ensures that the data stream maintains strict sequential consistency in the subsequent serialization write-back process, supporting precise exception handling and data recovery in out-of-order execution environments.

[0074] In this embodiment, through the collaborative control logic between memory access and computation, invalid computations, logical conflicts, and operational anomalies caused by incomplete data, illegal instructions, or timing misalignments are effectively avoided, strictly ensuring the sequential consistency of the entire data flow from "serialized memory access → ordered computation → serialized write-back". Simultaneously, by leveraging dynamic adaptation of computation modes, real-time response to abnormal states, and decoupling mechanisms for computation and write-back timing, the computational utilization and reliability of the arithmetic logic unit are significantly improved, ultimately achieving high determinism, high collaborative efficiency, and low resource waste in data processing for the AI ​​accelerator.

[0075] Based on the foregoing embodiments, a fourth embodiment of the serialized memory access and computation co-operation method of the present invention is proposed. In this embodiment, step S40 may include the following sub-steps S401~S403: Sub-step S401: When it is detected that the calculation result stored in the temporary result cache unit configured by the arithmetic logic unit has completed the calculation exception verification and the main storage interface is in an idle and writable state, the serialization write-back process is triggered; the legality verification of the preset serialization access rules is performed simultaneously.

[0076] In the specific implementation, when it is detected that the calculation result stored in the temporary result cache unit configured by the Arithmetic Logic Unit (ALU) has completed the operation exception verification (such as overflow, illegal opcode, etc.), and the main storage interface is in an idle and writable state, the serialization write-back process is triggered. At this time, it is necessary to combine the original memory access serialization information corresponding to the address request signal to perform a legality verification on the preset serialization access rules (the verification content includes target address range, access permissions, data alignment constraints, etc.).

[0077] Sub-step S402: After the verification is passed, the calculation results stored in the temporary result cache unit are format converted and aligned according to the data storage format requirements of the main storage to generate the processed calculation results that conform to the main storage interface protocol.

[0078] In the specific implementation, after all verifications pass, the calculation results stored in the temporary result cache unit can be format converted and aligned according to the data storage format requirements of the main storage (such as byte order, bit width alignment, ECC encoding, etc.) to generate processed calculation results that conform to the main storage interface protocol, ensuring that the data can be correctly parsed and stored by the main storage.

[0079] It is important to note that if any of the above verification steps (including verification of computational anomalies in the calculation results, verification of the legality of the preset serialization access rules, and verification of the format compliance of the calculation results) fail, the write-back process must be terminated immediately, the exception handling mechanism must be triggered, and an error log (including key information such as error type, verification failure step, and current operation sequence identifier) ​​must be recorded to provide a basis for subsequent troubleshooting.

[0080] Sub-step S403: Based on the write address sequence generated by the preset serialization access rules, the processed calculation results are written back to the target physical address corresponding to the main storage in sequence, thus completing the serialized data write-back operation.

[0081] In practical implementation, the processed calculation results can be written back to the target physical address in the main storage in order according to the write address sequence generated by the preset serialization access rules (a set of ordered physical addresses generated according to the preset serialization access rules, used to guide the target data to be written to a specific location in the main storage in a specific order), thus completing the serialized data write-back operation; thereby, through this series of processes, the sequential consistency and data integrity of the write-back operation can be effectively guaranteed.

[0082] In this embodiment, the synergistic effect of three dimensions—validation of the write address sequence (ensuring correct operation), conversion of the data format of ALU output (ensuring format compliance), and timing coordination to ensure the timing of write-back (ensuring appropriate timing)—can effectively ensure the sequential consistency and data integrity of the write-back operation, thus completing the serialized write-back operation.

[0083] Based on the foregoing embodiments, a fifth embodiment of the serialized memory access and computation co-operation method of the present invention is proposed. In this embodiment, after step S40, the method may further include the following steps S50~S60: Step S50: After confirming that all calculation results to be written back have been written to the corresponding target physical address of the main storage in sequence, a task completion signal is generated and fed back to the main controller, and the integrity verification of the data written back to the main storage is performed synchronously.

[0084] In the specific implementation, the completion status of the serialized data write-back operation can be detected in real time. When it is confirmed that all calculation results to be written back have been written to the corresponding target physical address in the main storage in order according to the write-back address, a task completion signal is generated and fed back to the main controller through a dedicated data channel. At the same time, the integrity of the data written back to the main storage is verified (such as by CRC check, checksum comparison, etc., comparing the data written back in the main storage with the original calculation results retained in the ALU temporary result cache unit to ensure that the data is not lost or tampered with).

[0085] In step S60, if the integrity verification passes, the temporary resources occupied by the current task are released to restore the context state of the AI ​​accelerator, and according to the preset scheduling strategy, the external commands or data of the next task are preloaded into the designated cache unit to enter the standby state to support the efficient pipelined processing of continuous tasks.

[0086] In the specific implementation, if the integrity check passes, the temporary resources occupied by the current task can be released immediately, including clearing the temporary result cache unit inside the ALU, releasing the address mapping table resources, and resetting the operation status register, so as to restore the context state of the AI ​​accelerator. Then, according to the preset scheduling strategy (such as by task priority, task submission order, etc.), the external command of the next task (to the request decoder / response decoder cache area) or target data (to the AI ​​accelerator local data cache unit) is preloaded, so that the AI ​​accelerator enters the standby state, thereby supporting the efficient pipelined processing of continuous tasks and improving the overall computing throughput.

[0087] In this embodiment, by performing integrity verification after writing data back to main storage, the reliability and consistency of the written-back data are ensured, thereby guaranteeing the stability of task switching, realizing efficient pipelined processing of continuous tasks, and ultimately improving the overall computing throughput of the AI ​​accelerator.

[0088] Based on the same inventive concept, the sixth embodiment of this invention also provides a serialized memory access and computational coordination system corresponding to the serialized memory access and computational coordination method of the foregoing embodiments. Since the principle of the system in the sixth embodiment of this invention for solving the problem is similar to the serialized memory access and computational coordination method of the foregoing embodiments of this invention, the implementation of the system can refer to the implementation of the method, and repeated details will not be elaborated further. Please refer to... Figure 4 This invention relates to a serialized memory access and computation co-processing system, applied to an AI accelerator that communicates with main memory. The AI ​​accelerator includes a request decoder, a response decoder, and an arithmetic logic unit. The system may include: The parsing module 10 is used to receive external commands issued by the main controller, parse the external commands through the request decoder, and generate an address request signal for accessing the main memory; at the same time, it parses the external commands through the response decoder and generates an operation control signal for controlling the arithmetic logic unit to perform specific operations. The memory access module 20 is used to read target data sequentially from the main memory according to the address request signal based on the preset serialization access rules, and complete the serialized data reading operation. The arithmetic module 30 is used to drive the arithmetic logic unit to perform corresponding logical operations on the target data according to the arithmetic control signal after the target data is ready, based on the collaborative control logic between memory access and computation, and generate computation results. The write-back module 40 is used to write the calculation results back to the main storage sequentially according to the preset serialization access rules, thereby completing the serialized data write-back operation.

[0089] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described serialized memory access and computational co-processing method.

[0090] Figure 5 This is a schematic block diagram of the electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device includes at least one processor 401, a memory 402, at least one network interface 403, and a user interface 405. The various components in the electronic device are coupled together via a bus system 404. It is understood that the bus system 404 is used to implement communication between these components. In addition to a data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 5 The general will label all buses as bus systems.

[0091] The user interface 405 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0092] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0093] In this embodiment of the invention, the memory 402 is used to store various types of data to support the operation of the electronic device 400. Examples of this data include: any executable program for operation on the electronic device 400, such as the operating system 4021 and application programs 4022; the operating system 4021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 4022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The serialization memory access and computation coordination method provided in this embodiment of the invention can be included in the application program 4022.

[0094] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in software form. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 401 may be a microprocessor or any conventional processor, etc. The steps of the serialization memory access and computation coordination method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0095] In an exemplary embodiment, the electronic device 400 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to perform the aforementioned method.

[0096] In summary, this invention uses two identical command parsers to parse the same external command in parallel, collaboratively generating address information for memory access and operation instructions for computational control. These two functions are complementary and synchronized in timing. Compared to traditional solutions that rely on OST buffers and their associated control logic, this architecture eliminates the need for additional cached control information or intermediate states, effectively saving the OST buffer and related read / write logic, significantly reducing the chip area overhead of the AI ​​accelerator; it also eliminates dynamic read / write power consumption and static leakage power consumption, significantly improving system energy efficiency. Furthermore, the triggering of memory access operations is independent of buffer capacity, meaning the number of memory accesses supported by a single task is no longer limited by the OST size, greatly enhancing the flexibility and parallelism of task processing.

[0097] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A serialized memory access and computation co-processing method, characterized in that, An AI accelerator connected in communication with main storage, the AI ​​accelerator including a request decoder, a response decoder, and an arithmetic logic unit, the method comprising: The system receives external commands from the master controller, parses the external commands through a request decoder, and generates an address request signal for accessing the main memory; at the same time, it parses the external commands through a response decoder and generates an operation control signal for controlling the arithmetic logic unit to perform specific operations. Based on the preset serialization access rules, the target data is read sequentially from the main storage according to the address request signal to complete the serialized data reading operation; Based on the collaborative control logic between memory access and computation, after the target data is ready, the arithmetic logic unit is driven to perform corresponding logical operations on the target data according to the operation control signal to generate the calculation result; According to the preset serialization access rules, the calculation results are written back to the main storage in sequence, completing the serialized data write-back operation.

2. The method according to claim 1, characterized in that, The step of parsing the external command through a request decoder to generate an address request signal for accessing main storage includes: The request decoder parses the external command to extract the base address register encoding, immediate offset, and read / write attributes contained therein; According to the address width standard specified by the main memory, the immediate value offset is subjected to sign extension or zero extension processing, and the extended immediate value is added to the register value pointed to by the base address register encoding to generate the target physical address; The target physical address and read / write attributes are encapsulated into an address request signal and sent to the main storage to trigger the corresponding data read operation.

3. The method according to claim 1, characterized in that, The step of parsing the external command through a response decoder to generate operation control signals for controlling the arithmetic logic unit to perform specific operations includes: The external command is parsed by the response decoder to extract the combination of opcodes and function fields contained therein in order to determine the target operation type; According to the target operation type, the opcode is decoded into the corresponding ALU operation signal, source operand selection signal and destination write enable signal; The ALU operation signal, the source operand selection signal, and the destination write enable signal are integrated, formatted, and encapsulated into an operation control signal, which is then sent to the data path to drive the arithmetic logic unit to perform the specified operation.

4. The method according to claim 1, characterized in that, The method, based on preset serialization access rules, sequentially reads target data from main storage according to the address request signal to complete the serialized data reading operation, including: Based on the address request signal, the target physical address, read / write attributes and valid enable bits contained therein are parsed, and combined with the preset serialization access rules, data dependency analysis and access priority sorting are performed on multiple target physical addresses involved in the current task to generate an ordered address access sequence with a strict sequential execution order. According to the ordered address access sequence, data read requests are initiated to the main storage in sequence; wherein, the next address read request is triggered only when the previous address read request has received a valid response from the main storage, or when the target data cached in the current data path has been completely consumed by the arithmetic logic unit, so as to avoid data path congestion. Receive each target data returned by the main storage and cache and align it according to the execution order of the ordered address access sequence to ensure that the data receiving order is completely consistent with the request initiation order; Once all data at all target physical addresses has been read and verified to be correct, a data read ready signal is output to indicate that the serialized data read operation is complete.

5. The method according to claim 1, characterized in that, The coordinated control logic between memory access and computation, after the target data is ready, drives the arithmetic logic unit to perform corresponding logical operations on the target data according to the operation control signal to generate calculation results, including: Once the target data is detected to be loaded and the operation control signal is successfully decoded, the collaborative operation process of memory access and computation is triggered, and the operation execution of the arithmetic logic unit is started. Based on the source operand selection signal obtained by decoding the operation control signal, the target data corresponding to the current operation is selected from the input data path of the arithmetic logic unit and used as the source operand of the arithmetic logic unit. Based on the ALU operation mode signal decoded from the operation control signal, the operation mode of the arithmetic logic unit is dynamically configured, and it is driven to perform specified operations on the loaded source operands to generate calculation results. Based on the target write enable signal obtained by decoding the operation control signal, the calculation result output by the arithmetic logic unit is written into the temporary result cache unit configured inside it. The hardware logic automatically associates and records the result sequence identifier corresponding to the current operation step during the writing process to ensure the sequential consistency of the data stream in the subsequent serialization write-back process.

6. The method according to claim 1, characterized in that, The step of writing the calculation results back to the main storage sequentially according to the preset serialization access rules, thereby completing the serialized data write-back operation, includes: When it is detected that the calculation result stored in the temporary result cache unit of the arithmetic logic unit has completed the operation exception verification, and the main storage interface is in an idle and writable state, the serialization write-back process is triggered; at the same time, the legality verification of the preset serialization access rules is performed. After the verification is passed, the calculation results stored in the temporary result cache unit are format converted and aligned according to the data storage format requirements of the main storage, and the processed calculation results that conform to the main storage interface protocol are generated. Based on the write address sequence generated by the preset serialization access rules, the processed calculation results are written back to the target physical address corresponding to the main storage in sequence, thus completing the serialized data write-back operation.

7. The method according to claim 1 or 6, characterized in that, After the step of writing the calculation results back to the main storage sequentially according to the preset serialization access rules to complete the serialized data write-back operation, the method further includes: After confirming that all calculation results to be written back have been written to the corresponding target physical address in the main storage in order according to the write-back address, a task completion signal is generated and fed back to the main controller, and an integrity check is performed on the data written back to the main storage. If the integrity check passes, the temporary resources occupied by the current task are released to restore the context state of the AI ​​accelerator. According to the preset scheduling strategy, the external commands or data of the next task are preloaded into the designated cache unit and put into standby state to support the efficient pipelined processing of continuous tasks.

8. A data processing system with a near-memory computing architecture, characterized in that, An AI accelerator connected in communication with main storage, the AI ​​accelerator including a request decoder, a response decoder, and an arithmetic logic unit, the system comprising: The parsing module is used to receive external commands issued by the main controller, parse the external commands through the request decoder, and generate an address request signal for accessing the main memory; at the same time, it parses the external commands through the response decoder and generates an operation control signal for controlling the arithmetic logic unit to perform specific operations. The memory access module is used to read target data sequentially from the main memory according to the address request signal based on the preset serialization access rules, and complete the serialized data reading operation. The arithmetic module is used to drive the arithmetic logic unit to perform corresponding logical operations on the target data according to the arithmetic control signal after the target data is ready, based on the coordinated control logic between memory access and computation, and generate the calculation result. The write-back module is used to write the calculation results back to the main storage sequentially according to the preset serialization access rules, thereby completing the serialized data write-back operation.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the processor to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed, performs the steps of the method according to any one of claims 1 to 7.

Images