In-memory computing AI accelerator design architecture based on RISC-V architecture and control method

Abstract

The invention provides an in-memory computing AI accelerator design architecture based on an RISC-V architecture and a control method. The architecture comprises an RISC-V processor used for executing a general-purpose computing task and a control instruction; the in-memory computing accelerator is used for executing a computing task in an AI algorithm, is connected with the RISC-V processor and comprises a tensor processing unit and a vector processing unit; the in-memory computing accelerator further comprises a data rearrangement unit which is used for carrying out format conversion and rearrangement on the data before the data are transmitted to the tensor processing unit or the vector processing unit. And the data format management unit is used for defining and managing a unified data format in the in-memory computing accelerator. According to the invention, the calculation performance and the resource utilization rate of the AI algorithm can be improved, and meanwhile, the accuracy of data transmission and the parallelism of processing are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of RISC-V processor architecture, and more specifically, to a design architecture and control method of an in-memory computing AI accelerator based on the RISC-V architecture. Background Art

[0002] With the rapid development of artificial intelligence technology, the demand for AI algorithms in various application scenarios is increasing, and the requirements for computing performance are also increasing. Existing computing platforms, especially CPUs and GPUs based on traditional architectures, have certain capabilities in processing complex computing tasks, but are often limited by data transmission bottlenecks and computing efficiency issues when facing large-scale parallel computing needs. When executing AI algorithms such as deep learning, these platforms require frequent data movement and high energy consumption, resulting in limited overall performance and difficulty in meeting the growing computing needs.

[0003] In this context, in-memory computing technology has emerged, aiming to reduce data transmission delays and energy consumption by executing computing tasks directly in memory. However, existing in-memory computing technologies are mostly focused on specific types of algorithms or applications, lacking versatility and flexibility. In-memory computing data management corresponding to specific types of algorithms or applications also faces the problem of low data storage and processing efficiency in in-memory computing accelerators when different data types or access orders appear, which seriously affects the execution speed of computing tasks. Summary of the invention

[0004] The present invention provides an in-memory computing AI accelerator design architecture and control method based on the RISC-V architecture, so as to provide an in-memory computing accelerator architecture with wider application and better performance.

[0005] In a first aspect of the present invention, an in-memory computing AI accelerator design architecture based on RISC-V architecture is provided.

[0006] It includes a RISC-V processor, at least one in-memory computing accelerator, and an expansion interface;

[0007] The RISC-V processor is used to execute general computing tasks and control instructions;

[0008] The in-memory computing accelerator is used to execute computing tasks in the AI ​​algorithm and is connected to the RISC-V processor. The in-memory computing accelerator includes a tensor processing unit and a vector processing unit;

[0009] The extended interface is used to realize communication between the RISC-V processor and the in-memory computing accelerator;

[0010] The in-memory computing accelerator also includes:

[0011] A data rearrangement unit, used for performing format conversion and rearrangement on data before transmitting the data to the tensor processing unit or the vector processing unit;

[0012] The data format management unit is used to define and manage the unified data format in the in-memory computing accelerator.

[0013] The data reordering unit supports multiple data types and formats to adapt to different AI algorithms and application scenarios. The data reordering unit is tightly integrated with the instruction FIFO unit. The data reordering unit automatically adjusts its operation according to the instructions of the instruction FIFO unit to ensure that the data is in the best state when it is transmitted to the tensor processing unit or the vector processing unit.

[0014] The data reordering unit supports conditional execution and dependency tracking; when the data reordering unit detects that the data is not ready or there is an unresolved dependency, the data is temporarily stored and processed when the conditions are met.

[0015] In a second aspect of the present invention, a method for controlling an in-memory computing AI accelerator design architecture based on a RISC-V architecture is provided, comprising:

[0016] Execute general computing tasks and control instructions on RISC-V processors;

[0017] The control instruction is transmitted to the in-memory computing accelerator through the instruction sending interface of the extended interface;

[0018] Parameters and results are transferred between the RISC-V processor and the in-memory computing accelerator through the source register interface and the destination register interface;

[0019] A data reordering unit in the in-memory computing accelerator is used to perform data format conversion and reordering before the data is transmitted to a tensor processing unit or a vector processing unit in the in-memory computing accelerator.

[0020] According to the above-mentioned embodiments of the present invention, at least the following beneficial effects are achieved: the in-memory computing AI accelerator design architecture based on the RISC-V architecture proposed in the present invention can achieve significant performance improvement and resource optimization through modular design and efficient interface mechanism. In this architecture, the close integration of the RISC-V processor and the in-memory computing accelerator ensures the efficiency of control instructions and data transmission. At the same time, the application of the data dependency detection module and the instruction FIFO unit can further improve the parallel processing capability and the flexibility of task scheduling. In addition, by introducing the data rearrangement unit, the data format conversion and rearrangement process are optimized, the delay in the data transmission process is reduced, and the utilization rate of the computing unit is improved. The management of the unified data format simplifies the data exchange process between different processing units and reduces the complexity of the system. Overall, the present invention can not only improve the execution speed and energy efficiency of the AI ​​algorithm, but also enhance the scalability and adaptability of the system, and provide an efficient, flexible and customizable hardware acceleration platform for AI applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0022] Figure 1 A schematic diagram of the structure of an in-memory computing AI accelerator design architecture provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of the structure of an instruction FIFO unit provided by an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of the structure of an extended interface provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of the structure of an instruction sending interface provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of a data format provided by an embodiment of the present invention;

[0027] Figure 6 A flow chart of a method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0029] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0030] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0031] Reference below Figure 1 , Figure 1 A schematic diagram of an in-memory computing AI accelerator design architecture based on the RISC-V architecture is provided in one embodiment of the present invention. Figure 1 As shown in the figure, an in-memory computing AI accelerator design architecture based on RISC-V architecture includes:

[0032] RISC-V processor, at least one in-memory computing accelerator, and expansion interfaces;

[0033] The RISC-V processor is used to execute general computing tasks and control instructions;

[0034] The in-memory computing accelerator is used to perform computing tasks in the AI ​​algorithm and is connected to the RISC-V processor. The in-memory computing accelerator includes a tensor processing unit and a vector processing unit. The tensor processing unit is used to perform large-scale matrix calculations, and the vector processing unit is used to perform activation functions, pooling, quantization, and normalization operations in the AI ​​algorithm.

[0035] The extended interface is used to implement communication between the RISC-V processor and the in-memory computing accelerator.

[0036] Among them, the extended interface may include:

[0037] Instruction sending interface, used for control instruction transmission between RISC-V processor and in-memory computing accelerator;

[0038] A data dependency detection module is used to ensure that multiple instructions can be sent simultaneously according to the data dependency when multiple instructions are sent simultaneously;

[0039] Source register interface, used for parameter transfer between RISC-V processor and in-memory computing accelerator;

[0040] The destination register interface is used to transfer calculation results between the in-memory computing accelerator and the RISC-V processor to realize return value reception.

[0041] In some embodiments, the in-memory computing AI accelerator design architecture based on the RISC-V architecture includes a series of components that work together to achieve efficient AI algorithm calculations. Specifically, the RISC-V processor not only performs general computing tasks, but is also responsible for issuing control instructions, which are passed to the in-memory computing accelerator through an extended interface.

[0042] Specifically, the RISC-V processor can be one or more cores that can handle various computing tasks and have the ability to execute specific control instructions of the AI ​​accelerator. In-memory computing accelerators include multiple processing units, such as tensor processing units and vector processing units, which are optimized for different types of computing tasks. The tensor processing unit may contain multiple processor arrays specifically for performing matrix multiplication operations; while the vector processing unit may contain dedicated hardware acceleration components for quickly performing operations such as activation functions.

[0043] Preferably, in the design of the extended interface, the instruction sending interface can be designed to support multi-channel communication, that is, a multi-instruction sending interface, to achieve high-speed data exchange between the RISC-V processor and the in-memory computing accelerator. The data dependency detection module can use advanced algorithms to predict and schedule tasks, determine the instructions that can be sent simultaneously, reduce waiting time, and improve the parallelism of instruction execution. The source register interface and the destination register interface can be designed to support multiple data widths to meet different data transmission requirements.

[0044] In some embodiments, the design of the in-memory computing accelerator allows it to be tightly integrated with the RISC-V processor to achieve efficient computing task execution. This integration can be achieved by sharing memory space or using a high-speed communication bus to reduce data transmission latency.

[0045] Specifically, the in-memory computing accelerator can share the same memory space with the RISC-V processor, or be connected through a cache coherent bus to achieve fast data exchange. The tensor processing unit and vector processing unit can be configured to support a variety of different data formats and sizes to meet the needs of different AI algorithms.

[0046] Preferably, the tensor processing unit can be designed to support dynamic adjustment of computing resource allocation to adapt to different matrix computing tasks. The vector processing unit can include multiple dedicated hardware accelerators for performing specific activation functions or normalization operations, and its operating frequency can be dynamically adjusted according to task requirements.

[0047] In some embodiments, the extended interface also includes a buffering mechanism for instruction and data transmission to further improve the overall performance of the system, namely, a configurable instruction FIFO unit in the in-memory computing accelerator is used to cache control instructions received from the RISC-V processor to achieve instruction prefetching and parallel processing of the in-memory computing accelerator.

[0048] Specifically, the instruction FIFO unit can be designed as a high-capacity first-in-first-out buffer to cache a large number of control instructions to ensure that the in-memory computing accelerator can continuously receive new computing tasks. The data reordering unit can dynamically adjust the data format and order according to the request of the instruction FIFO unit to meet the real-time computing needs of the tensor processing unit and the vector processing unit.

[0049] Preferably, the data format management unit can be designed to support custom data formats, allowing developers to define data storage and access methods based on specific AI algorithm requirements. This flexibility can significantly improve the efficiency of data access and reduce unnecessary data conversion overhead.

[0050] like Figure 2 As shown, Figure 2 A schematic diagram of the structure of the instruction FIFO unit is shown.

[0051] In some embodiments, the instruction FIFO unit in the in-memory computing AI accelerator design architecture is a key component for caching control instructions. The unit receives control instructions from the RISC-V processor and stores them in a FIFO (first-in-first-out) structure, thereby enabling pre-fetching of instructions and parallel processing of the accelerator.

[0052] Specifically, the instruction FIFO unit can be designed to have multiple cache lines or buffers, each of which can store a certain number of instructions. These cache lines can be hardware-implemented registers or memory blocks that can be quickly accessed and processed by the in-memory computing accelerator. Control instructions can be stored in the FIFO unit in the order in which they are received and taken out for execution when needed.

[0053] Preferably, the instruction FIFO unit can be designed to support a variety of different instruction types and lengths to meet the needs of complex AI algorithms. The unit can include dedicated hardware logic for decoding and prefetching instructions to ensure that instructions can be executed in the optimal order.

[0054] In some embodiments, the instruction FIFO unit is tightly integrated with other components of the in-memory computing accelerator (such as the instruction scheduling unit) to achieve efficient instruction processing. The unit can work in conjunction with the instruction scheduling unit to dynamically adjust the execution order of instructions according to the current state of the in-memory computing accelerator and task requirements.

[0055] Specifically, a high-speed communication link may be designed between the instruction FIFO unit and the instruction scheduling unit to quickly exchange instruction status and execution information. The instruction scheduling unit may adjust the priority of instructions in the instruction FIFO unit according to the feedback from the data dependency detection module to ensure that key tasks can be executed first.

[0056] Preferably, the instruction FIFO unit can also be designed to support interrupt and exception handling mechanisms. When a high-priority interrupt or exception is detected, the unit can pause the current instruction flow, quickly respond to and handle these events, and then resume the original instruction execution process.

[0057] Furthermore, the in-memory computing accelerator also includes a data reordering unit for performing format conversion and reordering on the data before the data is transmitted to the tensor processing unit or the vector processing unit, so as to optimize the access efficiency and computing performance of the data.

[0058] In some embodiments, the data display unit in the in-memory computing AI accelerator design architecture is a key component specifically designed to improve data access efficiency and computing performance. The data reordering unit is responsible for performing necessary format conversion and reordering on the data before sending it to the tensor processing unit or vector processing unit.

[0059] Specifically, the data reordering unit can dynamically adjust the layout and format of data according to the specific needs of the tensor processing unit or the vector processing unit. For example, if the tensor processing unit needs to perform a multiplication operation according to a specific matrix shape, the data reordering unit can convert the original data from row-major order to column-major order to optimize the memory access pattern and improve computing efficiency.

[0060] Preferably, the data rearrangement unit may include a series of hardware accelerators, such as dedicated sorting circuits, cache systems, and direct memory access (DMA) engines. These accelerators can work in parallel to achieve fast data rearrangement operations. The data rearrangement unit can also be designed to support multiple data types and formats to adapt to different AI algorithms and application scenarios.

[0061] In some embodiments, the data reordering unit is tightly integrated with the instruction FIFO unit to achieve an efficient data processing flow. The instruction FIFO unit can provide instructions on the required data format and arrangement order to the data reordering unit.

[0062] Specifically, the control instructions in the instruction FIFO unit may contain all the information required for data rearrangement, such as the expected shape of the data, the data type, and the rearrangement rules. The data rearrangement unit automatically adjusts its operation according to these instructions to ensure that the data is in the best state when it is transmitted to the tensor processing unit or the vector processing unit.

[0063] Preferably, the data reordering unit can also be designed to support conditional execution and dependency tracking. This means that if the data reordering unit detects that some data is not yet ready or there are dependencies that have not been resolved, it can temporarily store the data and process it when the conditions are met. This flexibility further improves the overall performance and efficiency of the in-memory computing AI accelerator design architecture.

[0064] Furthermore, the instruction FIFO unit is configured as follows:

[0065] Receive control instructions from the RISC-V processor;

[0066] According to the content of the control instruction, independently or with the assistance of the instruction scheduling unit, a request for data format conversion and rearrangement is issued to the data rearrangement unit;

[0067] The requested data is sent to the data reordering unit to implement real-time format conversion and reordering of the data of the tensor processing unit or the vector processing unit.

[0068] Specifically, the instruction FIFO unit is designed to receive multiple types of control instructions, and parse and cache these instructions. The unit may contain multiple buffers or queues to store instructions to be processed. Each control instruction may contain specific parameters or opcodes to indicate the type of data operation required, such as data format conversion or rearrangement in a specific order.

[0069] Preferably, the instruction FIFO unit can independently parse the control instruction and generate a data reordering request. This independence reduces the dependence on other components and improves the response speed and efficiency of the entire system. In addition, the instruction FIFO unit may also include a parsing engine that can identify the data operation requirements in the instruction and automatically generate corresponding requests.

[0070] More specifically, after parsing the control instruction, the instruction FIFO unit generates one or more data operation requests and sends them to the data reordering unit. These requests may include specific format requirements for the data, reordering order, or other related parameters. Based on these requests, the data reordering unit performs real-time format conversion and reordering of the data to be sent to the tensor processing unit or vector processing unit to ensure that the data is calculated and processed in the optimal format.

[0071] In some embodiments, the instruction FIFO unit works closely with the instruction scheduling unit to achieve more efficient task scheduling and resource management. The instruction scheduling unit can provide information about the current state of the system, such as the availability of processing units and priority tasks, to help the instruction FIFO unit optimize the generation and sending of requests.

[0072] Specifically, the instruction scheduling unit can provide feedback to the instruction FIFO unit to indicate the current workload and processing capacity of the data reordering unit. Based on this information, the instruction FIFO unit can adjust the sending rate or priority of the request to avoid overloading the data reordering unit and ensure that critical tasks can be processed in a timely manner.

[0073] Furthermore, the expansion interface may be a plurality of instruction sending interfaces for simultaneously sending a plurality of control instructions to the in-memory computing accelerator.

[0074] like Figure 3 As shown, Figure 3 A schematic diagram of the structure of the extended interface is shown.

[0075] In some embodiments, the design of the extended interface allows multiple instruction sending interfaces to work in parallel to achieve efficient control of multiple processing units in the in-memory computing accelerator. These instruction sending interfaces can send multiple control instructions to the tensor processing unit or the vector processing unit at the same time.

[0076] Specifically, each instruction sending interface can independently communicate with one or more processing units in the in-memory computing accelerator. In this way, the RISC-V processor can issue control instructions to multiple processing units at the same time, thereby achieving parallel processing and improving overall computing efficiency. Each instruction sending interface may contain dedicated hardware circuits, such as multiplexers and buffer registers, to support the rapid distribution of instructions.

[0077] like Figure 4 As shown, Figure 4 A schematic diagram of the structure of the instruction sending interface is shown.

[0078] The instruction transmission interface can be designed to support different communication protocols and data widths to adapt to the specific needs of different processing units. This allows each interface to be optimized according to the characteristics of the target processing unit, such as processing speed and data throughput. In addition, the instruction transmission interface can also include flow control and error detection mechanisms to ensure the reliability of instruction transmission.

[0079] More specifically, each instruction sending interface in the extended interface can be tightly integrated with the instruction scheduling unit to achieve dynamic task allocation and load balancing. The instruction scheduling unit can intelligently allocate control instructions to each instruction sending interface based on the current workload and processing power of each processing unit. This dynamic allocation mechanism can improve the resource utilization of the in-memory computing accelerator and reduce the total time required for task completion.

[0080] In some embodiments, the expansion interface may further include a centralized control unit for coordinating the operations of all instruction sending interfaces. The control unit may implement global policies, such as priority scheduling and task allocation, to ensure that all processing units can work efficiently and avoid potential resource conflicts.

[0081] Specifically, the centralized control unit may include a scheduling algorithm that optimizes the distribution of instructions based on real-time performance feedback and prediction models. The control unit may also implement a feedback mechanism to adjust the rate and order of instruction delivery based on the response of the processing unit, thereby dynamically adapting to different computing needs and system conditions.

[0082] Preferably, the design of the expansion interface can also include support for scalability, allowing more instruction sending interfaces to be added as needed. This design enables the in-memory computing AI accelerator design architecture to adapt to growing processing requirements and technological advances.

[0083] Furthermore, the above instruction scheduling unit is configured as follows:

[0084] Manage the execution flow of control instructions in the instruction FIFO unit;

[0085] Ensure that the data reordering request issued by the instruction FIFO unit matches the current state and task requirements of the in-memory computing accelerator;

[0086] Adjusts data reordering requests issued by the instruction FIFO unit to optimize the performance and resource utilization of in-memory computing accelerators.

[0087] In some embodiments, the instruction scheduling unit is responsible for efficiently managing the execution flow of the control instructions. The unit is configured to monitor the instruction queue in the instruction FIFO unit and intelligently schedule these control instructions according to the current state of the in-memory computing accelerator and task requirements.

[0088] Specifically, the instruction scheduling unit can be designed to include one or more schedulers, which are responsible for tracking the instruction status in the instruction FIFO unit and selecting the next instruction to be executed based on priority and other scheduling strategies. The scheduler can use various algorithms, such as polling, priority queue, or task dependency-based scheduling algorithms to achieve the optimal instruction execution order.

[0089] Preferably, the instruction scheduling unit can monitor the resource usage of the in-memory computing accelerator in real time, including the status of the tensor processing unit and the vector processing unit. Based on this information, the instruction scheduling unit can dynamically adjust the data reordering request issued by the instruction FIFO unit to avoid resource conflicts and improve computing performance.

[0090] More specifically, the instruction scheduling unit can be co-designed with the data dependency detection module to ensure that the data reordering request matches the current state and task requirements of the in-memory computing accelerator. This co-design allows the instruction scheduling unit to rearrange the execution order of instructions when data dependency or resource conflict is detected, thereby ensuring the correctness and efficiency of the computing task.

[0091] In some embodiments, the instruction scheduling unit can also implement a predictive scheduling function, by analyzing the instruction sequence in the instruction FIFO unit, predicting future computing needs, and performing resource allocation and task scheduling in advance. This predictive scheduling can significantly reduce waiting time and improve overall computing throughput.

[0092] Specifically, the instruction scheduling unit can use machine learning algorithms or statistical models to identify patterns and trends in instruction execution and predict future computing needs. Based on these predictions, the instruction scheduling unit can adjust the instruction queue in the instruction FIFO unit in advance, optimize the sending time of data reordering requests, and allocate processing resources.

[0093] Preferably, the instruction scheduling unit can also be designed to support configurability and flexibility, allowing its scheduling strategy and parameters to be adjusted according to specific application requirements and system conditions. This configurability enables the in-memory computing accelerator to adapt to different computing tasks and performance requirements, improving its versatility and applicability.

[0094] Furthermore, the in-memory computing AI accelerator design architecture also includes:

[0095] A data format management unit, used to define and manage a unified data format in the in-memory computing accelerator;

[0096] The unified data format includes parameterized base address Base_address, sizes Size_dim0, Size_dim1, Size_dim2, and intervals Stride_dim1, Stride_dim2.

[0097] like Figure 5 As shown, Figure 5 A schematic diagram of data structure in some embodiments is shown.

[0098] In some embodiments, the data format management unit is a key component in the in-memory computing AI accelerator design architecture, responsible for defining and managing a unified data format to optimize data access and processing efficiency. The unit ensures the consistency and accuracy of data exchange between different components.

[0099] Specifically, the data format management unit can be designed to include a set of parameterized defined attributes that describe the layout and organization of data in the memory. For example, the base address (Base_address) specifies the starting position of the data block in the memory, and the size (Size_dim0, Size_dim1, Size_dim2) defines the expansion of the data block in each dimension.

[0100] Preferably, the data format management unit may further define the intervals between data elements (Stride_dim1, Stride_dim2), which allows the processing unit in the in-memory computing accelerator to efficiently access data elements in a non-continuous manner, thereby adapting to different data processing modes and algorithm requirements.

[0101] More specifically, the data format management unit can work in conjunction with the instruction FIFO unit and the instruction scheduling unit to dynamically adjust the data format according to the requirements of the control instruction. For example, when the instruction FIFO unit issues a data reordering request, the data format management unit can provide necessary parameters to instruct the data reordering unit to reorganize the data according to the specified format.

[0102] In some embodiments, the data format management unit may also support multiple data types and structures to accommodate diverse computing tasks in AI algorithms. This may include the definition of data structures such as tensors, matrices, and vectors, as well as their storage order and access mode in memory.

[0103] Specifically, the data format management unit can be designed to be flexibly configured, allowing developers to customize the data format according to specific application scenarios and performance requirements. This may involve programming the base address, size and interval parameters, as well as optimizing the data storage format.

[0104] Preferably, the data format management unit can also implement an automated data format conversion function to reduce the data conversion workload required by the processing unit when performing tasks. For example, when reading data from the memory or writing data to the memory, the data format management unit can automatically adjust the layout of the data to match the expected format of the processing unit.

[0105] Another embodiment of the present invention, Figure 6The present invention provides a flow chart of a control method of an in-memory computing AI accelerator design architecture based on a RISC-V architecture. The control method can be executed by the design architecture in the above embodiment, such as Figure 6 As shown, the control method in this embodiment includes the following steps: executing general computing tasks and control instructions on a RISC-V processor;

[0106] The control instruction is transmitted to the in-memory computing accelerator through the instruction sending interface of the extended interface;

[0107] Parameters and results are transferred between the RISC-V processor and the in-memory computing accelerator through the source register interface and the destination register interface;

[0108] A data reordering unit in an in-memory computing accelerator is used to convert and reorder data formats before transmitting the data to a tensor processing unit or a vector processing unit.

[0109] Furthermore, the method further comprises:

[0110] The control instructions received from the RISC-V processor are cached in the instruction FIFO unit of the in-memory computing accelerator to

[0111] Implement parallel processing of RISC-V processors and in-memory computing accelerators to optimize the instruction execution process.

[0112] Furthermore, the data reordering unit in the in-memory computing accelerator performs data format conversion and reordering before the data is transmitted to the tensor processing unit or the vector processing unit, specifically including:

[0113] According to the content of the control instruction, the instruction FIFO unit sends a request for data format conversion and rearrangement to the data rearrangement unit;

[0114] The data processed by the data rearrangement unit is sent to the tensor processing unit or the vector processing unit in real time to match the needs of the tensor processing unit or the vector processing unit.

[0115] The control method of this embodiment can achieve efficient instruction and data operation efficiency of the in-memory computing accelerator, so that the accelerator has a wide range of applications, and the instruction data processing efficiency is relatively high, which effectively improves the computing performance and resource utilization efficiency of the in-memory accelerator.

[0116] It is understandable that the various steps of the in-memory computing AI accelerator design architecture control method based on RISC-V architecture are similar to the reference Figure 1The various module units in the in-memory computing AI accelerator design architecture based on the RISC-V architecture described above correspond to each other. Therefore, the structure, characteristics and beneficial effects described above for the in-memory computing AI accelerator design architecture based on the RISC-V architecture are also applicable to the in-memory computing AI accelerator design architecture control method based on the RISC-V architecture, and will not be repeated here.

[0117] The electronic devices in some embodiments of the present invention may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The terminal device is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0118] Electronic devices may include processing devices (e.g., central processing units, graphics processing units, etc.), which may perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) or programs loaded from a storage device into a random access memory (RAM). In RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0119] Typically, the following devices may be connected to the I / O interface: input devices including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and communication devices. The communication devices may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data.

[0120] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0121] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention to form a technical solution.

Claims

1. An in-memory computing AI accelerator design architecture based on RISC-V architecture, characterized in that: It includes a RISC-V processor, at least one in-memory computing accelerator, and an expansion interface; The RISC-V processor is used to execute general computing tasks and control instructions; The in-memory computing accelerator is used to execute computing tasks in the AI ​​algorithm and is connected to the RISC-V processor. The in-memory computing accelerator includes a tensor processing unit and a vector processing unit; The extended interface is used to realize communication between the RISC-V processor and the in-memory computing accelerator; The in-memory computing accelerator also includes: A data rearrangement unit, used for performing format conversion and rearrangement on data before transmitting the data to the tensor processing unit or the vector processing unit; The data format management unit is used to define and manage the unified data format in the in-memory computing accelerator.

2. The in-memory computing AI accelerator design architecture according to claim 1, characterized in that: The in-memory computing accelerator also includes: an instruction FIFO unit, which is used to cache control instructions received from the RISC-V processor to achieve parallel processing of the control instructions by the RISC-V processor and the in-memory computing accelerator.

3. The in-memory computing AI accelerator design architecture according to claim 2, characterized in that: The in-memory computing accelerator further includes: an instruction scheduling unit, wherein the instruction scheduling unit is configured to: Manage the execution flow of control instructions in the instruction FIFO unit; Ensure that the data reordering request issued by the instruction FIFO unit matches the current state and task requirements of the in-memory computing accelerator; and Adjusting data reordering requests issued by the instruction FIFO unit to optimize the performance and resource utilization of the in-memory computing accelerator; The instruction FIFO unit is specifically configured as follows: Receive control instructions from the RISC-V processor; According to the content of the control instruction, independently or with the assistance of the instruction scheduling unit, a request for data format conversion and rearrangement is issued to the data rearrangement unit; The requested data is sent to the data reordering unit to implement real-time format conversion and reordering of the data of the tensor processing unit or the vector processing unit.

4. The in-memory computing AI accelerator design architecture according to claim 1, characterized in that: The tensor processing unit is used to perform large-scale matrix calculations, and the vector processing unit is used to execute activation functions, pooling, quantization and normalization operations in AI algorithms.

5. The in-memory computing AI accelerator design architecture according to any one of claims 1-4, characterized in that: The extended interface includes: Instruction sending interface, used for control instruction transmission between RISC-V processor and in-memory computing accelerator; Source register interface, used for parameter transfer between RISC-V processor and in-memory computing accelerator; The destination register interface is used to transfer calculation results between the in-memory computing accelerator and the RISC-V processor to realize return value reception.

6. The in-memory computing AI accelerator design architecture according to claim 5, characterized in that: The instruction sending interface is a plurality of instruction sending interfaces, which are used to simultaneously send a plurality of control instructions to the in-memory computing accelerator; The expansion interface also includes: a data dependency detection module, which is used to determine whether multiple instructions can be sent simultaneously according to the data dependency.

7. The in-memory computing AI accelerator design architecture according to claim 1, characterized in that: The unified data format includes parameterized base address Base_address, sizes Size_dim0, Size_dim1, Size_dim2, and intervals Stride_dim1, Stride_dim2.

8. A control method for an in-memory computing AI accelerator design architecture based on RISC-V architecture, characterized in that: The following steps are involved: Execute general computing tasks and control instructions on RISC-V processors; The control instruction is transmitted to the in-memory computing accelerator through the instruction sending interface of the extended interface; Parameters and results are transferred between the RISC-V processor and the in-memory computing accelerator through the source register interface and the destination register interface; A data reordering unit in the in-memory computing accelerator is used to perform data format conversion and reordering before the data is transmitted to a tensor processing unit or a vector processing unit in the in-memory computing accelerator.

9. The method according to claim 8, characterized in that The method further comprises: The control instructions received from the RISC-V processor are cached in the instruction FIFO unit of the in-memory computing accelerator to achieve parallel processing of instructions by the RISC-V processor and the in-memory computing accelerator.

10. The method according to claim 9, characterized in that The data rearrangement unit in the in-memory computing accelerator is used to perform data format conversion and rearrangement before the data is transmitted to the tensor processing unit or the vector processing unit, specifically including: According to the content of the control instruction, the instruction FIFO unit sends a request for data format conversion and rearrangement to the data rearrangement unit; The data processed by the data rearrangement unit is sent to the tensor processing unit or the vector processing unit in real time to match the needs of the tensor processing unit or the vector processing unit.

Images