In-memory computing accelerator and system

Abstract

The utility model discloses an in-memory computing accelerator and system, the in-memory computing accelerator comprises a four-dimensional in-memory computing processor and a configuration control register file, and the configuration control register file is connected with the four-dimensional in-memory computing processor; the four-dimensional in-memory computing processor comprises four in-memory computing units, and the four in-memory computing units form a dynamic parallel structure; the configuration control register file is used for acquiring target configuration corresponding to a target neural network and sending the target configuration to the four-dimensional in-memory calculation processor; and the four-dimensional in-memory computing processor is used for performing configuration according to the target configuration, so that the four in-memory computing units form a target parallel structure. The four-dimensional in-memory calculation unit in the accelerator can execute at least one neural network reasoning calculation process on the input data aiming at the input data of a certain neural network, so that the utilization rate of different operator calculation requirements is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of processor architecture, and in particular to an in-memory computing accelerator and system. BACKGROUND

[0002] With the rapid development of neural networks (NNs), various advanced models based on convolutional neural networks (CNN) and Transformers are emerging and widely applied in computer vision (CV), natural language processing (NLP), and multi-modal fields. These models have achieved significant performance breakthroughs in image classification, object detection, speech recognition, machine translation, and other tasks, promoting the widespread application of artificial intelligence technology. However, various operators in CNN and Transformers exhibit significant differences in computational dimensions, computational modes, and data access methods, which further leads to different latency problems and power consumption bottlenecks during model running. Traditional computing architectures (such as CPU and GPU) often struggle to efficiently support diversified computing modes and data access modes when processing these complex models. In particular, in terms of data transmission between memory bandwidth and computing units, traditional architectures have significant performance bottlenecks, leading to low computing efficiency and increased energy consumption. To address these challenges, researchers have begun to explore more efficient and flexible hardware architectures to achieve efficient acceleration of neural network computation. Computing-in-Memory (CIM) technology, as an emerging computing paradigm, embeds computing units directly into memory to perform computing operations at data storage locations, thereby significantly reducing data transfer overhead and improving computing efficiency and energy efficiency.

[0003] However, existing in-memory computing accelerators are mostly optimized for specific models or operators, lacking flexible support for diversified computing modes and data access modes, resulting in low CIM utilization and inability to fully utilize its peak performance in actual running. This is mainly reflected in two aspects: spatial utilization and temporal utilization. For spatial utilization, existing CIM accelerators are usually designed with fixed parallel structures (such as fixed computing unit arrays), which cannot be dynamically adjusted to adapt to the diversified computing needs of different operators or models, resulting in idle states of some computing units during actual running, causing resource waste.

[0004] Therefore, the prior art still needs to be improved and developed. CONTENT OF THE INVENTION

[0005] The main purpose of the present application is to provide an in-memory computing accelerator and system, aiming to solve the problem that the existing in-memory computing accelerator is only optimized for specific models or operators, lacks flexible support for diversified computing modes and data access modes, and results in low utilization of in-memory computing in actual operation.

[0006] The first aspect of the embodiment of the present application provides an in-memory computing accelerator, which comprises a four-dimensional in-memory computing processor and a configuration control register stack connected with the four-dimensional in-memory computing processor; the four-dimensional in-memory computing processor comprises four in-memory computing units connected in series, which are switched to corresponding parallel structures according to a switching signal; the configuration control register stack is used to obtain a target switching signal and send it to the four-dimensional in-memory computing processor; the four-dimensional in-memory computing processor is used to adjust according to the target switching signal, so that the four in-memory computing units are switched to a target parallel structure.

[0007] Optionally, in an embodiment of the present application, the four-dimensional in-memory computing processor further comprises four groups of input switches, two groups of zeroth layer output switches, a group of first layer output switches and four groups of accumulation units; the four groups of input switches, the four in-memory computing units and the four groups of accumulation units are connected one-to-one, each group of zeroth layer output switches is connected with two accumulation units respectively, and the first layer output switch is connected with the two groups of zeroth layer output switches.

[0008] Optionally, in an embodiment of the present application, each in-memory computing unit comprises nine storage and calculation macro structures, each group of input switches comprises nine input switches, and each group of accumulation units comprises three accumulation units; nine storage and calculation macro structures are connected one-to-one with nine input switches, and every three storage and calculation macro structures are connected with an accumulation unit.

[0009] Optionally, in an embodiment of the present application, the in-memory computing accelerator further comprises the global buffer and the vector processor, the configuration control register stack is connected with the global buffer and the vector processor respectively, the four-dimensional in-memory computing processor is connected with the global buffer, and the global buffer is connected with the vector processor; four groups of accumulation units are connected with the vector processor, and the global buffer is connected with four groups of input switches.

[0010] Optionally, in an embodiment of the present application, the global buffer comprises three groups of static random access memories, and each group of static random access memories is connected with a corresponding input feature value distributor and an output feature value collector respectively.

[0011] Optionally, in an embodiment of the present application, the vector processor comprises a single data processor and a cross data processor, and the single data processor is connected with the cross data processor.

[0012] Optionally, in an embodiment of the present application, the vector processor further comprises a pooling unit, and the pooling unit is connected with the cross data processor.

[0013] Optionally, in an embodiment of the present application, the in-memory computing accelerator further comprises a direct memory access module, the configuration control register stack is connected with the direct memory access module, and the direct memory access module is connected with the global buffer.

[0014] The second aspect of the embodiments of the present application further provides a data processing system, wherein the data processing system comprises the in-memory computing accelerator, the compiler and the microprocessor according to any one of the above schemes, the compiler is connected with the microprocessor, and the configuration control register stack in the in-memory computing accelerator is connected with the microprocessor.

[0015] Optionally, in an embodiment of the present application, the data processing system further comprises an off-chip memory, and the off-chip memory is connected with the direct memory access module in the in-memory computing accelerator; the off-chip memory is used to provide input data to the in-memory computing accelerator.

[0016] Beneficial effects: the present application provides an in-memory computing accelerator and system, four in-memory computing units can be switched to corresponding parallel structures according to a switching signal, the configuration control register stack sends a target switching signal to the four-dimensional in-memory computing processor, and the four-dimensional in-memory computing processor adjusts according to the target switching signal, so that the four in-memory computing units are switched to the target parallel structure (to improve the space utilization rate), the four-dimensional in-memory computing unit in the accelerator of the present application can execute at least one neural network inference calculation process on the input data of a certain neural network, and improve the utilization rate when facing different operator calculation demands. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0018] Figure 1 The overall architecture schematic diagram of the preferred embodiment of the in-memory computing accelerator of the present application;

[0019] Figure 2 FIG. 1 is a schematic diagram of a 4D-CIM processor with dynamic parallelism in a preferred embodiment of an in-memory computing accelerator of the present application.

[0020] Legend of reference signs:

[0021] 100, in-memory computing accelerator; 10, in-memory computing processor; 110, in-memory computing unit; 111, compute-in-memory macro; 120, input switch; 130, accumulation unit; 141, zeroth layer output switch; 142, first layer output switch; 20, configuration control register file; 30, global buffer; 31, input eigenvalue distributor; 32, output eigenvalue collector; 40, vector processor; 50, direct memory access module. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and effects of the present application clearer and more explicit, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. The described embodiments are only possible technical implementations of the present application, not all possible implementations. Based on the embodiments in the present application, those skilled in the art can certainly combine the embodiments of the present application to obtain other embodiments without creative labor, and these embodiments are also within the protection scope of the present application.

[0023] The present application aims to solve the deficiencies of existing in-memory computing (CIM) accelerators in terms of space utilization and time utilization when facing diversified operators or networks. In terms of space utilization, existing CIM accelerators usually adopt fixed hardware design, lack flexibility, and are difficult to adapt to the diversity of different operators in terms of computing dimension, computing mode and data access mode, resulting in low utilization of CIM resources when mapping diversified tasks. In terms of time utilization, CIM accelerators are limited by the external memory access (EMA) delay of eigenvalues and the computing delay of non-CIM operators, resulting in low time utilization. The present application aims to achieve high efficiency in accelerating diversified neural network models by a flexible CIM accelerator architecture that supports dynamic utilization rate adaptation, so as to achieve high efficiency in accelerating diversified neural network models.

[0024] The following introduces the terms involved in the embodiments of the present application:

[0025] EMA (External Memory Access): external memory access;

[0026] MAC (Multiply Accumulation): multiply-accumulate operation, which is one of the basic operations in neural network computation, involving two steps of multiplication and accumulation;

[0027] 4D-CIM: Four-Dimensional In-Memory Computing, refers to the introduction of an additional parallel dimension in in-memory computing architectures to improve computational efficiency and flexibility.

[0028] SRAM: Static Random Access Memory, is a type of memory that can store data without the need for refresh circuits.

[0029] DMA: Direct Memory Access, is a technology that allows hardware subsystems (such as processors) to directly read and write to main memory (RAM) without the need for CPU intervention.

[0030] MCU: Microcontroller Unit, is a small computer that typically includes a processor, memory, and input / output peripherals for use in embedded systems.

[0031] SIMD: Single Instruction, Multiple Data, is a parallel computing technique that allows a single instruction to operate on multiple data items simultaneously.

[0032] VP: Vector Processor, is a processor specifically designed for handling vector operations.

[0033] SDP: Single Data Processor, refers to a processor unit that handles a single data element.

[0034] CDP: Cross Data Processor, refers to a processor unit that can handle operations across multiple data elements.

[0035] IF: Input Feature, refers to the data or features input into a neural network.

[0036] OF: Output Feature, refers to the results or features processed by a neural network.

[0037] IC: Input Channel, typically refers to the dimension or channel of input data in a neural network.

[0038] OC: Output Channel, typically refers to the dimension or channel of output data in a neural network.

[0039] LN (Layer Normalization): Layer normalization is a technique used in neural network training to normalize the inputs of a layer so that they have a stable distribution.

[0040] ACCU (Accumulator): Accumulator is a register used to store the result of accumulation.

[0041] H-tree: A tree-shaped network structure.

[0042] IS (Input Switch): Input switch is used to control the routing and multiplexing of input data.

[0043] OS (Output Switch): Output switch is used to control the routing and multiplexing of output data.

[0044] Online softmax algorithm and online LN algorithm: Refers to the optimization of performance in softmax and layer normalization calculation by reducing data traversal and memory access times. The following describes an in-memory computing accelerator and system according to an embodiment of the application. In view of the problem that the in-memory computing accelerator in the related art is only optimized for specific models or operators, and lacks flexible support for diversified computing modes and data access modes, resulting in low utilization of in-memory computing in actual operation, the present application provides an in-memory computing accelerator. In the in-memory computing accelerator, the four in-memory computing units can be switched to the corresponding parallel structure according to a switching signal. After the target switching signal is sent to the four-dimensional in-memory computing processor, the four-dimensional in-memory computing processor adjusts according to the target switching signal, so that the four in-memory computing units are switched to the target parallel structure (to improve the space utilization). The four-dimensional in-memory computing unit in the accelerator can execute at least one neural network inference calculation process on the input data of a neural network, improving the utilization rate when facing different operator computing demands. Thus, the technical problem of low utilization of in-memory computing in the related art is solved, which is caused by the in-memory computing accelerator only being optimized for specific models or operators, and lacking flexible support for diversified computing modes and data access modes.

[0045] The present application realizes an in-memory computing accelerator that can adaptively adjust resource utilization and support end-to-end efficient computation through adaptive CIM parallelism, adaptive feature value multiplexing, and linear operator architecture-algorithm collaborative optimization, thereby dynamically adapting to different operator computing demands and achieving efficient acceleration and optimization in diversified neural network models.

[0046] The technical solutions of the present application will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described in detail in some examples.

[0047] As shown in Figure 1 The in-memory computing accelerator 100 provided by the embodiment of the present application includes a four-dimensional in-memory computing processor 10 (i.e., 4D-CIM processor, four-dimensional in-memory computing processor 10) and a configuration control register stack 20 connected with the four-dimensional in-memory computing processor 10; the four-dimensional in-memory computing processor 10 includes four in-memory computing units 110 connected with each other, and the four in-memory computing units 110 are switched to a corresponding parallel structure according to a switching signal; the configuration control register stack 20 is used to obtain a target switching signal and send it to the four-dimensional in-memory computing processor 10; and the four-dimensional in-memory computing processor 10 is used to adjust according to the target switching signal, so that the four in-memory computing units 110 are switched to a target parallel structure.

[0048] It should be noted that the four in-memory computing units 110 form a dynamic parallel structure, and the target configuration corresponding to the target neural network is the target switching signal, the configuration control register stack 20 obtains the target configuration (i.e., the target switching signal) corresponding to the target neural network and sends it to the four-dimensional in-memory computing processor 10, and the four-dimensional in-memory computing processor 10 is configured according to the target configuration, so that the four in-memory computing units 110 are switched to form the target parallel structure.

[0049] In an embodiment of the present application, as shown in Figure 1 The in-memory computing accelerator 100 further includes the global buffer 30 and the vector processor 40, the configuration control register stack 20 is connected with the global buffer 30 and the vector processor 40 respectively, the four-dimensional in-memory computing processor 10 is connected with the global buffer 30, and the global buffer 30 is connected with the vector processor 40.

[0050] Specifically, the 4D-CIM processor is connected with the global buffer 30, the input feature value distributor 31, and the output feature value collector 32, and implements a multiply-accumulate (MAC) operation without precision loss in the digital domain, which is the core computing unit of the accelerator. The spatial utilization rate is improved by the 4D-CIM processor with dynamic parallelism, that is, the utilization rate of CIM resources is optimized by dynamically adjusting the CIM parallelism level (within a single CIM core and in two dimensions across multiple CIM cores), and this flexibility is supported by the IF and OF networks based on H-trees and the reconfigurable input switch 120 (IS) and output switch (OS).

[0051] Specifically, the global buffer 30 is connected with the input feature value distributor 31, the output feature value collector 32, and the DMA module, and is used to store the input feature values and the output feature values, and support the reuse and fast access of the feature values. The flexible memory system of adaptive feature reuse formed by the global buffer 30 and the 4D-CIM processor can improve the time utilization: in the local register reuse, the reuse of the feature values is realized through the IS, the OS and the accumulation unit 130 (ACCU), and the data access is reduced; in the global buffer 30 reuse, the compiler and the hardware controller cooperatively manage, support multiple reuse modes (such as IF reuse, OF reuse, layer fusion reuse), and reduce the EMA overhead.

[0052] Specifically, the vector processor 40 (VP) is connected with the global buffer 30 and the DMA module, and performs the element-by-element operation, the softmax, the layer normalization (LN) and the pooling operation, and processes the non-CIM operator. The vector processor 40 optimized through the algorithm-architecture cooperation improves the time utilization: the algorithm optimization, the online softmax and the online LN algorithm are designed, and the data traversal and the memory access times are reduced; the architecture optimization, the pipelining parallel design is used between two traversal pipelines by using the online algorithm, and the calculation throughput of the non-CIM operator is improved.

[0053] Specifically, the input feature value distributor 31 and the output feature value collector 32 are connected with the global buffer 30 and the 4D-CIM processor respectively, and the distributor and the collector are responsible for the loading, distribution and collection of the feature values, and optimize the data flow. The input feature value distributor 31 is responsible for reading the input feature values from the global buffer 30, and distributing them to the CIM processor for calculation according to a specific order, which helps to ensure that the data is processed in the correct order and format. The output feature value collector 32 is responsible for receiving the calculation results generated by the CIM processor, and writing them back to the global buffer 30 or passing them to other processing units, which helps to ensure the correctness and integrity of the calculation results.

[0054] Specifically, the configuration control register heap 20 is connected with the 4D-CIM processor, the vector processor 40 and the DMA module, and is used to receive the configuration from the customized compiler and load it into the register heap at runtime, so as to realize the dynamic configuration.

[0055] In the embodiments of the present application, through the 4D-CIM processor design of dynamic parallelism, the accelerator can flexibly adapt to the calculation requirements of different operators, significantly improving the space utilization rate of the CIM; through the flexible memory system supporting adaptive eigenvalue multiplexing and the vector processor 40 design of algorithm-architecture collaborative optimization, the off-chip memory access (EMA) overhead of the eigenvalue and the calculation delay of the nonlinear operator are effectively reduced, thereby greatly improving the time utilization rate of the CIM. Efficient resource utilization is achieved in both space and time dimensions, providing efficient acceleration support for diversified neural network models.

[0056] In an embodiment of the present application, the in-memory computing accelerator 100 further includes a direct memory access (DMA) module (i.e., a DMA module), the configuration control register stack 20 is connected with the direct memory access module 50, and the direct memory access module 50 is connected with the global buffer 30; the off-chip memory is used to provide input data to the in-memory computing accelerator 100.

[0057] Specifically, the DMA module is connected with the global buffer 30 and the AXI4 interface, can efficiently manage memory access, and reduce data transfer delay.

[0058] In an embodiment of the present application, as shown in Figure 1 and Figure 2 The four-dimensional in-memory computing processor 10 further includes four groups of input switches 120, two groups of zero-layer output switches 141, a group of first-layer output switches 142, and four groups of accumulation units 130; the four groups of input switches 120, the four in-memory computing units 110, and the four groups of accumulation units 130 are connected one-to-one, each group of zero-layer output switches 141 is connected with two accumulation units 130, and the first-layer output switch 142 is connected with two zero-layer output switches 141.

[0059] In an embodiment of the present application, each in-memory computing unit 110 includes nine memory-computing macro structures 111, each group of input switches 120 includes nine input switches 120, and each group of accumulation units 130 includes three accumulation units 130; the nine memory-computing macro structures 111 are connected one-to-one with the nine input switches 120, and every three memory-computing macro structures 111 are connected with one accumulation unit 130. The four groups of accumulation units 130 are connected with the vector processor 40, and the global buffer 30 is connected with the four groups of input switches 120.

[0060] Specifically, the 4D-CIM processor includes four CIM cores (i.e., in-memory computing units 110), each with high parallel processing capabilities. Each CIM core contains nine CIM macros (i.e., in-memory macro structures 111), interconnected via an internal network. These CIM macros are SRAM-based pure digital in-memory arrays used to perform lossless multiply-accumulate (MAC) operations in the digital domain. To support flexible parallelism adjustment, the 4D-CIM processor is equipped with an H-tree-based input / output network and reconfigurable input switches 120 (IS) and output switches (OS). These switches allow dynamic adjustment of the parallelism level within a single CIM core and across multiple CIM cores in two dimensions. IS and OS are connected to the input and output terminals of the CIM macros, respectively, to control data inflow and outflow and to enable flexible parallelism adjustment. The input / output switches (IS / OS) support flexible parallelism adjustment, enabling the 4D-CIM processor to dynamically adjust computing resources according to different computing needs, thereby improving space utilization.

[0061] Furthermore, the level of CIM parallelism can be achieved in two dimensions within a single CIM core. Figure 2 (1D, 2D) and / or two dimensions across multiple CIM cores ( Figure 2 The CIM is dynamically adjusted in both 3D and 4D dimensions to optimize CIM utilization. This flexibility is supported by H-tree-based IF and OF networks and reconfigurable input switches (IS) and output switches (OS). For example, the parallelism of CIM in the third and fourth dimensions can be flexibly configured to accumulate via adders in the input channel (IC) direction or sequentially extract the generated OF via multiplexers in the output channel (OC) direction.

[0062] In one embodiment of this application, the global buffer 30 includes three sets of static random access memory, each set of static random access memory being connected to a corresponding input feature value distributor 31 and output feature value collector 32.

[0063] Specifically, the global buffer 30 includes 18 pseudo-dual-port SRAMs, divided into 3 columns, connected by internal buses or crossbars, which are used to store input and output feature values, aiming to improve the efficiency and flexibility of data access. The input feature value distributor 31 loads and distributes feature values from the global buffer 30 to the CIM processor in a specific order. The calculation results generated by the CIM are further processed by the output feature value collector 32 and written back to the global buffer 30. The 32-way SIMD vector processor 40 (Vector Processor, VP) includes single data processors (Single Data Processor, SDP) for element-wise operations, cross data processors (Cross Data Processor, CDP) for softmax and layer normalization (Layer Normalization, LN), and pooling units. The DMA module is used to handle memory access through the AXI4 interface. The custom compiler generates optimal configurations for the target network through automatic tuning, which are loaded into the configuration register stack by the microcontroller (Microcontroller Unit, MCU) at runtime to achieve the required operations.

[0064] Further, with the flexible memory system (global buffer and 4D-CIM processor cooperation) with adaptive feature multiplexing, EMA in different operators (with different data access patterns) can be reduced. Throughout the memory hierarchy, from local registers to the global buffer 30, extensive feature multiplexing options are provided: multiplexing in local registers is achieved through IS, OS, and accumulation units 130 (ACCU), for example, registers in IS implement IF multiplexing during the sliding process of convolution operations, and registers in OS implement partial sum multiplexing. The adaptive multiplexing mechanism in the global buffer 30 is managed by the compiler (through the configuration registers) and the hardware controller in coordination, supporting multiple multiplexing modes such as input feature (IF) multiplexing, output feature (OF) multiplexing, and layer fusion multiplexing. Among them, OF multiplexing is achieved through read-modify-write modules near the buffer, which can complete the accumulation operation of partial sums in two cycles; layer fusion multiplexing is achieved through flexible allocation of the buffer, which directly multiplexes the output feature (OF) of the previous layer through buffer exchange between adjacent layers, thereby reducing data transfer and memory access overhead. This hybrid management mode significantly improves data multiplexing efficiency and optimizes computing performance.

[0065] In an embodiment of the present application, the vector processor 40 includes a single data processor and a cross data processor, and the single data processor is connected with the cross data processor.

[0066] In an embodiment of the present application, the vector processor 40 further comprises a pooling unit connected with the cross data processor.

[0067] It can be understood that if there is no corresponding calculation in some neural networks, the corresponding calculation unit can be skipped by configuring the register, such as when there is no pooling operation in the network, the result of the cross data processor can be directly transmitted to the global buffer 30. The vector processor 40 should process 32 data each time; for a larger neural network, the processing process needs to be repeated many times to complete a reasoning, that is, different neural networks all need to go through at least one process of the neural network calculation method of the present application, but the number of reasoning times of different types of neural networks is not completely the same.

[0068] Specifically, in the vector processor 40, the single data processor (SDP) is used for element-by-element operation, that is, the same operation is performed on each element in the vector; the cross data processor (CDP) is used for complex calculations such as softmax and layer normalization (LN) that require cross-element operations; the pooling unit is used to perform pooling operations such as max pooling or average pooling to reduce the dimension of data and extract key features. The SDP, CDP and pooling unit are connected with each other inside the vector processor 40 through a high-speed internal bus or data channel to realize high-speed transmission and sharing of data; the vector processor 40 is connected with external structures such as the global buffer 30, the DMA module and the configuration control register stack 20. The single data processor is used to perform element-by-element operations such as addition, multiplication, comparison, etc., which can independently calculate each element in the vector; the cross data processor is used to process complex calculations that require cross-element operations, such as softmax and layer normalization, which usually involve combining and transforming multiple elements in the vector to extract useful information or reduce the dimension of data; the role of the pooling unit is to perform pooling operations to reduce the dimension of data and extract key features to reduce the amount of calculation and improve the generalization ability of the model.

[0069] Further, for the widely used softmax and LN layers in the Transformer, the algorithm-architecture collaborative optimization method is used: in the algorithm optimization, an online LN algorithm based on the online softmax algorithm is designed. In the conventional LN, the IF vector needs to be traversed three times to calculate the mean, variance and final OF value respectively. In contrast, the online LN algorithm only needs two traversals, and the mean and variance are calculated in a single traversal. Through the online algorithm, the data traversal and memory access in LN and softmax are reduced by 1 / 3. In the architecture optimization, the above online algorithm is used, and pipeline parallel design is further used between the two traversal pipelines to further improve the calculation throughput of non-CIM operators.

[0070] In the present application, by using the 4D-CIM processor with dynamic parallelism, the accelerator can flexibly adapt to the calculation requirements of different operators, significantly improving the space utilization of CIM; at the same time, with the help of a flexible memory system supporting adaptive eigenvalue reuse, and through the vector processor 40 designed by algorithm-architecture collaborative optimization, the off-chip memory (EMA) overhead of eigenvalues and the calculation delay of nonlinear operators are effectively reduced, thereby greatly improving the time utilization of CIM. This design realizes efficient resource utilization in both space and time dimensions, providing efficient acceleration support for diversified neural network models.

[0071] Based on the above embodiments, the present application further provides a data processing system, wherein the data processing system comprises the in-memory computing accelerator 100, the compiler and the microprocessor according to any one of the above schemes, the compiler is in communication connection with the microprocessor, and the configuration control register stack 20 in the in-memory computing accelerator 100 is connected with the microprocessor.

[0072] It should be noted that the optimal configuration is generated by the customized compiler to realize dynamic utilization rate adaptation; the microprocessor (MCU) loads the configuration into the configuration register stack to realize the required operation.

[0073] Further, the customized compiler generates the optimal configuration for the target network through automatic tuning, and these configurations are loaded into the configuration register stack by the microprocessor (Microcontroller Unit, MCU) at runtime to realize the required operation.

[0074] The data processing system provided by the present application has all the beneficial effects of the above-mentioned technical solutions, and thus further description is omitted here.

[0075] In the description of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood in a broad sense, for example, can be fixedly connected, or can be detachably connected, or can be integrated; can be mechanically connected, or can be electrically connected or can be in communication with each other; can be directly connected, or can be indirectly connected through an intermediate medium; can be the internal connection of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meanings of the above-mentioned terms in the present application can be understood according to the specific circumstances.

[0076] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate relative or positional relationships based on the orientation or position shown in the drawings, and are merely intended to facilitate the description of the present application and simplify the description, and therefore cannot be understood as indicating or implying that the devices or elements indicated must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0077] In addition, the terms "first", "second", "third", "fourth" and the like in the description of the present application, if any, are used for distinguishing between similar objects and do not necessarily have an implied sequential or chronological precedence. It is to be understood that the data so distinguished can occur in any sequence.

[0078] It should be noted that in the present application, unless otherwise explicitly specified and limited, the "on" or "under" of the first feature to the second feature can be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the "over", "above" and "on" of the first feature to the second feature can be that the first feature is directly above or obliquely above the second feature, or only indicates that the horizontal height of the first feature is higher than that of the second feature. The "under", "below" and "under" of the first feature to the second feature can be that the first feature is directly below or obliquely below the second feature, or only indicates that the horizontal height of the first feature is less than that of the second feature.

[0079] The terms "first", "second", "third", "fourth" and the like in the description of the present application and in the claims, if any, are used for distinguishing between similar objects and do not necessarily have an implied sequential or chronological precedence. It is to be understood that the data so distinguished can occur in any sequence.

[0080] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present specification, the exemplary description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0081] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An in-memory computing accelerator, comprising: The in-memory computing accelerator comprises a four-dimensional in-memory computing processor and a configuration control register stack connected with the four-dimensional in-memory computing processor; The four-dimensional in-memory computing processor comprises four in-memory computing units connected in series, and the four in-memory computing units are switched into corresponding parallel structures according to a switching signal; The configuration control register stack is configured to obtain a target switching signal and send the target switching signal to the four-dimensional in-memory computing processor; The four-dimensional in-memory computing processor is configured to adjust according to the target switching signal, so that the four in-memory computing units are switched into a target parallel structure.

2. The in-memory computing accelerator of claim 1, wherein, The four-dimensional in-memory computing processor further comprises four groups of input switches, two groups of zeroth-layer output switches, one group of first-layer output switches, and four groups of accumulation units; The four groups of input switches, the four in-memory computing units, and the four groups of accumulation units are connected one-to-one, each group of zeroth-layer output switches is connected with two accumulation units respectively, and the first-layer output switch is connected with the two groups of zeroth-layer output switches.

3. The in-memory computing accelerator of claim 2, wherein, Each in-memory computing unit comprises nine memory-computing macro structures, each group of input switches comprises nine input switches, and each group of accumulation units comprises three accumulation units; The nine memory-computing macro structures are connected one-to-one with the nine input switches, and each three memory-computing macro structures are connected with one accumulation unit.

4. The in-memory computing accelerator of claim 2, wherein, The in-memory computing accelerator further comprises a global buffer and a vector processor, the configuration control register stack is connected with the global buffer and the vector processor respectively, the four-dimensional in-memory computing processor is connected with the global buffer, and the global buffer is connected with the vector processor; The four groups of accumulation units are connected with the vector processor, and the global buffer is connected with the four groups of input switches.

5. The in-memory computing accelerator of claim 4, wherein, The global buffer comprises three groups of static random access memories, and each group of static random access memories is connected with a corresponding input feature value distributor and an output feature value collector respectively.

6. The in-memory compute accelerator of claim 4, wherein, The vector processor comprises a single data processor and a cross data processor, and the single data processor is connected with the cross data processor.

7. The in-memory computing accelerator of claim 6, wherein, The vector processor further comprises a pooling unit, and the pooling unit is connected with the cross data processor.

8. The in-memory compute accelerator of claim 4, wherein, The in-memory computing accelerator further comprises a direct memory access module, the configuration control register stack is connected with the direct memory access module, and the direct memory access module is connected with the global buffer.

9. A data processing system, characterized by The data processing system comprises the in-memory computing accelerator of any one of claims 1 to 8, a compiler, and a microprocessor, the compiler is connected in communication with the microprocessor, and the configuration control register stack in the in-memory computing accelerator is connected with the microprocessor.

10. The data processing system of claim 9, wherein, The data processing system further comprises an off-chip memory connected with the direct memory access module in the in-memory computing accelerator; and the off-chip memory is configured to provide input data to the in-memory computing accelerator.

Images