Software and hardware joint simulation optimization system for memristor storage and calculation integration

Abstract

The invention belongs to the technical field of storage and calculation integration, and particularly discloses a memristor storage and calculation integration software and hardware joint simulation optimization system which comprises a basic configuration module, a hardware circuit simulation library, a software operator simulation library, a hardware architecture definition module, an algorithm code definition module, a model compiling module and a software and hardware performance evaluation module. According to the application, a hardware architecture can be constructed based on basic configuration parameters of hardware circuit simulation and a model of a circuit module through the hardware architecture definition module, a software algorithm can be constructed based on basic configuration parameters of software operator simulation and an operator model through the algorithm code definition module, and then the model compilation module is used for compiling the model. A software algorithm can be compiled into a storage and calculation integrated instruction sequence which can be executed on a hardware architecture, then performance evaluation can be carried out through a software and hardware performance evaluation module, a hardware performance evaluation result and a software performance evaluation result are obtained, and joint simulation optimization for memristor storage and calculation integrated software and hardware is achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of memory computing integration, and more particularly to a system for simulating and optimizing the software and hardware of memory computing integration based on memristors. BACKGROUND

[0002] Memory computing integration has become a key candidate for breaking through the "memory wall" problem under the Von Neumann architecture. The core idea is to directly implement the computing process on the memory, reducing the data transfer process between the memory and the computing unit under the traditional architecture. The existing memory computing integration has a wide variety of carrier media. New types of memories represented by memristors show great potential in the field of memory computing integration due to their high density and low power consumption. Among them, memristors refer to a series of storage media that realize data storage in the form of resistance, including resistive random access memory (RRAM), phase-change memory (PCM), and magnetic memory.

[0003] Although many algorithms have shown potential applications in memory computing based on memristors, the verification process of the algorithms on the memory computing platform is still limited by the development of the industry. On the one hand, the design of the memory computing architecture based on memristors shows diversification, and the design and exploration process of the hardware platform needs to be implemented at the architecture level. On the other hand, due to factors such as noise inherent to memristor devices and quantization noise introduced in memristor computing, different algorithm architectures and hardware characteristics will have different effects on the implementation of the algorithm. The hardware implementation of the memristor algorithm and the software computing function are strongly coupled, and changes in the hardware layer also need to be adjusted appropriately in the software algorithm to meet the needs of algorithm implementation. In related fields, there are simulation methods for the software and hardware system of memory computing integration based on SRAM, but the working principle of SRAM is very different from that of memristors, and this scheme is not suitable for simulation of memristors. Moreover, the hardware simulation in this method is based on verilog, which is more difficult to design and less flexible for architecture. How to realize the joint simulation and optimization of the software and hardware of memory computing integration based on memristors is a technical problem that needs to be solved in this field. SUMMARY

[0004] In view of the defects of the prior art, the purpose of the present application is to realize the joint simulation and optimization of the software and hardware of memory computing integration based on memristors.

[0005] To achieve the above purpose, in a first aspect, the present application provides a system for simulating and optimizing the software and hardware of memory computing integration based on memristors, comprising: a basic configuration module, a hardware circuit simulation library, a software operator simulation library, a hardware architecture definition module, an algorithm code definition module, a model compilation module, and a software and hardware performance evaluation module. The basic configuration module is used to provide basic configuration parameters for hardware circuit simulation and basic configuration parameters for software operator simulation. The hardware circuit simulation library is used to provide models of various circuit modules.​ a software operator simulation library configured to provide various operator models subject to non-ideal factors; a hardware architecture definition module configured to construct a hardware architecture based on the base configuration parameters of the hardware circuit simulation provided by the base configuration module and the models of the circuit modules provided by the hardware circuit simulation library; an algorithm code definition module configured to construct a software algorithm based on the base configuration parameters of the software operator simulation provided by the base configuration module and the operator models provided by the software operator simulation library; a model compilation module configured to compile the software algorithm into an in-memory-computing instruction sequence executable on the hardware architecture; a software and hardware performance evaluation module configured to perform performance evaluation based on the hardware architecture, the software algorithm, and the corresponding in-memory-computing instruction sequence, and obtain hardware performance evaluation results and software performance evaluation results.

[0006] In a possible implementation, the base configuration parameters of the hardware circuit simulation include a system operating frequency, a size of a memristor array, a memristor storage space, and circuit process node performance parameters. The base configuration parameters of the software operator simulation include device base performance and quantization-related parameters.

[0007] In a possible implementation, the hardware circuit simulation library includes models of in-memory-computing units and related circuits, models of memories, models of interconnection circuits, models of functional circuits, and a hardware circuit integration model. The functional circuits include shifters, accumulators, adders, and multipliers.

[0008] In a possible implementation, the software operator simulation library includes non-ideal factor models of devices and arrays, in-memory-computing unit precision compensation models, vector matrix multiplication models, and functional operator models. The functional operator models include fully connected layer operators, convolution operators, Transformer operators, and recurrent neural network operators.

[0009] In a possible implementation, the hardware architecture definition module is specifically configured to: extract the base configuration parameters of the hardware circuit simulation from the base configuration module; based on the base configuration parameters of the hardware circuit simulation, instantiate the models of the various circuit modules in the hardware circuit simulation library to obtain corresponding instances of the various circuit modules; based on the hardware circuit integration model in the hardware circuit simulation library, integrate the instances of the various circuit modules to determine a topology of the hardware architecture; based on the topology of the hardware architecture, determine an in-memory-computing instruction, the in-memory-computing instruction indicating a basic operation of the hardware architecture.

[0010] In a possible implementation, the algorithm code definition module is specifically configured to: determine the non-ideal factors of the basic circuit by extracting the basic configuration parameters of the software operator simulation from the basic configuration module; call the operator model in the software operator simulation library to construct the software algorithm.

[0011] In a possible implementation, the model compiling module is specifically configured to: abstract the data structure and data flow in the software algorithm operation process to obtain the intermediate file corresponding to the software algorithm; based on the intermediate file corresponding to the software algorithm and the basic configuration parameters of the hardware circuit simulation, configure the storage matrix in the storage-computation to the corresponding memristor array; based on the storage-computation integrated instruction of the hardware architecture and the position of the storage matrix, convert the calculation flow in the software algorithm operation process into the storage-computation integrated instruction sequence.

[0012] In a possible implementation, the software and hardware performance evaluation module is specifically configured to: based on the software algorithm and the calculation limitation caused by the non-ideal factors, perform software simulation to obtain the software performance evaluation result; based on the hardware architecture and the storage-computation integrated instruction sequence, model and count the hardware resource consumption to obtain the hardware performance evaluation result.

[0013] In a possible implementation, the system further includes a search optimization module configured to: based on the hardware architecture and the software algorithm, determine the solution space, which represents the set of software and hardware configurations; based on the defined optimization target, the solution space, and the current hardware performance evaluation result and software performance evaluation result, iteratively adjust the parameters in the basic configuration module by using the optimization algorithm to seek the optimal software and hardware collaborative design.

[0014] In a second aspect, the application provides a method for memristor storage-computation integrated software and hardware joint simulation optimization, which is applied to the system for memristor storage-computation integrated software and hardware joint simulation optimization described in the first aspect or any possible implementation of the first aspect, and includes: based on the basic configuration parameters of the hardware circuit simulation provided by the basic configuration module and the model of the circuit module provided by the hardware circuit simulation library, construct the hardware architecture; based on the basic configuration parameters of the software operator simulation provided by the basic configuration module and the operator model provided by the software operator simulation library, construct the software algorithm; compile the software algorithm into the storage-computation integrated instruction sequence that can be executed on the hardware architecture; Based on the hardware architecture, software algorithm and corresponding memory calculation integrated instruction sequence, performance evaluation is carried out to obtain hardware performance evaluation result and software performance evaluation result.

[0015] Overall, compared with the prior art, the above technical solutions conceived by the present application have the following beneficial effects: Through the hardware architecture definition module, the hardware architecture can be constructed based on the basic configuration parameters of the hardware circuit simulation provided by the basic configuration module and the model of the circuit module provided by the hardware circuit simulation library. Through the algorithm code definition module, the software algorithm can be constructed based on the basic configuration parameters of the software operator simulation provided by the basic configuration module and the operator model provided by the software operator simulation library. Then, through the model compilation module, the software algorithm can be compiled into a memory calculation integrated instruction sequence that can be executed on the hardware architecture. Then, through the software and hardware performance evaluation module, performance evaluation can be carried out based on the hardware architecture, software algorithm and corresponding memory calculation integrated instruction sequence to obtain hardware performance evaluation result and software performance evaluation result, realizing the joint simulation optimization of the memory calculation integrated software and hardware for the memristor. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a structural schematic diagram of a joint simulation optimization system for the memory calculation integrated software and hardware for the memristor provided by the embodiment of the present application; Figure 2 is a basic structure schematic diagram of a hardware circuit simulation library for the joint simulation of the memory calculation integrated software and hardware for the memristor provided by the embodiment of the present application; Figure 3 is a basic structure schematic diagram of a software operator for the joint simulation of the memory calculation integrated software and hardware for the memristor provided by the embodiment of the present application; Figure 4 is a basic flowchart of the hardware architecture configuration in the memory calculation integrated system provided by the embodiment of the present application; Figure 5 is a software code implementation flowchart in the memory calculation integrated system provided by the embodiment of the present application; Figure 6 is a data compilation flowchart in the simulation system provided by the embodiment of the present application; Figure 7 is a joint simulation framework diagram of the memory calculation integrated algorithm software and hardware in the simulation system provided by the embodiment of the present application; Figure 8 is an algorithm optimization process implementation diagram based on the joint simulation system provided by the embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present application and not to limit the present application.

[0018] In the embodiments of the present application, the words such as "exemplary" or "for example" are used to mean serving as an example, instance, or illustration, any embodiment or aspect described as "exemplary" or "for example" in the embodiments of the present application is not necessarily to be construed as preferred or advantageous over other embodiments or aspects. Rather, use of words such as "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0019] In the description of the embodiments of the present application, unless otherwise specified, "a plurality of" means two or more, for example, a plurality of processing units means two or more processing units, and the like; a plurality of elements means two or more elements, and the like.

[0020] The embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

[0021] The present application provides a hardware and software joint simulation optimization system for resistive memory computing integrated, as shown in Figure 1 The system comprises a basic configuration module 110, a hardware circuit simulation library 180, a software operator simulation library 190, a hardware architecture definition module 120, an algorithm code definition module 130, a model compilation module 140, and a hardware and software performance evaluation module 150. The basic configuration module 110 is configured to provide basic configuration parameters for hardware circuit simulation and basic configuration parameters for software operator simulation. The hardware circuit simulation library 180 is configured to provide models of various circuit modules. The software operator simulation library 190 is configured to provide various operator models, which are limited by non-ideal factors. The hardware architecture definition module 120 is configured to construct a hardware architecture based on the basic configuration parameters for hardware circuit simulation provided by the basic configuration module and the models of the circuit modules provided by the hardware circuit simulation library. The algorithm code definition module 130 is configured to construct a software algorithm based on the basic configuration parameters for software operator simulation provided by the basic configuration module and the operator models provided by the software operator simulation library. The model compilation module 140 is configured to compile the software algorithm into a resistive memory computing integrated instruction sequence executable on the hardware architecture. The hardware and software performance evaluation module 150 is configured to perform performance evaluation based on the hardware architecture, the software algorithm, and the corresponding resistive memory computing integrated instruction sequence, and obtain hardware performance evaluation results and software performance evaluation results.

[0022] It can be understood that, by means of the hardware architecture definition module, the hardware architecture can be constructed based on the basic configuration parameters of the hardware circuit simulation provided by the basic configuration module and the model of the circuit module provided by the hardware circuit simulation library, by means of the algorithm code definition module, the software algorithm can be constructed based on the basic configuration parameters of the software operator simulation provided by the basic configuration module and the operator model provided by the software operator simulation library, and then by means of the model compilation module, the software algorithm can be compiled into an operation-computation integrated instruction sequence capable of being executed on the hardware architecture, and then by means of the software and hardware performance evaluation module, the performance evaluation can be performed based on the hardware architecture, the software algorithm and the corresponding operation-computation integrated instruction sequence, the hardware performance evaluation result and the software performance evaluation result are obtained, and the joint simulation optimization for the operation-computation integrated software and hardware of the memristor is realized.

[0023] In a possible implementation, the basic configuration parameters of the hardware circuit simulation include: a running frequency of the system, a size of the memristor array, a memristor storage space and a circuit process node performance parameter. The basic configuration parameters of the software operator simulation include: a device basic performance and a quantization related parameter.

[0024] Here, the running frequency of the system is exemplarily illustrated, the working main frequency of the operation-computation integrated system is defined, and the main frequency configuration affects the trade-off between the throughput and the power consumption of the calculation.

[0025] Here, the size of the memristor array is exemplarily illustrated, the row and column dimensions (such as 128x128) of the memristor array are defined, and the array scale determines the parallelism of a single matrix operation and needs to be matched with the memory bandwidth.

[0026] Here, the memristor storage space is exemplarily illustrated, and the storage space (such as 256Kb) of the overall memristor array is defined.

[0027] Here, the circuit process node performance parameter is exemplarily illustrated, and the process used for simulation (such as 28nm CMOS process) is defined, and the process node performance affects the area and the power consumption of the operation-computation integrated system.

[0028] Here, the device basic performance is exemplarily illustrated, including: A high conductance value and a low conductance value of the memristor; A read noise of the memristor, defining a standard deviation of the Gaussian noise of the high and low conductance values; A failure ratio of the memristor, including a failure ratio at the high conductance value and a failure ratio at the low conductance value; A drift of the conductance value of the memristor, including a direction of the drift of the conductance value and a rate of the drift of the conductance value.

[0029] Quantization-related parameters are exemplarily described herein, including weight quantization bit width, input quantization bit width, digital-to-analog converter precision, analog-to-digital converter precision, and the like.

[0030] In a possible implementation, the hardware circuit simulation library includes: a model of a memory computing unit and related circuits, a model of a memory, a model of an interconnection circuit, a model of a functional circuit, and a hardware circuit integration model. The functional circuit includes a shifter, an accumulator, an adder, and a multiplier.

[0031] The memory computing unit and related circuits are described herein. The memory computing unit adopts a cross array structure, and the memristor is located at the cross point of the array. The related circuits include a controller, a digital-to-analog converter, and an analog-to-digital converter. The controller is configured to select a corresponding row and column in the memory computing unit and control the input and output of the memory computing unit. When performing a multiplication operation, the digital-to-analog converter converts the input data into a voltage vector and inputs the voltage vector into the memory computing unit to perform the multiplication operation. The current vector output by the memory computing unit is converted into data by the analog-to-digital converter, and the data is the multiplication operation result.

[0032] The model of the memory is described herein. The model of the memory includes a global data cache and a local data cache.

[0033] The interconnection circuit is described herein. The interconnection circuit is configured to transmit data between different memory computing units.

[0034] The hardware circuit integration model is configured to implement the calling process of the foregoing hardware circuit.

[0035] In a possible implementation, the software operator simulation library includes: a non-ideal factor model of a device and an array, a memory computing unit precision compensation model, a vector matrix multiplication model, and a functional operator model. The functional operator model includes a fully connected layer operator, a convolution operator, a Transformer operator, and a recurrent neural network operator.

[0036] The non-ideal factor model of the device and the array is described herein. The non-ideal factor model of the device includes, but is not limited to, the read noise of the memristor, the failure rate of the memristor, the drift of the conductance value of the memristor, and the like in the foregoing software operator simulation parameters.

[0037] The non-ideal factor model of the array includes line resistance and potential path. The line resistance is described here. There is a non-negligible resistance (line resistance) in the metal wires in the storage-computing integrated unit. When current flows through these wires, a voltage drop is generated, so that the actual received read / write voltage of the unit far from the voltage source is lower than expected, resulting in operation failure or serious error. The potential path is described here. The unselected devices in the storage-computing integrated unit are programmed by mistake due to the leakage current of the potential path.

[0038] The storage-computing integrated unit precision compensation model is used to compensate the calculation error of the array circuit through specific circuit design at the hardware level, and is also used to correct the calculation result through a correction algorithm at the software level.

[0039] The storage-computing integrated unit precision compensation model compensates at the hardware level, including but not limited to the write verification method.

[0040] The storage-computing integrated unit precision compensation model corrects at the software level, including but not limited to hardware-aware training.

[0041] In a possible implementation, the hardware architecture definition module is specifically configured to: extract a basic configuration parameter of hardware circuit simulation from the basic configuration module; based on the basic configuration parameter of hardware circuit simulation, instantiate the model of each circuit module in the hardware circuit simulation library to obtain the instance corresponding to each circuit module; based on the hardware circuit integration model in the hardware circuit simulation library, integrate the instances corresponding to each circuit module to determine the topology structure of the hardware architecture; based on the topology structure of the hardware architecture, determine the storage-computing integrated instruction, and the storage-computing integrated instruction indicates the basic operation of the hardware architecture.

[0042] The instantiation process is exemplarily described as follows. Based on the basic configuration parameter of hardware circuit simulation, the system selects the corresponding circuit model from the hardware circuit simulation library, configures the pre-configured power consumption, area and delay lookup table or calculation formula in the circuit model, and creates a specific circuit module instance. Each instance contains complete performance attributes (such as power consumption, area, delay, etc.).

[0043] The process of determining the topology structure is exemplarily described as follows. The hardware circuit integration model is called, at least one storage-computing integrated unit instance generated in the foregoing step is combined with the instances of other functional circuits according to the predefined processing core template in the hardware circuit integration model, to construct a processing core instance; further, one or more processing core instances are connected to form a processing core array by calling the interconnection circuit model; the processing core array is connected with the global data cache to establish the path between the instances in the hardware architecture.

[0044] The process of determining the compute-in-memory instruction is exemplarily illustrated as follows. The topology of the hardware architecture determined in the foregoing steps is analyzed to identify the functions, constraints and connection relationships of each circuit module instance. Based on the functions, constraints and connection relationships, a set of compute-in-memory instructions is defined for the hardware architecture, including at least: a calculation instruction for driving the compute-in-memory unit instance to perform vector-matrix multiplication and addition operation; a data movement instruction for data transmission between the processing core instance and the data cache instance; and a control instruction for controlling the execution flow of the hardware architecture.

[0045] In a possible implementation, the algorithm code definition module is specifically configured to: extracting the basic configuration parameters of the software operator simulation from the basic configuration module to determine the non-ideal factors of the basic circuit; calling the operator model in the software operator simulation library to construct the software algorithm.

[0046] The process of determining the non-ideal factors of the basic circuit is exemplarily illustrated as follows. The non-ideal factor model of the corresponding device and array is called from the software operator simulation library, and the configured non-ideal factor model is globally bound with all the functional operator models (such as full connection layer operator, convolution operator, etc.) in the software operator simulation library. This binding ensures that the behavior of any calculation layer instantiated by the functional operator will be automatically affected by these configured non-ideal effects, thereby accurately simulating the physical limitations of the hardware at the software level.

[0047] The process of calling the operator model to construct the software algorithm is exemplarily illustrated as follows. The topology structure of a deep learning network algorithm is defined through a graphical interface or a script language. For example, a convolutional neural network (CNN) including an “input layer-convolution layer-activation layer-pooling layer-full connection layer-output layer” sequence is defined. The algorithm code definition module traverses each calculation layer in the algorithm architecture defined by the user, and for each layer, it calls the corresponding functional operator model from the software operator simulation library to create an instance. For example, for the “convolution layer”, the module calls the “convolution operator” model and inputs the specific parameters of the layer (such as the number of input channels, the number of output channels, the size of the convolution kernel, the step, etc.). The module connects the operators instantiated in the previous step according to the data flow order defined in the algorithm architecture. That is, the output of the previous operator instance is specified as the input of the next operator instance, and finally a complete and directed acyclic software algorithm computation graph is constructed. This software algorithm computation graph is the mathematical representation of the software algorithm in the simulation system, and the operation of each node of the software algorithm computation graph has been constrained by the foregoing non-ideal factors.

[0048] In a possible implementation, the model compiling module is specifically configured to: abstract the data structure and data flow in the software algorithm operation process, and obtain the intermediate file corresponding to the software algorithm; based on the intermediate file corresponding to the software algorithm and the basic configuration parameter of the hardware circuit simulation, configure (split, deploy, and store) the storage matrix in the storage and calculation to the corresponding memristor array; based on the storage and calculation integrated instruction of the hardware architecture and the position of the storage matrix, convert the calculation flow in the software algorithm operation process into a storage and calculation integrated instruction sequence.

[0049] Here, the above abstract process is exemplarily described, including: traversing the foregoing generated software algorithm computation graph, identifying the nodes (such as convolution, full connection, and the like) and edges (data tensor and its dependency relationship) in the graph; decomposing the high-level functional operators in the computation graph into more bottom-level basic operation combinations that are more matched with the hardware calculation mode; converting and saving the decomposed basic operations and their new data dependency relationship into a standardized intermediate representation format that is independent of specific hardware, that is, generating an intermediate file. This file clearly describes all the basic calculation steps and data flow paths required by the algorithm, and shields the complexity of the upper algorithm framework.

[0050] Here, the configuration process of the storage matrix in the storage and calculation is exemplarily described. Extract all the storage matrices that need to be mapped to the memristor conductance value from the foregoing generated intermediate file, and obtain the size of the memristor array from the basic configuration module; based on the dimension of the storage matrix to be mapped and the dimension of the memristor array (hardware matrix), logically cut the storage matrix according to the size of the memristor array to generate a plurality of sub-matrices; and assign a specific position to each sub-matrix in the storage and calculation integrated unit instance of the hardware architecture.

[0051] For example, the dimension of the storage matrix can be padded to ensure that the dimension of the storage matrix is an integer multiple of the dimension of the memristor array, so that the storage matrix can be logically cut according to the size of the memristor array to generate a plurality of sub-matrices; and a specific position is assigned to each sub-matrix in the storage and calculation integrated unit instance of the hardware architecture, which aims to optimize data locality and maximize parallelism. Finally, the system generates and maintains a mapping table recording the correspondence between each part of the original storage matrix and its deployment position on the memristor array.

[0052] The transformation process of the above computing flow is exemplarily illustrated here. The computing flow in the aforementioned intermediate file is parsed. For each basic operation (such as vector-matrix multiplication), the aforementioned generated mapping table is queried to determine where the weight data involved in the operation is deployed on the compute-in-memory cell instances. A series of specific compute-in-memory instructions are generated for various operations. According to the original data dependency relationship of the computing flow, all generated instructions are sorted to ensure the logical correctness of the operation, and finally a complete and sequential compute-in-memory instruction sequence executable on the hardware architecture is output. The instruction sequence will be directly input to the software and hardware performance evaluation module.

[0053] In a possible implementation, the software and hardware performance evaluation module is specifically used for: based on the software algorithm and the calculation limitation caused by the non-ideal factor, performing software simulation to obtain a software performance evaluation result; based on the hardware architecture and the compute-in-memory instruction sequence, modeling and counting hardware resource consumption to obtain a hardware performance evaluation result.

[0054] The process of obtaining the software performance evaluation result is exemplarily illustrated here. The software algorithm computation graph constructed by the algorithm code definition module and bound with the non-ideal factor model is called. In the simulation environment, a preset test data set is input and the computation graph is executed. When each functional operator in the execution graph is operated, the bound non-ideal factor model is simultaneously enabled to simulate the calculation error caused by the conductance noise, drift, failure of the memristor device and the physical effects such as the line resistance and the latent path of the array at the functional level. The algorithm output data obtained after execution is compared with the baseline of the test data set. Based on the comparison result, one or more performance indicators for measuring the effectiveness of the algorithm are calculated, and the performance indicators include but are not limited to accuracy, precision or recall. The set of indicators is output as the software performance evaluation result.

[0055] The process of obtaining the hardware performance evaluation result is exemplarily illustrated here. The compute-in-memory instruction sequence generated by the model compilation module is parsed piece by piece. For each instruction in the sequence, the hardware module instance called by the instruction is identified, and the basic delay and basic energy consumption values required for executing the instruction are queried from the performance attributes of the instance. The delay in the execution process of the entire instruction sequence is accumulated, and the parallelism and dependency relationship between instructions are considered to calculate the total delay of the critical path required for completing the entire software algorithm. The energy consumption generated by executing all instructions is accumulated to calculate the total energy consumption for completing the entire software algorithm. The area attributes of all hardware module instances in the hardware architecture are counted and summed to obtain the total chip area. The total delay, total energy consumption and total area are output as the hardware performance evaluation result.

[0056] In a possible implementation, the system further includes a search optimization module 160, configured to: Based on the hardware architecture and the software algorithm, determine the solution space, which represents a set of soft and hardware configurations; Based on the (user) defined optimization target, the solution space, and the current hardware performance evaluation result and software performance evaluation result, iteratively adjust the parameters in the basic configuration module through an optimization algorithm to seek the optimal soft and hardware collaborative design.

[0057] Illustratively, iteratively adjusting the parameters in the basic configuration module through an optimization algorithm includes: (1) defining a search space and an optimization target; (2) selecting an optimization strategy and initializing; (3) performing evaluation iteration; (4) feedback and updating; and (5) looping and converging.

[0058] (1) Defining a search space and an optimization target. Receive user-defined value ranges of one or more parameters to be optimized to form a multi-dimensional design solution space; at the same time, receive user-defined optimization targets, which are constraint conditions or optimization functions of one or more performance indicators.

[0059] (2) Select an optimization strategy and initialize. Start a preset intelligent optimization algorithm, such as a genetic algorithm, a Bayesian optimization, or a simulated annealing algorithm. The algorithm selects one or a group of initial soft and hardware configuration parameter combinations from the solution space.

[0060] (3) Perform evaluation iteration. For each selected configuration parameter, call the core process of the system, i.e., sequentially trigger the hardware architecture definition module, the algorithm code definition module, the model compilation module, and the soft and hardware performance evaluation module to perform a complete simulation evaluation, thereby obtaining the software performance evaluation result and the hardware performance evaluation result under the configuration.

[0061] (4) Feedback and update. Feed back the obtained evaluation results to the optimization algorithm. The algorithm evaluates the pros and cons of the current solution according to the feedback and the preset optimization target, and generates one or a group of new configuration parameter combinations for the next generation according to its internal logic (such as selection, crossover, and mutation in a genetic algorithm).

[0062] (5) Loop and converge. Repeat (3) evaluation iteration and (4) feedback and update until the preset termination condition (such as reaching the maximum number of iterations or the performance indicator converging) is met. Finally, output one or more optimal soft and hardware configuration parameter combinations that meet the optimization target.

[0063] The application also provides a method for simulating and optimizing a memory and computing integrated soft and hardware system, which is applied to any one of the above-mentioned memory and computing integrated soft and hardware simulation and optimization systems, and includes the following steps: constructing a hardware architecture based on the base configuration parameters of the hardware circuit simulation provided by the base configuration module and the model of the circuit module provided by the hardware circuit simulation library; constructing a software algorithm based on the base configuration parameters of the software operator simulation provided by the base configuration module and the operator model provided by the software operator simulation library; compiling the software algorithm into an integrated computing and calculation instruction sequence capable of being executed on the hardware architecture; performing performance evaluation based on the hardware architecture, the software algorithm, and the corresponding integrated computing and calculation instruction sequence, to obtain hardware performance evaluation results and software performance evaluation results.

[0064] The system and method for simulating and optimizing the integrated hardware and software of the memristor computing and calculation scheme will be described below through several examples.

[0065] The deployment and implementation of the memristor computing and calculation scheme depend on the collaborative optimization process of the hardware system and the software algorithm. In the positioning of the key design bottleneck of the computing and calculation hardware, the simulation of the hardware level relying on the software simulation tool and system can greatly reduce the cost and shorten the design cycle. Based on this, the present application provides a system for simulating and optimizing the integrated hardware and software of the memristor computing and calculation scheme, which is used to support the joint simulation and optimization process of the memristor hardware design and the software algorithm implementation. The system includes a base configuration module, a hardware circuit simulation library, a software operator simulation library, a hardware architecture definition module, an algorithm code definition module, a model compilation module, a software and hardware performance evaluation module, and a search optimization module, as shown in Figure 1 .

[0066] The base configuration module is used to configure the basic hardware of the bottom design, such as the running frequency of the system, the size of the array, the memristor storage space, the basic performance of the device, the circuit process node performance, and other configuration parameters. These parameters are used to support the consistency of the simulation environment in the subsequent hardware architecture and algorithm implementation process.

[0067] The hardware circuit simulation library is a basic module for abstract implementation of the circuit modules required for calculation based on the circuit dynamics model modeling or circuit behavior characteristic simulation. These modules implement a unified calling interface based on standard programming languages (including C, C++, Python, etc.), as shown in Figure 2 .

[0068] Specifically, the circuit dynamics model refers to simulating the behavior characteristics of device electrons, holes, and other carriers to obtain the current, resistance, and voltage model of the device. The circuit behavior characteristic simulation refers to establishing a mathematical model of the external behavior of the circuit without considering the internal factors of the circuit operation.

[0069] The modules in the hardware circuit simulation library include memory-computing integrated units and their basic circuit models, memory models, register models, interconnection circuit models, functional circuit models, and hardware circuit integration models. The functional circuits include shifters, accumulators, adders, multipliers, etc. The hardware circuit integration model is to realize the calling process of the foregoing hardware circuits through the hardware programming model.

[0070] The models of the circuit modules in the hardware circuit simulation library can realize the calculation process of the circuit power consumption and delay parameters under a certain operation after instantiation, alone or jointly. The circuit modules can also realize the calculation process of the hardware calculation area through function calling.

[0071] The software operator simulation library refers to the simulation based on the dynamic model or behavior model of the memristor device, array, and circuit, and constructs the basic algorithm simulation operator model including various non-ideal factors of memory-computing circuits. As shown in Figure 3 The operator model includes non-ideal factor models of devices and arrays, memory-computing integrated unit precision compensation models, vector matrix multiplication models, and functional operator models. The functional operator model includes operator types such as full connection layer operators, convolution operators, Transformer operators, and recurrent neural network operators.

[0072] The process of software operator function simulation considers the precision conversion, data quantization, voltage drop of the memristor array, noise of device storage, and non-ideal models such as device and circuit failure based on the implementation of vector matrix multiplication of the memristor array.

[0073] The operator provides a unified calculation calling interface. Specifically, the functional operator includes an automatic construction scheme of a calculation graph in a running process, which can support the forward propagation and backward derivation processes of artificial neural network algorithms. The gradient of the backward derivation process is based on the calculation gradient of the memristor hardware level or the full-precision gradient based on the CMOS calculation unit.

[0074] The calculation operator can realize the calling process alone or jointly. The operator can realize the non-ideal characteristic evaluation of the memory-computing hardware through port calling under a specific programming language.

[0075] The precision compensation model includes a scheme for compensating for the calculation error of the array circuit through a specific circuit design at the hardware level, and a scheme for correcting the calculation result through a correction algorithm at the software level. The software layer also includes using low-precision memory-computing integrated units to realize higher-precision data type data representation and calculation process, for example, storing a matrix data on multiple arrays when storing data on the array; for input data, converting the input data into continuous binary data, etc. In the simulation implementation, the input data is directly quantized to the corresponding data precision. However, due to the limitation of non-ideal factors, the simulation of multiple memristor arrays needs to be realized separately.

[0076] The workflow of the hardware architecture definition module is shown in Figure 4 The main steps include the following.

[0077] S410: Extract the basic parameters of the hardware circuit architecture setting from the basic configuration module, which are used to support the hardware characteristics of the storage-computing integrated hardware architecture simulation.

[0078] S420: On the basis of the above hardware parameters, each computing module of the storage-computing circuit (belonging to the module in the hardware circuit simulation library) is instantiated.

[0079] S430: Define the topology of the storage-computing circuit operation. The topology refers to the connection relationship, data flow direction, communication mode, etc. of each code module (the instance obtained in step S420). The implementation of the hardware topology relationship definition is based on the hardware circuit integration model in the hardware circuit simulation library.

[0080] S440: Based on the definition of the storage-computing hardware computing architecture, a unified storage-computing integrated instruction calling interface is implemented. The storage-computing integrated instruction refers to the basic operation mode on the storage-computing integrated hardware.

[0081] After defining the hardware structure of the calculation, the storage-computing integrated operator is called to implement the basic function for the calculation algorithm to be implemented. The workflow is shown in Figure 5 The main steps include the following two steps.

[0082] S510: Extract the basic circuit non-ideal factors for software simulation from the basic configuration module, which are used to support the simulation process of the subsequent operators.

[0083] S520: Call the storage-computing integrated basic operator to implement the algorithm function.

[0084] According to the defined software algorithm, it is compiled into a basic instruction that can be executed on the storage-computing integrated hardware. The algorithm compilation process includes three steps, as shown in Figure 6

[0085] S610: First, according to the implementation of the software algorithm, the data structure and data flow in the operation process are abstracted to form the intermediate file of the algorithm. The abstracted result is saved in a fixed file format.

[0086] S620: Then, according to the structure of the algorithm abstraction and the basic configuration of the storage-computing hardware, the storage matrix in the storage-computing is split, deployed, and stored to the corresponding memristor array.

[0087] S630: According to the hardware storage-computing instruction set architecture, the calculation flow is converted into pseudo-instructions in the operation process in combination with the position of the storage matrix.

[0088] ​The flow of system software and hardware joint simulation is shown in Figure 7 The simulation process mainly contains two paths. One path is the software simulation part, which is mainly used to call the function model of the storage calculation operator to obtain the algorithm performance under the non-ideal characteristics of the memristor storage calculation. On the other hand, according to the pseudo instruction compiled by the software algorithm, the performance evaluation model of the memristor hardware is called to evaluate the hardware performance and the utilization rate of the overall hardware system. The hardware performance includes specific indicators such as energy consumption and delay in the implementation process of the algorithm.

[0089] In addition, the system also provides a software and hardware joint optimization scheme, which is realized based on the simulation results of the hardware architecture and the simulation results of the algorithm function. The optimization process is shown in Figure 8 The optimization targets include low delay and low power consumption of the memristor hardware and high accuracy of the algorithm. Therefore, according to the corresponding optimization target, an optimization function can be defined, which is determined by the following formula.

[0090] ; Where H refers to the evaluation result of the hardware layer, and A refers to the evaluation result of the software algorithm. is a system-level comprehensive evaluation function established based on the evaluation results of hardware and algorithm. The function is determined by the specific optimization target. Further, according to the implementation of the hardware computing architecture and the software algorithm, the solution space of the optimization is determined. The solution space refers to the results that may occur when the software and hardware are configured, for example, the size of the system-level cache, the bit width of the bus, the parallelism of the system, the precision of the input and weight quantization, etc. These solution spaces support the optimization process of the algorithm. Further, according to the solution space, a targeted optimization algorithm such as genetic algorithm can be implemented. The optimization algorithm will adjust the direction of optimization according to the current simulation results of software and hardware, so that the optimal solution can be reached as soon as possible. According to the optimization results generated by the optimization algorithm, the design of the hardware system and the software algorithm can be further adjusted to achieve the purpose of system-level optimization.

[0091] Overall, the system and method provided in the present application can realize the definition and implementation process of flexible storage calculation integrated architecture and the simulation process of various non-ideal factors of storage calculation integrated algorithm. Based on the present application, any storage calculation integrated architecture and storage calculation integrated software computing scheme can be realized by using the storage calculation integrated instruction. Based on the present application, the design space exploration and optimization process of the software and hardware under the determined optimization target can be realized.

[0092] It can be understood that the various digital numbers involved in the embodiments of the present application are only used for differentiation for convenience of description, and do not limit the scope of the embodiments of the present application.

[0093] Those skilled in the art can easily understand that the above description is only the preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A hardware and software co-simulation optimization system for memristor in-memory computing, characterized in that, The method comprises the following steps: The basic configuration module, hardware circuit simulation library, software operator simulation library, hardware architecture definition module, algorithm code definition module, model compilation module and software and hardware performance evaluation module are provided. The basic configuration module is used to provide the basic configuration parameters of hardware circuit simulation and the basic configuration parameters of software operator simulation. The hardware circuit simulation library is used to provide models of various circuit modules. The software operator simulation library is used to provide various operator models, which are limited by non-ideal factors. The hardware architecture definition module is used to construct a hardware architecture based on the basic configuration parameters of hardware circuit simulation provided by the basic configuration module and the models of circuit modules provided by the hardware circuit simulation library. The algorithm code definition module is used to construct a software algorithm based on the basic configuration parameters of software operator simulation provided by the basic configuration module and the operator models provided by the software operator simulation library. The model compilation module is used to compile the software algorithm into an in-memory-computing instruction sequence that can be executed on the hardware architecture. The software and hardware performance evaluation module is used to perform performance evaluation based on the hardware architecture, software algorithm and corresponding in-memory-computing instruction sequence, and obtain hardware performance evaluation results and software performance evaluation results.

2. The system according to claim 1, wherein, The basic configuration parameters of hardware circuit simulation include the running frequency of the system, the size of the memristor array, the memristor storage space and the circuit process node performance parameters. The basic configuration parameters of software operator simulation include the basic performance of the device and the quantization related parameters.

3. The system according to claim 1, wherein, The hardware circuit simulation library includes models of in-memory-computing units and related circuits, models of memories, models of interconnection circuits, models of functional circuits and hardware circuit integration models. The functional circuits include shifters, accumulators, adders and multipliers.

4. The system according to claim 1, wherein, The software operator simulation library includes non-ideal factor models of devices and arrays, in-memory-computing unit precision compensation models, vector matrix multiplication models and functional operator models. The functional operator models include fully connected layer operators, convolution operators, Transformer operators and recurrent neural network operators.

5. The system according to claim 1, wherein, The hardware architecture definition module is specifically used to: extract the basic configuration parameters of hardware circuit simulation from the basic configuration module; based on the basic configuration parameters of hardware circuit simulation, instantiate the models of various circuit modules in the hardware circuit simulation library to obtain corresponding instances of the various circuit modules; based on the hardware circuit integration model in the hardware circuit simulation library, integrate the corresponding instances of the various circuit modules to determine the topology structure of the hardware architecture; based on the topology structure of the hardware architecture, determine in-memory-computing instructions, which indicate the basic operations of the hardware architecture.

6. The system according to claim 1, wherein, The algorithm code definition module is specifically used to: determine the basic circuit non-ideal factors by extracting the basic configuration parameters of software operator simulation from the basic configuration module; call the operator models in the software operator simulation library to construct a software algorithm.

7. The system according to claim 1, wherein, The model compilation module is specifically used to: abstract the data structure and data flow in the software algorithm operation process to obtain the intermediate file corresponding to the software algorithm; based on the intermediate file corresponding to the software algorithm and the basic configuration parameters of hardware circuit simulation, configure the storage matrix in in-memory-computing to the corresponding memristor array. Based on the hardware architecture, the storage matrix and the position of the storage and calculation integrated instruction, the calculation flow in the software algorithm operation process is converted into a storage and calculation integrated instruction sequence.

8. The system according to claim 1, wherein, The software and hardware performance evaluation module is specifically configured to: Based on the software algorithm and the calculation limitation caused by the non-ideal factor, software simulation is performed to obtain a software performance evaluation result; Based on the hardware architecture and the storage and calculation integrated instruction sequence, modeling and statistics of hardware resource consumption are performed to obtain a hardware performance evaluation result.

9. The hardware-in-the-loop simulation system for the optimization of the resistive memory computing integrated system according to any one of claims 1-8, wherein, Further comprising: The search optimization module is configured to: Based on the hardware architecture and the software algorithm, a solution space is determined, and the solution space represents a set of software and hardware configurations; Based on the defined optimization target, the solution space, and the current hardware performance evaluation result and the software performance evaluation result, the parameters in the basic configuration module are iteratively adjusted by an optimization algorithm to seek an optimal software and hardware collaborative design.

10. A method for memristive storage-computing integrated hardware-software co-simulation optimization, comprising: The system is applied to the memory resistance storage and calculation integrated software and hardware joint simulation optimization system as claimed in any one of claims 1-9, and comprises: Based on the basic configuration parameters of the hardware circuit simulation provided by the basic configuration module and the model of the circuit module provided by the hardware circuit simulation library, a hardware architecture is constructed; Based on the basic configuration parameters of the software operator simulation provided by the basic configuration module and the operator model provided by the software operator simulation library, a software algorithm is constructed; The software algorithm is compiled into a storage and calculation integrated instruction sequence that can be executed on the hardware architecture; Based on the hardware architecture, the software algorithm and the corresponding storage and calculation integrated instruction sequence, performance evaluation is performed to obtain a hardware performance evaluation result and a software performance evaluation result.

Images