A hardware-software co-optimization method for a hybrid in-memory architecture

Abstract

The application discloses a kind of hardware and software collaborative optimization methods of hybrid in-memory architecture, comprising the following steps: the joint feature representation of target AI algorithm and in-memory computing architecture is carried out, context features are extracted, and the in-memory computing unit configuration of in-memory computing architecture, neural network processor pipeline, multi-core interconnection topology and storage hierarchy interface are parameterized;Offline benchmark dataset is constructed and predictive agent model is trained, the model uses embedding method to process discrete architecture parameters, uses encoder with self-attention mechanism to learn feature association, and realizes the joint prediction of multidimensional PPA index through parallel prediction network, and introduces feasible region constraint learning mechanism in training;The trained agent model is embedded in multi-objective evolutionary algorithm, with energy efficiency, average computing power utilization, model execution delay as optimization target, while meeting chip area efficiency and power consumption constraints, searching for Pareto optimal in-memory computing architecture configuration set.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer architecture and artificial intelligence, and particularly relates to a hardware-software co-optimization method for a hybrid in-memory architecture. BACKGROUND

[0002] With the rapid development of artificial intelligence, especially deep learning, large language models (such as Transformer) and emerging complex models (such as Mamba), unprecedented challenges have been posed to the computing power, energy efficiency and bandwidth of computing hardware. The traditional von Neumann architecture faces bottlenecks in processing large-scale AI computing due to the limitations of the "memory wall" and "power wall". In-memory computing (IMC) technology, by integrating computing units directly near or inside the memory units, reduces the delay and power consumption caused by data transfer, and is considered as one of the effective ways to cope with the challenges of AI computing.

[0003] The in-memory computing chip architecture design is complex, and it contains a large number of configurable parameters and microstructure options inside, such as the type, number and precision configuration of in-memory computing units, the pipeline design of neural network processors, the interconnection topology of multi-core systems, and the storage hierarchy interface. Different combinations of these parameters constitute an extremely broad design space. At the same time, different AI algorithms have different characteristics of hardware architecture, and their model context features such as computation graph characteristics, tensor space dimensions, sparsity and mixed precision operation proportion are not the same, resulting in significant differences in the adaptability and execution efficiency of specific in-memory computing architecture configurations for different AI algorithms.

[0004] Therefore, how to scientifically and efficiently explore the in-memory computing architecture design space has become a key technical problem to be solved in the current in-memory computing chip design field. Existing technologies have made some explorations in this regard. For example, in order to accelerate design space exploration (DSE), researchers have begun to use novel generative models to directly produce excellent hardware designs. Wang et al. proposed a DSE method based on diffusion model (Diffusion Model) (Y. Wang et al., "DiffuSE: Cross-Layer Design Space Exploration of DNN Accelerator via Diffusion-Driven Optimization," Proceedings of the 56th Annual IEEE / ACM International Symposium on Microarchitecture (MICRO'23).), which learns the distribution of existing data to generate high-performance DNN accelerator configurations in an iterative denoising manner.

[0005] However, such methods still have deficiencies: first, they mainly generate macro parameters for general DNN accelerators, without delving into the micro level of the hybrid in-memory computing architecture concerned by the present application, such as the precise proportion of digital-analog hybrid computing units, ADC / DAC precision and other unique parameters, and the complex interactions among these parameters are difficult to be captured by general generation models; second, the method lacks a clear mechanism to systematically use the "context features" (such as sparsity, operation type distribution, etc.) of AI algorithms as conditional inputs to guide the generation process, thereby limiting its ability to fine-tune optimization for specific algorithm categories; finally, directly generating a configuration set that satisfies the multi-objective Pareto optimality is a challenge for diffusion models, while the present application embeds a proxy model into the framework of traditional multi-objective optimization algorithms, which can more directly perform multi-objective trade-offs.

[0006] On the other hand, in the field of hardware-software co-design, Geng et al. proposed a co-exploration framework named Gibbon (Y. Geng et al., "Gibbon: An efficient co-exploration framework of nn model and processing-in-memory architecture," 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA).), which can simultaneously search for neural network (NN) model structures and in-memory processing (PIM) hardware architectures.

[0007] However, this technology also has its inherent limitations: first, its problem setting is "co-searching", that is, changing both the algorithm and the hardware at the same time, which has its advantages when looking for completely new solutions, but cannot solve the problem of how to deeply optimize and adapt the hardware architecture for AI algorithms (such as Transformers) that are relatively fixed, which is the scenario that the present application focuses on; second, in order to make the huge "algorithm-hardware" joint search space controllable, the co-searching framework usually needs to simplify the parameter space of the hardware, making it difficult to fine-tune the parameter configuration and exploration of the microstructure of in-memory computing as the present application does; third, this method aims to find one or a few optimal "algorithm-hardware" matching pairs, while the present application aims to output a set of Pareto optimal hardware architecture solutions, providing designers with multiple choices for trade-offs among different PPA targets when meeting the needs of specific algorithms.

[0008] In summary, the existing generation model is difficult to adapt to the complexity and refinement parameters of the hybrid in-memory computing architecture, or to search for "algorithm-hardware" collaboration through simplified hardware models. The ability to refine the multi-objective optimization of the in-memory computing architecture for specific algorithm context is generally lacking. SUMMARY

[0009] The present application aims to solve the design and optimization problems caused by the broad design space and parameter configuration complexity of existing in-memory computing architectures when adapting to diversified artificial intelligence (AI) algorithms, especially complex large models such as Transformers and Mamba. A hardware-software collaborative optimization algorithm for hybrid in-memory computing architecture is provided. The core of this method is to build and use a data-driven predictive proxy model. Through a multi-objective optimization framework, the key configurable parameters and microstructure of the in-memory computing architecture are fine-tuned. Specifically, it includes: joint feature representation of target AI algorithms (such as Transformers, Mamba, etc.) and in-memory computing architecture, extracting algorithm computation graph features, tensor space dimensions, sparsity and mixed precision operation proportion, etc. Context features, and parameterizing in-memory computing unit configuration, neural network processor pipeline, multi-core interconnection topology and storage hierarchy interface of in-memory computing architecture; build an offline benchmark dataset containing architecture feasible region information and corresponding performance, power, area (Power, Performance, and Area, PPA) indicators; develop and train a high-fidelity predictive proxy model with context awareness. This model uses an embedding method to handle discrete architecture parameters, uses an encoder with self-attention mechanism to learn feature associations, and uses a parallel prediction network to achieve joint prediction of multi-dimensional PPA indicators. A feasible region constraint learning mechanism is introduced during training; embed the trained proxy model into a multi-objective evolutionary algorithm, with energy efficiency, average algorithm power utilization, model execution delay as optimization objectives, while meeting chip area efficiency and power consumption constraints, searching for Pareto optimal in-memory computing architecture configuration set. The present application can significantly improve the execution efficiency and adaptability of AI algorithms on in-memory computing chips, providing a scientific architecture optimization scheme for high-performance in-memory computing chip design, and providing hardware basis for subsequent compiler development.

[0010] The finally output verified optimization architecture of the application provides a hardware basis for subsequent compiler development. The compiler will map the AI model to the hardware instruction of the compute-in-memory unit efficiently and accurately according to the optimized architecture characteristics, including the proportion and capacity of the mixed precision unit of pure digital in-memory calculation or analog-digital hybrid in-memory calculation, the specific parameters of the internal static random access memory pipeline of the neural network processor, the multi-core heterogeneous topology (if applicable), and the optimized data path between the static random access memory and the three-dimensional stacked dynamic random access memory.

[0011] The application is implemented by at least one of the following technical solutions.

[0012] A hardware-software co-optimization method for a hybrid in-memory architecture includes the following steps:

[0013] S1, extract the context features of the target AI algorithm to form an AI algorithm context feature vector, and parameterize the configurable parameters and microstructure of the in-memory computing architecture to obtain multiple sets of in-memory computing architecture parameter configuration vectors and form an in-memory computing architecture parameter set;

[0014] S2, based on the in-memory computing architecture parameter set of step S1, construct an offline benchmark data set, which contains multiple sets of in-memory computing architecture parameter configurations sampled from the in-memory computing architecture parameter set, corresponding AI algorithm context feature vectors, PPA indicators obtained through hardware simulation, and architecture feasible region information;

[0015] S3, based on the offline benchmark data set, train a predictive proxy model with context awareness, which can predict the corresponding in-memory computing indicators according to the input in-memory computing architecture parameter configuration vector and AI algorithm context feature vector; the training process includes using embedding method to process discrete architecture parameters, using encoder with self-attention mechanism to learn the complex relationship between features, using parallel prediction network to realize joint prediction of multi-dimensional PPA indicators, and introducing feasible region constraint learning mechanism to learn the relationship between PPA of feasible configurations in supervised learning architecture, and introducing adversarial negative sampling mechanism to discriminative learning of infeasible configurations;

[0016] S4, embed the trained predictive proxy model as an evaluation function into a multi-objective optimization algorithm framework;

[0017] S5, define multiple optimization objectives for the in-memory computing architecture, which are based on PPA indicators and meet the chip area efficiency and power consumption constraints;

[0018] S6, running the multi-objective optimization algorithm, using the predictive proxy model to quickly evaluate the PPA indicators of candidate in-memory computing architecture configurations, searching and outputting a set of Pareto optimal in-memory computing architecture parameter configuration schemes that meet the preset optimization objectives.

[0019] Further, the context features include computation graph features, tensor space dimensions, sparsity, and mixed precision operation proportion.

[0020] Further, the configurable parameters of the in-memory computing architecture include in-memory computing unit configuration parameters, neural network processor pipeline parameters, multi-core interconnection topology, and memory hierarchy interface parameters.

[0021] Further, the neural network processor pipeline parameters include the organization of the internal static random access memory and the number of pipeline stages; the multi-core interconnection topology includes the connection mode and bandwidth between different cores; and the memory hierarchy interface parameters include the optimized data path characteristics and prefetch logic between the static random access memory and the three-dimensional stacked dynamic random access memory.

[0022] Further, the offline benchmark dataset distinguishes between feasible architecture configuration points and infeasible architecture configuration points, the feasible architecture configuration points refer to configurations that can successfully compile, map, and execute AI algorithms and whose PPA indicators are within a preset reasonable range, and the infeasible architecture configuration points refer to configurations that cannot meet the basic requirements due to physical constraints, compilation errors, or extremely low performance.

[0023] Further, in step S3, the feasible region constraint learning mechanism aims to minimize the prediction error of the predictive proxy model on the feasible architecture configuration points through supervised learning, and to punish the tendency of the infeasible architecture configuration points to produce good PPA indicator predictions through discriminative learning, to ensure that the predictive proxy model has accurate architecture evaluation capability;

[0024] The adversarial negative sampling mechanism is used to enhance the identification ability and robustness of the predictive proxy model for infeasible configurations.

[0025] Further, the structure of the predictive proxy model includes: mapping the discrete in-memory computing architecture parameter configuration vector through an embedding layer to a continuous vector; using an encoder layer with a self-attention mechanism to process the continuous vector to generate a unified architecture feature representation; fusing the unified architecture feature representation with the AI algorithm context feature vector; inputting the fused features into a parallel multi-head prediction network and aggregating the outputs of each prediction head through a top-level attention mechanism or a direct output method to generate a multi-dimensional PPA indicator prediction vector.

[0026] The system for implementing the hybrid in-memory architecture software and hardware co-optimization method includes:

[0027] A feature extraction and parameterization module is configured to extract a target AI algorithm context feature vector and parameterize a configurable parameter and microstructure of the in-memory computing architecture;

[0028] A dataset construction module is configured to construct an offline benchmark dataset, which contains a plurality of sets of in-memory computing architecture parameter configurations, corresponding AI algorithm context feature vectors, and PPA indicators obtained through hardware simulation, and records architecture feasible region information;

[0029] A proxy model training module is configured to train a predictive proxy model with context awareness capability, which can predict the corresponding PPA indicators according to the input in-memory computing architecture parameter configuration and AI algorithm context feature vector;

[0030] A multi-objective optimization module is configured to embed the trained predictive proxy model as an evaluation function into a multi-objective optimization algorithm framework, run the multi-objective optimization algorithm according to a plurality of predefined optimization objectives based on the PPA indicators, quickly evaluate the PPA indicators of candidate in-memory computing architecture configurations using the predictive proxy model, and search and output a set of Pareto optimal in-memory computing architecture parameter configuration schemes.

[0031] The computer device of the present application comprises a memory and a processor, the memory is electrically connected with the processor, the memory stores a computer program, and the computer program is executed by the processor to realize the method.

[0032] The computer readable storage medium of the present application stores a computer program, and the processor realizes the method when the computer program is executed by the processor.

[0033] The present application has the following advantages and effects relative to the prior art:

[0034] 1) By performing more detailed joint feature representation of AI algorithms (such as Transformer, Mamba, etc.) and in-memory computing architecture (including computation graph features, mixed precision proportion, multi-core interconnection topology, etc.), and constructing a proxy model with context awareness capability, the PPA indicators of in-memory computing architecture parameters in different AI algorithm scenarios can be more accurately captured, thereby realizing fine optimization for specific AI algorithm adaptability;

[0035] 2) Using high-fidelity predictive proxy model (with embedding, self-attention encoder, and parallel prediction network) to replace time-consuming hardware simulation for PPA evaluation, and introducing feasible region constraint learning mechanism (including adversarial negative sampling), significantly accelerating the optimization process of multi-objective optimization algorithm, improving the accuracy of model prediction and the recognition ability of feasible region, making it possible to find excellent architecture configuration in a wide and complex constraint design space;

[0036] 3) Using a multi-objective optimization framework, multiple conflicting PPA indicators (energy efficiency ratio, performance, area efficiency, delay) can be optimized simultaneously, and actual power consumption and area efficiency constraints are considered to obtain a set of Pareto optimal solutions, providing designers with a variety of trade-off options;

[0037] 4) Through fine tuning of internal key configurable parameters and microstructure of in-memory computing architecture, the comprehensive performance of in-memory computing chip in executing target AI algorithms can be significantly improved, such as higher energy efficiency ratio, computing power utilization rate, lower delay, and meeting area and power consumption requirements;

[0038] 5) The optimized architecture configuration output by the method is verified by actual tests, which can directly provide clear hardware basis for subsequent compiler development, guiding the compiler to implement efficient and accurate mapping of AI models to hardware instructions according to the optimized architecture characteristics (such as mixed precision unit ratio, neural network processor pipeline parameters, multi-core topology, storage channel, etc.), thereby forming a closed loop of software and hardware co-design. BRIEF DESCRIPTION OF DRAWINGS

[0039] Fig. 1 is an architecture schematic diagram of an embodiment of a predictive proxy model.

[0040] Fig. 2 is a flowchart of an embodiment of a soft and hardware co-optimization method of a hybrid in-memory architecture.

[0041] Fig. 3 is a flowchart of an embodiment of a multi-objective optimization method. DETAILED DESCRIPTION

[0042] The present application will be further described below in conjunction with the drawings and embodiments.

[0043] As shown in Figs. 1-3 , the soft and hardware co-optimization method of the hybrid in-memory architecture of the embodiment includes the following steps:

[0044] S1, for a target AI algorithm (such as a typical model of Transformer, Mamba, etc.), extract its model context features, including computation graph features (such as operation type distribution, dependency relationship complexity), tensor space dimension, sparsity, and mixed precision operation proportion, etc., to form an AI algorithm context feature vector a. At the same time, the configurable parameters and microstructure of the in-memory computing architecture are parameterized to obtain a plurality of in-memory computing architecture parameter configuration vectors, and the plurality of in-memory computing architecture parameter configuration vectors form an in-memory computing architecture parameter set; the configurable parameters of the in-memory computing architecture include in-memory computing unit configuration parameters (such as the proportion and capability of pure digital in-memory computing or mixed analog-digital in-memory computing mixed precision units), neural network processor pipeline parameters (such as specific parameters of internal static random access memory pipeline), multi-core interconnection topology (such as connection mode and bandwidth between different cores), and storage hierarchy interface parameters (such as optimized data path characteristics and prefetch logic between static random access memory and three-dimensional stacked dynamic random access memory).

[0045] As an embodiment, in the AI algorithm and in-memory computing architecture feature characterization stage, first, the features of the target AI algorithm are extracted. Taking the Transformer model as an example, the feature set of the target AI algorithm is defined as a=(c1, c2, …, c i ), where each feature c i represents a quantifiable feature of the AI algorithm. Taking the Transformer model as an example, the selected key features include c1-c6, where c1 is the computation graph feature (such as operation type distribution, critical path length), c2 is the model parameter quantity, c3 is the ratio of attention layer to feedforward network layer, c4 is the average tensor dimension, c5 is the mixed precision operation proportion (such as the proportion of FP16, INT8 operation), and c6 is the average sparsity. At the same time, the configurable parameters of the in-memory computing architecture are parameterized, and the in-memory computing architecture parameter set is defined as x=(p1, p2, …, p6). The selected key in-memory computing architecture parameters include p1-p6, where p1 is the number of core computing units in the mixed precision in-memory computing unit, p2 is the conversion precision of the analog-digital converter or digital-analog converter, p3 is the number of stages of the neural network processor pipeline, p4 is the number of in-memory computing cores, p5 is the link bandwidth of the on-chip network, and p6 is the prefetch buffer size of the static random access memory to the three-dimensional stacked dynamic random access memory level storage interface. The entire design space X is the set of all possible combinations of in-memory computing architecture parameters.

[0046] S2. Based on the AI ​​algorithm features and in-memory computing architecture parameters defined in step S1, construct an offline benchmark dataset D through systematic and precise hardware simulation experiments or actual hardware tests. Each entry in dataset D contains an in-memory computing architecture configuration x and its corresponding PPA (Power Performance, Performance, Area Efficiency) index y. This dataset also contains design points D that have been verified as feasible. feasible (i.e., the set of feasible architectural configuration points with effective performance metrics y) and infeasible design points D. infeasible (i.e., the set of infeasible architecture configuration points, without effective performance metrics), that is, D = D feasible ∪D infeasible A feasible architecture configuration point refers to an architecture configuration point that can successfully compile, map, and execute AI algorithms, and whose PPA (Power, Performance, and Area) metrics are within a preset reasonable range. An infeasible architecture configuration point is an architecture configuration point that cannot meet the basic requirements due to physical constraints, compilation errors, or extremely poor performance.

[0047] In the stage of building the offline benchmark dataset, a large-scale experiment was conducted using a precise hardware simulation platform (such as a Cycle-accurate simulator) to build the offline benchmark dataset D for a series of AI algorithm context feature vectors a and in-memory computing architecture parameter configurations x.

[0048] Dataset D is further divided into a set of feasible architecture configuration points D feasible and the set of infeasible architecture configuration points D infeasible The feasible architecture configuration point refers to a configuration that can successfully compile, map, and execute the AI ​​algorithm and whose PPA (Programme-Performance Ratio) is within a preset reasonable range. The infeasible architecture configuration point refers to a configuration that cannot meet the basic requirements due to physical constraints, compilation errors, or extremely poor performance. That is, D = D feasible ∪D infeasible =={(x1,y1),(x2,y2),…,((x M ,y M ))∪{x′1,x′2,…,x′ N}. M and N are the number of feasible and infeasible architecture configuration points, respectively, where y M It is a multi-dimensional PPA indicator vector that includes energy efficiency ratio, performance (such as throughput or execution time), area efficiency, etc. M It is the parameter vector of the Mth feasible architecture configuration point in the dataset, x′ N It is the parameter vector of the Nth infeasible architecture configuration point in the dataset.

[0049] S3. Train a context-aware predictive agent model f using the offline benchmark dataset constructed in step S2. θ(x), where x represents the in-memory computing architecture configuration, and represents the parameters of the surrogate model. The function of the surrogate model is to predict the corresponding PPA indicator vector (e.g. latency), i.e. The process of the surrogate model is as follows:

[0050] The input in-memory computing architecture configuration x is mapped to a continuous vector through an embedding layer, and the continuous parameters of the continuous vector can be processed through a linear layer.

[0051] The continuous vector is processed by an encoding layer containing a self-attention mechanism to learn the complex correlation between features and generate a unified architecture feature representation E arch ; The architecture feature representation E arch is fused (e.g. spliced and then passed through a fully connected layer) with the AI algorithm context feature vector a to obtain a fused feature E fused .

[0052] The fused feature E fused is input into a parallel multi-head prediction network, and each head is responsible for predicting one PPA dimension (e.g. one head predicts energy efficiency, one head predicts latency, and one head predicts area efficiency), and finally generates a multi-dimensional PPA indicator prediction vector On the feasible data points, i.e. the set of feasible architecture configuration points D feasible , the prediction PPA indicator prediction vector is minimized through supervised learning (e.g. mean square error) between the real PPA indicator y; At the same time, an adversarial negative sampling mechanism is introduced, and the infeasible configuration points, i.e. the set of infeasible architecture configuration points D infeasible , are used as negative samples for discriminative learning, and a penalty term is introduced in the loss function to suppress the tendency of the predictive surrogate model to produce good PPA indicator predictions for these infeasible points, ensuring that the model has accurate architecture evaluation capability and recognition ability for the feasible region.

[0053] During the training process, the training target of the surrogate model is to minimize the difference between the predicted performance indicator and the real performance indicator, and effectively distinguish between feasible and infeasible configurations, while avoiding overly optimistic estimates for unseen design points. The feasible region constraint learning mechanism is introduced in the training process, which aims to make the predictive surrogate model minimize the prediction error through supervised learning on the feasible architecture configuration points, and use the infeasible architecture configuration points as negative samples to punish their tendency to produce good PPA indicator predictions through discriminative learning, to ensure that the model has accurate architecture evaluation capability. The adversarial negative sampling mechanism is used to enhance the model's recognition ability for infeasible configurations and the robustness of the prediction.

[0054] The loss function The design incorporates supervised learning loss and feasible region discrimination loss, and is not limited to the set of feasible architecture configuration points D. feasible The standard supervised learning loss term also includes an additional regularization term to adjust the model's behavior on two special classes of data points:

[0055] Among them, the loss function of the standard supervised learning loss term The definition is shown in formula (2):

[0056]

[0057] in, It is in the set of feasible architecture configuration points D feasible The supervised learning loss, expressed in the norm form of the mean squared error, is used to minimize the model's dependence on the input x. i Predictive performance index f θ (x i ) and the true performance index vector y i The differences between them. It is a conservative term. Opt(f) θ ) represents an attempt to find a model f that makes the current learning model f θ (x) predicts the approximately stochastic optimization process for the optimal performance index. This is the negative sample configuration generated by the process. By minimizing the negative value of this term, the model is encouraged to predict higher costs for these potentially overestimated points, thereby suppressing overly optimistic estimates of unknown regions. α is a hyperparameter and satisfies α>0.

[0058] Final loss function As shown in formula (3):

[0059]

[0060] Where β is a hyperparameter satisfying β>0, Used to integrate the set of infeasible architecture configuration points D infeasible Information. By using a method similar to formula (2), the model is encouraged to consider these infeasible configurations x′. i The predicted higher cost value f θ (x′ i This is used to penalize the model for its tendency to predict good performance index values ​​for infeasible configuration points.

[0061] S4, in the predictive agent model f θ (x) After training, it is embedded into a multi-objective optimization algorithm framework and used as an evaluation function. The multi-objective optimization algorithm used is a decomposition-based multi-objective evolutionary algorithm. The optimization problem is defined as: finding a set of Pareto optimal in-memory computing architecture parameter configurations x. *such that a set of predefined optimization objectives O = {obj1, obj2, …, objK} are simultaneously optimized, objk(f(x, a)) is the k-th optimization objective, f(x, a) is the performance indicator vector predicted by the proxy model for a given in-memory computing configuration x and a feature vector a of the target AI algorithm, and x is the configuration of the in-memory computing architecture. k} while satisfying the constraints Area(x) ≤ γ, Power(x) ≤ μ, and Feasible(x) = 1, where γ is the area threshold, Power(.) is the PPA value of the in-memory computing architecture with configuration x, and Feasible(.) is a function that returns 1 if the configuration x is feasible and 0 otherwise. k For the k-th objective, as shown in equation (4):

[0062] x * ∈ ParetoOptimalSet(obj1(f θ (x, a)), obj2(f θ (x, a)), …, obj k (f θ (x, a))) (4)

[0063] where ParetoOptimalSet(.) denotes the Pareto optimal set, which contains a set of optimal solutions such that for any solution in the set, it is not possible to improve any other objective without sacrificing at least one objective, a is the feature vector of the target AI algorithm, and f θ (x, a) is the performance indicator vector predicted by the proxy model for a given in-memory computing configuration x and a feature vector a of the target AI algorithm. The function obj k (·) computes the value of the k-th optimization objective from the predicted performance indicator vector.

[0064] As an example, for a Transformer inference, multiple optimization objectives are defined for the in-memory computing architecture, which are based on PPA indicators and satisfy the chip area efficiency and power consumption constraints. The chip area efficiency refers to the effective computing power or performance provided per unit area, and the power consumption constraint refers to the upper limit of the total power consumption of the chip under typical workloads.

[0065] The optimization objectives can be set as: obj1 is to maximize the energy efficiency ratio, obj2 is to maximize the average computing utilization, and obj3 is to minimize the model execution delay. At the same time, the chip area and power consumption constraints need to be satisfied: Area(x) ≤ γ, Power(x) ≤ μ, and Feasible(x) = 1. γ is the area threshold, Area(.) is the chip area when the in-memory computing architecture parameter configuration is x, Power(.) is the chip PPA indicator value when the in-memory computing architecture parameter configuration is x, and Feasible(.) is whether the current in-memory computing architecture parameter configuration x is feasible, 1 represents feasible, and 0 represents infeasible.

[0066] In the iteration process of the multi-objective optimization algorithm, the performance indicator evaluation of the candidate architecture configuration is completed quickly by the proxy model, thereby significantly accelerating the optimization process. The specific objective function is shown in equation (1):

[0067]

[0068] S5, the multi-objective optimization algorithm generates a population of candidate in-memory computing architecture configurations through iteration, quickly evaluates the PPA indicators of the candidate in-memory computing architecture configurations using the predictive surrogate model, searches and outputs a set of Pareto optimal in-memory computing architecture parameter configuration schemes that meet the preset optimization objectives. In each generation, the performance indicators of each individual in the population are evaluated by the surrogate model f θ (x) Fast evaluation. The algorithm selects, crosses and mutates according to the decomposed scalar optimization sub-problems and neighborhood information, gradually converging to the Pareto front.

[0069] S6, after the multi-objective optimization algorithm is run, a set of Pareto optimal in-memory computing architecture configuration schemes are output represents the qth Pareto optimal in-memory computing architecture configuration scheme. This set of schemes represents a variety of preferred configurations that trade off between different optimization objectives. A representative scheme is selected and verified in detail using an accurate hardware simulation platform to confirm its performance indicator data under real running conditions and to verify whether the preset core technology indicators are met. Further performance attribution analysis is performed to quantify the specific contribution of the optimized configuration to reducing the computational bottleneck of the target AI model (such as Transformer) and to evaluate the performance robustness of the optimized architecture under different AI workloads and potential process disturbances.

[0070] The verified optimized architecture configuration will provide a solid hardware basis for subsequent compiler development. Compiler designers can fully utilize these optimized architecture characteristics, such as: explicit pure digital in-memory computing or mixed analog-digital in-memory computing mixed precision unit ratio and computing power, specific parameters of the neural network processor internal static random access memory pipeline (such as depth, width, access mode), determined multi-core heterogeneous topology and inter-core communication mechanism, and optimized data path and prefetch strategy between static random access memory and three-dimensional stacked dynamic random access memory. Based on these accurate hardware information, the compiler can develop more efficient mapping strategies, instruction scheduling algorithms and data arrangement schemes, thereby achieving efficient and accurate mapping of AI models (such as Transformer, Mamba, etc.) to target in-memory computing hardware instructions, and fully exploiting the potential of the optimized architecture.

[0071] The embodiment also provides a system for implementing a hardware-software co-optimization algorithm for a hybrid in-memory architecture, comprising:

[0072] A feature extraction and parameterization module is configured to perform feature extraction on a target AI algorithm (including a Transformer, Mamba, etc.) to extract features including a computational graph feature, a tensor space dimension, a sparsity, and a mixed precision operation proportion, to form an AI algorithm context feature vector; and to perform parameterization description on configurable parameters and microstructures of an in-memory computing architecture, the in-memory computing architecture parameters including in-memory computing unit configuration parameters, neural network processor pipeline parameters, multi-core interconnection topology parameters, and memory hierarchy interface parameters, to form an in-memory computing architecture parameter set;

[0073] A dataset construction module is configured to construct an offline benchmark dataset through large-scale simulation experiments, the dataset containing multiple groups of in-memory computing architecture parameter configurations, corresponding AI algorithm context feature vectors, and PPA indicators obtained through accurate hardware simulation, and recording architecture feasible region information;

[0074] A proxy model training module is configured to train a predictive proxy model with context awareness based on the offline benchmark dataset, the proxy model being capable of predicting a corresponding PPA indicator according to input in-memory computing architecture parameter configurations and AI algorithm context feature vectors;

[0075] A multi-objective optimization module is configured to embed the trained predictive proxy model as an evaluation function into a multi-objective optimization algorithm framework, to run the multi-objective optimization algorithm according to a plurality of predefined optimization objectives based on PPA indicators, to quickly evaluate PPA indicators of candidate in-memory computing architecture configurations by using the predictive proxy model, and to search and output a group of Pareto optimal in-memory computing architecture parameter configuration schemes.

[0076] The above preferred embodiment is described in detail, but only expresses one feasible implementation of the present application, and does not limit the scope of the present application. It should be noted that researchers and engineers in the field can add several modifications or improvements on the basis of the preferred embodiment within the framework of the present application, but these are within the scope of the present application. The protection scope of the present application should be subject to the appended claims.

Claims

1. A hardware-software co-optimization method for a hybrid in-memory architecture, characterized in that, Includes the following steps: S1. Extract the context features of the target AI algorithm to form the AI ​​algorithm context feature vector, and perform parameterized description of the configurable parameters and microstructure of the in-memory computing architecture to obtain multiple sets of in-memory computing architecture parameter configuration vectors, forming an in-memory computing architecture parameter set. S2. Based on the in-memory computing architecture parameter set of step S1, construct an offline benchmark dataset. The offline benchmark dataset includes multiple sets of in-memory computing architecture parameter configurations sampled from the in-memory computing architecture parameter set, corresponding AI algorithm context feature vectors, PPA indicators obtained through hardware simulation, and architecture feasible domain information. The PPA indicators are multi-dimensional indicators including energy efficiency ratio, performance, and area efficiency. S3. Based on the offline benchmark dataset, train a predictive agent model with context awareness. The predictive agent model can predict the corresponding in-memory computing metrics according to the input in-memory computing architecture parameter configuration vector and AI algorithm context feature vector. The training process includes using an embedding method to process discrete architecture parameters, using an encoder with a self-attention mechanism to learn the complex relationships between features, using a parallel prediction network to achieve joint prediction of multi-dimensional PPA metrics, and introducing a feasible domain constraint learning mechanism. While supervising the learning of PPA relationships between feasible architecture configurations, the mechanism introduces an adversarial negative sampling mechanism to discriminately learn infeasible configurations. The structure of the predictive agent model includes: mapping discrete in-memory computing architecture parameter configuration vectors into continuous vectors through an embedding layer; processing the continuous vectors with an encoder layer containing a self-attention mechanism to generate a unified architecture feature representation; fusing the unified architecture feature representation with the AI ​​algorithm context feature vector; inputting the fused features into a parallel multi-head prediction network and aggregating the outputs of each prediction head through a top-level attention mechanism or direct output method to generate a multi-dimensional PPA index prediction vector; S4. The trained predictive agent model is used as an evaluation function and embedded into the multi-objective optimization algorithm framework; S5. Define multiple optimization objectives for the in-memory computing architecture. The optimization objectives are based on the PPA index and meet chip area efficiency and power consumption constraints. S6. Run the multi-objective optimization algorithm, use the predictive surrogate model to quickly evaluate the PPA index of the candidate in-memory computing architecture configuration, search and output a set of Pareto optimal in-memory computing architecture parameter configuration schemes that meet the preset optimization objectives.

2. The hardware and software co-optimization method for a hybrid in-memory architecture according to claim 1, characterized in that, The contextual features include computation graph features, tensor space dimension, sparsity, and the proportion of mixed precision operations.

3. The hardware and software co-optimization method for a hybrid in-memory architecture according to claim 1, characterized in that, The configurable parameters of the in-memory computing architecture include in-memory computing unit configuration parameters, neural network processor pipeline parameters, multi-core interconnect topology, and storage hierarchy interface parameters.

4. The hardware and software co-optimization method for a hybrid in-memory architecture according to claim 3, characterized in that, Neural network processor pipeline parameters include the organization of its internal static random access memory and the number of pipeline stages; multi-core interconnect topology includes the connection methods and bandwidth between different cores; storage hierarchy interface parameters include the optimized data path characteristics and prefetch logic between static random access memory and three-dimensional stacked dynamic random access memory.

5. The hardware and software co-optimization method for a hybrid in-memory architecture according to claim 1, characterized in that, The offline benchmark dataset distinguishes between feasible architecture configuration points and infeasible architecture configuration points. Feasible architecture configuration points refer to configurations that can successfully compile, map, and execute AI algorithms and whose PPA indicators are within a preset reasonable range. Infeasible architecture configuration points refer to configurations that cannot meet the basic requirements due to physical constraints, compilation errors, or extremely poor performance.

6. The hardware-software co-optimization method for a hybrid in-memory architecture according to claim 1, characterized in that, In step S3, the feasible domain constraint learning mechanism aims to enable the predictive agent model to minimize the prediction error at feasible architecture configuration points through supervised learning, and to punish the tendency of infeasible architecture configuration points to produce good PPA index predictions through discriminative learning, so as to ensure that the predictive agent model has accurate architecture evaluation capabilities. The adversarial negative sampling mechanism is used to enhance the predictive agent model's ability to identify infeasible configurations and its robustness in prediction.

7. A system for implementing the hardware-software co-optimization method for a hybrid in-memory architecture as described in claim 1, characterized in that, include: The feature extraction and parameterization module is used to extract the context feature vector of the target AI algorithm and to provide a parameterized description of the configurable parameters and microstructure of the in-memory computing architecture. The dataset construction module is used to build an offline benchmark dataset. The dataset contains multiple sets of in-memory computing architecture parameter configurations, corresponding AI algorithm context feature vectors, and PPA indicators obtained through hardware simulation, and records architecture feasibility domain information. The proxy model training module is used to train a predictive proxy model with context awareness capabilities. The proxy model can predict the corresponding PPA index based on the input in-memory computing architecture parameter configuration and AI algorithm context feature vector. The multi-objective optimization module is used to embed the trained predictive agent model as an evaluation function into the multi-objective optimization algorithm framework. Based on multiple predefined optimization objectives based on PPA indicators, the multi-objective optimization algorithm is run to quickly evaluate the PPA indicators of candidate in-memory computing architecture configurations using the predictive agent model, and to search for and output a set of Pareto optimal in-memory computing architecture parameter configuration schemes.

8. A computer device comprising a memory and a processor, the memory being electrically connected to the processor, the memory storing a computer program, characterized in that: When the computer program is executed by the processor, it causes the processor to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor implements the method as described in any one of claims 1 to 6.

Images