Storage and calculation fusion architecture parameter configuration method and device of neural network model, and medium

Abstract

The invention provides a storage and calculation fusion architecture parameter configuration method for a neural network model, and is applied to the technical field of electronic design automation and integrated circuits. Comprising the steps that pruning is conducted on a first feature space according to a feature importance vector of the storage and calculation fusion architecture, a second feature space is obtained, and a plurality of elements included in the feature importance vector are used for representing relative importance of a plurality of hardware parameters of the storage and calculation fusion architecture; determining a plurality of first features in the second feature space, wherein the first features comprise parameter values of the hardware parameters; for each first feature, according to the first feature and the model parameter feature of the neural network model, predicting the operation performance of the neural network model on the storage and calculation fusion architecture to obtain a first performance value corresponding to each first feature; determining a target feature in the plurality of first features according to the plurality of first performance values; and performing parameter configuration on the storage and calculation fusion architecture according to the parameter value of each hardware parameter included in the target feature.

Description

Technical Field

[0001] The present disclosure relates to the fields of electronic design automation and integrated circuit technologies, and particularly to a method, device, and medium for configuring the parameters of a memory-computation fusion architecture of a neural network model. Background Art

[0002] As a new type of computing architecture, the design parameters of the memory-computation fusion architecture are not completely the same as those of the traditional architecture. When designing a memory-computation fusion neural network acceleration architecture, not only the cache and computing power configuration problems existing in the traditional neural network acceleration architecture need to be considered, but also the storage and bandwidth resource configuration problems unique to the memory-computation fusion architecture need to be considered. In related technologies, the rich experience of architects is relied on to configure the various parameters of the memory-computation fusion architecture.

[0003] In the process of implementing the concept of the present disclosure, at least the following problems exist in related technologies: Different application scenarios have different performance requirements and hardware constraints, and as the scale of the memory-computation fusion architecture increases, its design parameters become more complex. Therefore, an efficient method for exploring the design space of the memory-computation fusion architecture is needed to provide a more efficient memory-computation fusion architecture for the neural network model. Summary of the Invention

[0004] In view of the above problems, the present disclosure provides a method, device, and medium for configuring the parameters of a memory-computation fusion architecture of a neural network model.

[0005] According to a first aspect of the present disclosure, there is provided a method for configuring parameters of a memory-computation fusion architecture for a neural network model, the method including: pruning a first feature space according to a feature importance vector of the memory-computation fusion architecture to obtain a second feature space, where the feature importance vector includes a plurality of elements respectively used to represent the relative importance of a plurality of hardware parameters of the memory-computation fusion architecture, and the first feature space is a value space of the plurality of hardware parameters; determining a plurality of first features in the second feature space, where the first feature includes a parameter value of each hardware parameter; for each first feature, predicting the running performance of the neural network model on the memory-computation fusion architecture according to the first feature and the model parameter features of the neural network model to obtain a first performance value corresponding to each first feature; determining a target feature among the plurality of first features according to the plurality of first performance values; and configuring the parameters of the memory-computation fusion architecture according to the parameter values of the respective hardware parameters included in the target feature.

[0006] According to an embodiment of the present disclosure, the second feature space includes a plurality of second features; determining a plurality of first features in the second feature space includes: performing a weighted transformation on the plurality of second features according to a preset weight vector to obtain a plurality of third features; and determining a plurality of representative third features among the plurality of third features to obtain a plurality of first features.

[0007] According to an embodiment of the present disclosure, determining a plurality of representative third features from a plurality of third features to obtain a plurality of first features includes: clustering the plurality of third features to obtain feature clusters; and determining a plurality of first features that are closer to the clustering centers from the feature clusters.

[0008] According to an embodiment of the present disclosure, pruning a first feature space according to a feature importance vector of a memory-computation fusion architecture to obtain a second feature space includes: normalizing the feature importance vector to obtain a target feature importance vector; determining the number of parameter candidate values of each hardware parameter according to each element of the target feature importance vector, where the number of parameter candidate values of each hardware parameter is positively correlated with the element; and uniformly sampling in the candidate value set of each hardware parameter according to each number of parameter candidate values to obtain a second feature space.

[0009] According to an embodiment of the present disclosure, predicting the running performance of a neural network model on a memory-computation fusion architecture according to a first feature and a model parameter feature of the neural network model to obtain a first performance value corresponding to each first feature includes: concatenating the model parameter feature and the first feature to obtain a model input feature; and inputting the model input feature into a trained performance prediction model to output a first performance value.

[0010] According to an embodiment of the present disclosure, the method further includes: extracting a plurality of fourth features from the first feature space; for each fourth feature, simulating the running performance of the neural network model on the memory-computation fusion architecture according to the fourth feature and the model parameter feature to obtain a second performance value corresponding to each fourth feature; determining a performance prediction model according to each fourth feature and the second performance value corresponding to each fourth feature; and quantifying the influence degree of each hardware parameter on the running performance according to the weight allocation mechanism inherent in the performance prediction model to obtain a feature importance vector.

[0011] According to an embodiment of the present disclosure, determining a performance prediction model according to each fourth feature and the second performance value corresponding to each fourth feature includes: normalizing each second performance value to obtain a plurality of third performance values; classifying each fourth feature and the second performance value corresponding to each fourth feature into a plurality of preset equal-width bins labeled with bin indices according to the plurality of third performance values, where the bin index is the feature importance vector; training a random forest classifier with each fourth feature as a training sample and the bin index corresponding to each fourth feature as a label to obtain a performance prediction model.

[0012] According to an embodiment of the present disclosure, the method further includes: performing feature encoding on the topological structure of the modular substructure in the neural network model to determine a model parameter feature.

[0013] The second aspect of the present disclosure provides an electronic device, including: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0014] The third aspect of the present disclosure further provides a computer-readable storage medium, on which a computer program or instruction is stored, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0015] According to the embodiments of the present disclosure, a feature importance vector is introduced to prune the first feature space representing hardware space parameters. By quantifying the dynamic influence weight of hardware parameters on the performance of the neural network model, the value range of hardware parameters with higher importance is retained in the second feature space, and the search space is compressed from exponential level to linear level. Multiple first features are determined in the second feature space to determine, from the second feature space, first features with higher importance composed of the parameter values of each hardware parameter. The first features and the model parameter features of the neural network model are combined to predict the running performance of the neural network model on the in-memory computing fusion architecture, and a first performance value corresponding to each first feature is obtained. Compared with the traditional method of optimizing neural network model parameters or hardware parameters in isolation, joint feature modeling can achieve cross-layer collaboration and break through the performance bottleneck of single-dimensional optimization. A target feature is determined from multiple first features according to multiple first performance values, and the in-memory computing fusion architecture is configured with parameter values of each hardware parameter included in the target feature, so as to provide a more efficient in-memory computing fusion architecture for the neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, the above content and other objects, features and advantages of the present disclosure will become clearer. In the drawings:

[0017] Figure 1 A schematic diagram of an application scenario of a method for configuring in-memory computing fusion architecture parameters of a neural network model according to an embodiment of the present disclosure is schematically shown.

[0018] Figure 2 A flowchart of a method for configuring in-memory computing fusion architecture parameters of a neural network model according to an embodiment of the present disclosure is schematically shown.

[0019] Figure 3 A schematic diagram of an in-memory computing fusion architecture according to an embodiment of the present disclosure is schematically shown.

[0020] Figure 4 A schematic diagram of determining a feature representation of a modular substructure according to an embodiment of the present disclosure is schematically shown.

[0021] Figure 5A schematic diagram schematically shows a modular sub - structure of a neural network model according to an embodiment of the present disclosure.

[0022] Figure 6 A flowchart schematically shows the determination of target features according to an embodiment of the present disclosure.

[0023] Figure 7 A block diagram schematically shows a structure of a device for configuring in - memory computing architecture parameters of a neural network model according to an embodiment of the present disclosure.

[0024] Figure 8 A block diagram schematically shows an electronic device suitable for implementing a method for configuring in - memory computing architecture parameters of a neural network model according to an embodiment of the present disclosure. Detailed implementation manners

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well - known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

[0026] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] In the case of using expressions such as "at least one of A, B, and C, etc.", generally, it should be interpreted according to the meaning usually understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but is not limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0029] In the industrial field, the exploration of the architecture design space has traditionally relied on the rich experience of architects. However, this experience-driven approach has gradually shown its inadequacy in the design of emerging processor architectures. With the explosion of the complexity of modern processor designs, the design space has expanded exponentially, and it has become difficult to cover all potential designs solely based on manual experience. In academia, attempts have been made to achieve rapid exploration by constructing lightweight interpretable models based on analysis, using mathematical formulas to describe the relationships between architecture design, workload, and design goals. However, this method not only requires a large amount of professional knowledge but also is difficult to generalize to new types of processors.

[0030] In recent years, the rapid development of statistics and machine learning technologies has injected new vitality into the exploration of the architecture design space. The data-driven modeling method treats the architecture as a "black box" and uses a large amount of data generated by simulation to build a surrogate model, thereby achieving performance prediction for a large number of design schemes at a relatively low cost. Compared with the traditional experience-based method, the data-driven method significantly improves the exploration efficiency. Its surrogate model can complete thousands of predictions within seconds and is embedded in the inner loop of the optimization process, providing strong support for the rapid search of the architecture design space. Nevertheless, the exponential growth of the design space still poses a non-negligible challenge to the optimization algorithm, imposing higher requirements on the scalability of the algorithm.

[0031] Applying neural network models to the in-memory computing architecture to form an in-memory computing neural network accelerator, there are obvious limitations in the research paradigm for exploring the architecture design space of past accelerators. Early work mainly focused on the independent optimization of hardware parameters, such as simply improving system performance by increasing the density of computing units or expanding the memory bandwidth, but ignored the deep impact of the dynamic characteristics and algorithm features of the neural network workload on hardware efficiency. This fragmented optimization strategy has exposed significant defects in practice: when the hardware parameters mismatch the algorithm's computing characteristics, it is not only difficult to fully exploit the performance potential of the hardware but may also cause serious resource waste and a decrease in the energy efficiency ratio.

[0032] Embodiments of the present disclosure provide a method for configuring parameters of a memory-computation fusion architecture of a neural network model. Pruning a first feature space according to a feature importance vector of the memory-computation fusion architecture to obtain a second feature space, where the feature importance vector includes a plurality of elements respectively used to characterize the relative importance of a plurality of hardware parameters of the memory-computation fusion architecture, and the first feature space is the value space of the plurality of hardware parameters; determining a plurality of first features in the second feature space, where the first feature includes the parameter values of the respective hardware parameters; for each of the first features, predicting the running performance of the neural network model on the memory-computation fusion architecture according to the first feature and the model parameter features of the neural network model to obtain a first performance value corresponding to each of the first features; determining a target feature among the plurality of first features according to the plurality of first performance values; and configuring the parameters of the memory-computation fusion architecture according to the parameter values of the respective hardware parameters included in the target feature.

[0033] Figure 1 Schematically shows an application scenario diagram of a method for configuring parameters of a memory-computation fusion architecture of a neural network model according to an embodiment of the present disclosure.

[0034] As Figure 1 shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0035] Users may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0037] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process data such as user requests received, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0038] It should be noted that the method for configuring the memory-computation fusion architecture parameters of the neural network model provided by the embodiments of the present disclosure can generally be executed by the server 105. The method for configuring the memory-computation fusion architecture parameters of the neural network model provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0039] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0040] are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. Figure 1 Based on the Figures 2 - 6 scenario described below, the method for configuring the memory-computation fusion architecture parameters of the neural network model of the disclosed embodiments will be described in detail through

[0041] Figure 2 FIG. schematically shows a flowchart of the method for configuring the memory-computation fusion architecture parameters of the neural network model according to the embodiments of the present disclosure.

[0042] As Figure 2 shown, the method for configuring the memory-computation fusion architecture parameters of the neural network model of this embodiment includes: operation S210 to operation S250.

[0043] In operation S210, the first feature space is pruned according to the feature importance vector of the memory-computation fusion architecture to obtain a second feature space, where the feature importance vector includes multiple elements respectively used to characterize the relative importance of multiple hardware parameters of the memory-computation fusion architecture, and the first feature space is the value space of the multiple hardware parameters.

[0044] In operation S220, multiple first features are determined in the second feature space, where the first features include the parameter values of each hardware parameter.

[0045] In operation S230, for each first feature, according to the first feature and the model parameter features of the neural network model, the running performance of the neural network model on the memory-computation fusion architecture is predicted to obtain a first performance value corresponding to each first feature.

[0046] In operation S240, a target feature is determined among a plurality of first features according to a plurality of first performance values.

[0047] In operation S250, the in-memory computing architecture is configured with parameter values of respective hardware parameters included in the target feature.

[0048] According to an embodiment of the present disclosure, the in-memory computing architecture is a hardware architecture for accelerating a neural network model.

[0049] According to an embodiment of the present disclosure, the multiple hardware parameters of the in-memory computing architecture may include the number of rows of the in-memory computing node array, the number of columns of the in-memory computing node array, the number of rows of the computing unit array within the in-memory computing node, the number of columns of the computing unit array within the in-memory computing node, the input buffer size within the in-memory computing node, the weight buffer size within the in-memory computing node, the output buffer size within the in-memory computing node, the storage capacity of the DRAM bank, the organization mode of the DRAM bank, and the total DRAM capacity. The organization mode of the DRAM bank may include a two-dimensional row-column mode, a bank group mode, etc. Among them, the two-dimensional row-column mode is an array mode in which a plurality of independent DRAM banks are formed in a "row * column" form; the bank group mode is to divide a plurality of DRAM banks into several groups, and the DRAM banks within the group share some circuits. For example, the DRAM banks within the group share an address decoder.

[0050] According to an embodiment of the present disclosure, the feature importance vector is composed of multiple elements. Among them, the elements may be in numerical form, and each element is used to characterize the relative importance between the respective hardware parameters in the in-memory computing architecture. The first feature space is a value space composed of multiple hardware parameters.

[0051] For example, the multiple hardware parameters of the in-memory computing architecture include parameter A and parameter B. The value range of parameter A includes a1 and a2, and the value range of parameter B includes b1 and b2. Then the first feature space includes: (a1, b1), (a1, b2), (a2, b1), (a2, b2).

[0052] According to an embodiment of the present disclosure, according to each element of the feature importance vector, the value space of the multiple hardware parameters in the first feature space is reduced to obtain a second feature space.

[0053] For example, multiple hardware parameters of the memory - computing integrated architecture include parameter A and parameter B. The value range of parameter A includes a1, a2, and a3, and the value range of parameter B includes b1 and b2. The feature importance vector is (1, 2), where the relative importance of parameter A is 1 and the relative importance of parameter B is 2. According to the ratio of the relative importance of parameter A and parameter B, the value ranges of parameter A and parameter B are reduced to obtain new value ranges of hardware parameters. That is, the new value range of parameter A can include a2, and the value range of parameter B can include b1 and b2. The second feature space includes: (a2, b1), (a2, b2).

[0054] According to an embodiment of the present disclosure, the first feature is a feature matrix composed of parameter values of multiple hardware parameters, and the first feature is used to characterize the parameter configuration of a representative memory - computing integrated architecture. For example, multiple hardware parameters of the memory - computing integrated architecture include parameter A and parameter B. In the second feature space, the value range of parameter A is a2, and the value range of parameter B includes b1, b2, and b3. The second feature space includes: (a2, b1), (a2, b2), (a2, b3). A few are selected from (a2, b1), (a2, b2), (a2, b3) to determine multiple first features: (a2, b1), (a2, b3).

[0055] According to an embodiment of the present disclosure, the model parameter feature of the neural network model is used to characterize the computing power and storage capacity of the neural network model. For each first feature, the first feature and the model parameter feature of the neural network model can be used as inputs, and the first performance value corresponding to the input can be predicted through a pre - trained model. Since the model parameter feature of the neural network model is fixed, the first performance value corresponds to the first feature.

[0056] According to an embodiment of the present disclosure, according to the first performance value of each first feature, the first feature with the smallest first performance value is selected from multiple first features as the target feature. The memory - computing integrated architecture is configured with parameters according to the parameter values of each hardware parameter included in the target feature to improve the execution efficiency of the neural network model on the memory - computing integrated architecture.

[0057] According to an embodiment of the present disclosure, a feature importance vector is introduced to prune the first feature space representing the hardware space parameters. By quantifying the dynamic influence weight of the hardware parameters on the performance of the neural network model, the value range of the hardware parameters with higher importance is retained in the second feature space, and the search space is compressed from exponential level to linear level. Multiple first features are determined in the second feature space to determine, from the second feature space, the first features with higher importance constituted by the parameter values of each hardware parameter. The first features and the model parameter features of the neural network model are combined to predict the running performance of the neural network model on the in-memory computing architecture, and a first performance value corresponding to each first feature is obtained. Compared with the traditional method of optimizing the neural network model parameters or hardware parameters in isolation, the joint feature modeling can achieve cross-layer cooperation and break through the performance bottleneck of single-dimensional optimization. The target feature is determined from the multiple first features according to the multiple first performance values, and the in-memory computing architecture is configured with parameters according to the parameter values of each hardware parameter included in the target feature, so as to provide a more efficient in-memory computing architecture for the neural network model.

[0058] Figure 3 FIG. schematically shows a schematic diagram of an in-memory computing architecture according to an embodiment of the present disclosure.

[0059] As Figure 3 shown, the in-memory computing architecture can be divided into a DRAM (Dynamic Random Access Memory) layer, a logic layer, and a router. The DRAM layer is used to provide a dynamic access space for each in-memory computing node; the logic layer is used to perform operations such as data calculation and data storage; the router is used for communication between in-memory computing nodes.

[0060] The in-memory computing architecture may include multiple in-memory computing nodes 300 shown in the figure. The in-memory computing node 300 includes a router 301, a storage controller 302, an input buffer 303, a weight buffer 304, a computing unit array 305, an output buffer 306, and a local DRAM bank 307. Among them, the local DRAM bank 307 is used to store the data that the in-memory computing node needs to save.

[0061] The router 301 communicates with other in-memory computing nodes, and the router 301 transmits the received communication information to the storage controller 302, and the storage controller 302 controls the data storage and operation on the in-memory computing node 300. Specifically, the input buffer 303 stores the data used for calculation, and the weight buffer 304 stores the weights of the neural network model used to participate in the calculation.

[0062] The storage controller 302 controls the array of computing units 305 to perform calculations based on the data cached in the input cache 303 and the weights cached in the weight cache 304, obtains the calculation results, and the output cache 306 is used to cache the calculation results. The storage controller 302 stores the calculation results cached in the output cache 306 into the local DRAM bank 307 based on a preset program when a certain preset condition is met.

[0063] According to an embodiment of the present disclosure, the second feature space includes a plurality of second features; determining a plurality of first features according to the second feature space includes: performing a weighted transformation on the plurality of second features according to a preset weight vector to obtain a plurality of third features; determining a plurality of representative third features from the plurality of third features to obtain a plurality of first features.

[0064] According to an embodiment of the present disclosure, each second feature included in the second feature space includes parameter values of respective hardware parameters, and the plurality of second features can traverse all combinations of the plurality of hardware parameters in the second feature space. Each element in the preset weight vector is used to represent the relative importance between different hardware features. The elements in the preset vector are used as the weights of the parameter values of the respective hardware parameters to perform a weighted transformation on the plurality of second features, so as to highlight the key hardware parameters in the second features, amplify the differences of the plurality of second features in the key hardware parameters, and obtain a plurality of third features.

[0065] According to an embodiment of the present disclosure, the first feature represents a third feature with a larger parameter value range on relatively important hardware parameters. Determine the value ranges of the respective hardware parameters according to the plurality of third features, calculate the intermediate values of the respective hardware parameters, and for each hardware parameter, use the plurality of values smaller than the preset threshold of the intermediate value of the hardware parameter as the new value range of the hardware parameter. Determine a plurality of first features according to the new value ranges of the respective hardware parameters.

[0066] According to an embodiment of the present disclosure, perform a weighted transformation on the plurality of second features according to a preset weight vector to adjust the contribution degrees of different hardware features, determine a plurality of representative third features from the plurality of third features to obtain a plurality of first features, so as to reduce the number of features without losing important features and improve the processing efficiency of the computer.

[0067] According to an embodiment of the present disclosure, determining a plurality of representative third features from the plurality of third features to obtain a plurality of first features includes: clustering the plurality of third features to obtain feature clusters; determining a plurality of first features closer to the cluster centers from the feature clusters.

[0068] According to an embodiment of the present disclosure, the k-means algorithm is used to cluster multiple third features to obtain feature clusters; the distances between each third feature and the cluster centers are determined, and the third features with smaller distances are screened from the multiple third features according to the distances between the third features and the cluster centers, so as to obtain multiple first features.

[0069] According to an embodiment of the present disclosure, a clustering algorithm is used to cluster multiple weighted third features to obtain feature clusters, and multiple third features closer to the cluster centers are determined from the feature clusters as representative points, that is, multiple first features are determined, and the centrality of the multiple first features in the feature clusters is used to improve the accuracy and interpretability of the features.

[0070] According to an embodiment of the present disclosure, the third features include weighted parameter values of multiple hardware parameters; multiple representative third features are determined from the multiple third features to obtain multiple first features, including: for each hardware parameter, clustering the weighted parameter values of the multiple hardware parameters included in the multiple third features respectively to obtain feature clusters corresponding to the multiple hardware parameters respectively; according to the distances between the weighted parameter values of the hardware parameters in each feature cluster and the cluster centers of the feature clusters, determining the target weighted parameter values of the respective hardware parameters, so as to obtain multiple first features.

[0071] According to an embodiment of the present disclosure, a feature weighted clustering method is used to generate an initial sample set that covers the whole but is locally dense in key dimensions, so as to provide high-quality prior data for subsequent exploration.

[0072] According to an embodiment of the present disclosure, pruning the first feature space according to the feature importance vector of the memory-computation fusion architecture to obtain a second feature space, including: normalizing the feature importance vector to obtain a target feature importance vector; according to each element of the target feature importance vector, determining the number of parameter candidate values of each hardware parameter, where the number of parameter candidate values of each hardware parameter is positively correlated with the element; uniformly sampling in the candidate value sets of each hardware parameter according to each number of parameter candidate values to obtain a second feature space.

[0073] According to an embodiment of the present disclosure, the feature importance vector includes elements corresponding to each hardware parameter, and each element of the feature importance vector is normalized so that each element can represent the relative importance of the hardware parameter. Specifically, the element of the target feature importance vector corresponding to the j-th hardware parameter can be the ratio of the element of the feature importance vector corresponding to the hardware parameter to the element with the largest value in the feature importance vector, where is greater than 0, and is greater than or equal to 1, j is a positive integer, and j is less than the total number of hardware parameters.

[0074] According to an embodiment of the present disclosure, assume that the j-th parameter has candidate values in the original design space. According to the element of the target feature importance vector corresponding to the j-th hardware parameter, the number of candidate values proportional to its importance is reserved for this hardware parameter , and the number of parameter candidate values of each hardware parameter is determined according to the same method.

[0075] According to an embodiment of the present disclosure, through a preset uniform sampling parameter, candidate values are screened from the candidate value set of each hardware parameter according to the number of candidate values corresponding to the candidate value set of this hardware parameter. Specifically, the algorithm sorts the candidate values and then selects candidate values at equal intervals, so as to make the updated second feature space cover the entire design space as evenly as possible.

[0076] According to an embodiment of the present disclosure, based on the first feature and the model parameter feature of the neural network model, the running performance of the neural network model on the in-memory computing architecture is predicted to obtain a first performance value corresponding to each first feature, including: splicing the model parameter feature and the first feature to obtain a model input feature; inputting the model input feature into the trained performance prediction model to output the first performance value.

[0077] According to an embodiment of the present disclosure, the model parameter feature is deeply analyzed to extract the key parameter information it contains, and multiple first features are standardized and normalized to eliminate the dimensional difference between different features. The processed model parameter feature and the first feature are horizontally connected or vertically stacked to realize the organic fusion of multi-dimensional features, so as to obtain a complete and strongly representative model input feature.

[0078] According to an embodiment of the present disclosure, the model input feature is used as input data and input into the trained performance prediction model. With its internal complex network structure and parameter weights learned through training, the performance prediction model abstracts and extracts features from the input features layer by layer, and through the forward propagation mechanism, deeply calculates and logically infers the model input features, and finally outputs the first performance value that can accurately reflect the performance of the target object, providing important data support for subsequent performance evaluation, decision-making analysis and other tasks.

[0079] According to an embodiment of the present disclosure, the method for configuring the computing-in-memory architecture parameters of a neural network model further includes: extracting a plurality of fourth features in a first feature space; for each fourth feature, simulating the running performance of the neural network model on the computing-in-memory architecture according to the fourth feature and the model parameter feature, to obtain a second performance value corresponding to each fourth feature; determining a performance prediction model according to each fourth feature and the second performance value corresponding to each fourth feature; and quantifying the influence degree of each hardware parameter on the running performance according to the weight distribution mechanism inherent in the performance prediction model, to obtain a feature importance vector.

[0080] According to an embodiment of the present disclosure, n fourth features are extracted from the first feature space by using Latin Hypercube Sampling (LHS), and the fourth features include the parameter values of each hardware parameter. Each fourth feature can be denoted as . Where i = 1, 2,..., n.

[0081] According to an embodiment of the present disclosure, the running performance of the neural network model on the computing-in-memory architecture is simulated according to the fourth feature and the model parameter feature, so as to obtain a second performance value corresponding to each fourth feature .

[0082] According to an embodiment of the present disclosure, the second performance value is used to represent that the fourth feature and the model parameter feature are combined into a unified input vector, and the running performance of the neural network model on the computing-in-memory architecture is simulated to obtain a second performance value corresponding to each fourth feature.

[0083] According to an embodiment of the present disclosure, the input vector formed by combining the fourth feature and the model parameter feature is used as a sample for training, and the second performance value is used as a sample label. A training sample is formed by using the sample and the sample label to train the performance prediction model until the performance prediction model reaches the expected effect.

[0084] According to an embodiment of the present disclosure, the basic model of the performance prediction model can be a random forest classifier. The feature importance extraction function of the random forest is called to extract the feature importance vector of the performance prediction model, so as to quantify the influence degree of each hardware parameter on the running performance. Each element of the feature importance vector v represents the influence degree of the jth hardware parameter on the running performance.

[0085] According to an embodiment of the present disclosure, when the decision tree in the random forest divides the fourth feature, the higher the purity improvement obtained by dividing the fourth feature by using the parameter value of a certain hardware parameter in the fourth feature, the greater the role of the hardware parameter in distinguishing the fourth feature.

[0086] According to an embodiment of the present disclosure, based on data-driven, the contribution degree of each hardware parameter to the running performance is objectively quantified, getting rid of the problems of strong subjectivity and low efficiency caused by relying on expert experience to set the feature importance vector.

[0087] According to an embodiment of the present disclosure, a plurality of fourth features are extracted in the first feature space, and the running performance of the neural network model on the in-memory computing architecture is simulated by using the fourth features and the model parameter features, obtaining second performance values respectively corresponding to the respective fourth features; thereby, the fourth features and the second performance values are used for model training to obtain a performance prediction model. According to the weight allocation mechanism inherent in the performance prediction model, the influence degree of each hardware parameter on the running performance is quantified, obtaining a feature importance vector that can fully capture important hardware parameters.

[0088] According to an embodiment of the present disclosure, determining the performance prediction model according to each of the fourth features and the second performance values respectively corresponding to the respective fourth features includes: normalizing each of the second performance values to obtain a plurality of third performance values; according to the plurality of third performance values, classifying each of the fourth features and the second performance values respectively corresponding to the respective fourth features into a plurality of preset equal-width bins marked with bin indices, wherein the bin index is the feature importance vector; using each of the fourth features as a training sample and using the bin index respectively corresponding to each of the fourth features as a label to train a random forest classifier to obtain a performance prediction model.

[0089] According to an embodiment of the present disclosure, the Min-Max normalization is used to map the second performance value to the interval [0, 1]. The normalized plurality of third performance values are discretized into ordered bin indices, and each of the fourth features and the second performance values respectively corresponding to the respective fourth features are classified into a plurality of preset equal-width bins marked with bin indices. Specifically, for each fourth feature, according to the second performance value obtained by its normalization, the bin index corresponding to the fourth feature is calculated.

[0090] For example, using equal-width bins, setting the bin boundaries as , with a total of boundaries, wherein is the equal-width bin, used to identify that the label corresponding to each equal-width bin is the bin index.

[0091] According to an embodiment of the present disclosure, using each of the fourth features as a training sample and using the bin index respectively corresponding to each of the fourth features as a label to train a random forest classifier to obtain a performance prediction model.

[0092] According to an embodiment of the present disclosure, using the discretized bin indices as above, taking a plurality of fourth features and the corresponding classification labels as inputs, and using a random forest classifier for modeling.

[0093] According to an embodiment of the present disclosure, a random forest classifier is used for modeling, and the prediction accuracy and uncertainty of potential optimization directions are evaluated based on Bayesian. The first feature space often has non-smooth characteristics and strong non-linear correlations among multi-dimensional performance parameters, which poses great challenges to traditional modeling methods. The tree ensemble model XGBoost based on the gradient boosting framework is selected as the surrogate model to determine the performance prediction model.

[0094] According to an embodiment of the present disclosure, the model complexity of the performance prediction model needs to match the scale of the training data set. During the Bayesian iterative optimization process, newly generated samples will be injected into the annotation process in real time, resulting in the scale of the training data set far exceeding the initial design amount. Based on this data evolution characteristic, the gradual adjustment of model parameters is particularly crucial. For the tree ensemble model, its complexity mainly depends on the number and maximum depth of the base learners.

[0095] According to an embodiment of the present disclosure, a phased optimization strategy is adopted: initially, a low-complexity model composed of shallow and limited-number weak learners is constructed; subsequently, according to the change of the data scale, the number of base learners is gradually increased and their depth is extended until the theoretical optimal threshold, so as to effectively control the model complexity. Let be the cumulative sample size at time , then the parameter adjustment strategy of the tree ensemble model XGBoost can be expressed by formulas (1) and (2):

[0096] (1);

[0097] (2);

[0098] where and are the benchmark parameters in the dynamic adjustment strategy of model complexity, representing the initial reference values of the number of base learners and depth respectively. is the depth growth coefficient, is the adjustment coefficient of the number of base learners, is the initial number of training samples. This adaptive parameter adjustment mechanism runs through the entire process of training the performance prediction model, constructing a dynamic modeling framework based on data evolution.

[0099] According to an embodiment of the present disclosure, the method for configuring the memory and computing fusion architecture parameters of the neural network model further includes: performing feature encoding on the topological structure of the modular sub-structures in the neural network model to determine the model parameter features.

[0100] The model parameter features of the neural network model are a multi-dimensional feature matrix composed of multiple model parameters that characterize the neural network. The model parameters of the neural network model can include the number of model multiplication and addition operations, the total number of model parameters, the number of input layer channels, the vertical dimension size of the input feature map, the horizontal dimension size of the input feature map, the feature representation of the modular sub-structure, and the repetition times of the modular sub-structure. Among them, the modular sub-structure represents the module structure that repeatedly appears in the neural network model. The modular sub-structure can be determined based on expert experience. The structure diagram of the modular sub-structure can be input into the encoder to obtain the feature representation of the modular sub-structure.

[0101] Figure 4 Schematically shows a schematic diagram of determining the feature representation of the modular sub-structure according to an embodiment of the present disclosure.

[0102] As Figure 4 shown, perform structural analysis on the original structure of the neural network model to obtain the modular sub-structure of the neural network model. The modular sub-structure of the neural network model can be a structure that repeatedly appears in the neural network model. For most neural network models used on the in-memory computing fusion architecture, it is usually composed of multiple repeated modular sub-structures. Among them, Figure 4 the size of each convolution kernel is only schematic. Among them, "matrix addition" is usually an operation in the residual connection, that is, element-wise addition of the input matrix and the output matrix after convolution processing by the 1x1 convolution layer and the 3x3 convolution layer.

[0103] Input the modular sub-structure into the graph encoder to obtain the feature representation [x1, x2, x3... x n .

[0104] The graph encoder can be a directed acyclic graph variational autoencoder based on the variational autoencoder, which effectively captures the dependency relationship of the computational flow when encoding the directed acyclic graph by using the asynchronous message passing mechanism of the directed acyclic graph variational autoencoder.

[0105] Figure 5 Schematically shows a schematic diagram of the modular sub-structure of the neural network model according to an embodiment of the present disclosure.

[0106] As Figure 5 shown, the modular sub-structure of the neural network model is (a) a residual link module, or (b) an inception module, where the size of each convolution kernel is only schematic.

[0107] Figure 6 Schematically shows a flowchart of determining the target feature according to an embodiment of the present disclosure.

[0108] As Figure 6As shown in the figure, the neural network features and the first feature space are used as inputs. The spatial pruning engine 601 prunes the first feature space according to the feature importance vector of the memory - computing fusion architecture to obtain the second feature space. The adaptive sampling module 602 determines multiple first features in the second feature space. For each first feature, according to the first feature and the model parameter features of the neural network model, the running performance of the neural network model on the memory - computing fusion architecture is predicted to obtain a first performance value corresponding to each first feature. And according to the multiple first performance values, target features are determined among the multiple first features and the target features are output. There can be multiple target features, and the target features can be in the form of a Pareto solution set.

[0109] Based on the above - mentioned method for configuring the memory - computing fusion architecture parameters of the neural network model, the present disclosure also provides a device for configuring the memory - computing fusion architecture parameters of the neural network model. The following will be combined with Figure 7 to describe this device in detail.

[0110] Figure 7 The structural block diagram of the device for configuring the memory - computing fusion architecture parameters of the neural network model according to an embodiment of the present disclosure is schematically shown.

[0111] As Figure 7 shown, the device 700 for configuring the memory - computing fusion architecture parameters of the neural network model in this embodiment includes a spatial pruning module 710, a feature determination module 720, a performance prediction module 730, a target determination module 740, and a parameter configuration module 750.

[0112] The spatial pruning module 710 is configured to prune the first feature space according to the feature importance vector of the memory - computing fusion architecture to obtain the second feature space, where the multiple elements included in the feature importance vector are respectively used to characterize the relative importance of multiple hardware parameters of the memory - computing fusion architecture, and the first feature space is the value space of the multiple hardware parameters.

[0113] The feature determination module 720 is configured to determine multiple first features in the second feature space, where the first feature includes the parameter values of each hardware parameter.

[0114] The performance prediction module 730 is configured to, for each first feature, predict the running performance of the neural network model on the memory - computing fusion architecture according to the first feature and the model parameter features of the neural network model, and obtain a first performance value corresponding to each first feature.

[0115] The target determination module 740 is configured to determine target features among the multiple first features according to the multiple first performance values.

[0116] The parameter configuration module 750 is configured to configure the parameters of the memory - computing fusion architecture according to the parameter values of each hardware parameter included in the target features.

[0117] According to an embodiment of the present disclosure, any multiple of the spatial pruning module 710, the feature determination module 720, the performance prediction module 730, the target determination module 740, and the parameter configuration module 750 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the spatial pruning module 710, the feature determination module 720, the performance prediction module 730, the target determination module 740, and the parameter configuration module 750 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging circuits, etc., implemented by hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the spatial pruning module 710, the feature determination module 720, the performance prediction module 730, the target determination module 740, and the parameter configuration module 750 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0118] Figure 8 A block diagram of an electronic device suitable for implementing a parameter configuration method for a memory-computation fusion architecture of a neural network model according to an embodiment of the present disclosure is schematically shown.

[0119] As Figure 8 shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage section 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 801 may also include on-board memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0120] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via the bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 802 and / or the RAM 803. It should be noted that the programs may also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 may also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0121] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, and the input / output (I / O) interface 805 is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read from it can be installed into the storage portion 808 as needed.

[0122] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0123] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include one or more memories other than the above-described ROM 802 and / or RAM 803 and / or ROM 802 and RAM 803.

[0124] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to enable the computer system to implement the method for configuring the parameters of the memory-computation fusion architecture of the neural network model provided by the embodiment of the present disclosure.

[0125] When the computer program is executed by the processor 801, it executes the above functions defined in the system / apparatus of the embodiment of the present disclosure. According to an embodiment of the present disclosure, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0126] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 809, and / or be installed from the removable medium 811. The program code contained in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0127] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or be installed from the removable medium 811. When the computer program is executed by the processor 801, it executes the above functions defined in the system of the embodiment of the present disclosure. According to an embodiment of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0128] In accordance with embodiments of the present disclosure, program code for executing the computer programs provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or alternatively, can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0130] Those skilled in the art can understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0131] The above describes the embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes and not for limiting the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the respective embodiments cannot be used advantageously in combination. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. A method for configuring parameters of a memory-computation fusion architecture of a neural network model, characterized in that, The method includes: Pruning a first feature space according to a feature importance vector of a memory - computing fusion architecture to obtain a second feature space, where the feature importance vector includes a plurality of elements respectively used to characterize the relative importance of a plurality of hardware parameters of the memory - computing fusion architecture, and the first feature space is a value space of the plurality of hardware parameters; Determining a plurality of first features in the second feature space, where the first features include parameter values of the respective hardware parameters; For each of the first features, predicting the running performance of the neural network model on the memory - computing fusion architecture according to the first feature and the model parameter features of the neural network model to obtain a first performance value corresponding to each of the first features; Determining a target feature from the plurality of first features according to the plurality of first performance values; Performing parameter configuration on the memory - computing fusion architecture according to the parameter values of the respective hardware parameters included in the target feature.

2. The method according to claim 1, wherein The second feature space includes a plurality of second features; The determining a plurality of first features in the second feature space includes: Performing weighted transformation on the plurality of second features according to a preset weight vector to obtain a plurality of third features; Determining a plurality of representative third features from the plurality of third features to obtain the plurality of first features.

3. The method according to claim 2, characterized in that, Determining a plurality of representative third features from the plurality of third features to obtain the plurality of first features includes: Clustering the plurality of third features to obtain feature clusters; Determining the plurality of first features that are closer to the cluster centers from the feature clusters.

4. The method according to claim 1, characterized in that The pruning the first feature space according to the feature importance vector of the memory - computing fusion architecture to obtain the second feature space includes: Normalizing the feature importance vector to obtain a target feature importance vector; Determining the number of parameter candidate values of the respective hardware parameters according to the elements of the target feature importance vector, where the number of parameter candidate values of the respective hardware parameters is positively correlated with the element; Performing uniform sampling in the candidate value sets of the respective hardware parameters according to the respective numbers of parameter candidate values to obtain the second feature space.

5. The method according to claim 1, wherein The predicting the running performance of the neural network model on the memory - computing fusion architecture according to the first feature and the model parameter features of the neural network model to obtain a first performance value corresponding to each of the first features includes: Concatenating the model parameter features and the first feature to obtain model input features; Inputting the model input features into a trained performance prediction model to output the first performance value.

6. The method according to claim 1, characterized in that, The method further includes: Extracting a plurality of fourth features in the first feature space; For each of the fourth features, simulating the running performance of the neural network model on the memory - computing fusion architecture according to the fourth feature and the model parameter features to obtain second performance values respectively corresponding to the respective fourth features; Determining a performance prediction model according to the respective fourth features and the second performance values respectively corresponding to the respective fourth features. Quantify the influence degree of each hardware parameter on the running performance according to the weight allocation mechanism inherent in the performance prediction model, and obtain the feature importance vector.

7. The method according to claim 6, wherein The determining the performance prediction model according to each of the fourth features and the second performance values respectively corresponding to the fourth features includes: Normalize each of the second performance values to obtain a plurality of third performance values; According to the plurality of third performance values, classify each of the fourth features and the second performance values respectively corresponding to the fourth features into a plurality of preset equal-width bins marked with bin indices, where the bin index is the feature importance vector; Use each of the fourth features as a training sample, and use the bin index respectively corresponding to each of the fourth features as a label to train a random forest classifier to obtain the performance prediction model.

8. The method according to claim 1, characterized in that, The method further includes: Perform feature encoding on the topological structure of the modular substructure in the neural network model to determine the model parameter features.

9. An electronic device, comprising: One or more processors; A memory for storing one or more computer programs, Characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 8..

Images