Core particle parameter library construction method, core particle parameter integration method, and related device

Abstract

The application discloses a kind of core particle parameter library construction method, core particle parameter integration method and related device, the method includes: obtaining target model;Wherein, target model includes multiple computing layers;For target model, target core particle parameter library is constructed;Wherein, target core particle parameter library includes several core particle design parameters, at least one core particle design parameter is adapted to at least two computing layers, the number of core particle design parameters in target core particle parameter library is less than the number of computing layers of target model, several core particle design parameters are selected from the adjusted candidate core particle parameters of each computing layer, and the adjusted candidate core particle parameters can be adapted to corresponding computing layer and at least one other computing layer.By the above mode, the application can realize the construction of core particle parameter library for model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for constructing a chip parameter library, a method for integrating chip parameters, and related apparatus. Background Technology

[0002] With the rapid development of large-scale artificial intelligence models, deep learning models are widely used in fields such as computer vision, natural language processing, intelligent vehicles, and robotics. These models are typically characterized by their large scale, complex structure, diverse operator types, and rapid update and iteration speed.

[0003] Currently, for target models with various multi-layer structures, the industry generally adopts a dedicated deployment mode of one model corresponding to one independent chip, which has technical drawbacks such as high design cost and poor versatility. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a method for constructing a core parameter library, a method for integrating core parameters, and related apparatus, which can realize the construction of a core parameter library for a model.

[0005] The first aspect of this application provides a method for constructing a particle parameter library. The method includes: obtaining a target model; wherein the target model includes multiple computational layers; constructing a target particle parameter library for the target model; wherein the target particle parameter library includes a plurality of particle design parameters, at least one of the particle design parameters is adapted to at least two computational layers, the number of particle design parameters in the target particle parameter library is less than the number of computational layers in the target model, the plurality of particle design parameters are selected from adjusted candidate particle parameters of each computational layer, and the adjusted candidate particle parameters are adaptable to the corresponding computational layer and at least one other computational layer.

[0006] The step of constructing a target core parameter library for the target model includes: obtaining candidate core parameters designed for each of the computing layers; designating each computing layer as the current layer; adjusting the candidate core parameters corresponding to the current layer according to the adjustment direction of adapting the candidate core parameters of the current layer to at least one other computing layer, so that the adjusted candidate core parameters can adapt to the current layer and at least one other computing layer; and selecting a plurality of core design parameters from the adjusted candidate core parameters to construct the target core parameter library.

[0007] The step of adjusting the candidate core parameters corresponding to the current layer according to the adjustment direction of adapting the candidate core parameters of the current layer to at least one other computing layer includes: under the condition of satisfying the core performance, adjusting the candidate core parameters corresponding to the current layer according to the adjustment direction of maximizing the candidate adaptation gain of the candidate core parameters of the current layer; wherein, the candidate adaptation gain represents the number of other computing layers to which the candidate core parameters are adapted.

[0008] The core particle performance condition includes: the degree of decrease in the adjusted first simulation execution performance is less than or equal to the target performance threshold, where the first simulation execution performance is the simulation execution performance of the candidate core particle parameters on the current layer.

[0009] The method for constructing a particle parameter library after adjusting the candidate particle parameters corresponding to the current layer further includes: for each of the other computing layers, obtaining a second simulation execution performance; wherein the second simulation execution performance is the simulation execution performance of the adjusted candidate particle parameters of the current layer on the other computing layers; and obtaining a third simulation execution performance; wherein the third simulation execution performance is the simulation execution performance of the candidate particle parameters of the other computing layers on the other computing layers; and determining, based on the second simulation execution performance and the third simulation execution performance, whether the adjusted candidate particle parameters corresponding to the current layer are compatible with the other computing layers.

[0010] The step of determining whether the adjusted candidate chip parameters corresponding to the current layer are compatible with the other computing layers based on the second simulation execution performance and the third simulation execution performance includes: determining the adjusted candidate chip parameters corresponding to the current layer to be compatible with the other computing layers in response to the second simulation execution performance being less than the product of the third simulation execution performance and a preset multiple; wherein the preset multiple is greater than 1; and determining that the adjusted candidate chip parameters corresponding to the current layer are not compatible with the other computing layers in response to the second simulation execution performance being greater than or equal to the product of the third simulation execution performance and the preset multiple.

[0011] The step of selecting the plurality of core design parameters from the adjusted candidate core parameters includes: using the adjusted candidate core parameters to form a core parameter set, and deleting a target number of candidate core parameters from the core parameter set with the goal of reducing design cost; wherein the design cost includes the current engineering cost and the current model performance of the core parameter set, the current engineering cost representing the engineering cost corresponding to the current number of candidate core parameters in the core parameter set, and the current model performance representing the simulation model performance of the current candidate core parameters in the core parameter set on the target model; for the deleted core parameter set, the step of deleting a target number of candidate core parameters from the core parameter set with the goal of reducing design cost is repeated until the target stopping iteration requirement is met, and the current candidate core parameters of the core parameter set are used as the plurality of core design parameters.

[0012] Wherein, the current engineering cost is related to the baseline model performance of the target model and the number of parameters of the current candidate core parameters in the core parameter set; and / or, the target stopping iteration requirement includes the number of parameters of the current candidate core parameters in the core parameter set meeting the target number threshold.

[0013] Wherein, the simulation execution performance includes latency overhead; and / or, the simulation model performance includes latency overhead.

[0014] A second aspect of this application provides a method for constructing a core parameter library. The method includes: obtaining a reference model set corresponding to a reference scenario; wherein the reference model set includes several reference models involved in the reference scenario; constructing a first reference core parameter library for each reference model; wherein the first reference core parameter library is constructed using the core parameter library construction method provided in the first aspect; and constructing a second reference core parameter library for the reference model set using each of the first reference core parameter libraries.

[0015] Wherein, the number of core parameters in the second reference core parameter library is less than the number of first references, the number of first references is the sum of the number of core parameters in each of the first reference core parameter libraries, the second reference core parameter library includes a number of second reference core parameters, and at least one second reference core parameter is adapted to at least two reference models.

[0016] Each of the reference models includes multiple computational layers, and the second reference core parameter adapting to at least two reference models includes: the reference model corresponding to the second reference core parameter adapting and at least one computational layer adapting to at least one other reference model.

[0017] The second reference core parameter library includes a plurality of second reference core parameters. The step of constructing the second reference core parameter library for the reference model set using each of the first reference core parameter libraries includes: using each of the reference models as the current model, and using the first reference core parameter library corresponding to each of the reference models as the current parameter library; for each of the first reference core parameters in the current parameter library, adjusting the first reference core parameters according to an adjustment direction that adapts the first reference core parameters to at least one other reference model, so that the adjusted first reference core parameters can adapt to the current model and at least one other reference model; and constructing the second reference core parameter library for the reference model set using the adjusted first reference core parameters.

[0018] Each of the reference models includes multiple computational layers. The adjusted first reference core parameter adapting to other reference models includes: the adjusted first reference core parameter adapting to at least one computational layer of the other reference model; adjusting the first reference core parameter according to the adjustment direction that adapts the first reference core parameter to at least one other reference model includes: adjusting the first reference core parameter according to the adjustment direction that maximizes the reference adaptation gain of the first reference core parameter; wherein, the reference adaptation gain represents the proportion of the number of computational layers of the adapted other reference model to the number of second references, and the number of second references is the number of computational layers of the current model.

[0019] A third aspect of this application provides a core parameter integration method, the method comprising: acquiring a core parameter library constructed for a first model set; wherein a plurality of first models in the first model set are models involved in a target scenario, and the core parameter library is constructed using the core parameter library construction method provided in the second aspect above; and acquiring a second model set corresponding to a business scenario; wherein the second model set includes at least one second model, the second model is a model involved in the business scenario, and the business scenario is a sub-scenario of the target scenario; and finding matching core parameters from the core parameter library that match the second model set.

[0020] The step of finding matching core parameters from the core parameter library that match the second model set includes: randomly selecting a preset number of core parameters from the core parameter library to form a current core parameter combination; determining a first task execution performance based on the current core parameter combination; finding at least one core parameter from the remaining core parameters in the core parameter library to replace at least one current core parameter in the current core parameter combination to obtain a replaced parameter combination; determining a second task execution performance based on the replaced parameter combination; selecting a new current core parameter combination from the current core parameter combination and the replaced parameter combination based on the first task execution performance and the second task execution performance, and re-executing the step of finding at least one core parameter from the remaining core parameters in the core parameter library to replace at least one current core parameter in the current core parameter combination to obtain a replaced parameter combination and subsequent steps, until the stopping iteration requirement is met, and using the current core parameter in the last current core parameter combination as the matching core parameter.

[0021] The step of selecting a new current core parameter combination from the current core parameter combination and the replaced parameter combination includes: in response to the second task execution performance being higher than the first task execution performance, or the second task execution performance being lower than the first task execution performance and meeting the retention condition, using the replaced parameter combination as the new current core parameter combination; wherein the retention condition includes at least one of the following: the difference between the second task execution performance and the first task execution performance is less than a difference threshold, and the temperature parameter of the selection algorithm is greater than a temperature threshold, wherein the selection algorithm is an algorithm used to execute the step of randomly finding a preset number of core parameters from the core parameter library, constructing the current core parameter combination and its subsequent steps; in response to the second task execution performance being lower than the first task execution performance and not meeting the retention condition, using the current core parameter combination as the new current parameter combination.

[0022] Wherein, the preset quantity is the number of chip slots on the simulation hardware template, the simulation hardware template is used to execute the simulation task; and / or, there are several simulation tasks, the second model is at least two, the at least two second models have different calling proportions in the business scenario, and several simulation tasks are generated according to the calling proportion of each second model.

[0023] The fourth aspect of this application provides an electronic device including a memory and a processor. The memory stores program instructions, and the processor executes the program instructions to implement the chip parameter library construction method provided in the first aspect, and / or the chip parameter library construction method provided in the second aspect, and / or the chip parameter integration method provided in the third aspect.

[0024] The fifth aspect of this application provides a computer-readable storage medium for storing program instructions that can be executed to implement the chip parameter library construction method provided in the first aspect above, and / or the chip parameter library construction method provided in the second aspect above, and / or the chip parameter integration method provided in the third aspect above.

[0025] In the above technical solution, the target kernel parameter library is designed and constructed specifically for the target model. That is, the target kernel parameter library is custom-designed around each computational layer of the target model. Therefore, several kernel design parameters in the target kernel parameter library are adapted to the computational needs of the target model. Consequently, when the target model has subsequent inference requirements, utilizing physical kernel hardware produced based on the target kernel parameter library to carry out the computations of the target model's computational layers can improve the inference performance of the target model.

[0026] Furthermore, several chip design parameters are selected from the adjusted candidate chip parameters for each computation layer. These adjusted candidate chip parameters are adaptable to the corresponding computation layer and at least one other computation layer. Since the adjusted candidate chip parameters are adaptable to the corresponding computation layer and at least one other computation layer, they can adapt to at least two computation layers. And because the several chip design parameters are selected from the adjusted candidate chip parameters for each computation layer, each chip design parameter can adapt to at least two computation layers.

[0027] Each core design parameter can be adapted to at least two computational layers, and the number of core design parameters in the target core parameter library is less than the number of computational layers in the target model. Compared to one core design parameter adapting to one computational layer, only a small number of core design parameters are needed to cover all computational layers of the target model, reducing the number of core design parameters designed for the target model. Furthermore, by reducing the number of core design parameters designed for the target model, the types of differentiated core hardware specifications that need to be developed are effectively reduced, and the one-time investment in design, verification, masking, etc., for multiple types of core hardware is reduced, thereby reducing the non-repetitive engineering (NRE) cost of core development. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating an embodiment of the chip parameter library construction method provided in this application; Figure 2 yes Figure 1 The flowchart of step S12 shown is a schematic diagram of one embodiment; Figure 3 This is a schematic diagram of another embodiment of the chip parameter library construction method provided in this application; Figure 4 This is a comparison chart of the first simulation execution performance before and after the adjustment of the candidate chip parameters provided in this application; Figure 5 This is a graph showing how the current engineering cost varies with the number of parameters in the current candidate core parameter set provided in this application; Figure 6 This is a flowchart illustrating another embodiment of the chip parameter library construction method provided in this application; Figure 7 yes Figure 6 The flowchart of step S63 shown is a schematic diagram of one embodiment. Figure 8 This is a flowchart illustrating an embodiment of the chip parameter integration method provided in this application; Figure 9 This is a schematic diagram of the structure of an embodiment of the simulation hardware template provided in this application; Figure 10 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application; Figure 11 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0029] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0030] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application. The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, "many" in this document means two or more. Additionally, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0031] Please see Figure 1 , Figure 1This is a flowchart illustrating an embodiment of the chip parameter library construction method provided in this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes: Step S11: Obtain the target model.

[0032] In this embodiment, a target model is obtained, which includes multiple computational layers. The target model is not limited and can be specifically set according to actual usage needs; for example, the target model can be a 50-layer residual network (ResNet50), a transformer model based on bidirectional encoder representation (BERT), a graph convolutional neural network (GCN), a wide & deep model, etc.

[0033] It's important to note that the target model comprises multiple layers. In some cases, only a subset of these layers can be used as computational layers; that is, not all layers are treated as computational layers. For example, taking a ResNet50 model as the target model, which includes convolutional layers, fully connected layers, and pooling layers, the convolutional and fully connected layers are used as computational layers. This is because convolutional and fully connected layers involve heavy computations centered on matrix / tensor multiplication, and their multiplication and accumulation computations increase exponentially, accounting for a large portion of the ResNet50 model's total inference time. Pooling layers, on the other hand, perform element-wise and maximum-value calculations without large-scale dense multiplications; they are simple point-by-point calculations of one-dimensional vectors, do not require large parallel computing arrays, and account for a very small portion of the ResNet50 model's inference time, having minimal impact on the overall latency. Another example is the BERT model, where the linear layers and matrix multiplication layers are used as computational layers. For example, taking the target model as the GCN model, the sparse matrix multiplication layer of the GCN model is used as the computation layer.

[0034] Alternatively, all model layers can be used as computation layers. No specific restrictions are imposed here, and the specific settings can be determined according to actual usage needs.

[0035] Furthermore, taking the ResNet50 model as an example, the ResNet50 model includes multiple convolutional layers. These convolutional layers can have different kernel sizes (e.g., 3×3 kernels, 2×2 kernels, etc.), different strides (e.g., strides of 1, 2, etc.), different input dimensions, or different output dimensions. In this application, each of the multiple convolutional layers in the ResNet50 model is used as a computational layer. The ResNet50 model also includes multiple fully connected layers. These fully connected layers can have different input dimensions or different output dimensions. In this application, each of the multiple fully connected layers in the ResNet50 model is used as a computational layer. When using other models such as BERT or GCN as the target model, the determination of the computational layers is similar to that of the ResNet50 model, and will not be elaborated further here.

[0036] Step S12: Construct a target core parameter library for the target model.

[0037] In this embodiment, a target core parameter library is constructed for the target model. The target core parameter library includes several core design parameters. At least one core design parameter is adapted to at least two computational layers. The number of core design parameters in the target core parameter library is less than the number of computational layers in the target model. The several core design parameters are selected from the adjusted candidate core parameters of each computational layer. The adjusted candidate core parameters can be adapted to the corresponding computational layer and at least one other computational layer.

[0038] The target kernel parameter library is designed and built specifically for the target model. That is, it is custom-designed around each computational layer of the target model. Therefore, several kernel design parameters in the target kernel parameter library are adapted to the computational needs of the target model. Consequently, when the target model has inference requirements, using physical kernel hardware produced based on the target kernel parameter library to support the computational layers of the target model can improve its inference performance.

[0039] Furthermore, several chip design parameters are selected from the adjusted candidate chip parameters for each computation layer. These adjusted candidate chip parameters are adaptable to the corresponding computation layer and at least one other computation layer. Since the adjusted candidate chip parameters are adaptable to the corresponding computation layer and at least one other computation layer, they can adapt to at least two computation layers. And because the several chip design parameters are selected from the adjusted candidate chip parameters for each computation layer, each chip design parameter can adapt to at least two computation layers.

[0040] Each core design parameter can be adapted to at least two computational layers, and the number of core design parameters in the target core parameter library is less than the number of computational layers in the target model. Compared to one core design parameter adapting to one computational layer, only a small number of core design parameters are needed to cover all computational layers of the target model, reducing the number of core design parameters designed for the target model. Furthermore, by reducing the number of core design parameters designed for the target model, the types of differentiated core hardware specifications that need to be developed are effectively reduced, and the one-time investment in design, verification, masking, etc., for multiple types of core hardware is reduced, thereby reducing the non-repetitive engineering (NRE) cost of core development.

[0041] There is no limit to the number of core design parameters in the target core parameter library.

[0042] It should be noted that the design parameters of several core components in the target core component parameter library are not finished physical core component hardware. Therefore, the target core component parameter library is a collection of design schemes storing configurations of several core component hardware parameters. Each core component design parameter in the target core component parameter library fully defines the hardware characteristics of a core component. If core component hardware needs to be deployed, the core component design parameters can be converted into hardware engineering files to produce physical core component hardware. After the physical core component hardware is mass-produced, based on the inference load requirements of the target model, the required physical core component hardware can be selected from the physical core component hardware produced based on the target core component parameter library and inserted into a hardware template (the hardware template includes multiple core component slots). Multiple physical core component hardware components are integrated on the hardware template, and the hardware template integrating multiple physical core component hardware components is used to deploy and execute the target model.

[0043] In addition, the computational layer of the target model is the computational unit of the software algorithm dimension; the core is the hardware physical unit used to carry the computation of the computational layer of the target model. That is, the core is the hardware carrier that carries the computation of the computational layer of the target model.

[0044] Please see Figure 2 , Figure 2 yes Figure 1 The flowchart shown is a schematic diagram of one embodiment of step S12. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily follow the same pattern. Figure 2 The illustrated process sequence is limited. For example... Figure 2 As shown, this embodiment includes: Step S21: Obtain the candidate core parameters designed for each computing layer.

[0045] In this embodiment, candidate core parameters designed for each computing layer are obtained.

[0046] Specifically, a design space is configured, which contains all adjustable chip configuration dimensions, fully covering the hardware elements for chip operation. In other words, the design space defines the complete search range for chip parameters.

[0047] The process of designing candidate core parameters for a computational layer can be as follows: Initiate a simulated annealing search process, randomly select a core configuration parameter within the design space, and input it into the simulator to obtain the corresponding hardware evaluation result; modify the core configuration parameter using a transformation operator and input it into the simulator to obtain the new hardware evaluation result; iterate through the process: repeatedly execute the transformation operator to generate new core configuration parameters → the simulator evaluates the hardware evaluation result; after stopping the iteration, select the core configuration parameter with the optimal hardware evaluation result as the candidate core parameter for the computational layer. Through this design process, the simulated annealing algorithm can be used to search for the optimal candidate core parameters for the computational layer within the design space.

[0048] Among these, hardware evaluation results can include latency overhead.

[0049] It should be noted that when there are two or more dimensions in the core configuration parameters, a transformation algorithm can be used to modify one or more dimensions to obtain new core configuration parameters.

[0050] In one implementation, such as Figure 3 As shown, Figure 3 This is a schematic diagram of another embodiment of the chip parameter library construction method provided in this application. The dimensions for constructing candidate chip design parameters include at least one of the following: dataflow architecture, number of rows in the PE array (Row#), number of columns in the PE array (Column#), accumulated cache size (AccbufSz), accumulated cache quantity (Accbuf#), global cache size (GbufSz), and global cache quantity (Gbuf#).

[0051] In one specific implementation, the data flow architecture can be TPU-style, Simba-style, MatRaptor-style, etc., and is not limited here.

[0052] For example, for a convolutional layer in a ResNet50 model, a kernel configuration parameter with flexible data partitioning and a Simba-style dataflow architecture can be explored in the design space as a candidate kernel parameter HW_conv. For a linear layer in a BERT model, a candidate kernel parameter HW_linear can be explored in the design space, characterized by a large PE array and static weight dataflow with a TPU-style dataflow architecture. For a sparse layer in a GCN model, a kernel configuration parameter with a MatRaptor-style dataflow architecture can be explored in the design space as a candidate kernel parameter.

[0053] Step S22: Set each computation layer as the current layer.

[0054] In this embodiment, each computing layer is designated as the current layer.

[0055] Step S23: Adjust the candidate core parameters corresponding to the current layer according to the adjustment direction of adapting the candidate core parameters of the current layer to at least one other computing layer, so that the adjusted candidate core parameters can be adapted to the current layer and at least one other computing layer.

[0056] In this embodiment, the candidate core parameters corresponding to the current layer are adjusted according to the adjustment direction of adapting the candidate core parameters of the current layer to at least one other computational layer, so that the adjusted candidate core parameters can adapt to the current layer and at least one other computational layer. Since the candidate core parameters of the current layer are designed and customized for the current layer, the candidate core parameters of the current layer are adapted to the current layer and have good performance on the current layer. If it is necessary for the candidate core parameters of the current layer to adapt to at least one other computational layer, the candidate core parameters of the current layer need to be adjusted or optimized to avoid overfitting to the corresponding current layer, improve the generalization ability of the candidate core parameters of the current layer, and enable them to have good performance on at least one other computational layer of the target model.

[0057] In one embodiment, the candidate kernel parameters corresponding to the current layer are adjusted according to the adjustment direction that adapts the candidate kernel parameters of the current layer to at least one other computing layer. Specifically, this may include: under the condition of satisfying kernel performance, adjusting the candidate kernel parameters corresponding to the current layer according to the adjustment direction that maximizes the candidate adaptation gain of the candidate kernel parameters of the current layer, wherein the candidate adaptation gain represents the number of other computing layers that the candidate kernel parameters adapt to. By adjusting the candidate kernel parameters corresponding to the current layer, the number of computing layers that the candidate kernel parameters of the current layer adapt to or can efficiently process is maximized, while ensuring that kernel performance is satisfied. This maximizes the total number of computing layers that the candidate kernel parameters of the current layer can adapt to, thereby further reducing the number of kernel design parameters in the target kernel parameter library, reducing the types of differentiated kernel hardware specifications that need to be developed, reducing the non-repetitive engineering (NRE) costs corresponding to kernel development from the source, and fully exploring the potential for kernel hardware reuse, taking into account both model operation performance and hardware design economy.

[0058] In one specific implementation, the particle performance condition includes: the degree of decrease in the adjusted first simulation execution performance is less than or equal to the target performance threshold, where the first simulation execution performance is the simulation execution performance of the candidate particle parameters on the current layer. By adjusting the candidate particle parameters corresponding to the current layer, the number of computational layers that the candidate particle parameters corresponding to the current layer can adapt to or efficiently process is maximized, while ensuring their simulation execution performance on the current layer, i.e., the original computational layer. This maximizes the total number of computational layers that the candidate particle parameters of the current layer can adapt to, thereby further reducing the number of particle design parameters in the target particle parameter library, reducing the types of differentiated particle hardware specifications that need to be developed, reducing the non-repetitive engineering (NRE) costs corresponding to particle development from the source, and fully exploring the potential for particle hardware reuse, taking into account both model computation performance and hardware design economy.

[0059] Before adjusting the candidate core parameters corresponding to the current layer, the candidate core parameters corresponding to the current layer have good performance on the current layer. After adjusting the candidate core parameters corresponding to the current layer for cross-layer compatibility, when the candidate core parameters corresponding to the current layer are used to perform operations on the current layer, that is, the original computing layer, the attenuation of its simulation execution performance does not exceed the pre-configured performance loss allowable threshold (i.e., the target performance threshold), ensuring that the basic computing performance of the current layer is in a usable state and avoiding unlimited sacrifice of the performance of the current layer in exchange for cross-layer reuse capability.

[0060] For example, the candidate kernel parameter for a computational layer (Conv2d layer) of the target model—ResNet50—is HW_conv. The candidate kernel parameter HW_conv of the Conv2d layer of the target model ResNet50 is adjusted so that it can adapt to as many other computational layers in the ResNet50 model as possible while maintaining the performance of the original Conv2d layer (e.g., the performance drop does not exceed 10%, i.e., slack=1.1).

[0061] For example, Figure 4 This is a comparison chart of the first simulation execution performance before and after the parameter adjustment of the candidate core provided in this application. Taking the target model as the VGG16 model and the first simulation execution performance as the latency as an example, as shown... Figure 4 As shown in Figure (a), Figure (a) is a schematic diagram of the first simulation performance before the candidate core parameter adjustment (fine-tuning). The X-axis represents each computational layer of the target model—VGG16 model, and the Y-axis represents the normalized delay. Before adjusting the candidate core parameters of each computational layer of the target model—VGG16 model, the candidate core parameters of each computational layer of the VGG16 model only have good performance on a few other computational layers; such as Figure 4As shown in Figure (b), Figure (b) is a schematic diagram of the first simulation execution performance after the candidate particle parameters are adjusted (fine-tuned). The adjusted candidate particle parameters of each computing layer have achieved better performance on more other computing layers. In other words, the performance of the adjusted candidate particle parameters of each computing layer on more other computing layers has been improved, and the average normalized latency has been significantly reduced.

[0062] In one specific implementation, simulation execution performance can be latency overhead.

[0063] In one specific implementation, after adjusting the candidate kernel parameters corresponding to the current layer, for each other computing layer, a second simulation execution performance is obtained, wherein the second simulation execution performance is the simulation execution performance of the adjusted candidate kernel parameters of the current layer on other computing layers; and a third simulation execution performance is obtained, wherein the third simulation execution performance is the simulation execution performance of the candidate kernel parameters of other computing layers on other computing layers; based on the second simulation execution performance and the third simulation execution performance, it is determined whether the adjusted candidate kernel parameters corresponding to the current layer are compatible with other computing layers. After completing the adjustment of the candidate kernel parameters of the current layer, by obtaining the second simulation execution performance of the adjusted candidate kernel parameters of the current layer on other computing layers and the third simulation execution performance corresponding to the original candidate kernel parameters of other computing layers, the cross-layer adaptability is quantitatively verified based on the optimal running performance of each computing layer itself, realizing an objective quantitative judgment of the adaptability of candidate kernel parameters; at the same time, the original optimal performance of each computing layer is used as the evaluation benchmark, which expands the scope of kernel cross-layer reuse while constraining the computational performance loss of each computing layer in the entire network, balancing hardware reuse efficiency and model inference performance, and improving the reuse value and overall reliability of the final kernel parameter library.

[0064] In one specific implementation, based on the second and third simulation execution performance, it is determined whether the adjusted candidate kernel parameters corresponding to the current layer are compatible with other computing layers. Specifically, this may include: in response to the second simulation execution performance being less than the product of the third simulation execution performance and a preset multiple, determining the adjusted candidate kernel parameters corresponding to the current layer to be compatible with other computing layers, wherein the preset multiple is greater than 1; in response to the second simulation execution performance being greater than or equal to the product of the third simulation execution performance and a preset multiple, determining that the adjusted candidate kernel parameters corresponding to the current layer are not compatible with other computing layers. Using the native optimal simulation performance of other computing layers (i.e., the third simulation execution performance) as a benchmark, a preset multiple greater than 1 is set as a performance tolerance upper limit. The performance of the adjusted candidate kernel parameters of the current layer in other computing layers (i.e., the second simulation execution performance) is reused. As long as it does not exceed the product of the native optimal simulation performance and the preset multiple, the adjusted candidate kernel parameters of the current layer are determined to be compatible with it; otherwise, they are not compatible.

[0065] In one specific implementation, the specific formula is as follows:

[0066] In the formula, M represents the set of all computational layers contained in the target model; Indicates the current layer The adjusted candidate core parameters; This indicates the performance of the second simulation execution, i.e., the current layer. Adjusted candidate core parameters In other computing layers On the simulation execution performance; Indicates other computing layers Candidate core parameters; This indicates the performance of the third simulation execution layer, i.e., other computational layers. Candidate core parameters In other computing layers The simulation execution performance is indicated by threshold, which represents the preset multiplier, such as 1.1, 1.5, etc., and is not limited here. This represents a 0-1 indicator function. If the inequality within the parentheses is true, the indicator function's value is 1; if the inequality within the parentheses is false, the indicator function's value is 0. The summation of these values ​​represents the total value for the current layer. Adjusted candidate core parameters The number of other computing layers that can be adapted.

[0067] Step S24: Select several core design parameters from the adjusted candidate core parameters to construct a target core parameter library.

[0068] In this embodiment, several core design parameters are selected from the adjusted candidate core parameters to construct a target core parameter library. After the previous cross-layer adaptation and optimization of the candidate core parameters for each computational layer, a large number of adjusted candidate core parameters with highly overlapping functions and adaptation ranges are generated. Selecting several core design parameters from the adjusted candidate core parameters to construct the target core parameter library can filter and remove duplicates, eliminating redundant candidate core parameters with overlapping adaptation capabilities and no new reuse value. The total number of candidate core parameters finally included in the library is significantly less than the total number of computational layers in the target model, thus compressing the size of the target core parameter library from the source.

[0069] In one embodiment, several core design parameters are selected from the adjusted candidate core parameters. Specifically, this may include: using the adjusted candidate core parameters to form a core parameter set, and deleting a target number of candidate core parameters from the core parameter set with the goal of reducing design cost. The design cost includes the current engineering cost and the current model performance of the core parameter set. The current engineering cost represents the engineering cost corresponding to the current number of candidate core parameters in the core parameter set, and the current model performance represents the simulation model performance of the current candidate core parameters in the core parameter set on the target model. For the deleted core parameter set, the process of deleting a target number of candidate core parameters from the core parameter set with the goal of reducing design cost is repeated until the target stopping iteration requirement is met. The remaining candidate core parameters in the core parameter set are then used as several core design parameters.

[0070] Specifically, the adjusted candidate core parameters are integrated to form a core parameter set. Each adjusted candidate core parameter in the core parameter set represents an independent core hardware solution. A large number of adjusted candidate core parameters in the core parameter set results in extremely high one-time engineering costs; therefore, iterative reduction and compression of core specifications within the library are necessary. The optimization goal of the entire iterative deletion operation is to reduce design costs, which consist of two parts: current engineering cost and current model performance. Current engineering cost is determined by the number of existing current candidate core parameters in the core parameter set. The more existing current candidate core parameters in the core parameter set, the higher the current engineering cost; the fewer existing current candidate core parameters in the core parameter set, the lower the current engineering cost. The existing current candidate core parameters in the core parameter set are input into the simulator to run all computational layers of the target model, obtaining the simulation model performance. Design cost is a balance function between the two. Reducing the number of existing candidate core parameters in the core parameter set will reduce engineering cost, but may increase model performance cost; increasing the number of existing candidate core parameters in the core parameter set will reduce performance cost, but will increase engineering cost. Iterative reduction is to find the balance point between engineering cost and model performance.

[0071] The target quantity can be 1, 2, etc., and is not limited here.

[0072] In one specific implementation, the current engineering cost is related to the baseline model performance of the target model and the number of current candidate core parameters in the core parameter set. Specifically, the current engineering cost can be the product of the baseline model performance of the target model, the number of current candidate core parameters in the core parameter set, and preset hyperparameters. The preset hyperparameters are not limited.

[0073] In one specific implementation, the specific formula for the design cost is as follows:

[0074] In the formula, S represents the set of core parameters consisting of the current candidate core parameters; M represents the target model; α represents the current model performance, that is, the simulation model performance of the current candidate core parameters of the core parameter set S on the target model M; α represents the hyperparameter. Indicates the performance of the benchmark model; This represents the number of current candidate core parameters in the core parameter set S; This represents the design cost. A penalty term is applied, taking into account both the target model's performance and the size of the core parameter set (NRE engineering cost), to find the optimal balance between performance and cost.

[0075] like Figure 5 As shown, Figure 5 This is a graph showing how the current engineering cost varies with the number of parameters in the current candidate core parameter set, as provided in this application. Figure 5 The graphs shown include the current project cost curve with and without the parameter quantity penalty term corresponding to the core parameter set. Figure 5 It also indicates the optimal number of parameters corresponding to the core parameter set. Figure 5 The horizontal axis of the curve represents the number of candidate core parameters in the core parameter set. Figure 5 Figure (a) in the diagram corresponds to the VGG (Visual Geometry Group) 16 model. Figure 5 Figure (b) in the diagram corresponds to the Transformer model. Figure 5 Figure (c) in the diagram corresponds to the Deep Learning Recommendation Model (DLRM). Figure 5 The (d) diagram in the diagram corresponds to a graph convolutional neural network (GCN).

[0076] In one specific implementation, the target stopping iteration requires that the number of parameters in the current candidate core parameter set, including the core parameter set, meets a target number threshold. The size of the target number threshold is not limited and can be set according to actual usage needs.

[0077] Specifically, the initial state is as follows: A core parameter set is formed using the adjusted candidate core parameters. The current number of candidate core parameters in the core parameter set is much larger than the target threshold K. A single round of greedy optimization is performed: the design cost of deleting any target number of candidate core parameters is calculated through simulation; the target number of candidate core parameters that maximize the reduction in design cost are selected, deleted, and the core parameter set is updated. A loop check is then performed: if the number of current candidate core parameters in the updated core parameter set is greater than the target threshold K, the next round of greedy iteration continues; otherwise, the iteration stops and the loop exits. After the loop ends, the remaining K current candidate core parameters in the core parameter set are used as several core design parameters and combined to form the target core parameter library. Compared to brute-force enumeration of all K-sized subsets, the greedy algorithm significantly reduces the hardware simulation computation load and solves the problem of computational power explosion caused by the large number of candidate core parameters.

[0078] In one specific implementation, the simulation model performance can be latency overhead.

[0079] In one specific implementation, the formula for calculating the current model performance corresponding to the core parameter set is as follows:

[0080] In the formula, M represents the target model; S represents the set of core parameters consisting of the current candidate core parameters; This represents the current model performance, that is, the simulation model performance of the current candidate core parameters of the core parameter set S on the target model M; it also represents the number of current candidate core parameters in the core parameter set S. Represents any computational layer in the target model M; This represents the i-th current candidate core parameter in the core parameter set S; Represents the parameters of the i-th current candidate core. In the computation layer On the simulation execution performance; Represents the parameters of the i-th current candidate core. For the computation layer, among all current candidate core parameters in the core parameter set S. The ranking of simulation execution performance, i.e., computational layer. From the perspective of the i-th current candidate core particle parameters The performance ranking within the core parameter set S. The above calculation formula summarizes the performance of each computational layer. The weighted harmonic average of the simulation performance on all current candidate kernel parameters in the kernel parameter set S. These are the weighting coefficients.

[0081] Please see Figure 6 , Figure 6This is a flowchart illustrating another embodiment of the chip parameter library construction method provided in this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 6 The illustrated process sequence is limited. For example... Figure 6 As shown, this embodiment includes: Step S61: Obtain the reference model set corresponding to the reference scene.

[0082] In this embodiment, a reference model set corresponding to the reference scenario is obtained, wherein the reference model set includes several reference models involved in the reference scenario.

[0083] There are no restrictions on the reference scenario; for example, the reference scenario could be an intelligent driving scenario. Furthermore, there are no restrictions on the number of reference models involved in the reference scenario or the specific reference models; for example, the reference scenario may involve three reference models: ResNet50, BERT, and GCN.

[0084] Step S62: Construct the first reference core parameter library for each reference model.

[0085] In this embodiment, a first reference core parameter library is constructed for each reference model. This library is constructed using the core parameter library construction method described above. Since the first reference core parameter library for each reference model is constructed using this method, it is designed and constructed specifically for the reference model. That is, the first reference core parameter library is custom-designed around each computational layer of the reference model. Therefore, several core parameters in the first reference core parameter library are adapted to the computational requirements of the reference model. Thus, when the reference model has inference requirements, using physical core hardware produced based on the first reference core parameter library constructed for it to carry out the computation of the reference model's computational layers can improve the inference performance of the reference model.

[0086] Furthermore, each kernel parameter in the first reference kernel parameter library is adapted to at least two computational layers, and the number of kernel parameters in the first reference kernel parameter library is less than the number of computational layers in the reference model. Compared to one kernel parameter adapting to one computational layer, only a small number of kernel parameters are needed to cover all computational layers of the reference model, reducing the number of kernel parameters designed for the reference model. Moreover, because the number of kernel parameters designed for the reference model is reduced, the types of differentiated kernel hardware specifications that need to be developed are effectively reduced, and the one-time investment in design, verification, masking, etc., corresponding to multiple types of kernel hardware is reduced, thereby reducing the non-repetitive engineering (NRE) cost of kernel development.

[0087] It should be noted that the details regarding the construction of the first reference core parameter library for each reference model can be found in the embodiments of the core parameter library construction method described above, and will not be repeated here.

[0088] Step S63: Construct a second reference core parameter library for the reference model set using the first reference core parameter libraries.

[0089] In this embodiment, a second reference core parameter library is constructed for the reference model set using each of the first reference core parameter libraries. Since a second reference core parameter library is constructed for the reference model set involved in the reference scenario, high-performance and high-efficiency inference can be achieved in multiple sub-business scenarios under the reference scenario.

[0090] In one embodiment, the number of kernel parameters in the second reference kernel parameter library is less than the number of first references, where the number of first references is the sum of the number of kernel parameters in each of the first reference kernel parameter libraries. The second reference kernel parameter library includes several second reference kernel parameters, and at least one second reference kernel parameter adapts to at least two reference models. Since at least one second reference kernel parameter in the second reference kernel parameter library adapts to at least two reference models, and the number of kernel parameters in the second reference kernel parameter library is less than the sum of the number of kernel parameters in each of the first reference kernel parameter libraries, all reference models in the second model set can be covered with a small number of second reference kernel parameters, reducing the number of second reference kernel parameters designed for the second model set. Furthermore, because the number of second reference kernel parameters designed for the second model set is reduced, the types of differentiated kernel hardware specifications that need to be developed are effectively reduced, and the one-time investment in design, verification, masking, etc., corresponding to multiple types of kernel hardware is reduced, thereby reducing the non-repetitive engineering (NRE) cost of kernel development.

[0091] In one specific implementation, each reference model includes multiple computational layers, and the second reference core parameter adaptation to at least two reference models includes: the reference model corresponding to the second reference core parameter adaptation and at least one computational layer adapting to at least one other reference model.

[0092] In one implementation, a second reference core parameter library can be obtained by combining the first reference core parameter library of each reference model, and the second reference core parameter library can be constructed for the reference model set.

[0093] In other implementations, such as Figure 7 As shown, Figure 7 yes Figure 6 The flowchart shown in step S63 is a schematic diagram of an embodiment. Using each of the first reference core parameter libraries, a second reference core parameter library is constructed for the reference model set. Specifically, it includes the following sub-steps: Step S71: Take each reference model as the current model, and take the first reference core parameter library corresponding to each reference model as the current parameter library.

[0094] In this embodiment, each reference model is used as the current model, and the first reference core parameter library corresponding to each reference model is used as the current parameter library.

[0095] Step S72: For each first reference core parameter in the current parameter library, adjust the first reference core parameter in the direction of adapting the first reference core parameter to at least one other reference model, so that the adjusted first reference core parameter can adapt to the current model and at least one other reference model.

[0096] In this embodiment, for each first reference core parameter in the current parameter library, the first reference core parameter is adjusted according to the adjustment direction of adapting the first reference core parameter to at least one other reference model, so that the adjusted first reference core parameter can adapt to the current model and at least one other reference model. Since the first reference core parameters in the current parameter library are designed and customized for the current model, the first reference core parameters in the current parameter library are adapted to the current model and have good performance on the current model. If it is necessary for the first reference core parameters in the current parameter library to adapt to at least one other reference model, the first reference core parameters in the current parameter library need to be adjusted or optimized to improve the generalization ability of the first reference core parameters in the current parameter library, so that it also has good performance on other reference models.

[0097] In one embodiment, each reference model includes multiple computational layers. The adjusted first reference core parameter adapting to other reference models can specifically involve: the adjusted first reference core parameter adapting to at least one computational layer of other reference models; and adjusting the first reference core parameter according to an adjustment direction that adapts the first reference core parameter to at least one other reference model. Specifically, this can include: adjusting the first reference core parameter according to an adjustment direction that maximizes the reference adaptation gain of the first reference core parameter. Here, the reference adaptation gain represents the proportion of the number of computational layers of the adapted other reference models to the number of computational layers in the second reference model, where the number of computational layers in the second reference model is the number of computational layers in the current model. The first reference core parameter is iteratively adjusted with the goal of maximizing the reference adaptation gain, thereby maximizing the proportion of computational layers in the other reference models adapted to the first reference parameter. A single first reference core parameter can simultaneously adapt to or be compatible with a large number of computational layers in multiple reference models, significantly reducing the total amount of differentiated core parameters required in multi-model scenarios, and eliminating the need to customize a dedicated core parameter library for each set of reference models.

[0098] It should be noted that the reference adaptation ratio for each current model is normalized according to the number of computational layers of the current model to ensure that reference models with fewer computational layers and reference models with more computational layers are treated equally.

[0099] In other implementations, the first reference core parameter can be adjusted in an adjustment direction that maximizes the reference adaptation gain of the first reference core parameter, while satisfying the model performance conditions. The model performance conditions can be: the degree of performance degradation of the adjusted current parameter library on the current model is less than or equal to a performance threshold.

[0100] In one specific implementation, the specific formula is as follows:

[0101] In the formula, M represents the reference model set; Any reference model in the middle; L M This represents the total number of computational layers across all reference models. This represents the computational layer of the current model (reference model) M. The adjusted first reference core parameters; This represents the computational layer of the current model (reference model) M. Adjusted first reference core parameters Computational layers of other reference models On the simulation execution performance; Represents the computational layer of other reference models First reference core parameters; Represents the computational layer of other reference models First reference core parameters Computational layers of other reference models The simulation execution performance is indicated by threshold, which represents the preset multiplier, such as 1.1, 1.5, etc., and is not limited here. This represents a 0-1 indicator function. If the inequality within the parentheses is true, the indicator function's value is 1; if the inequality within the parentheses is false, the indicator function's value is 0. This indicates that the summation of the overall values ​​represents the computational layer of the current model (reference model) M. Adjusted candidate core parameters The number of computational layers in one of the other reference models to be adapted.

[0102] Step S73: Using the adjusted parameters of each first reference core, construct a second reference core parameter library for the reference model set.

[0103] In this embodiment, a second reference core parameter library is constructed for the reference model set using the adjusted parameters of each first reference core. The adjusted parameters of each first reference core corresponding to each reference model are aggregated, and similar tuning and selection are performed at the model set level to obtain the core parameter library constructed for the reference model set, i.e., the second reference parameter library.

[0104] For example, using the adjusted first reference particle parameters, a second reference particle parameter library is constructed for the reference model set. This library may contain a particle parameter that is good at convolution, a particle parameter that is good at linear computation, a particle parameter that is good at sparse computation, and a general-purpose particle parameter.

[0105] In one embodiment, a second reference particle parameter library is constructed for the reference model set using the adjusted first reference particle parameters. Specifically, this may include: using the adjusted first reference particle parameters to form a reference particle parameter set, and deleting a reference number of first reference particle parameters from the reference particle parameter set with the goal of reducing reference design cost. The reference design cost includes the current reference engineering cost and the current reference model performance of the reference particle parameter set. The current reference engineering cost represents the engineering cost corresponding to the current number of first reference particle parameters in the reference particle parameter set, and the current reference model performance represents the simulation model performance of the current first reference particle parameters on the reference model set. For the deleted reference particle parameter set, the process of deleting a reference number of first reference particle parameters from the reference particle parameter set with the goal of reducing reference design cost is repeated until the reference stopping iteration requirement is met. The second reference parameter library is then constructed using the current first reference parameters of the reference particle parameter set.

[0106] Specifically, the adjusted parameters of each first reference particle are integrated to form a reference particle parameter set. Each adjusted first reference particle parameter in the reference particle parameter set represents an independent particle hardware solution. A large number of adjusted first reference particle parameters in the reference particle parameter set will result in extremely high one-time engineering costs. Therefore, iterative reduction and compression of particle specifications in the library are necessary. The optimization goal of the entire iterative deletion operation is to reduce the reference design cost, which consists of two parts: the current reference engineering cost and the current reference model performance. The current reference engineering cost is determined by the number of existing first reference particle parameters in the reference particle parameter set. The more existing first reference particle parameters in the reference particle parameter set, the higher the current reference engineering cost; the fewer existing first reference particle parameters in the reference particle parameter set, the lower the current reference engineering cost. The existing first reference particle parameters in the reference particle parameter set are input into the simulator, and the simulation model performance of the current first reference particle parameters in the reference particle parameter set on the reference model set is also considered. The reference design cost is a balance function between the two. Reducing the number of existing first reference core parameters in the reference core parameter set will reduce engineering costs, but may increase the performance cost of the simulation model. Increasing the number of existing first reference core parameters in the reference core parameter set will reduce performance costs, but will increase engineering costs. Iterative reduction is to find the balance point between engineering costs and model performance.

[0107] The reference quantity can be 1, 2, etc., and is not limited here.

[0108] In one specific implementation, the specific formula for the reference design cost is as follows:

[0109] In the formula, This represents the set of reference core parameters that constitute the parameters of the current first reference core. Represents the reference model set; This represents the current performance of the reference model, i.e., the set of reference core parameters. The current first reference core parameters in the reference model set The simulation model performance; β represents the hyperparameter; Represents the reference core parameter set The number of current first reference core parameters; This indicates the cost of the reference design.

[0110] In one specific implementation, the reference stopping iteration requirement includes the requirement that the number of parameters of the current first reference core parameter in the reference core parameter set meets a reference number threshold. The value of the reference number threshold is not limited and can be set according to actual usage needs.

[0111] Specifically, the initial state is as follows: a reference core parameter set is formed using the adjusted parameters of each first reference core. The current number of first reference core parameters in the reference core parameter set is much greater than the reference quantity threshold Q. A single round of greedy optimization is performed: the reference design cost of deleting any number of first reference core parameters is simulated and calculated; the number of first reference core parameters that maximize the reduction in reference design cost is selected and deleted, updating the reference core parameter set. A loop check is performed: if the current number of first reference core parameters in the newly acquired reference core parameter set is greater than the reference quantity threshold Q, the next round of greedy iteration continues; otherwise, the iteration stops and the loop exits. After the loop ends, a second reference core parameter library is constructed using the remaining Q current first reference core parameters in the reference core parameter set.

[0112] In one specific implementation, the formula for calculating the performance of the current reference model corresponding to the reference core parameter set is as follows:

[0113] In the formula, This represents the current performance of the reference model, i.e., the set of reference core parameters. The current first reference core parameters in the reference model set The performance of the simulation model on the surface; This represents the total number of reference models in the reference model set; M represents the set of reference models; M represents any reference model in the set of reference models. This represents the baseline model performance of the reference model M; Indicates the reference core parameter set The simulation model performance of the reference model M is given in the above formula. The formula statistically represents the average simulation model performance of all reference models in the reference model set. To avoid inhomogeneity caused by model size, the average simulation model performance is calculated within the reference core parameter set. The simulation model performance of the reference model M is standardized against the performance of the optimal ideal model, i.e., the baseline model performance.

[0114] Please see Figure 8 , Figure 8 This is a schematic flowchart of an embodiment of the chip parameter integration method provided in this application. It should be noted that if substantially the same result is obtained, this embodiment does not necessarily reflect that outcome. Figure 8 The illustrated process sequence is limited. For example... Figure 8 As shown, this embodiment includes: Step S81: Obtain the core parameter library constructed for the first model set.

[0115] In this embodiment, a core parameter library constructed for the first model set is obtained, wherein several first models in the first model set are models involved in the target scene, and the core parameter library is constructed using the core parameter library construction method described above.

[0116] Step S82: Obtain the second model set corresponding to the business scenario.

[0117] In this embodiment, a second model set corresponding to a business scenario is obtained. The second model set includes at least one second model, which is a model involved in the business scenario, and the business scenario is a sub-scenario of the target scenario.

[0118] There is no limitation on the number of second models included in the second model set; for example, one, two, etc. Furthermore, there is no limitation on the business scenario corresponding to the second model; for example, the business scenario could be a computer vision application scenario, a natural language processing service scenario, etc.

[0119] Step S83: Find the matching core parameters that match the second model set from the core parameter library.

[0120] In this embodiment, matching kernel parameters that match the second model set are found from the kernel parameter library. For a specific business scenario under the target scenario, the optimal kernel parameters for the second model set are found from the pre-built kernel parameter library. Scenario-driven kernel parameter selection enables on-demand customization and maximized utilization of hardware resources.

[0121] like Figure 9 As shown, Figure 9 This is a schematic diagram of an embodiment of the simulation hardware template provided in this application. The simulation hardware template is used to simulate and execute the simulation task corresponding to the second model set. The simulation hardware template includes an I / O chiplet and multiple accelerator chip sockets. For a specific business scenario in the target scenario, the optimal or matching chip parameters are selected from the pre-built chip parameter library for the multiple chip sockets on the simulation hardware template.

[0122] Specifically, such as Figure 9 As shown, the simulation hardware template has an I / O chip in the center, containing a DDR interface, PCIe, D2D interconnect, and general-purpose I / O. Eight accelerator chip slots are evenly distributed around it for inserting accelerator chips. DDR is a double data rate memory interface. D2D is an abbreviation for Die-to-Die. Figure 9 The vertically arranged D2D marking areas on the left and right sides are used for high-speed data communication between the I / O core and the surrounding eight accelerator cores, serving as a high-speed transmission channel between cores. General purpose I / O refers to... Figure 9 Misc.I / O in the middle. Figure 9 The PCIe4x16 in the I / O chip is a 16-channel PCIe4.0 high-speed host bus located at the bottom of the I / O chip.

[0123] In one embodiment, finding matching core parameters from the core parameter library that match the second model set can specifically involve: randomly selecting a preset number of core parameters from the core parameter library to form a current core parameter combination; determining the first task execution performance based on the current core parameter combination; finding at least one core parameter from the remaining core parameters in the core parameter library to replace at least one current core parameter in the current core parameter combination, resulting in a replaced parameter combination; determining the second task execution performance based on the replaced parameter combination; selecting a new current core parameter combination from the current core parameter combination and the replaced parameter combination based on the first and second task execution performance, and re-executing the process of finding at least one core parameter from the remaining core parameters in the core parameter library to replace at least one current core parameter in the current core parameter combination, resulting in a replaced parameter combination and subsequent steps, until the iteration stop requirement is met, and using the current core parameter in the final current core parameter combination as the matching core parameter.

[0124] In one specific implementation, selecting a new current core parameter combination from the current core parameter combination and the replaced parameter combination may specifically include: in response to the second task execution performance being higher than the first task execution performance, or the second task execution performance being lower than the first task execution performance and meeting the retention condition, using the replaced parameter combination as the new current core parameter combination, wherein the retention condition includes at least one of the following: the difference between the second task execution performance and the first task execution performance is less than a difference threshold, or the temperature parameter of the selection algorithm is greater than a temperature threshold, and the selection algorithm is an algorithm used to randomly select a preset number of core parameters from the core parameter library, construct the current core parameter combination and its subsequent steps; in response to the second task execution performance being lower than the first task execution performance and not meeting the retention condition, using the current core parameter combination as the new current core parameter combination.

[0125] The difference threshold and temperature threshold are not limited in size and can be set according to actual usage needs. Furthermore, the temperature parameter for selecting the algorithm can be understood as the scheduling parameter for selecting the algorithm.

[0126] In one specific implementation, there are several simulation tasks and at least two second models. These at least two second models have different call percentages in the business scenario. Several simulation tasks are generated according to the call percentage of each second model. For example, the second model set includes a ResNet50 model and a RetinaNet model. The ResNet50 model has a call percentage of 70% (high priority), and the RetinaNet model has a call percentage of 30% (medium priority). Of the several simulation tasks, 70% are tasks adapted to the ResNet50 model, and 30% are tasks adapted to the RetinaNet model.

[0127] Taking a business scenario of computer vision application, a second model set including ResNet50 and RetinaNet models, with ResNet50 models accounting for 70% of calls (high priority) and RetinaNet models accounting for 30% of calls (medium priority) as an example: From the kernel parameter library, a preset number of kernel parameters are randomly selected to form the current kernel parameter combination. Specifically, the eight kernel slots of the simulation hardware template are filled with [NPU-Generic, NPU-Generic, NPU-Generic, NPU-Generic, NPU-CV, NPU-CV, NPU-CV, NPU-Sparse]. Then, a simulator is run to simulate the arrival of tasks in this business scenario (following a Poisson distribution). The scheduler allocates simulation tasks to the kernel parameters in the kernel slots according to the priority and SLA requirements of the simulation tasks; for example, the convolutional layer of the ResNet50 model is assigned to a kernel in a certain kernel slot for execution, and its execution time is recorded. The proportion of simulation tasks that meet the SLA requirements out of the total number of simulation tasks is counted, i.e., the SLA satisfaction rate.

[0128] Iterative optimization: Randomly select a kernel slot (e.g., kernel slot 1) and replace its kernel parameters with another random kernel parameter from the kernel parameter library (e.g., replace NPU-Generic with NPU-CV), forming a replaced parameter combination. Then, evaluate the SLA satisfaction rate of the replaced parameter combination. If the replaced parameter combination performs better, it is used as the new current kernel parameter combination; if the performance is worse but the retention condition is met, it is used as the new current kernel parameter combination to avoid getting trapped in local optima; if the performance is worse but the retention condition is not met, it is discarded.

[0129] Repeat the above iterative optimization and gradually reduce the temperature parameter of the selection algorithm until the selection algorithm gradually converges.

[0130] When the selection algorithm ends, the current kernel parameters in the final current kernel parameter combination are used as the matching kernel parameters, that is, as the optimal kernel parameter combination for matching the second model set. For example, the final result might be [6 x NPU-CV, 2x NPU-Generic]. This kernel parameter combination makes extensive use of NPU-CV, which is good at convolution, perfectly matching the business scenario's requirement of ResNet50 as the main model, thus achieving the highest SLA satisfaction rate.

[0131] It should be noted that the selection algorithm starts with a random combination of kernel parameters and iteratively replaces kernel parameters in kernel slots and evaluates performance to find the optimal combination of kernel parameters that maximizes the service level agreement (SLA) satisfaction rate. Simulated annealing is used in the evaluation process. To evaluate the performance of a kernel parameter combination, a heuristic multi-task scheduler for heterogeneous kernel systems is proposed to help determine the kernel parameter allocation for each simulation task, thereby evaluating the SLA satisfaction rate in specific business scenarios. The core idea of ​​the scheduler's scheduling algorithm is to assign high-priority simulation tasks to their preferred kernel parameters. The scheduler scores simulation tasks based on their priority, SLA deadline, and waiting time, prioritizing high-scoring simulation tasks and assigning them to their preferred kernels for execution whenever possible, thus simulating a realistic multi-tenant inference environment.

[0132] The second model set, which includes ResNet50, BERT, and GCN models, will be used as an example for illustration: Model set: Define a second model set, which includes ResNet50 model, BERT model, and GCN model.

[0133] Core parameter library construction: Construct a core parameter library Slib_initial with a size of Q=3. The core parameter library contains the following core parameters: NPU-CNN, NPU-Transformer, and NPU-Sparse.

[0134] Deployment in Business Scenario A. Business Scenario A—image recognition service scenario, is 100% a ResNet50 model task. The optimal combination of kernel parameters is selected from the kernel parameter library Slib_initial. The optimal combination of kernel parameters can be [8 x NPU-CNN], which achieves extremely high SLA satisfaction rate and energy efficiency.

[0135] Deployment in Business Scenario B. Business Scenario B—Natural Language Processing Service Scenario—is 100% a task of the BERT model. The optimal combination of kernel parameters is selected from the kernel parameter library Slib_initial. The optimal combination of kernel parameters can be [8 x NPU-Transformer], which also achieves high performance.

[0136] Model set changes. Suppose a new important model, Wide&Deep (containing a large number of linear layers), has emerged in the market and needs to be added to the second model set. We can directly use the old kernel parameter library Slib_initial to handle the Wide&Deep model task. Because its computational characteristics are similar to the BERT model, the NPU-Transformer kernel parameters can handle it well.

[0137] New Scenario Deployment. Define a new target scenario—a recommendation system service scenario, comprising 60% Wide&Deep and 40% DLRM (another recommendation model, also primarily linear). Select the optimal combination of kernel parameters from the kernel parameter library Slib_initial. The optimal kernel parameter combination can be [8 x NPU-Transformer] or a similar kernel parameter combination. Although this kernel parameter combination is not specifically designed for the Wide&Deep model, it can still provide a fairly good SLA satisfaction rate.

[0138] The core parameter library constructed in this application has good robustness and generalization ability. When a new model appears, it is not necessarily necessary to rebuild the library at a high cost immediately. Existing libraries can often provide effective support, which greatly extends the life cycle of the core library and reduces long-term R&D costs.

[0139] Please see Figure 10 , Figure 10 This is a schematic diagram of an embodiment of the electronic device provided in this application. The electronic device 100 includes a memory 101 and a processor 102 coupled to each other. The processor 102 is used to execute program instructions stored in the memory 101 to implement the steps of any of the above-described chip parameter library construction method and / or chip parameter integration method embodiments. In a specific implementation scenario, the electronic device 100 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 100 may also include mobile devices such as laptops and tablets, which are not limited here.

[0140] Specifically, processor 102 controls itself and memory 101 to implement the steps of any of the above-described chip parameter library construction method and / or chip parameter integration method embodiments. Processor 102 may also be referred to as a CPU (Central Processing Unit). Processor 102 may be an integrated circuit chip with signal processing capabilities. Processor 102 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 102 may be implemented using integrated circuit chips.

[0141] Please see Figure 11 , Figure 11 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 110 of this application embodiment stores program instructions 111. When executed, these program instructions 111 implement the methods provided by any embodiment of the chip parameter library construction method and / or chip parameter integration method of this application, as well as any non-conflicting combination thereof. The program instructions 111 can form a program file and be stored in the aforementioned computer-readable storage medium 110 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) can execute all or part of the steps of the methods of various embodiments of this application. The aforementioned computer-readable storage medium 110 includes various media capable of storing program code, such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.

[0142] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for constructing a core parameter library, characterized in that, The method includes: Obtain the target model; wherein the target model includes multiple computational layers; A target core parameter library is constructed for the target model; wherein, the target core parameter library includes several core design parameters, and the number of core design parameters in the target core parameter library is less than the number of computational layers of the target model; The step of constructing a target core parameter library for the target model includes: Obtain candidate core parameters designed for each of the aforementioned computational layers; Each of the aforementioned computational layers is designated as the current layer; Adjust the candidate core parameters corresponding to the current layer in accordance with the adjustment direction of adapting the candidate core parameters of the current layer to at least one other computing layer, so that the adjusted candidate core parameters can be adapted to the current layer and at least one other computing layer; From the adjusted candidate core parameters, select several core design parameters to construct the target core parameter library.

2. The method according to claim 1, characterized in that, The step of adjusting the candidate core parameters corresponding to the current layer according to the adjustment direction of adapting the candidate core parameters of the current layer to at least one other computing layer includes: Under the condition of satisfying the chip performance, the candidate chip parameters corresponding to the current layer are adjusted in the adjustment direction that maximizes the candidate adaptation gain of the candidate chip parameters of the current layer; wherein, the candidate adaptation gain represents the number of other computing layers to which the candidate chip parameters are adapted.

3. The method according to claim 2, characterized in that, The core performance condition includes: the degree of decrease in the adjusted first simulation execution performance is less than or equal to the target performance threshold, where the first simulation execution performance is the simulation execution performance of the candidate core parameters on the current layer.

4. The method according to claim 1, characterized in that, After adjusting the candidate core parameters corresponding to the current layer, the method further includes: For each of the other computing layers, a second simulation execution performance is obtained; wherein, the second simulation execution performance is the simulation execution performance of the candidate core parameters of the current layer on the other computing layers after adjustment; In addition, a third simulation execution performance is obtained; wherein the third simulation execution performance is the simulation execution performance of the candidate core parameters of the other computing layers on the other computing layers; Based on the second simulation execution performance and the third simulation execution performance, determine whether the adjusted candidate core parameters corresponding to the current layer are suitable for the other computing layers.

5. The method according to claim 4, characterized in that, The step of determining whether the adjusted candidate core parameters corresponding to the current layer are suitable for the other computing layers based on the second simulation execution performance and the third simulation execution performance includes: In response to the second simulation execution performance being less than the product of the third simulation execution performance and a preset multiple, the adjusted candidate core parameters corresponding to the current layer are determined to adapt to the other computing layers; wherein, the preset multiple is greater than 1; In response to the second simulation execution performance being greater than or equal to the product of the third simulation execution performance and a preset multiple, the adjusted candidate core parameters corresponding to the current layer are determined, which are not compatible with the other computing layers.

6. The method according to claim 1, characterized in that, The step of selecting the plurality of core design parameters from the adjusted candidate core parameters includes: A set of core parameters is formed using the adjusted candidate core parameters. With the goal of reducing design cost, a target number of candidate core parameters are deleted from the core parameter set. The design cost includes the current engineering cost and the current model performance of the core parameter set. The current engineering cost represents the engineering cost corresponding to the current number of candidate core parameters in the core parameter set, and the current model performance represents the simulation model performance of the current candidate core parameters in the core parameter set on the target model. For the deleted core parameter set, the process of deleting a target number of candidate core parameters from the core parameter set with the goal of reducing design cost is repeated until the target stopping iteration requirement is met, and the current candidate core parameters of the core parameter set are used as the several core design parameters.

7. The method according to claim 6, characterized in that, The current engineering cost is related to the performance of the baseline model of the target model and the number of parameters of the current candidate core parameters in the core parameter set; And / or, the target stopping iteration requirement includes the number of parameters of the current candidate core parameters in the core parameter set satisfying the target number threshold.

8. The method according to any one of claims 3-7, characterized in that, The simulation execution performance includes latency overhead; And / or, the performance of the simulation model includes latency overhead.

9. A method for constructing a core parameter library, characterized in that, The method includes: Obtain a reference model set corresponding to the reference scenario; wherein the reference model set includes several reference models involved in the reference scenario; A first reference core parameter library is constructed for each of the aforementioned reference models; wherein, the core parameter library is constructed using the core parameter library construction method as described in any one of claims 1-8; Using each of the first reference core parameter libraries, a second reference core parameter library is constructed for the reference model set; The second reference core parameter library includes several second reference core parameters; the step of constructing the second reference core parameter library for the reference model set using each of the first reference core parameter libraries includes: Each of the aforementioned reference models is used as the current model, and the first reference core parameter library corresponding to each of the aforementioned reference models is used as the current parameter library. For each first reference core parameter in the current parameter library, the first reference core parameter is adjusted in the direction of adapting the first reference core parameter to at least one other reference model, so that the adjusted first reference core parameter can adapt to the current model and at least one other reference model. Using the adjusted parameters of each of the first reference core particles, a second reference core particle parameter library is constructed for the reference model set.

10. The method according to claim 9, characterized in that, The number of core parameters in the second reference core parameter library is less than the number of first references. The number of first references is the sum of the number of core parameters in each of the first reference core parameter libraries. The second reference core parameter library includes several second reference core parameters, and at least one second reference core parameter is adapted to at least two reference models.

11. The method according to claim 10, characterized in that, Each of the aforementioned reference models includes multiple computational layers. The second reference core parameter adaptation to at least two reference models includes: the reference model corresponding to the second reference core parameter adaptation and at least one computational layer adapting to at least one other reference model.

12. The method according to claim 9, characterized in that, Each of the aforementioned reference models includes multiple computational layers. Adapting the adjusted first reference core parameters to other reference models includes: adapting the adjusted first reference core parameters to at least one computational layer of the other reference model; adjusting the first reference core parameters according to an adjustment direction that adapts the first reference core parameters to at least one other reference model includes: The first reference core parameter is adjusted in an adjustment direction that maximizes the reference adaptation gain of the first reference core parameter; wherein, the reference adaptation gain represents the proportion of the number of computational layers of other adapted reference models to the number of second references, and the number of second references is the number of computational layers of the current model.

13. A method for integrating core parameters, characterized in that, The method includes: Obtain a core parameter library constructed for a first model set; wherein, a plurality of first models in the first model set are models involved in the target scene, and the core parameter library is constructed using the core parameter library construction method as described in any one of claims 9-12; In addition, a second model set corresponding to the business scenario is obtained; wherein the second model set includes at least one second model, the second model is a model involved in the business scenario, and the business scenario is a sub-scenario of the target scenario; Find the matching core parameters that match the second model set from the core parameter library; The step of finding matching core parameters from the core parameter library that match the second model set includes: A preset number of core parameters are randomly selected from the core parameter library to form the current core parameter combination; Based on the current combination of core parameters, determine the execution performance of the first task in the simulation task; From the remaining core parameters in the core parameter library, find at least one core parameter and replace at least one current core parameter in the current core parameter combination to obtain the replaced parameter combination; Based on the replaced parameter combination, the execution performance of the second task for performing the simulation task is determined; Based on the execution performance of the first task and the execution performance of the second task, a new current core parameter combination is selected from the current core parameter combination and the replaced parameter combination. The process of finding at least one core parameter from the remaining core parameters in the core parameter library and replacing at least one current core parameter in the current core parameter combination is repeated to obtain the replaced parameter combination and subsequent steps until the iteration stop requirement is met. The current core parameter in the last current core parameter combination is taken as the matching core parameter.

14. The method according to claim 13, characterized in that, The step of selecting a new current core parameter combination from the current core parameter combination and the replaced parameter combination includes: In response to the second task's execution performance being higher than the first task's execution performance, or the second task's execution performance being lower than the first task's execution performance and meeting the retention conditions, the replaced parameter combination is used as the new current core particle parameter combination; wherein, the retention conditions include at least one of the following: the difference between the second task's execution performance and the first task's execution performance is less than a difference threshold, or the temperature parameter of the selection algorithm is greater than a temperature threshold, wherein the selection algorithm is used to execute the algorithm for randomly selecting a preset number of core particle parameters from the core particle parameter library, constructing the current core particle parameter combination, and subsequent steps; In response to the second task's execution performance being lower than the first task's execution performance and not meeting the retention condition, the current core parameter combination is used as the new current core parameter combination.

15. The method according to claim 13, characterized in that, The preset quantity is the number of chip slots on the simulation hardware template, which is used to perform the simulation task. And / or, there are several simulation tasks, and at least two second models, with different call ratios for the at least two second models in the business scenario, and several simulation tasks are generated according to the call ratio of each second model.

16. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores program instructions, and the processor executes the program instructions to implement the chip parameter library construction method as described in any one of claims 1-8, and / or the chip parameter library construction method as described in any one of claims 9-12, and / or the chip parameter integration method as described in any one of claims 13-15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed to implement the chip parameter library construction method as described in any one of claims 1-8, and / or the chip parameter library construction method as described in any one of claims 9-12, and / or the chip parameter integration method as described in any one of claims 13-15.

Images