Method, device, electronic device and computer readable storage medium for graph optimization

Abstract

Embodiments of the present application disclose a graph optimization method and device, electronic equipment and a computer readable storage medium. The method comprises: obtaining first information and second information determined offline, wherein the first information comprises a first subgraph equivalence rule set, and the second information comprises performance acceleration indicators of each subgraph equivalence rule in the first subgraph equivalence rule set; determining a second subgraph equivalence rule set from the first subgraph equivalence rule set, wherein the second subgraph equivalence rule set is a set of subgraph equivalence rules effective on a hardware platform to which an input model is to be deployed; and optimizing the input model according to the second subgraph equivalence rule set and the performance acceleration indicators of each subgraph equivalence rule in the second subgraph equivalence rule set, to obtain an optimized model. Through the present application, the problem of poor real-time performance of the optimization process caused by using absolute performance indicators for graph optimization in the related art is solved.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and more particularly to a method, apparatus, electronic device, and computer-readable storage medium for graph optimization. Background Technology

[0002] In classic approaches, fixed graph optimization rules are manually set and applied equally to all models. The problems with this approach are: a fixed set of rules lacks versatility, and the optimization results for some new models are unpredictable; it also has poor generalization ability on new platforms; furthermore, the optimization rules are coupled with each other, and some rules may not be suitable for certain networks, leading to high maintenance costs.

[0003] To address the aforementioned issues, related technologies propose using absolute performance metrics based on cost model feedback to evaluate all optimization strategies and find the optimal implementation. This approach can solve the generality and maintainability problems of the classic solutions mentioned above. However, the cost model is often time-consuming, and the optimization process may require deployment time ranging from minutes to hours, which seriously affects the real-time performance of the deployment.

[0004] The problem of poor real-time performance in the optimization process caused by the use of absolute performance metrics in graph optimization in related technologies has not yet been effectively solved. Summary of the Invention

[0005] To address the aforementioned technical problems, embodiments of the present invention aim to provide a method, apparatus, electronic device, and computer-readable storage medium for graph optimization, which can solve the problem of poor real-time performance in the optimization process caused by the use of absolute performance indicators in graph optimization in related technologies.

[0006] This invention provides a graph optimization method, comprising: acquiring offline determined first information and second information, wherein the first information includes a first subgraph equivalence rule set, and the second information includes performance acceleration indicators of each subgraph equivalence rule in the first subgraph equivalence rule set; determining a second subgraph equivalence rule set from the first subgraph equivalence rule set, wherein the second subgraph equivalence rule set is a set of subgraph equivalence rules valid on the hardware platform where the input model will be deployed; and optimizing the input model according to the second subgraph equivalence rule set and the performance acceleration indicators of each subgraph equivalence rule in the second subgraph equivalence rule set to obtain an optimized model.

[0007] Optionally, determining the second subgraph equivalence rule set from the first subgraph equivalence rule set includes: obtaining the hardware description file of the hardware platform to which the input model will be deployed, wherein the hardware description file describes the specifications supported by the hardware platform, and the specifications supported by the hardware platform include at least one of the following: operator (OP) type constraints, specific type OP operation mode constraints, OP input / output size constraints, and OP connection relationship constraints; for each subgraph equivalence rule in the first subgraph equivalence rule set that matches the input model, determining whether the operators of the target subgraph in the subgraph equivalence rule satisfy the hardware description file for each OP; and obtaining the second subgraph equivalence rule set based on the determination result, wherein all operators of each target subgraph in the second subgraph equivalence rule set satisfy the hardware description file.

[0008] Optionally, obtaining the second subgraph equivalence rule set based on the judgment result includes: selecting from the judgment result a subgraph equivalence rule set in which all operators of the target subgraph satisfy the hardware description file; and determining the subgraph equivalence rule set as the second subgraph equivalence rule set.

[0009] Optionally, the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set included in the second information is determined offline, including: setting test data for each subgraph equivalence rule in the first subgraph equivalence rule set; obtaining the performance acceleration index of the subgraph equivalence rule under the test data in a preset hardware test environment, thereby obtaining the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set, wherein the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set is stored in an offline database.

[0010] Optionally, optimizing the input model based on the second subgraph equivalence rule set and the performance acceleration index of each subgraph equivalence rule in the second subgraph equivalence rule set to obtain the optimized model includes: executing the following iterative steps until the number of times the output model has been optimized reaches the termination condition and / or the output model does not change: determining all matching points of the input model; traversing all matching points, for each matching point, selecting the target subgraph of the subgraph equivalence rule in the second subgraph equivalence rule set whose performance acceleration index meets the preset condition, replacing the current subgraph of the matching point, to obtain the output model; using the output model as the input model for the next iterative step.

[0011] Optionally, the first set of subgraph equivalence rules included in the first information is determined offline by: given a subgraph size, constructing a set of subgraphs within the range of the subgraph size; comparing the subgraph equivalence relations in the subgraph sets pairwise; recording equivalent subgraph pairs to obtain the first set of subgraph equivalence rules.

[0012] Optionally, after determining the first subgraph equivalence rule set included in the first information offline, the method further includes: determining whether the target subgraph in the first subgraph equivalence rule set contains OPs and OP connections that are not supported by the hardware platform where the input model will be deployed; if the determination result is yes, obtaining the target subgraph containing OPs and OP connections that are not supported by the hardware platform, and the subgraph equivalence rule corresponding to the target subgraph, removing the subgraph equivalence rule from the first subgraph equivalence rule set to obtain the third subgraph equivalence rule set.

[0013] This invention also provides a graph optimization apparatus, comprising: a first online optimization module, configured to acquire first information and second information determined offline, wherein the first information includes a first subgraph equivalence rule set, and the second information includes performance acceleration indicators of each subgraph equivalence rule in the first subgraph equivalence rule set; a second online optimization module, configured to determine a second subgraph equivalence rule set from the first subgraph equivalence rule set, wherein the second subgraph equivalence rule set is a set of subgraph equivalence rules valid on the hardware platform on which the input model will be deployed; and a third online optimization module, configured to optimize the input model according to the second subgraph equivalence rule set and the performance acceleration indicators of each subgraph equivalence rule in the second subgraph equivalence rule set, to obtain an optimized model.

[0014] Optionally, the second online optimization module includes: a first acquisition unit, for acquiring a hardware description file of the hardware platform to which the input model will be deployed, wherein the hardware description file describes the specifications supported by the hardware platform, and the specifications supported by the hardware platform include at least one of the following: operator (OP) type constraints, specific type OP operation mode constraints, OP input / output size constraints, and OP connection relationship constraints; a judgment unit, for judging each OP of the subgraph equivalence rules in the first subgraph equivalence rule set that match the input model, whether the operators of the target subgraph in the subgraph equivalence rules satisfy the hardware description file; and a second acquisition unit, for obtaining a second subgraph equivalence rule set based on the judgment result, wherein all operators of each target subgraph in the second subgraph equivalence rule set satisfy the hardware description file.

[0015] The second acquisition unit includes: a selection subunit, configured to select from the judgment results a set of subgraph equivalence rules for which all operators of the target subgraph satisfy the hardware description file; and a determination subunit, configured to determine the set of subgraph equivalence rules as the second set of subgraph equivalence rules.

[0016] The performance acceleration metrics of each subgraph equivalence rule in the first subgraph equivalence rule set included in the second information are determined offline, including: setting test data for each subgraph equivalence rule in the first subgraph equivalence rule set; obtaining the performance acceleration metrics of the subgraph equivalence rule under the test data in a preset hardware test environment, thereby obtaining the performance acceleration metrics of each subgraph equivalence rule in the first subgraph equivalence rule set, wherein the performance acceleration metrics of each subgraph equivalence rule in the first subgraph equivalence rule set are stored in an offline database;

[0017] The third online optimization module includes a processing unit for executing the following iterative steps until the number of times the output model has been optimized reaches the termination condition and / or the output model remains unchanged: determining all matching points of the input model; traversing all matching points, for each matching point, selecting the target subgraph of the subgraph equivalence rule that satisfies the preset condition in the second subgraph equivalence rule set, replacing the current subgraph of that matching point, and obtaining the output model; using the output model as the input model for the next iterative step;

[0018] The first subgraph equivalence rule set included in the first information is determined offline, including: given a subgraph size, constructing a set of subgraphs within the range of the subgraph size; comparing the subgraph equivalence relations in the subgraph sets pairwise; recording equivalent subgraph pairs to obtain the first subgraph equivalence rule set; after determining the first subgraph equivalence rule set included in the first information offline, it further includes: determining whether the target subgraph in the first subgraph equivalence rule set contains operators (OPs) and OP connections that are not supported by the hardware platform where the input model will be deployed; if the determination result is yes, obtaining the target subgraph containing OPs and OP connections that are not supported by the hardware platform, and the subgraph equivalence rule corresponding to the target subgraph, removing the subgraph equivalence rule from the first subgraph equivalence rule set to obtain the third subgraph equivalence rule set.

[0019] This invention provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to perform the steps of any of the above methods.

[0020] This invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of any of the above methods.

[0021] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for graph optimization. The method includes: acquiring offline determined first information and second information, wherein the first information includes a first subgraph equivalence rule set, and the second information includes performance acceleration metrics for each subgraph equivalence rule in the first subgraph equivalence rule set; determining a second subgraph equivalence rule set from the first subgraph equivalence rule set, wherein the second subgraph equivalence rule set is a set of subgraph equivalence rules valid on the hardware platform where the input model will be deployed; and optimizing the input model based on the second subgraph equivalence rule set and the performance acceleration metrics of each subgraph equivalence rule in the second subgraph equivalence rule set to obtain an optimized model. In other words, in each optimization process, the embodiments of the present invention obtain the effective subgraph equivalence rules on the hardware platform and the performance acceleration index (e.g., speedup ratio or speedup level) of each subgraph equivalence rule as a relative index for graph optimization, instead of relying on the absolute performance index of the absolute time consumption of the cost model simulation. This solves the problem of poor real-time performance of the optimization process caused by the use of absolute performance indexes for graph optimization in related technologies, and achieves the technical effect of improving the real-time performance of the graph optimization optimization process. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0023] Figure 1 A flowchart illustrating a graph optimization method provided in an embodiment of the present invention;

[0024] Figure 2 A schematic diagram (a) of a graph optimization device provided for an embodiment of the present invention;

[0025] Figure 3 A schematic diagram (II) of a graph optimization device provided for an embodiment of the present invention;

[0026] Figure 4 A schematic diagram (III) of a graph optimization device provided for an embodiment of the present invention;

[0027] Figure 5 A schematic diagram (four) of a graph-optimized device provided for an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish different objects, rather than to limit a specific order.

[0030] This invention provides a graph optimization method. Graph optimization belongs to the field of deep learning. Current deep learning algorithm models can be represented using graph data structures (or computation graphs). Graph optimization optimizes and transforms the model structure based on the computation graph, thereby improving specific properties of the computation graph. These property metrics are related to specific tasks. For example, in deep learning deployment tasks, the purpose of graph optimization is to make the computation graph more closely aligned with specific hardware (Graphics Processing Unit (GPU), Central Processing Unit (CPU), Distributed Processing Unit (DPU), etc.), thereby enabling more efficient inference execution on that hardware.

[0031] Optionally, the application scenarios of the above graph optimization method include, but are not limited to, face recognition models, action detection models, speech recognition models, etc., without any limitation.

[0032] Figure 1 This is a flowchart illustrating a graph optimization method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the graph optimization method provided in this application includes the following steps:

[0033] S102, obtain the first information and the second information determined offline, wherein the first information includes a first subgraph equivalence rule set, and the second information includes the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set;

[0034] It should be noted that, in this embodiment, the aforementioned performance acceleration metrics include, but are not limited to, speedup ratio and speedup level. The aforementioned first and second information can be stored in an offline database, which can provide a data source for the optimization model.

[0035] Optionally, the performance acceleration metrics of each subgraph equivalence rule in the first subgraph equivalence rule set included in the second information are determined offline, including:

[0036] S11, Set test data for each subgraph equivalence rule in the first subgraph equivalence rule set;

[0037] S12, Under a preset hardware test environment, the performance acceleration index of the subgraph equivalence rule under the test data is obtained respectively, and the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set is obtained. The performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set is stored in an offline database.

[0038] Optionally, in this embodiment, the testing method of the above-mentioned hardware testing environment includes, but is not limited to, the actual hardware executing the subgraph in the first subgraph equivalence rule set, or obtaining the time consumption data of the subgraph in the first subgraph equivalence rule set through hardware modeling, simulation, time consumption prediction and other technologies.

[0039] If a new hardware platform is used to execute the above subgraph, only the hardware needs to be switched for testing, and the other processes remain unchanged.

[0040] Through the above steps S11~S12, it can be ensured that the graph optimization method provided in this embodiment has transferability to new computational graphs or new hardware, and improves generalization ability.

[0041] It should be noted that the test data mentioned above are based on possible input values ​​obtained from real-world business scenarios where the current model (e.g., a neural network model) might be applied. For example, in the field of video surveillance, possible input image sizes include 4K, 2K, 1080p, 720p, etc., and no specific limitation is made here.

[0042] The following example, using 1000 sets of test data, illustrates the above S11~S12.

[0043] In this example, the method for obtaining the performance acceleration metrics (i.e., the first information mentioned above) for all equivalent subgraphs includes the following steps:

[0044] S21. Considering that the actual execution utilization of hardware such as Neural-network Processing Unit (NPU) is related to the specific input data size (data size, alignment, etc.), the test data constructed in this example needs to cover the size within the typical numerical range as much as possible. In particular, this example selects the form of randomly constructing 1000 sets of data sizes.

[0045] S22, based on the batch test data of S21, is the actual hardware test of the performance speedup ratio of the current equivalent rule, which can yield batch performance index data.

[0046] S23. Summarize the performance speedup data from 1000 sets to determine the performance improvement of the equivalence rule under general conditions. The summarization method can be varied, such as averaging the speedup, least squares regression, multinomial fitting, or even using lightweight machine learning fitting methods. (It is necessary to ensure the real-time acquisition of performance improvement metrics).

[0047] S24. For the above set of equivalence rules, repeat steps S21 to S23 to obtain the performance acceleration metrics of all subgraph equivalence rules.

[0048] Optionally, the set of first subgraph equivalence rules included in the aforementioned first information is determined offline and includes:

[0049] S31, Given the size of a subgraph, construct a set of subgraphs within the range of that size;

[0050] S32, compare the subgraph equivalence relations in the subgraph set pairwise;

[0051] It should be noted that the above subgraph equivalence relation means that two subgraphs have completely identical outputs when the inputs are the same. If s represents the subgraph equivalence relation, then the subgraph equivalence relation s is g_1 = g_2, where g is a small computation graph composed of 0 or more interconnected operators.

[0052] S33, record the pairs of equivalent subgraphs, and obtain the set of equivalence rules for the first subgraph.

[0053] By using steps S31 to S33 above, the subgraph equivalence rules are determined offline, which can guarantee a large optimization space and further improve the quality of graph optimization.

[0054] Optionally, in this embodiment, after determining the first subgraph equivalence rule set included in the first information offline, the method further includes:

[0055] S41, determine whether the target subgraph in the first subgraph equivalence rule set contains OPs and OP connections that are not supported by the hardware platform where the input model will be deployed;

[0056] S42, if the judgment result is yes, obtain the target subgraph containing OPs and OP connections that are not supported by the hardware platform, as well as the subgraph equivalence rules corresponding to the target subgraph, remove the subgraph equivalence rules from the first subgraph equivalence rule set, and obtain the third subgraph equivalence rule set.

[0057] In other words, in this embodiment, the subgraph equivalence rule set can be pre-screened offline before graph optimization, which can reduce the burden during online optimization.

[0058] S104, determine the second subgraph equivalence rule set from the first subgraph equivalence rule set, wherein the second subgraph equivalence rule set is the set of subgraph equivalence rules that are valid on the hardware platform on which the input model will be deployed;

[0059] It should be noted that the input models mentioned above include, but are not limited to, deep learning models, and the hardware platforms mentioned above include, but are not limited to, AI chips, Google TPUs, Hisense Nnies, and other hardware or chips containing dedicated neural network computing cores. Deployment technology refers to the process of running a trained deep learning model or neural network on specific hardware and the technology stack involved.

[0060] Optionally, determining the second subgraph equivalence rule set from the first subgraph equivalence rule set may include:

[0061] S51, Obtain the hardware description file of the hardware platform to which the input model will be deployed, wherein the hardware description file is used to describe the specifications supported by the hardware platform.

[0062] It should be noted that the specifications supported by the aforementioned hardware platform can be the set of capabilities that the hardware (such as the NPU or the vendor's underlying software) can support. This set of capabilities includes the range / boundaries of various functions the hardware can support; for example, if the hardware supports convolution operations, there will be constraints such as the maximum supported convolution input / output size, kernel size, and contextual relationships. These are determined by the hardware and are independent of the input model. Generally, specification constraints include OP type constraints, constraints on the operation mode of specific OP types, OP input / output size constraints, and OP connection relationship constraints. For example, a JSON file can be used to record the various parameter limitations of the OPs supported by the hardware, forming a hardware description file.

[0063] S52, for the subgraph equivalence rules in the first subgraph equivalence rule set that match the input model, each operation determines whether the operator of the target subgraph in the subgraph equivalence rule satisfies the hardware description file;

[0064] S53, based on the judgment result, the second subgraph equivalence rule set is obtained, wherein all operators of each target subgraph in the second subgraph equivalence rule set satisfy the hardware description file.

[0065] Optionally, S53 may include: S531, selecting from the judgment results a set of subgraph equivalence rules for which all operators of the target subgraph satisfy the hardware description file; S532, determining the subgraph equivalence rule set as the second subgraph equivalence rule set. This step effectively determines the second subgraph equivalence rule set from the first subgraph equivalence rule set, providing a foundation for graph optimization.

[0066] Optionally, the first subgraph equivalence rule set mentioned above may be the same as the second subgraph equivalence rule set mentioned above, or the second subgraph equivalence rule set may be a subset of the first subgraph equivalence rule set mentioned above.

[0067] For example, if the first subgraph equivalence rule set is {s1, s2, s3}, then the second subgraph equivalence rule set can be {s1, s2, s3}, {s1, s2}, or {s3}, where s1, s2, and s3 each represent a subgraph equivalence rule.

[0068] For example, suppose the source subgraph is A+B+C, and we replace it with the target subgraph ABC using a subgraph equivalence rule s1, where "+" and "-" represent the "+" operator and the "-" operator, respectively. During hardware validity assessment, we need to check whether each operator in the target subgraph ABC satisfies the hardware description file. Taking the OP input / output size as an example, suppose the input data A of the first operator is 1024, and the hardware requires the corresponding data size to be less than or equal to 1024. Then we can determine that the input of the first operator matches the hardware. Next, we use the same method to check whether the input of the second operator matches the hardware, until all OPs satisfy the hardware description file. In this case, the target subgraph is a valid equivalent subgraph on the hardware platform, and the corresponding subgraph equivalence rule s1 is a valid subgraph equivalence rule.

[0069] For example, suppose the source subgraph is A×3, and a subgraph equivalence rule s2 is used to replace it with the target subgraph A+A+A, where "×" and "+" represent the "×" operator and the "+" operator, respectively. Then, during hardware validity judgment, it is necessary to check whether the operators of the target subgraph A+A+A satisfy the hardware description file one by one. Taking the OP input and output size as an example, suppose the size of the input data A of the first operator is 2048, while the hardware requires the corresponding data size to be less than or equal to 1024. Then it can be determined that the input of the first operator does not match the hardware. Therefore, the target subgraph is an invalid equivalent subgraph on the hardware platform, and the corresponding subgraph equivalence rule s2 is an invalid subgraph equivalence rule.

[0070] Therefore, the hardware specification constraint problem is further solved through the subgraph equivalence rule filtering mechanism in steps S51 to S53 above.

[0071] S106. Based on the set of equivalence rules for the second subgraph and the performance acceleration index of each subgraph equivalence rule in the set of equivalence rules for the second subgraph, optimize the input model to obtain the optimized model.

[0072] It should be noted that the source subgraphs in the aforementioned first or second subgraph equivalence rule set can exceed the hardware rules. For example, the input model M0 might contain subgraph g11, which is not supported by the hardware. However, if rule 11 exists: g11 = g0, then g11 in M0 can be replaced with g0, allowing the optimized model to run on the hardware.

[0073] Optionally, in this embodiment, S106 may include:

[0074] S61, Execute the following iterative steps until the number of times the output model has been optimized reaches the termination condition and / or the output model remains unchanged:

[0075] S611, determine all matching points of the input model; S612, traverse all matching points, for each matching point, select the target subgraph of the subgraph equivalence rule that satisfies the preset condition in the second subgraph equivalence rule set, and replace the current subgraph of the matching point to obtain the output model; S613, use the output model as the input model for the next loop step.

[0076] For example, suppose the initial model is ABCD, which becomes A-(b1-b2)-CD after one optimization iteration. After this optimization iteration, it is found that b2-C ​​has a better equivalent subgraph E, so it will iterate to A-b1-ED until the number of iterations reaches the termination condition and / or the model does not change, that is, the model optimization is complete. Here, "-" represents an operator, and b1-b2 is an equivalent subgraph of B.

[0077] Optionally, the aforementioned preset conditions may include: greater than a certain preset threshold, equal to a certain preset threshold, or an optimal value. These preset conditions can be set according to the application scenario and efficiency metrics, and are not limited here.

[0078] For example, assuming the current matching point is M, and the performance acceleration metrics of subgraph equivalence rules s1, s2, and s3 in the second subgraph equivalence rule set are 0.8, 1.2, and 1 respectively, and the above preset condition is set to be greater than 0.9, then for the matching point M, the above subgraph equivalence rules s2 and s3 can be set as the target subgraph equivalence rules to replace the corresponding existing subgraph.

[0079] For example, assuming the current matching point is M, and the performance acceleration metrics of subgraph equivalence rules s1, s2, and s3 in the second subgraph equivalence rule set are 0.8, 1.2, and 1 respectively, and the above preset condition is set to equal to 1.2, then the above subgraph equivalence rule s2 can be set as the target subgraph equivalence rule to replace the corresponding existing subgraph.

[0080] Each iteration of S61 and S611~S613 ensures that the model's performance is improved, resulting in a high-quality final model that can be well executed on specific hardware.

[0081] In summary, through the above steps S102~S106, first and second information determined offline are obtained. The first information includes a first subgraph equivalence rule set, and the second information includes the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set. A second subgraph equivalence rule set is determined from the first subgraph equivalence rule set, wherein the second subgraph equivalence rule set is a set of subgraph equivalence rules valid on the hardware platform where the input model will be deployed. Based on the second subgraph equivalence rule set and the performance acceleration index of each subgraph equivalence rule in the second subgraph equivalence rule set, the input model is optimized to obtain the optimized model. That is to say, in each optimization process, this embodiment of the invention obtains the valid subgraph equivalence rules on the hardware platform and the performance acceleration index (e.g., speedup ratio or speedup level) of each subgraph equivalence rule as a relative index for graph optimization, instead of relying on the absolute performance index of the absolute time consumption of the cost model simulation. This solves the problem of poor real-time performance in the optimization process caused by the use of absolute performance indexes in graph optimization in related technologies, and achieves the technical effect of improving the real-time performance of the graph optimization optimization process.

[0082] The following examples illustrate this embodiment.

[0083] Example 1

[0084] In Example 1, a method for graph optimization is provided, which includes the following steps:

[0085] S71. Given a set of neural network operators (op), generate possible optimization strategies offline (corresponding to the set of equivalent rules for the first subgraph mentioned above).

[0086] S72. During online deployment, the optimizer evaluates all optimization strategies based on the performance acceleration metrics of the above optimization strategies and selects the best optimization strategy.

[0087] S73. Using the above-mentioned optimal optimization strategy, optimize the input model to obtain the optimized model.

[0088] This example addresses the generality and maintainability issues of classic solutions mentioned in the background section. Furthermore, because it enumerates all possible optimization strategies, it is also well-suited for models using Neural Processing Units (NPUs).

[0089] Example 1 above is an example of the main solution of the embodiment of this application. The specific implementation solution of this embodiment will be illustrated below with reference to Example 2 and Example 3.

[0090] Example 2

[0091] In Example 2, a method for graph optimization is provided, which includes the following steps:

[0092] S81, offline generation of the first equivalent rule set S (containing 1000 equivalent rules);

[0093] S82, the relative performance label of each equivalence rule in test S, that is, the performance acceleration index of each equivalence rule;

[0094] S83 optimizes the input model M0:

[0095] S831, find a matching point p0 in M0, with a corresponding subgraph g0. (A matching point refers to a subgraph structure contained in M0, which is the source subgraph of some equivalent rules in S.) Figure 1 To);

[0096] Correspondingly, M0 can be represented as: M0_0 (before g0) + g0 + M0_1 (after g0)

[0097] S832, matching points p0 and g0 match the five equivalent rules in S, namely...

[0098] rule0: g0=g1, speedup ratio 1.1;

[0099] rule 1: g0 = g2, speedup ratio 1.2;

[0100] Rule 2: g0 = g3, speedup ratio 1.3;

[0101] Rule 3: g0 = g4, acceleration ratio 1.4;

[0102] Rule 4: g0 = g5, speedup ratio 1.05;

[0103] S833 performs hardware validity checks on the above five equivalence rules, namely:

[0104] Rule0: g1 meets the hardware requirements, the rule is valid;

[0105] Rule 1: g2 contains matrix multiplication operations, which are not supported by the hardware, so the rule is invalid.

[0106] Rule 2: g3 contains an addition operator, and the hardware supports a maximum input of 1024; however, if a replacement is performed, the input of this addition operator is 2048, which is not supported by the hardware, and the rule is invalid.

[0107] Rule 3: g4 contains a structure that connects pooled ops to tanh ops, which is not supported by the hardware, so the rule is invalid.

[0108] Rule 4: g5 meets the hardware requirements, the rule is valid;

[0109] In S835, between rule0 and rule4, select rule0 and perform the replacement. Replace g0 in M0 with g1 to obtain the new model M1, which can be represented as M0_0 (before g0) + g1 + M0_1 (after g0).

[0110] S836, take M1 as the input for the next iteration, and repeat the process from S831 to S836.

[0111] It should be noted that in graph optimization, multiple matching points may be found in a single iteration; only one has been described above, and the rest are similar. Furthermore, graph optimization can involve multiple iterations; only one has been described above, and the rest are similar.

[0112] Example 3

[0113] In Example 3, a method for graph optimization is provided, which includes the following steps:

[0114] S91, generates the first equivalent rule set S offline (containing 1000 equivalent rules).

[0115] S92 performs preliminary filtering on S. The filtering is based on the fact that if the target subgraph contains ops or op connections (topologies) that are not supported by the hardware, they are not adopted, resulting in 800 rules S1.

[0116] S93 tests the relative performance metrics of each equivalence rule in S1, i.e., the performance acceleration metrics of each equivalence rule.

[0117] S94 optimizes the input model M0;

[0118] S941, find a matching point p0 in M0, with a corresponding subgraph g0. (A matching point means a subgraph structure in M0 that corresponds to a source subgraph of certain rules in S1.) Figure 1 To);

[0119] S942, the rules that match p0 and g0 include 3 rules: rule0, rule2, and rule4. The other two rules were filtered out before optimization.

[0120] S943, perform validity checks on the above 3 rules and filter to ReLU2;

[0121] S944, select ReLU2, perform the replacement, replace g0 contained in M0 with g1, and obtain the new model M1, which can be represented as M0_0 (before g0) + g1 + M0_1 (after g0).

[0122] S945, take M1 as the input for the next iteration, and repeat S941~S945.

[0123] It should be noted that in graph optimization, multiple matching points may be found in a single iteration; only one has been described above, and the rest are similar. Furthermore, graph optimization can involve multiple iterations; only one has been described above, and the rest are similar.

[0124] In summary, the graph optimization method provided in this embodiment does not rely on hardware simulation to obtain absolute execution time, ensuring that a graph optimization strategy of similar quality can be obtained in a shorter time. It possesses better real-time performance and engineering practice potential, aligns well with the design of automatic graph optimization schemes for NPUs, and is more compatible with the AI ​​chip hardware deployment process. It addresses hardware specification constraints through a rule-based filtering mechanism. Furthermore, compared to expert rule-based schemes, it exhibits superior optimization quality and generalization ability.

[0125] This invention also provides a graph optimization apparatus. Graph optimization belongs to the field of deep learning. Current deep learning algorithm models can be represented using graph data structures (or computation graphs). Graph optimization optimizes and transforms the model structure based on the computation graph, thereby improving specific properties of the computation graph. These property indicators are related to specific tasks. For example, in deep learning deployment tasks, the purpose of graph optimization is to make the computation graph more closely aligned with specific hardware (Graphics Processing Unit (GPU), Central Processing Unit (CPU), Distributed Processing Unit (DPU), etc.), thereby enabling more efficient inference execution on that hardware.

[0126] Optionally, the application scenarios of the above-mentioned graph-optimized device include, but are not limited to, face recognition models, action detection models, speech recognition models, etc., without any limitation.

[0127] Figure 2This is a schematic diagram of a graph-optimized device provided in an embodiment of the present invention, such as... Figure 2 As shown, the diagram optimization apparatus provided in this application includes:

[0128] The first online optimization module 22 is used to obtain first information and second information determined offline, wherein the first information includes a first subgraph equivalence rule set, and the second information includes the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set;

[0129] It should be noted that, in this embodiment, the aforementioned performance acceleration metrics include, but are not limited to, speedup ratio and speedup level. The aforementioned first and second information can be stored in an offline database, which can provide a data source for the optimization model.

[0130] Optionally, the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set included in the second information is determined offline, including: setting test data for each subgraph equivalence rule in the first subgraph equivalence rule set; obtaining the performance acceleration index of the subgraph equivalence rule under the test data in a preset hardware test environment, thereby obtaining the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set. The performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set is stored in an offline database, thus ensuring that the graph optimization method provided in this embodiment has transferability to new computational graphs or new hardware, and improving generalization ability.

[0131] Optionally, in this embodiment, the preset hardware test environment can be a currently known hardware environment or a new hardware environment. When a new hardware environment appears, incremental testing will be performed to update the first information.

[0132] Optionally, the first set of subgraph equivalence rules included in the first information is determined offline, including: given a subgraph size, constructing a set of subgraphs within that size range; comparing the subgraph equivalence relations in the set pairwise; and recording equivalent subgraph pairs to obtain the first set of subgraph equivalence rules. Determining subgraph equivalence rules offline ensures a large optimization space and further improves the quality of graph optimization.

[0133] It should be noted that the above subgraph equivalence relation means that two subgraphs have completely identical outputs when the inputs are the same. If s represents the subgraph equivalence relation, then the subgraph equivalence relation s is g_1 = g_2, where g is a small computation graph composed of 0 or more interconnected operators.

[0134] Optionally, after determining the first subgraph equivalence rule set included in the first information offline, the method further includes: determining whether the target subgraph in the first subgraph equivalence rule set contains OPs and OP connections that are not supported by the hardware platform where the input model will be deployed; if the determination result is yes, obtaining the target subgraph containing OPs and OP connections that are not supported by the hardware platform, and the subgraph equivalence rule corresponding to the target subgraph, removing the subgraph equivalence rule from the first subgraph equivalence rule set to obtain the third subgraph equivalence rule set. This allows for offline pre-screening of the subgraph equivalence rule set before graph optimization, thus reducing the burden during online optimization.

[0135] The second online optimization module 24 is used to determine a second subgraph equivalence rule set from the first subgraph equivalence rule set, wherein the second subgraph equivalence rule set is a set of subgraph equivalence rules that are valid on the hardware platform where the input model will be deployed;

[0136] It should be noted that the input models mentioned above include, but are not limited to, deep learning models, and the hardware platforms mentioned above include, but are not limited to, AI chips, Google TPUs, Hisense Nnies, and other hardware or chips containing dedicated neural network computing cores. Deployment technology refers to the process of running a trained deep learning model or neural network on specific hardware and the technology stack involved.

[0137] like Figure 3 As shown, the second online optimization module 24 includes:

[0138] The first acquisition unit 242 is used to acquire the hardware description file of the hardware platform to which the input model will be deployed, wherein the hardware description file is used to describe the specifications supported by the hardware platform.

[0139] It should be noted that the specifications supported by the aforementioned hardware platform can be the set of capabilities that the hardware (such as the NPU or the vendor's underlying software) can support. This set of capabilities includes the range / boundaries of various functions the hardware can support; for example, if the hardware supports convolution operations, there will be constraints such as the maximum supported convolution input / output size, kernel size, and contextual relationships. These are determined by the hardware and are independent of the input model. Generally, specification constraints include OP type constraints, constraints on the operation mode of specific OP types, OP input / output size constraints, and OP connection relationship constraints. For example, a JSON file can be used to record the various parameter limitations of the OPs supported by the hardware, forming a hardware description file.

[0140] The judgment unit 244 is used to determine, one operation at a time, whether the operator of the target subgraph in the subgraph equivalence rule that matches the input model in the first subgraph equivalence rule set satisfies the hardware description file.

[0141] Optionally, the first subgraph equivalence rule set mentioned above may be the same as the second subgraph equivalence rule set mentioned above, or the second subgraph equivalence rule set may be a subset of the first subgraph equivalence rule set mentioned above.

[0142] For example, if the first subgraph equivalence rule set is {s1, s2, s3}, then the second subgraph equivalence rule set can be {s1, s2, s3}, {s1, s2}, or {s3}, where s1, s2, and s3 each represent a subgraph equivalence rule.

[0143] For example, suppose the source subgraph is A+B+C, and we replace it with the target subgraph ABC using a subgraph equivalence rule s1, where "+" and "-" represent the "+" operator and the "-" operator, respectively. During hardware validity assessment, we need to check whether each operator in the target subgraph ABC satisfies the hardware description file. Taking the OP input / output size as an example, suppose the input data A of the first operator is 1024, and the hardware requires the corresponding data size to be less than or equal to 1024. Then we can determine that the input of the first operator matches the hardware. Next, we use the same method to check whether the input of the second operator matches the hardware, until all OPs satisfy the hardware description file. In this case, the target subgraph is a valid equivalent subgraph on the hardware platform, and the corresponding subgraph equivalence rule s1 is a valid subgraph equivalence rule.

[0144] For example, suppose the source subgraph is A×3, and a subgraph equivalence rule s2 is used to replace it with the target subgraph A+A+A, where "×" and "+" represent the "×" operator and the "+" operator, respectively. Then, during hardware validity judgment, it is necessary to check whether the operators of the target subgraph A+A+A satisfy the hardware description file one by one. Taking the OP input and output size as an example, suppose the size of the input data A of the first operator is 2048, while the hardware requires the corresponding data size to be less than or equal to 1024. Then it can be determined that the input of the first operator does not match the hardware. Therefore, the target subgraph is an invalid equivalent subgraph on the hardware platform, and the corresponding subgraph equivalence rule s2 is an invalid subgraph equivalence rule.

[0145] The second acquisition unit 246 is used to obtain the second subgraph equivalence rule set according to the judgment result, wherein all operators of each target subgraph in the second subgraph equivalence rule set satisfy the hardware description file;

[0146] Optionally, such as Figure 4As shown, the second acquisition unit 246 includes: a selection subunit 2462, used to select from the judgment result a set of subgraph equivalence rules for which all operators of the target subgraph satisfy the hardware description file; and a determination subunit 2464, used to determine the subgraph equivalence rule set as the second subgraph equivalence rule set. Figure 4 The apparatus shown can effectively determine the second subgraph equivalence rule set from the first subgraph equivalence rule set, providing a foundation for graph optimization.

[0147] Therefore, the hardware specification constraint problem is further solved through the subgraph equivalence rule filtering mechanism of the second online optimization module 24.

[0148] The third online optimization module 26 is used to optimize the input model based on the second subgraph equivalence rule set and the performance acceleration index of each subgraph equivalence rule in the second subgraph equivalence rule set, so as to obtain the optimized model.

[0149] It should be noted that the source subgraphs in the aforementioned first or second subgraph equivalence rule set can exceed the hardware rules. For example, the input model M0 might contain subgraph g11, which is not supported by the hardware. However, if rule 11 exists: g11 = g0, then g11 in M0 can be replaced with g0, allowing the optimized model to run on the hardware.

[0150] like Figure 5 As shown, the third online optimization module 26 includes a processing unit 262, which performs the following iterative steps until the number of times the output model has been optimized reaches the termination condition and / or the output model does not change: determine all matching points of the input model; traverse all matching points, and for each matching point, select the target subgraph of the subgraph equivalence rule that satisfies the preset condition in the second subgraph equivalence rule set, and replace the current subgraph of the matching point to obtain the output model; use the output model as the input model for the next iterative step.

[0151] For example, suppose the initial model is ABCD, which becomes A-(b1-b2)-CD after one optimization iteration. After this optimization iteration, it is found that b2-C ​​has a better equivalent subgraph E, so it will iterate to A-b1-ED until the number of iterations reaches the termination condition and / or the model does not change, that is, the model optimization is complete. Here, "-" represents an operator, and b1-b2 is an equivalent subgraph of B.

[0152] Optionally, the aforementioned preset conditions may include: greater than a certain preset threshold, equal to a certain preset threshold, or an optimal value. These preset conditions can be set according to the application scenario and efficiency metrics, and are not limited here.

[0153] For example, assuming the current matching point is M, and the performance acceleration metrics of subgraph equivalence rules s1, s2, and s3 in the second subgraph equivalence rule set are 0.8, 1.2, and 1 respectively, and the above preset condition is set to be greater than 0.9, then for the matching point M, the above subgraph equivalence rules s2 and s3 can be set as the target subgraph equivalence rules to replace the corresponding existing subgraph.

[0154] For example, assuming the current matching point is M, and the performance acceleration metrics of subgraph equivalence rules s1, s2, and s3 in the second subgraph equivalence rule set are 0.8, 1.2, and 1 respectively, and the above preset condition is set to equal to 1.2, then the above subgraph equivalence rule s2 can be set as the target subgraph equivalence rule to replace the corresponding existing subgraph.

[0155] Each iteration of the aforementioned processing unit 262 ensures that the model's performance is improved, resulting in a final model with good optimization quality, which in turn enables the optimized model to be well executed on specific hardware.

[0156] In summary, the above-described apparatus acquires offline determined first and second information. The first information includes a first set of subgraph equivalence rules, and the second information includes performance acceleration metrics for each subgraph equivalence rule in the first set. A second set of subgraph equivalence rules is determined from the first set, wherein the second set is a set of subgraph equivalence rules valid on the hardware platform where the input model will be deployed. Based on the second set of subgraph equivalence rules and the performance acceleration metrics for each subgraph equivalence rule in the second set, the input model is optimized to obtain an optimized model. In other words, in each optimization process, this embodiment of the invention acquires the valid subgraph equivalence rules on the hardware platform and the relative metrics (e.g., speedup ratio or speedup level) of each subgraph equivalence rule as relative metrics for graph optimization, rather than relying on the absolute performance metric of the absolute simulation time of the cost model. This solves the problem of poor real-time performance in the optimization process caused by using absolute performance metrics in related technologies, achieving the technical effect of improving the real-time performance of the graph optimization process.

[0157] This invention also provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to perform the steps of any of the above methods. The processor can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), a GPU, a CPU, a DPU, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0158] This invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of any of the above methods.

[0159] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A method of graph optimization, characterized by, The method comprises: obtaining first information and second information determined offline, wherein the first information comprises a first subgraph equivalence rule set, and the second information comprises performance acceleration indicators of subgraph equivalence rules in the first subgraph equivalence rule set; the performance acceleration indicators are determined offline by setting test data for each subgraph equivalence rule in the first subgraph equivalence rule set, and obtaining performance acceleration indicators of the subgraph equivalence rules under the test data in a preset hardware test environment to obtain performance acceleration indicators of the subgraph equivalence rules in the first subgraph equivalence rule set, wherein the performance acceleration indicators of the subgraph equivalence rules in the first subgraph equivalence rule set are stored in an offline database; determining a second subgraph equivalence rule set from the first subgraph equivalence rule set, comprising: obtaining a hardware description file of a hardware platform to which an input model is to be deployed, wherein the hardware description file is used to describe specifications supported by the hardware platform, and the specifications supported by the hardware platform comprise at least one of the following: operator OP type constraint, specific type OP operation mode constraint, OP input / output size constraint, and OP connection relationship constraint; for a subgraph equivalence rule in the first subgraph equivalence rule set that matches the input model, judging whether operators in a target subgraph in the subgraph equivalence rule satisfy the hardware description file OP by OP; and obtaining the second subgraph equivalence rule set according to the judgment result, wherein all operators of each target subgraph in the second subgraph equivalence rule set satisfy the hardware description file, and the second subgraph equivalence rule set is a set of subgraph equivalence rules that are effective on the hardware platform to which the input model is to be deployed; optimizing the input model according to the second subgraph equivalence rule set and the performance acceleration indicators of the subgraph equivalence rules in the second subgraph equivalence rule set to obtain an optimized model.

2. The method of claim 1, wherein, The obtaining of the second subgraph equivalence rule set according to the judgment result comprises: selecting, from the judgment result, a subgraph equivalence rule set in which all operators of the target subgraph satisfy the hardware description file; determining the subgraph equivalence rule set as the second subgraph equivalence rule set.

3. The method of claim 1, wherein, The optimizing of the input model according to the second subgraph equivalence rule set and the performance acceleration indicators of the subgraph equivalence rules in the second subgraph equivalence rule set to obtain an optimized model comprises: performing the following loop steps until the number of times of optimization of an output model reaches a termination condition and / or the output model does not change: determining all matching points of the input model; traversing all the matching points, and for each matching point in the matching points, selecting a target subgraph of a subgraph equivalence rule that satisfies a preset condition in terms of performance acceleration indicators from the second subgraph equivalence rule set to replace a current subgraph of the matching point to obtain an output model; taking the output model as an input model for the next loop step.

4. The method of claim 1, wherein, The first subgraph equivalence rule set comprised in the first information is determined offline by constructing a subgraph set within a subgraph scale range given the subgraph scale; comparing subgraph equivalence relations in the subgraph set two by two; recording equivalent subgraph pairs to obtain the first subgraph equivalence rule set.

5. The method of claim 4, wherein, After determining the first subgraph equivalence rule set included in the first information offline, further comprising: determining whether a target subgraph in the first subgraph equivalence rule set contains an OP and an OP connection that are not supported by a hardware platform to which the input model is to be deployed; in the case of a positive determination, obtaining the target subgraph containing the OP and the OP connection not supported by the hardware platform, and the subgraph equivalence rule corresponding to the target subgraph, removing the subgraph equivalence rule from the first subgraph equivalence rule set to obtain a third subgraph equivalence rule set.

6. An apparatus for graph optimization, the apparatus comprising: comprising: a first online optimization module for obtaining first information and second information determined offline, wherein the first information includes a first subgraph equivalence rule set, the second information includes a performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set, and the performance acceleration index is determined offline by setting test data for each subgraph equivalence rule in the first subgraph equivalence rule set, and obtaining the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set under the test data in a preset hardware test environment, wherein the performance acceleration index of each subgraph equivalence rule in the first subgraph equivalence rule set is saved in an offline database; a second online optimization module for determining a second subgraph equivalence rule set from the first subgraph equivalence rule set, wherein the second subgraph equivalence rule set is a set of subgraph equivalence rules effective on a hardware platform to which an input model is to be deployed; a third online optimization module for optimizing the input model according to the second subgraph equivalence rule set and the performance acceleration index of each subgraph equivalence rule in the second subgraph equivalence rule set to obtain an optimized model; the second online optimization module comprises: a first obtaining unit for obtaining a hardware description file of a hardware platform to which an input model is to be deployed, wherein the hardware description file is used to describe specifications supported by the hardware platform, and the specifications supported by the hardware platform include at least one of the following: operator OP type constraints, specific type OP operation mode constraints, OP input / output size constraints, and OP connection relationship constraints; a judging unit for judging whether an operator of a target subgraph in a subgraph equivalence rule in the first subgraph equivalence rule set that matches the input model satisfies the hardware description file; and a second obtaining unit for obtaining the second subgraph equivalence rule set according to the judgment result, wherein all operators of each target subgraph in the second subgraph equivalence rule set satisfy the hardware description file.

7. An electronic device, comprising: comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the steps of any one of the methods of claims 1-5.

8. A computer-readable storage medium having stored thereon instructions, the computer-readable storage medium comprising: The instructions, when executed by the processor, implement the steps of the method of any one of claims 1-5.

Images