Model compiling method and device

Abstract

The invention provides a model compiling method and device. The method comprises the following steps: determining a hardware feature vector of target hardware and analysis data of a to-be-operated model on the target hardware; based on the hardware feature vector and the analysis data of the to-be-operated model, generating a first optimization strategy and a second optimization strategy by using a decision rule and a machine learning model; fusing the first optimization strategy and the second optimization strategy to obtain a target optimization strategy; and under the condition that the confidence evaluation result of the target optimization strategy meets the requirement, the target optimization strategy is applied to the compiler, so that the compiler compiles the to-be-operated model according to the target optimization strategy.

Description

Technical Field

[0001] This application relates to the field of computer software technology, and in particular to a method and apparatus for model compilation. Background Technology

[0002] With the development of deep learning technology, how to efficiently deploy models on various hardware has become a key issue. Model compilation is a necessary step in connecting the model with the hardware, and its optimization effect directly affects the model's performance on the target hardware.

[0003] In related technologies, some compilation frameworks support the deployment of models from different deep learning frameworks to multiple hardware platforms and provide automatic tuning mechanisms. For example, the AutoTVM module in the TVM framework explores the search space to find the optimal configuration, but it relies on a predefined search space, the tuning process is time-consuming, and it is difficult to dynamically adjust according to different hardware characteristics. In addition, this compilation process lacks sufficient analysis of hardware and models, resulting in a lack of targeted optimization strategies and affecting compilation performance. Therefore, a new method is urgently needed to improve the above problems. Summary of the Invention

[0004] One objective of this disclosure is to provide a method for compiling a model to address the problem that insufficient analysis of hardware and models in the compilation process of related schemes leads to a lack of targeted optimization strategies and affects compilation performance; another objective is to provide a device for compiling a model; a third objective is to provide an electronic device; a fourth objective is to provide a computer-readable storage medium; and a fifth objective is to provide a computer program product.

[0005] To achieve the above objectives, the technical solution adopted in this disclosure is as follows: This disclosure proposes a method for model compilation, which includes: determining the hardware feature vector of the target hardware and the analysis data of the model to be run on the target hardware; generating a first optimization strategy and a second optimization strategy based on the hardware feature vector and the analysis data of the model to be run using decision rules and a machine learning model, respectively; fusing the first optimization strategy and the second optimization strategy to obtain a target optimization strategy; and applying the target optimization strategy to the compiler if the confidence evaluation result of the target optimization strategy meets the requirements, so that the compiler compiles the model to be run according to the target optimization strategy.

[0006] Based on the aforementioned technical methods, two optimization strategies are first generated by combining hardware feature vectors and analysis data of the model to be run, along with decision rules and machine learning models. These strategies are then merged into a single objective optimization strategy. This approach allows for a more comprehensive consideration of hardware characteristics and the structure of the model, improving the adaptability and accuracy of the optimization strategy. Furthermore, the objective optimization strategy is only applied when the confidence assessment results meet certain conditions, preventing the adoption of inefficient or unstable strategies and thus improving compilation efficiency and model performance.

[0007] Furthermore, the target optimization strategy is applied to the compiler so that the compiler compiles the runtime model according to the target optimization strategy. This includes: adding an optimization module corresponding to the target optimization strategy to the compiler so that the runtime model can be compiled through the optimization module corresponding to the target optimization strategy. The optimization module includes at least one of the following modules: memory optimization module, layout change module, hardware-aware operator fusion module, hardware-aware loop optimization module, and instruction set optimization module.

[0008] Furthermore, the model to be run is compiled by the optimization module corresponding to the target optimization strategy, including: in at least one of the Relay graph optimization stage, Tensor expression optimization stage and automatic tuning stage, the model to be run is compiled by calling the optimization module corresponding to the target optimization strategy.

[0009] Furthermore, the method also includes: obtaining the runtime data of the compiled model to be run; the runtime data includes the execution time of the compiled model to be run on the target hardware; and adjusting the optimization parameters in the target optimization strategy using the runtime data until the runtime data meets the requirements.

[0010] Furthermore, the method also includes: updating the adjusted target optimization strategy to the optimization strategy library; the optimization strategy library includes the adjusted target optimization strategy corresponding to running the model on the target hardware; and using the optimization strategy library as prior data to iteratively train the machine learning model.

[0011] Furthermore, the analysis data of the model to be run on the target hardware is determined, including: parsing the computation graph of the model to be run to obtain the analysis data of the model to be run; the analysis data of the model to be run includes operator type characteristics, data dependencies in the model to be run, and target nodes, where the target nodes are nodes with a computational load greater than a preset threshold.

[0012] Further, determining the hardware feature vector of the target hardware includes: testing the target hardware to obtain test data of the target hardware; the test data includes at least one of computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data; quantizing and normalizing at least one of the computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data to obtain the hardware feature vector of the target hardware, and storing it in the hardware feature database.

[0013] This disclosure provides an apparatus for model compilation, which includes a determining unit, a generating unit, a fusing unit, and an application unit. The determination unit is used to determine the hardware feature vector of the target hardware and the analysis data of the model to be run on the target hardware; The generation unit is used to generate a first optimization strategy and a second optimization strategy based on the analysis data of the hardware feature vector and the model to be run, respectively, using decision rules and machine learning models. The fusion unit is used to fuse the first optimization strategy and the second optimization strategy to obtain the target optimization strategy; The application unit is used to apply the target optimization strategy to the compiler when the confidence evaluation result of the target optimization strategy meets the requirements, so that the compiler compiles the model to be run according to the target optimization strategy.

[0014] This disclosure provides an electronic device, which includes a processor and a memory configured to store a computer program capable of running on the processor, wherein the processor is configured to execute the steps of the aforementioned method when running the computer program.

[0015] This disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method.

[0016] This disclosure provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the aforementioned method.

[0017] The model compilation method provided in this disclosure determines the hardware feature vector of the target hardware and the analysis data of the model to be run. Based on the above data, a first optimization strategy and a second optimization strategy are generated using decision rules and a machine learning model, respectively. These two strategies are then fused to obtain the target optimization strategy. Finally, if the confidence evaluation result of the target optimization strategy meets the requirements, the target optimization strategy is applied to the compiler to guide the compiler in compiling the model to be run. This fully considers the hardware characteristics of the target hardware and the model structure of the model to be run, improving the adaptability and accuracy of the optimization strategy. Furthermore, applying the target optimization strategy only when the confidence evaluation result meets the requirements avoids the adoption of inefficient or unstable optimization strategies, thereby improving compilation efficiency and model performance. Compared to related technologies that rely on manual tuning or fixed search spaces, this disclosure reduces manual intervention and improves the intelligence level of the compilation process by automatically extracting features and generating optimization strategies. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a model compilation method provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating another model compilation method provided in this embodiment of the disclosure; Figure 3 This is a flowchart illustrating another model compilation method provided in this embodiment of the disclosure; Figure 4A This is a schematic diagram of a TVM compilation process provided in an embodiment of this disclosure; Figure 4B This is a schematic diagram of another TVM compilation process provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure; Figure 6 A schematic diagram of the structure of a model compilation apparatus provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this disclosure.

[0019] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this disclosure clearer, the disclosure will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the specific technical solutions of the application will be further described in detail below with reference to the accompanying drawings of the embodiments of this disclosure. The following embodiments are used to illustrate this disclosure, but are not intended to limit the scope of this disclosure.

[0022] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0023] In the following description, the terms "first," "second," and "third" are used only to distinguish different objects and do not represent a specific order of objects or have any chronological limitation. It is understood that "first," "second," and "third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0025] This disclosure provides methods, apparatus, systems, devices, storage media, and program products for model compilation. In practical applications, the model compilation method can be implemented by a vehicle control device, and the functional entities in the vehicle control device can be collaboratively implemented by the hardware resources of electronic devices (such as computing resources and communication resources such as the vehicle or its processor). For example, the electronic device can be a controller in the vehicle.

[0026] The following describes the model compilation method provided in the embodiments of this disclosure, using an electronic device as the execution subject.

[0027] Figure 1 This is a flowchart illustrating a model compilation method provided in an embodiment of the present disclosure, as shown below. Figure 1 As shown, the method for compiling this model can be implemented through steps 101-104: Step 101: Determine the hardware feature vector of the target hardware and the analysis data of the model to be run on the target hardware.

[0028] Here, the hardware feature vector can be a set of hardware characteristic data obtained after testing the target hardware. For example, this hardware feature data includes, but is not limited to, computing power, memory bandwidth, cache structure, parallel processing capability, and supported instruction sets. These hardware features can then be combined to form a high-dimensional vector, i.e., the hardware feature vector. This hardware feature vector is used to represent the performance indicators of the target hardware.

[0029] For example, when the target hardware is a CPU, hardware characteristics may include: the number of cores, the size of the L1 cache, the size of the L2 cache, the size of the L3 cache, and memory bandwidth. When the target hardware is a GPU, hardware characteristics may include: the number of cores in the Compute Unified Device Architecture (CUDA), the size of shared memory, the number of registers, and whether it supports Tensor Cores. Hardware feature vectors can then be formed based on the numerical values ​​of these hardware characteristics.

[0030] The analysis data for the model to be run is the data extracted after parsing the model. This analysis data reflects the structural complexity and computational requirements of the model itself, and can be used to generate target optimization strategies.

[0031] In some embodiments, the process by which an electronic device determines the hardware feature vector of the target hardware and the analysis data of the model to be run on the target hardware may include: the electronic device determining the hardware feature vector of the target hardware and determining the analysis data of the model to be run on the target hardware.

[0032] By determining the hardware feature vectors of the target hardware and the analytical data of the model to be run, accurate and comprehensive data support can be provided for the subsequent generation of target optimization strategies. Moreover, based on the hardware feature vectors of the target hardware and the analytical data of the model to be run, the optimization space can be effectively narrowed, optimization efficiency can be improved, and ultimately the performance of the model to be run on the target hardware can be enhanced.

[0033] Step 102: Based on the hardware feature vector and the analysis data of the model to be run, generate the first optimization strategy and the second optimization strategy using decision rules and machine learning models respectively.

[0034] In some embodiments, such as Figure 2As shown, after obtaining the hardware feature vectors and the analysis data of the model to be run, a first optimization strategy and a second optimization strategy can be generated through the strategy generation engine in the electronic device. The hardware feature vectors include at least the computing power feature vector, memory bandwidth feature vector, cache size feature vector, and parallel capability feature vector. For example, the computing power feature vector is [0.8, 0.6, 0.9]; the memory bandwidth feature vector is [0.5, 0.7, 0.3]; the cache size feature vector is [0.4, 0.8, 0.6]; and the parallel capability feature vector is [0.9, 0.7, 0.8]. The analysis data of the model to be run includes at least the operator type distribution, computation graph topology, memory access patterns, and data dependencies.

[0035] The policy generation engine includes two different paths: the decision rule path (also known as the rule engine path) and the machine learning path.

[0036] For example, decision rules can be logical rules built based on expert experience or existing knowledge bases to derive optimization strategies from the analysis data of hardware feature vectors and the model to be run. For instance, the decision rule path may include an IF-THEN rule base, expert experience rules, and hardware-specific rules.

[0037] Here, the IF-THEN rule base can include multiple "production rules". Each rule in the IF-THEN rule base has the structure IF<condition>THEN<conclusion / action>. IF<condition> represents one or more conditions (facts, states), which can be connected by logical connectors (AND, OR, NOT). THEN<conclusion / action> indicates that if the IF<condition> is satisfied, the corresponding conclusion or action is output. For example, after determining the hardware feature vector of the target hardware, if this feature vector indicates that the target hardware has low memory bandwidth, the first optimization strategy output by the decision rule based on this hardware feature vector could include: adopting a more aggressive operator fusion strategy to reduce the number of memory accesses. If the hardware feature vector indicates that the target hardware has a large cache capacity, the first optimization strategy output by the decision rule could include: selecting a larger cyclic block parameter to fully utilize cache resources.

[0038] The process by which an electronic device generates a second optimization strategy through a machine learning path may include: calling a machine learning model to perform feature preprocessing on hardware feature vectors and analysis data of the model to be run, and obtaining the processing results; then performing model inference processing on the processing results to obtain the inference results; and finally performing strategy recommendation processing on the inference results to obtain the second optimization strategy.

[0039] In some embodiments, the machine learning model in the machine learning path can be a model that is trained and can automatically output a second optimization strategy based on hardware feature vectors and analysis data of the model to be run. For example, the type of machine learning model can be a decision tree, random forest, or neural network, and the machine learning model can be trained using a large amount of sample data, thereby giving the machine learning model high accuracy and generalization ability.

[0040] In some embodiments, the sample data used to train the machine learning model may include at least: the hardware feature vector of the sample hardware, the analysis data of the model running on the sample hardware, and the corresponding optimization strategy.

[0041] The sample data used to train a machine learning model may also include at least: the hardware feature vector of the sample hardware, the analysis data of the model running on the sample hardware, the corresponding optimization strategy, and the performance data corresponding to the optimization strategy. The performance data corresponding to the optimization strategy can be the execution time of the model on the sample hardware. For example, supervised learning methods can be used to train a machine learning model by using historical hardware, corresponding optimization strategies, and the performance data of those strategies as training samples. This model is used to predict the optimal optimization strategy under current conditions (i.e., the model to be run is running on the target hardware).

[0042] In some embodiments, the process of generating a first optimization strategy and a second optimization strategy based on hardware feature vectors and analysis data of the model to be run, respectively, using decision rules and a machine learning model, may include: the electronic device generating a first optimization strategy by invoking decision rules based on hardware feature vectors and analysis data of the model to be run; and generating a second optimization strategy by invoking a machine learning model.

[0043] Decision rules and machine learning models can output a variety of optimization strategies, which can both utilize the experience and knowledge of experts and make predictions with the help of models, thereby improving the robustness and adaptability of strategy generation.

[0044] Step 103: Combine the first optimization strategy and the second optimization strategy to obtain the target optimization strategy.

[0045] In some embodiments, since decision rules and machine learning models each have their own advantages and disadvantages, the first optimization strategy generated by the decision rules and the second optimization strategy generated by the machine learning model can be fused to obtain the target optimization strategy, so as to make the formed target optimization strategy more robust and efficient.

[0046] For example, the fusion method for combining the first optimization strategy and the second optimization strategy may include at least one of weighted averaging, voting mechanisms, and confidence-weighted averaging. For instance, an electronic device can set weights for the first and second optimization strategies respectively, and calculate the target optimization strategy based on these weights. The weights of the first and second optimization strategies can be obtained based on their historical accuracy rates. Of course, the weights of the first and second optimization strategies can also be dynamically adjusted according to the accuracy rates.

[0047] For example, an electronic device can also select one optimization strategy from the first optimization strategy and the second optimization strategy as the target optimization strategy.

[0048] In some embodiments, such as Figure 2 As shown, the electronic device also includes a hybrid decision module. After the strategy generation engine generates a first optimization strategy and a second optimization strategy, the hybrid decision module fuses the first optimization strategy and the second optimization strategy to obtain a target optimization strategy. Then, the confidence level of the target optimization strategy is evaluated. After the confidence level evaluation, the strategy can be selected for each specific optimization parameter in the target optimization strategy, thereby outputting the target optimization strategy configuration.

[0049] In some embodiments, an uncertainty assessment mechanism may be introduced during the fusion of the first and second optimization strategies. The electronic device scores the effectiveness of each optimization strategy to ensure that the final fused target optimization strategy has high reliability and consistency. For example, for certain target hardware, if a certain optimization strategy has historically performed poorly, the electronic device can reduce the weight of that optimization strategy during fusion to avoid negatively impacting the overall optimization strategy.

[0050] By integrating the first and second optimization strategies, the limitations of a single optimization strategy can be overcome, the overall quality of the target optimization strategy can be improved, and its applicability in different scenarios can be enhanced.

[0051] Step 104: If the confidence evaluation result of the target optimization strategy meets the requirements, apply the target optimization strategy to the compiler so that the compiler compiles the model to be run according to the target optimization strategy.

[0052] In some embodiments, after fusing the first optimization strategy generated by the decision rule and the second optimization strategy generated by the machine learning model to obtain the target optimization strategy, a confidence evaluation can be performed on the target optimization strategy. Exemplarily, the confidence evaluation can be performed by comparing historical running data, simulation execution results, etc. Only when the confidence evaluation result of the target optimization strategy meets a set threshold is the target optimization strategy sent to the encoder so that the encoder can compile the model to be run using the target optimization strategy. When the confidence evaluation result of the target optimization strategy does not meet the set threshold, the optimization strategy is regenerated.

[0053] like Figure 2 As shown, the output target optimization strategy configuration can include at least an operator fusion strategy, a tensor layout strategy, a cyclic block strategy, and a parallelization strategy. The operator fusion strategy can include a fusion mode (aggressive / conservative), a fusion rule set, and a fusion priority. The tensor layout strategy can include an inner coarse layout (NCHW / NHWC), data alignment, and a layout transformation sequence. The cyclic block strategy can include block size, cyclic reordering, and vectorization parameters. The parallelization strategy can include thread configuration, task partitioning, and a synchronization strategy.

[0054] By evaluating the confidence level of the target optimization strategy, we can ensure that the final applied target optimization strategy has sufficient reliability, thereby avoiding performance degradation caused by incorrect strategies.

[0055] In some embodiments, such as Figure 3 As shown, the electronic device first performs hardware feature vector construction and computational graph analysis of the model to be run. Hardware feature vector construction includes running a micro-benchmark program to extract hardware features (at least including computational unit features, memory system features, cache hierarchy features, and instruction set features), and constructing hardware feature vectors. Computational graph analysis of the model to be run includes parsing the computational graph of the model, extracting operator type features, analyzing data flow dependencies, and identifying computationally intensive nodes. Then, optimization strategy generation is performed. This includes applying decision rules and machine learning models to generate target optimization strategies (at least including operator fusion strategies, tensor layout transformation strategies, loop partitioning strategies, and parallelization strategies), and outputting the target optimization strategy configuration. Next, TVM compilation optimization is performed, which includes a Relay graph optimization stage, a Tensor expression optimization stage, an automatic tuning stage, and a code generation stage. Finally, performance feedback and optimization are performed. This includes running the model on the target hardware and collecting performance data; analyzing the optimization effect based on the performance data; and finally updating the optimization strategy library based on the optimization effect. The target optimization strategies in the optimization strategy library can be used to further train the machine learning model.

[0056] The model compilation method provided in this disclosure first determines the hardware feature vector of the target hardware and the analysis data of the model to be run. Second, based on the above data, a first optimization strategy and a second optimization strategy are generated using decision rules and a machine learning model, respectively. Then, these two strategies are fused to obtain the target optimization strategy. Finally, if the confidence evaluation result of the target optimization strategy meets the requirements, the target optimization strategy is applied to the compiler to guide the compiler in compiling the model to be run. This fully considers the hardware characteristics of the target hardware and the model structure of the model to be run, improving the adaptability and accuracy of the optimization strategy. Furthermore, applying the target optimization strategy only when the confidence evaluation result meets the requirements avoids the adoption of inefficient or unstable optimization strategies, thereby improving compilation efficiency and model performance. Compared to related technologies that rely on manual tuning or fixed search spaces, this disclosure reduces manual intervention and improves the intelligence level of the compilation process by automatically extracting features and generating optimization strategies.

[0057] The following section explains the process of applying the target optimization strategy to the compiler in step 104 so that the compiler compiles the model to be run according to the target optimization strategy.

[0058] In one possible implementation, the process may include, but is not limited to, step 201 below.

[0059] Step 201: Based on the target optimization strategy, add an optimization module corresponding to the target optimization strategy to the compiler so that the model to be run can be compiled through the optimization module corresponding to the target optimization strategy.

[0060] In some embodiments, the electronic device can add an optimization module to the compiler that matches the target optimization strategy. Since this optimization module is designed based on the hardware feature vectors of the target hardware and the analysis data of the model to be run on the target hardware, more efficient code generation can be achieved during the compilation process.

[0061] For example, if the target hardware has high memory bandwidth, a memory optimization module can be added; if the hardware supports a specific instruction set (such as AVX), an instruction set optimization module can be added. This modular design allows the compiler to flexibly adjust optimization methods according to different hardware platforms, improving code execution efficiency.

[0062] Here, the optimization module includes at least one of the following modules: memory optimization module, layout change module, hardware-aware operator fusion module, hardware-aware loop optimization module, and instruction set optimization module. Each module is responsible for handling different optimization tasks. For example, the memory optimization module can reduce the number of memory accesses and improve cache hit rate; the layout change module can adjust the storage format of tensors to make the adjusted storage format more suitable for the access patterns of the target hardware. Of course, the optimization module may also include other modules, and this disclosure does not limit this.

[0063] In this embodiment, by adding optimization modules corresponding to the target optimization strategy to the compiler, the compiler can automatically adapt to different hardware environments without manual configuration or reliance on AutoTVM for time-consuming searches. This improves compilation efficiency and enhances the system's portability and performance across different platforms.

[0064] The following section explains the process of compiling the model to be run using the optimization module corresponding to the target optimization strategy in step 201.

[0065] In one possible implementation, the process may include, but is not limited to, step 301 below.

[0066] Step 301: In at least one of the Relay graph optimization stage, Tensor expression optimization stage, and automatic tuning stage of the compilation, compile the model to be run by calling the optimization module corresponding to the target optimization strategy.

[0067] like Figure 4A As shown, the compiler's compilation process includes multiple stages, such as the front-end import stage, the Relay graph optimization stage (also known as the hardware-aware Relay graph optimization stage), the Tensor expression optimization stage (also known as the Tensor expression lowering stage), the automatic tuning stage (also known as the hardware-aware AutoTVM tuning stage), the code generation stage, and the performance evaluation stage.

[0068] The front-end import stage is used for loading the model to be run and converting the format of the model to be run in order to obtain a data format that the compiler can compile.

[0069] The Relay graph optimization phase is used to perform high-level optimizations on the computation graph of the model to be run, such as operator fusion and layout transformation.

[0070] The Tensor expression optimization phase is used to further optimize specific tensor operations, such as block partitioning and parallelization.

[0071] The automatic tuning stage utilizes tools such as AutoTVM to select the optimal configuration parameters based on hardware feature vectors.

[0072] The code generation phase transforms high-level programming languages ​​or computation graphs into low-level machine code that can be executed on specific hardware. This phase includes LLVM IR generation, CUDA / OpenCL code generation, and executable generation. In the LLVM IR generation phase, the source code or computation graph is parsed and converted into a target hardware-independent intermediate representation (IR). Next, if the target hardware involves GPU parallel computing, the CUDA / OpenCL code generation phase begins. This phase converts the LLVM IR into parallel code suitable for GPU execution. Finally, in the executable generation phase, the compiler compiles and links the output into the final executable file.

[0073] The performance evaluation phase assesses the runtime data of the compiled model to be run. Based on this data, the target optimization strategy in the optimization strategy library can be continuously optimized, thereby updating the optimization modules corresponding to the target optimization strategy in the compiler. It's understandable that updating the optimization modules corresponding to the target optimization strategy in the compiler can involve updating optimization modules in multiple compilation phases (e.g., the Relay graph optimization phase, the Tensor expression optimization phase, and the automatic tuning phase).

[0074] In some embodiments, the compiler may call different optimization modules at different stages described above. For example, such as Figure 4A The Relay graph optimization phase calls the memory optimization module, the layout change module (also known as the intelligent layout change module), and the hardware-aware operator fusion module. The Tensor expression optimization phase calls the hardware-aware loop optimization module, the cache-friendly array packing module, and the instruction set optimization module. The automatic tuning phase calls the post-pruning search space module, the hardware-guided search module, and the fast convergence tuning module.

[0075] For example, in the Relay graph optimization phase, the hardware-aware operator fusion module can determine whether to merge multiple adjacent operators into a larger operator based on the hardware's cache size and computing power, thereby reducing intermediate data transmission overhead. In the Tensor expression optimization phase, the hardware-aware loop optimization module can adjust the loop block size and parallelism based on the hardware's parallel capabilities to maximize hardware resource utilization. By integrating these optimization modules into the compilation process described above, the performance of the final generated code can be significantly improved without affecting the original compilation logic.

[0076] In some embodiments, such as Figure 4B As shown, another TVM compilation flow is provided, through comparison. Figure 4A and Figure 4B It can be seen that, Figure 4B TVM compilation process and Figure 4A The difference in the TVM compilation process lies in the different optimization modules in each optimization stage. Figure 4B The optimization modules in the Relay graph optimization stage include constant folding, dead code elimination, and general operator fusion; the optimization modules in the Tensor expression optimization stage include loop optimization, number packing, and intrinsic insertion; and the optimization modules in the automatic tuning stage include search space definition, random search / reinforcement learning, and performance evaluation. A comparison shows that... Figure 4A The TVM compilation process shown introduces several optimization modules based on the target hardware and the module to be run, thereby enabling... Figure 4A The TVM compilation flow shown is compared to Figure 4B The TVM compilation process shown yields better compilation results.

[0077] In this embodiment, by introducing different optimization modules at different stages, the compiler can continuously consider the target hardware and the model to be run throughout the compilation process, thereby generating more efficient and customized executable files. This also reduces unnecessary search processes, thus accelerating model compilation and improving the overall system deployment efficiency and model inference performance.

[0078] In some embodiments, the method further includes steps 401-402, wherein: Step 401: Obtain the runtime data of the compiled model to be run.

[0079] Here, runtime data includes the execution time of the compiled model on the target hardware.

[0080] In some embodiments, the runtime data refers to performance metrics collected when the compiled model to be run is actually executed on the target hardware. This runtime data is used to evaluate the actual performance of the optimization strategy. The runtime data reflects the execution efficiency and resource consumption of the compiled model to be run on the target hardware platform, and can be used to adjust the target optimization strategy. For example, when running the module to be run (e.g., a deep learning model) on a target device (such as an edge device or embedded chip), the execution time of the deep learning model can be used to determine whether the target optimization strategy needs further adjustment.

[0081] Here, execution time refers to the time required for the compiled model to complete one full forward computation, which can be measured in milliseconds (ms). Execution time reflects the computational efficiency of the compiled model on the target hardware; a shorter execution time means higher performance. On different hardware platforms, due to differences in the number of computing units, memory bandwidth, cache structure, etc., the execution time of the same compiled model may vary significantly.

[0082] By acquiring operational data, electronic devices can provide feedback and evaluation on the current target optimization strategy. For example, in the TVM framework, when using AutoTVM for automatic tuning, the electronic device can analyze operational data to determine whether the current search space is reasonable and whether further pruning or expansion of the search range is needed. Furthermore, operational data can be used as input features for machine learning models, allowing for model training and the generation of more intelligent optimization strategies.

[0083] Step 402: Use the running data to adjust the optimization parameters in the target optimization strategy until the running data meets the requirements.

[0084] In some embodiments, the target optimization strategy is a set of optimization rules and parameter configurations generated based on hardware feature vectors and analysis data of the model to be run, used to guide the compiler's compilation process. For example, the optimization parameters in the target optimization strategy may include loop block size, number of threads, memory layout method, and operator fusion rules, etc.

[0085] After obtaining the runtime data of the compiled model to be run, the runtime data is compared with a preset performance threshold, and the optimization parameters in the target optimization strategy are adjusted based on the comparison results. For example, if the execution time of the compiled model to be run is higher than the preset performance threshold, the block size, parallelism, or tensor layout method in the optimization parameters can be adjusted to reduce memory access latency and improve computational efficiency. The process of adjusting the optimization parameters can be carried out iteratively. After adjusting the optimization parameters in the target optimization strategy once, the model to be run is recompiled according to the adjusted optimization parameters, and the recompiled model is run on the target hardware to record the new execution time, until the runtime data meets the preset performance threshold.

[0086] In some embodiments, the preset performance threshold can be a performance target set according to the application scenario. For example, it could be to control the execution time within a certain range, or to ensure that the compiled model to be run completes a certain number of task processing within a limited time. The preset performance threshold can be an absolute value (e.g., execution time ≤ 50ms) or a relative value (e.g., 30% faster than the initial version). In addition, the adjustment of the target optimization strategy should take into account the balance between performance improvement and resource consumption, and avoid over-optimization that leads to hardware overload or excessive power consumption. In this embodiment, by acquiring the runtime data of the compiled model on the target hardware and dynamically adjusting the optimization parameters in the target optimization strategy accordingly, closed-loop optimization based on performance feedback can be achieved. This allows the system to continuously approach the optimal configuration, thereby improving the adaptability and execution efficiency of the compiled model on different hardware platforms, significantly shortening the model's runtime, and enhancing the overall system's intelligence and automation level.

[0087] In some embodiments, the method further includes: Step 501: Update the adjusted target optimization strategy to the optimization strategy library.

[0088] The optimization strategy library includes the adjusted target optimization strategy when running the model on the target hardware.

[0089] In some embodiments, the optimization strategy library is a data structure that stores different models and their corresponding target optimization strategies for different hardware platforms. The optimization strategy library includes the target optimization strategies corresponding to running the models on different hardware.

[0090] In some embodiments, the electronic device may include an optimization policy library, which may include target optimization policies. For example... Figure 4A As shown, after generating the compiled model to be run, the performance of the model to be run can be evaluated, and the optimization strategy library can be continuously updated based on the performance evaluation results through a performance feedback loop. New optimization modules can be introduced in each compilation stage according to the updated optimization strategy library to improve the compilation effect of the compiler.

[0091] The process of continuously updating the optimization strategy library based on performance evaluation results may include: updating the performance evaluation results to the performance results corresponding to the target optimization strategy in the optimization strategy library; and updating the target optimization strategy according to the performance evaluation results so that the updated target optimization strategy can help the compiler to better compile the model to be run.

[0092] It is understandable that there is a data mapping relationship between the optimization strategy library and the target hardware. When an electronic device detects a new hardware environment, it can query the optimization strategy library based on this new hardware environment to obtain the target optimization strategy suitable for the new hardware environment. Simultaneously, after generating new optimization strategies, these strategies can be added to the optimization strategy library for use in subsequent identical or similar hardware environments. In some embodiments, the optimization policy library can be deployed on a cloud server, a local database, or an embedded device. For example, in a cloud-edge collaboration scenario, a central server maintains a global optimization policy library, and each edge node uploads performance data. The central server updates the optimization policy library based on the performance data and distributes the updated optimization policy library to other edge devices, thereby achieving cross-device policy sharing and optimization reuse.

[0093] Step 502: Use the optimization strategy library as prior data to iteratively train the machine learning model.

[0094] In some embodiments, the process of iteratively training a machine learning model using an optimization policy library as prior data may include: an electronic device inputting the target optimization policy in the optimization policy library as training data into the machine learning model to train the machine learning model, thereby improving the prediction accuracy and generalization ability of the machine learning model.

[0095] For example, if a target hardware has successfully run multiple models and accumulated a certain number of target optimization strategies, the target optimization strategies can be used as prior data to train a machine learning model on another similar hardware, thereby reducing search time and improving optimization efficiency.

[0096] Each record in the optimization strategy library can be considered a sample. Each sample includes a hardware feature vector, analytical data of the model to be executed, and a target optimization strategy that matches the hardware feature vector and the analytical data of the model to be executed. The machine learning model learns from these samples to establish a mapping function from the hardware feature vector and the analytical data of the model to the target optimization strategy. As the optimization strategy library is continuously updated, the machine learning model will continuously iterate and train based on new sample data, thereby adapting to more diverse scenarios.

[0097] In some embodiments, various models, such as decision trees, neural networks, and random forests, can be used to construct machine learning models. For example, in the AutoTVM module of TVM, the electronic device uses the optimization policy library as prior data to quickly generate an initial search space, thereby significantly reducing tuning time and resource consumption.

[0098] In this embodiment, the adjusted target optimization strategy is updated to the optimization strategy library, and the optimization strategy library is used as prior data to iteratively train the machine learning model. This improves the efficiency and accuracy of model compilation and optimization, thereby supporting a wider range of hardware platforms and application scenarios.

[0099] In some embodiments, step 101 determines the analysis data of the model to be run on the target hardware, including step 1011, wherein: Step 1011: Analyze the computational graph of the model to be run to obtain the analysis data of the model to be run.

[0100] Here, the data analysis includes operator type characteristics, data dependencies in the model to be run, and target nodes; target nodes are nodes whose computational cost exceeds a preset threshold.

[0101] In some embodiments, the computation graph of the model to be run is a graphical representation of the internal structure of the deep learning model. The computation graph consists of nodes and edges. Nodes are used to represent operations (operators), and edges are used to represent the direction of data flow and tensor dimensions.

[0102] Electronic devices can extract analytical data from the computational graph of the model to be run by parsing it. This analytical data includes operator type characteristics, data dependencies, and target nodes. Target nodes are those with a computational load exceeding a preset threshold; these target nodes can also be called computationally intensive nodes. Figure 2 As shown, the analysis data of the model to be run may also include computation graph topology and memory access patterns. This disclosure does not limit the content of the analysis data of the model to be run.

[0103] In some embodiments, the process of parsing the computation graph of the model to be run may include: an electronic device calling an API provided by a deep learning framework (such as Relay IR in TVM) to parse the computation graph of the model to be run. By parsing the computation graph of the model to be run, the model structure can be automatically identified, avoiding the tedious process of manually labeling or pre-setting the model structure, thereby improving the degree of automation and applicability.

[0104] In some embodiments, operator type features are the specific categories and attributes of each operator, such as convolution, pooling, fully connected layers, activation functions, etc. Different types of operators have different computational complexities and resource consumption characteristics. Classifying different types of operators can help generate target optimization strategies. For example, convolution operations can be parallelized, while fully connected layers can be fused to reduce memory accesses.

[0105] Data dependencies refer to the data transfer relationships between operators in the model to be run; that is, the output of one operator serves as the input of the next. Data dependencies determine the execution order and potential parallelizable directions of the model.

[0106] Target nodes are those with high computational demands in the model to be run. These target nodes often have a significant impact on the performance of the model and can be prioritized for optimization. For example, target nodes can be identified by counting the number of floating-point operations (FLOPS) or memory accesses for each node, and a preset threshold can be set to filter them.

[0107] Here, the preset threshold can be a user-defined or empirically set value used to distinguish between ordinary nodes and computationally intensive nodes. When the computational load of a node exceeds the preset threshold, that node is marked as a target node. Electronic devices can dynamically adjust the preset threshold based on different hardware platforms; for example, a higher threshold can be set on high-performance GPUs, while a lower threshold can be set on low-power edge devices, adapting to optimization needs in different scenarios. Identifying the target nodes in electronic devices that have the greatest impact on overall performance helps optimize resource allocation and scheduling strategies, allowing electronic devices to focus on optimizing these target nodes, thereby improving model operating efficiency and response speed.

[0108] In this embodiment, by parsing the computation graph of the model to be run, operator type features, data dependencies, and target nodes are extracted to generate analysis data for the model. This allows for targeted optimization strategies to be designed, improving the operational efficiency of the model.

[0109] In some embodiments, step 101: determining the hardware feature vector of the target hardware includes steps 1021-1022, wherein: Step 1021: Test the target hardware and obtain the test data of the target hardware.

[0110] The test data includes at least one of the following: computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data.

[0111] In some embodiments, the process of testing the target hardware and obtaining test data may include: the electronic device running a micro-benchmark program to test the target hardware and obtain test data. For example, testing the target hardware includes computational capability testing, memory system testing, parallel capability testing, cache hierarchy testing, and instruction set testing. Computational capability testing is used to evaluate the target hardware's floating-point arithmetic capabilities (e.g., FLOPS), integer arithmetic capabilities, etc.; memory system testing is used to measure the bandwidth and latency of different memory levels (L1, L2, L3, main memory); parallel capability testing is used to evaluate the target hardware's maximum number of threads, cores, number of SIMD units, etc.; cache hierarchy testing is used to identify cache size, structure, access latency, etc.; and instruction set testing is used to detect whether specific extended instruction sets (e.g., AVX, SSE, NEON, Tensor Core, etc.) are supported.

[0112] like Figure 5 As shown, the electronic device also includes a hardware detection module, a feature extraction module, and a feature storage module. The hardware detection module is used to perform computing power testing, memory system testing, and parallel capability testing on the target hardware. The feature extraction module is used to quantify hardware features and normalize test results to construct a hardware feature vector of the target hardware. The feature storage module is used to store the hardware feature vector of the target hardware in a hardware feature database, while simultaneously constructing a feature index and providing a fast retrieval interface.

[0113] The computing power test includes floating-point performance testing, integer performance testing, and special instruction testing. Memory system testing includes memory bandwidth testing, cache hierarchy testing, and access latency testing. Parallelism capability testing includes multi-core scalability testing, thread efficiency testing, and synchronization overhead testing. The constructed hardware feature vector should at least include computing power feature vectors, memory bandwidth feature vectors, cache size feature vectors, and parallelism capability feature vectors. It may also include other types of feature vectors, such as energy efficiency feature vectors, which will not be elaborated here.

[0114] The above tests can yield the core performance indicators of the target hardware, reflecting its capability boundaries and performance characteristics, and providing a reliable basis for subsequent model compilation and optimization.

[0115] Step 502: Quantize and normalize at least one of the following test data: computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data, to obtain the hardware feature vector of the target hardware, and store it in the hardware feature database.

[0116] After obtaining the computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data, the computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data can be quantized and normalized to obtain a hardware feature vector that includes at least one of the following dimensions: computing feature dimension, memory feature dimension, parallel feature dimension, cache feature dimension, and instruction set feature dimension.

[0117] In some embodiments, quantization and normalization are processes for standardizing computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data. This transforms data with different units and dimensions into a unified numerical range, facilitating subsequent comparisons and modeling. For example, it converts different memory bandwidth values ​​into values ​​between 0 and 1, or maps the number of computing units in different architectures to a unified scoring system.

[0118] The processed data forms a high-dimensional vector, namely the hardware feature vector, which is used to characterize the overall performance characteristics of the target hardware. The hardware feature vector is then stored in a hardware feature database for use in the generation of subsequent optimization strategies.

[0119] In this embodiment, a micro-benchmarking program is first run to acquire test data of the target hardware across multiple dimensions. The test data is then quantized and normalized, transforming it into hardware feature vectors in a unified format. These feature vectors are subsequently stored in a hardware feature database for use in generating subsequent optimization strategies. This allows the electronic device to more accurately match the target hardware with the target optimization strategy, enabling the final compiled model to execute efficiently on the target hardware.

[0120] The following section, using a specific application scenario, elaborates on the training data generation method provided in this disclosure through an embodiment.

[0121] Currently, performance optimization is a challenge when deploying deep learning models across multiple hardware platforms. TVM, as an open-source deep learning compiler, supports deploying models from various frameworks (such as TensorFlow and PyTorch) to various hardware (CPU, GPU, dedicated accelerators). However, in TVM, the optimization process often requires extensive manual tuning or searching using AutoTVM. Searching with AutoTVM is time-consuming, and TVM struggles to automatically adjust optimization strategies based on different hardware characteristics. Existing technologies lack deep awareness of hardware features and automated optimization strategy generation, resulting in low compilation and optimization efficiency for TVM-generated code, and the code's performance is not optimal.

[0122] This disclosure aims to provide a hardware-aware deep learning model compilation optimization method to solve the problems of low compilation optimization efficiency and insufficient utilization of hardware features in the prior art.

[0123] The method proposed in this disclosure can achieve the following beneficial effects: automatically extracting hardware features and generating optimization strategies, reducing human intervention; significantly reducing AutoTVM search time through hardware-aware search space pruning; the method proposed in this disclosure is applicable to a variety of hardware platforms, including CPUs, GPUs, and edge AI chips; and improving model inference speed and reducing memory usage by using targeted optimization strategies.

[0124] To achieve the above objectives, the present disclosure adopts the following technical solution: A model compilation method includes the following steps: 1. Hardware Feature Profile Construction: By running the micro-benchmark program, the hardware features of the target hardware are extracted, including the number of computing units, memory bandwidth, cache hierarchy, supported instruction sets, etc., and a hardware feature vector is constructed.

[0125] In some embodiments, the micro-benchmark program includes: memory bandwidth testing: measuring the bandwidth of different memory levels (L1, L2, L3, main memory); computational performance testing: measuring floating-point operation performance, integer operation performance, etc.; cache size testing: probing cache size by accessing arrays of different sizes; instruction set testing: detecting supported instruction sets (such as AVX, NEON, etc.). The test results are normalized to construct a hardware feature vector. For example, for a GPU, the feature vector may include: the number of CUDA cores, global memory bandwidth, shared memory size, number of registers, and whether Tensor Cores are supported.

[0126] 2. Model Graph Analysis: Analyze the computation graph of the input deep learning model to extract features such as operator type, data flow, and computationally intensive nodes.

[0127] 3. Optimization Strategy Generation: Based on hardware feature vectors and model graph features, optimization strategies are generated using predefined decision rules or machine learning models, including: Operator Fusion Strategy: Determining fusion rules based on hardware memory characteristics. Tensor Layout Transformation: Selecting the tensor layout best suited to the hardware memory access pattern. Cyclic Blocking Strategy: Determining cyclic blocking parameters based on hardware computing units and cache size. Parallelization Strategy: Selecting thread configurations based on hardware parallelism capabilities.

[0128] In some embodiments, a mapping relationship is established between the target hardware, the model to be run, and the target optimization strategy. For example, if the hardware memory bandwidth is limited, a more aggressive operator fusion approach is preferred to reduce memory reads and writes. If the hardware has a large cache, a larger chunk size can be chosen. If the hardware supports Tensor Cores, a matrix computation library (such as cuBLAS) is used, and the data layout is adjusted to match Tensor Core requirements. This mapping relationship can be learned using decision trees, rule engines, or simple neural networks. The initial mapping relationship can be obtained through expert experience or offline learning.

[0129] 4. Compilation Optimization: The generated optimization strategies are applied to the TVM compilation process, including Relay graph optimization, Tensor expression optimization, and automatic tuning (AutoTVM) stages.

[0130] For example, in the TVM compilation process, optimization modules can be introduced in the Relay graph optimization and AutoTVM stages. For instance, in the Relay graph optimization stage, operator fusion and layout transformation are performed according to the target optimization strategy. In the AutoTVM stage, a hardware-aware search space is used to search only configurations that match hardware features, thereby accelerating the optimization process.

[0131] 5. Performance Feedback and Optimization: Run the compiled model on the target hardware, collect performance data, and use this data to update the optimization strategy, forming a closed-loop optimization.

[0132] The following specific embodiment further illustrates the implementation of this disclosure: Taking a mobile GPU (e.g., Mali-G (Mali-G)77) as an example, researchers extract hardware features of the mobile GPU (e.g., Mali-G77) through micro-benchmarks, including key parameters such as the number of computing units and memory bandwidth. Then, for a convolutional neural network (e.g., MobileNet), an optimization strategy for model optimization is generated. Due to the limited memory bandwidth of mobile GPUs, developers adopt an aggressive operator fusion strategy and convert the data layout from the default format to NHWC, as the NHWC data layout is specifically optimized in the Mali-G77 graphics processing unit. The thread configuration and block size of the mobile GPU are searched in AutoTVM, and the search space is pruned to contain only configurations suitable for the mobile GPU. Experiments show that, compared to the default TVM optimization, the method of this disclosure achieves better performance in the same search time, or requires a shorter search time to achieve the same performance. In summary, this disclosure provides a model compilation method. This method can fully consider the hardware characteristics of the target hardware and the model structure of the model to be run, and can improve the adaptability and accuracy of the optimization strategy. Furthermore, the target optimization strategy is only applied when the confidence evaluation results meet the requirements, avoiding the adoption of inefficient or unstable optimization strategies, thereby improving compilation efficiency and model performance. Compared with related technologies that rely on manual tuning or fixed search spaces, this disclosure reduces manual intervention and improves the intelligence level of the compilation process by automatically extracting features and generating optimization strategies.

[0133] Secondly, embodiments of this disclosure provide an apparatus for model compilation, with reference to Figure 6 The apparatus for compiling the model, as shown, may include a determining unit 601, a generating unit 602, a fusing unit 603, and an application unit 604.

[0134] The determining unit 601 is used to determine the hardware feature vector of the target hardware and the analysis data of the model to be run on the target hardware; The generation unit 602 is used to generate a first optimization strategy and a second optimization strategy based on the analysis data of the hardware feature vector and the model to be run, respectively, using decision rules and machine learning models. The fusion unit 603 is used to fuse the first optimization strategy and the second optimization strategy to obtain the target optimization strategy; Application unit 604 is used to apply the target optimization strategy to the compiler when the confidence evaluation result of the target optimization strategy meets the requirements, so that the compiler compiles the model to be run according to the target optimization strategy.

[0135] In some embodiments, the application unit 604 is further configured to add an optimization module corresponding to the target optimization strategy to the compiler according to the target optimization strategy, so as to compile the model to be run through the optimization module corresponding to the target optimization strategy. The optimization module includes at least one of the following modules: memory optimization module, layout change module, hardware-aware operator fusion module, hardware-aware loop optimization module, and instruction set optimization module.

[0136] In some embodiments, the application unit 604 is further configured to compile the model to be run by calling the optimization module corresponding to the target optimization strategy in at least one of the Relay graph optimization stage, Tensor expression optimization stage, and automatic tuning stage during compilation.

[0137] In some embodiments, the generation unit 602 is further configured to acquire the execution data of the compiled model to be run; the execution data includes the execution time of the compiled model to be run on the target hardware; and to adjust the optimization parameters in the target optimization strategy using the execution data until the execution data meets the requirements.

[0138] In some embodiments, application unit 604 is further configured to update the adjusted target optimization strategy to the optimization strategy library; the optimization strategy library includes the adjusted target optimization strategy corresponding to the model to be run when running on the target hardware; and to use the optimization strategy library as prior data to iteratively train the machine learning model.

[0139] In some embodiments, the determining unit 601 is further configured to parse the computation graph of the model to be run to obtain the analysis data of the model to be run; the analysis data of the model to be run includes operator type features, data dependencies in the model to be run, and target nodes, wherein the target nodes are nodes whose computational load is greater than a preset threshold.

[0140] In some embodiments, the determining unit 601 is further configured to test the target hardware to obtain test data of the target hardware; the test data includes at least one of computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data; and to quantize and normalize at least one of the computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data to obtain a hardware feature vector of the target hardware, and store it in a hardware feature database.

[0141] It should be noted that the model compilation apparatus provided in this disclosure includes all the units included, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc.

[0142] The description of the above apparatus embodiments is similar to that of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the apparatus embodiments of this disclosure, please refer to the description of the method embodiments of this disclosure for understanding.

[0143] It should be noted that, in the embodiments of this disclosure, if the above-described vehicle driving control method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several voice commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk. Thus, the embodiments of this disclosure are not limited to any specific hardware and software combination.

[0144] Thirdly, this disclosure provides another electronic device that can implement the model compilation method provided in the first aspect above.

[0145] In one example, reference Figure 7 The electronic device 70, as shown, includes: a processor 701, at least one communication bus 702, a user interface 703, at least one external communication interface 704, and a memory 705. The communication bus 702 is configured to enable communication between these components. The user interface 703 may include a display screen, a microphone, etc. The external communication interface 704 may include standard wired and wireless interfaces.

[0146] The memory 705 is configured to store voice commands and applications executable by the processor 701, and can also cache data to be processed or already processed by the processor 701 and various modules in the electronic device (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0147] In some embodiments, this disclosure provides a vehicle including a processor and a memory, the memory storing a computer program or voice instructions, wherein when the computer program is executed by the processor, it implements the method provided in the first aspect.

[0148] Fourthly, embodiments of this disclosure provide a storage medium, namely a computer-readable storage medium, on which a computer program or voice instruction is stored, wherein when the computer program or voice instruction is executed by a processor, the steps in any of the model compilation methods provided in the first aspect of the above embodiments are implemented.

[0149] Fifthly, embodiments of this disclosure provide a computer program product, which includes a computer program or voice instructions. When the computer program or voice instructions are executed by a processor, they implement the steps in any of the model compilation methods provided in the first aspect of the above embodiments.

[0150] It should be noted that the descriptions of the above embodiments of storage media, devices, apparatuses, and program products are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of storage media, devices, apparatuses, and program products of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0151] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above-described embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0152] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0153] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0154] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0155] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0156] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program voice commands. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0157] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several voice commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0158] The above are merely embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

[0159] It should be understood that if this disclosure references any user data and personal information (including but not limited to device information, behavioral data, location information, etc.) and before applying the technical solutions described in the embodiments of this disclosure, the relevant products or services should comply with the laws and regulations concerning the protection of user data and personal information, strictly process users' personal information and data in accordance with the provisions of applicable laws and regulations throughout the entire data processing lifecycle, follow the principles of legality, legitimacy, necessity, good faith, openness, and transparency, and adopt reasonable privacy design schemes and technical measures to ensure the security of user data and personal information, protect users' legitimate rights and interests, and prevent the risks of leakage, theft, or tampering of user data and personal information. Specifically, the company must publish and display its privacy policy in a prominent position on the user interface, clearly informing users of the types, purposes, uses, and methods of processing personal information, as well as other matters that should be disclosed as required by laws and regulations; obtain users' prior informed consent or explicit authorization for data processing through user-initiated interaction (such as confirmation pop-ups); process or store user data securely within the legally required timeframe; adopt a series of security technologies and management measures, including but not limited to data encryption and access control; share and transfer user data within the scope permitted by law and in a legally required manner; and process user rights, including the rights to query, access, correct, delete, withdraw authorization and consent, cancel registration, and obtain copies of personal information, within the legally required timeframe.

Claims

1. A method for compiling a model, characterized in that, The method includes: Determine the hardware feature vector of the target hardware and the analysis data of the model to be run on the target hardware; Based on the hardware feature vectors and the analysis data of the model to be run, a first optimization strategy and a second optimization strategy are generated using decision rules and machine learning models, respectively. The first optimization strategy and the second optimization strategy are fused to obtain the target optimization strategy; If the confidence evaluation result of the target optimization strategy meets the requirements, the target optimization strategy is applied to the compiler so that the compiler compiles the model to be run according to the target optimization strategy.

2. The method according to claim 1, characterized in that, The step of applying the target optimization strategy to the compiler, so that the compiler compiles the model to be run according to the target optimization strategy, includes: According to the target optimization strategy, an optimization module corresponding to the target optimization strategy is added to the compiler so as to compile the model to be run through the optimization module corresponding to the target optimization strategy. The optimization module includes at least one of the following modules: memory optimization module, layout change module, hardware-aware operator fusion module, hardware-aware loop optimization module, and instruction set optimization module.

3. The method according to claim 2, characterized in that, The step of compiling the model to be run using the optimization module corresponding to the target optimization strategy includes: In at least one of the Relay graph optimization stage, Tensor expression optimization stage, and automatic tuning stage of the compilation, the model to be run is compiled by calling the optimization module corresponding to the target optimization strategy.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Obtain the execution data of the compiled model to be run; the execution data includes the execution time of the compiled model to be run on the target hardware; The optimization parameters in the target optimization strategy are adjusted using the operational data until the operational data meets the requirements.

5. The method according to claim 4, characterized in that, The method further includes: The adjusted target optimization strategy is updated to the optimization strategy library; the optimization strategy library includes the adjusted target optimization strategy corresponding to running the model to be run on the target hardware. The optimization strategy library is used as prior data to iteratively train the machine learning model.

6. The method according to claim 1, characterized in that, The analysis data for determining the model to be run on the target hardware includes: The computation graph of the model to be run is parsed to obtain the analysis data of the model to be run; the analysis data of the model to be run includes operator type features, data dependencies in the model to be run, and target nodes, wherein the target nodes are nodes whose computational cost is greater than a preset threshold.

7. The method according to any one of claims 1-6, characterized in that, The determination of the hardware feature vector of the target hardware includes: The target hardware is tested to obtain test data for the target hardware; the test data includes at least one of the following: computing power test data, memory system test data, parallel capability test data, cache hierarchy test data, and instruction set test data. At least one of the following test data—the computing power test data, the memory system test data, the parallel capability test data, the cache hierarchy test data, and the instruction set test data—is quantized and normalized to obtain the hardware feature vector of the target hardware, which is then stored in the hardware feature database.

8. An apparatus for model compilation, characterized in that, The apparatus for compiling the model includes a determining unit, a generating unit, a fusing unit, and an application unit; The determining unit is used to determine the hardware feature vector of the target hardware and the analysis data of the model to be run on the target hardware; The generation unit is used to generate a first optimization strategy and a second optimization strategy based on the hardware feature vector and the analysis data of the model to be run, respectively, using decision rules and machine learning models. The fusion unit is used to fuse the first optimization strategy and the second optimization strategy to obtain the target optimization strategy; The application unit is configured to apply the target optimization strategy to the compiler if the confidence evaluation result of the target optimization strategy meets the requirements, so that the compiler compiles the model to be run according to the target optimization strategy.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory storing processor-executable instructions; when the instructions are executed by the processor, the method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images