Method and apparatus for optimizing deep learning model

Abstract

Embodiments of the present application provide a method and device for optimizing a deep learning model. The method comprises: based on a first agent, analyzing and processing a received target deep learning model to generate a model analysis report of the target deep learning model; based on a second agent, according to the model analysis report and a preset optimization target, optimizing the target deep learning model to obtain a plurality of candidate optimized models adapted to a target hardware platform; and based on a third agent, determining a final model adapted to deployment of the target hardware platform from the candidate optimized models. The method is used to improve the efficiency and quality of model optimization.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for optimizing a deep learning model. Background Technology

[0002] With the rapid development of autonomous driving technology, deep learning has become a core technology supporting autonomous driving perception systems, and is widely used in key tasks such as object detection, semantic segmentation, and lane recognition. While deep learning models can achieve high recognition accuracy, they also bring problems such as high computational complexity, large memory consumption, and slow inference speed. They cannot be directly deployed on in-vehicle edge computing platforms with limited computing power and storage resources, and must undergo targeted compression optimization and hardware adaptation to meet real-time requirements.

[0003] Currently, the optimization and deployment of deep learning models mainly rely on experienced algorithm engineers to complete manually. However, this manual optimization method has a long optimization cycle and low efficiency. The optimization process of a complex model usually takes several weeks or even months, which is difficult to meet the needs of rapid iteration of autonomous driving technology. Furthermore, the optimization quality is highly dependent on the individual experience and technical level of the engineers, and the optimization results of different engineers vary greatly, resulting in unstable optimization quality. At the same time, optimization and adaptation need to be carried out again for different hardware platforms, which seriously restricts the large-scale industrial deployment of autonomous driving models.

[0004] Therefore, there is an urgent need for a model optimization method that can improve optimization efficiency, shorten optimization cycle, ensure optimization quality, and adapt to various heterogeneous hardware platforms without human intervention, so as to support the rapid iteration and large-scale deployment of autonomous driving systems. Summary of the Invention

[0005] The deep learning model optimization method and device provided in this application are used to improve the efficiency and quality of model optimization.

[0006] In a first aspect, embodiments of this application provide a method for optimizing a deep learning model, comprising:

[0007] Based on the first intelligent agent, the received target deep learning model is analyzed and processed to generate a model analysis report of the target deep learning model;

[0008] Based on the second intelligent agent, the target deep learning model is optimized according to the model analysis report and the preset optimization target to obtain multiple candidate optimized models adapted to the target hardware platform.

[0009] Based on a third intelligent agent, a final model adapted to the deployment of the target hardware platform is determined from the candidate optimization models.

[0010] In one possible implementation, the second intelligent agent includes a fourth intelligent agent and a fifth intelligent agent. Based on the second intelligent agent, and according to the model analysis report and preset optimization objectives, the target deep learning model is optimized to obtain multiple candidate optimized models adapted to the target hardware platform, including:

[0011] Based on the fourth intelligent agent, and according to the model analysis report, the optimization objective, and the preset decision logic, multiple target optimization strategies for the target deep learning model are determined; wherein, the target optimization strategies include at least one of the following strategies: structured pruning, quantization, knowledge distillation, and operator fusion.

[0012] Based on the fourth intelligent agent, the target deep learning model is optimized according to the target optimization strategy to obtain the optimized model;

[0013] Based on the fifth intelligent agent, and according to the target hardware platform, the optimized model is subjected to hardware adaptation processing to obtain the candidate optimized model.

[0014] In one possible implementation, the step of performing hardware adaptation processing on the optimized model based on the fifth intelligent agent and the target hardware platform to obtain the candidate optimized model includes:

[0015] Based on the fifth intelligent agent, the platform information of the target hardware platform is determined according to the target hardware platform and the preset hardware feature description library;

[0016] Based on the fifth intelligent agent, according to the platform information of the target hardware platform, the optimized model is compiled and optimized to obtain the candidate optimized model; wherein, the compilation and optimization process includes at least one of the following operations: model format conversion operation, operator compatibility verification operation, inference performance prediction operation, and inference parameter determination operation.

[0017] In one possible implementation, the step of optimizing the target deep learning model based on the fourth agent and according to the target optimization strategy to obtain the optimized model includes:

[0018] Based on the fourth agent, the target optimization strategy is sent to the third agent;

[0019] Based on the third intelligent agent, the target optimization strategy is reviewed, and the review result is returned;

[0020] Based on the fourth intelligent agent, the target deep learning model is optimized according to the review results to obtain the optimized model.

[0021] In one possible implementation, the step of analyzing and processing the received target deep learning model based on the first intelligent agent to generate a model analysis report of the target deep learning model includes:

[0022] Based on the first intelligent agent, the structure of the target deep learning model is parsed according to the parsing method corresponding to the format of the target deep learning model, so as to obtain the computation graph of the target deep learning model.

[0023] Based on the first intelligent agent, the architecture type of the target deep learning model is determined according to the combination pattern of operators in the computation graph;

[0024] Based on the first intelligent agent, according to the architecture analysis template matched by the architecture type, at least one analysis process is performed on the computation graph to generate the model analysis report; wherein, the analysis process includes layer-by-layer parameter statistical processing, sparsity detection processing, sensitivity assessment processing, bottleneck location processing, and baseline establishment processing; the model analysis report includes at least one of the following information: architecture description information, layer-by-layer parameter statistical information, channel importance score information, quantization sensitivity classification information, bottleneck location annotation information, layer-by-layer optimization suggestions, and baseline data.

[0025] In one possible implementation, the third intelligent agent includes a sixth intelligent agent and a seventh intelligent agent; the step of determining the final model adapted to the deployment of the target hardware platform from the candidate optimization models based on the third intelligent agent includes:

[0026] Based on the sixth intelligent agent, the candidate optimization models are evaluated to obtain the evaluation result of each candidate optimization model;

[0027] Based on the seventh agent, when the preset iteration termination condition is met, the final model is determined from the candidate optimization models according to the evaluation results of the candidate optimization models.

[0028] In one possible implementation, determining the final model from the candidate optimization models based on the evaluation results of the candidate optimization models, using the seventh agent, includes:

[0029] Based on the seventh agent, a three-dimensional coordinate space is constructed using the accuracy, inference latency, and memory usage in the evaluation results. Then, according to the evaluation results of each candidate optimization model, the candidate optimization model is mapped to coordinate points in the three-dimensional coordinate space.

[0030] Based on the seventh agent, all coordinate points that satisfy the Pareto optimality condition among the coordinate points are taken as the Pareto front.

[0031] Based on the seventh agent, for the candidate optimization model on the Pareto front, the score of the candidate optimization model is determined according to the evaluation result of the candidate optimization model;

[0032] The candidate optimization model with the highest score is selected as the final model.

[0033] In one possible implementation, the method further includes:

[0034] Based on the seventh agent, when the iteration termination condition is not met, a new target optimization strategy is generated according to the evaluation results of the candidate optimization model and sent to the second agent, so that the second agent can perform iterative optimization processing on the target deep learning model according to the new target optimization strategy.

[0035] In one possible implementation, the method further includes:

[0036] Based on the third intelligent agent, model information of the final model is generated; wherein, the model information includes the model file, deployment configuration file, and optimization report of the final model.

[0037] Secondly, embodiments of this application provide an optimization apparatus for a deep learning model, comprising:

[0038] The analysis module is used to analyze and process the received target deep learning model based on the first intelligent agent, and generate a model analysis report of the target deep learning model.

[0039] An optimization module is used to optimize the target deep learning model based on the second intelligent agent, according to the model analysis report and preset optimization objectives, to obtain multiple candidate optimized models adapted to the target hardware platform.

[0040] The determination module is used to determine, based on a third intelligent agent, the final model adapted to the deployment of the target hardware platform from the candidate optimization models.

[0041] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0042] The memory stores computer-executed instructions;

[0043] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0044] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0045] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0046] The deep learning model optimization method and device provided in this application embodiment completes a comprehensive quantitative analysis of the target deep learning model and outputs a standardized model analysis report through a first intelligent agent. A second intelligent agent automatically executes various differentiated optimization strategy combinations based on the model analysis report and optimization objectives, generating multiple hardware-adapted candidate optimization models. A third intelligent agent performs multi-dimensional objective evaluation of all candidate optimization models and selects the optimal model adapted for deployment on the target hardware platform. This approach completely eliminates the reliance on human experience in traditional model optimization processes. Engineers are no longer required to manually analyze model structure, identify performance bottlenecks, select optimization strategies, or repeatedly debug parameters, significantly shortening the model optimization cycle and improving optimization efficiency. Simultaneously, the standardized analysis process and data-driven decision logic ensure the stability and consistency of optimization quality, avoiding significant differences in optimization results among different engineers. It can also simultaneously consider multiple conflicting optimization objectives such as accuracy, inference latency, and memory usage, finding a globally optimal trade-off solution. Furthermore, it automatically adapts the model to the target hardware platform, eliminating the need for re-optimization for different hardware platforms, further reducing development costs and effectively meeting the urgent needs of rapid model iteration and large-scale industrial deployment in fields such as autonomous driving. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0048] Figure 1 A system architecture diagram of a model optimization system provided in this application;

[0049] Figure 2 A flowchart illustrating the optimization method for the deep learning model provided in this application. Figure 1 ;

[0050] Figure 3 A flowchart illustrating the optimization method for the deep learning model provided in this application. Figure 2 ;

[0051] Figure 4A flowchart illustrating an optimization method for a deep learning model provided in this application;

[0052] Figure 5 A schematic diagram of an optimization strategy selection decision tree provided in this application;

[0053] Figure 6 A schematic diagram of the structure of the optimization device for the deep learning model provided in this application;

[0054] Figure 7 A schematic diagram of the structure of the electronic device provided in this application.

[0055] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0057] First, the terms used in the embodiments of this application will be explained.

[0058] An agent is a software entity with autonomous perception, decision-making, and execution capabilities. In this application, it refers to a collaborative module that undertakes a specific model optimization task.

[0059] Post-Training Quantization (PTQ) is a technique that converts model weights from high precision (such as FP32) to low precision (such as INT8) without retraining after the model has been trained.

[0060] Quantization-Aware Training (QAT) is a quantization method that simulates quantization operations during training to adapt the model to low-precision representations.

[0061] Average Precision (AP) is a core metric for measuring the accuracy of single-class detection in computer vision object detection tasks. It is obtained by calculating the average precision under different recall thresholds and is used to evaluate the model's ability to identify and locate objects in a single class.

[0062] Mean Average Precision (mAP) is a commonly used comprehensive evaluation metric in object detection tasks, used to measure the detection accuracy of a model across multiple categories.

[0063] Precision is a core evaluation metric in computer vision and machine learning classification and detection tasks. It refers to the proportion of samples that are actually positive out of all samples predicted as positive by the model. It is used to measure the accuracy of the model's prediction results and reflects the model's ability to avoid false detections (misclassifying negative samples as positive samples).

[0064] Recall, also known as recall, is a core evaluation metric in computer vision and machine learning classification and detection tasks. It refers to the proportion of samples that are correctly predicted as positive by the model out of all samples that are actually positive. It is used to measure the model's ability to cover positive samples and reflects the model's ability to avoid missing detections (misclassifying positive samples as negative samples).

[0065] Floating Point Operations (FLOPs) is a metric used to measure the computational complexity of a model, representing the total number of floating point operations required for a single inference iteration.

[0066] Neural Architecture Search (NAS) is a technique for automatically searching for the optimal neural network structure.

[0067] TensorRT (Inference Optimization Engine) is a deep learning inference optimizer that optimizes trained models into inference engines that run efficiently on graphics processing units.

[0068] OpenVINO is a cross-platform deep learning inference optimization toolkit.

[0069] Open Neural Network Exchange (ONNX) is a cross-framework standard for representing neural network models, supporting model conversion between different deep learning frameworks.

[0070] The L1 norm (L1 Norm), the sum of the absolute values ​​of the elements of a vector or matrix, is used in this application to measure the importance of convolutional channels.

[0071] Batch Normalization (BN) is a technique used in deep neural networks to accelerate and stabilize the training process. Its scaling factor can be used to evaluate channel importance.

[0072] Knowledge distillation is a technique that transfers knowledge from a large model (teacher model) to a small model (student model), enabling the student model to achieve performance close to that of the teacher model while maintaining a smaller size.

[0073] Structured pruning is a model compression technique that removes redundant or low-contribution channels, layers, or attention heads in a neural network. It can be divided into unstructured pruning and structured pruning.

[0074] Pareto optimality is a set of states in multi-objective optimization where no objective can be improved without reducing the other objectives. The Pareto front consists of all Pareto optimal solutions.

[0075] The Ascend Computing Language (ACL) is a library of operators for accelerating inference.

[0076] Half-precision floating-point, 8-bit integer, and 4-bit integer (FP16 / INT8 / INT4) are commonly used numerical precision representation formats in deep learning. The lower the precision, the smaller the model size and computational cost, but the higher the risk of precision loss.

[0077] Intersection over Union (IoU) is a core metric in computer vision used to measure the degree of overlap between two regions. It is calculated by dividing the area of ​​the intersection of the predicted region and the ground truth region by the area of ​​their union. It is a fundamental evaluation metric for tasks such as object detection, semantic segmentation, and instance segmentation.

[0078] Mean Intersection over Union (mIoU) is a commonly used evaluation metric in semantic segmentation tasks, used to measure the degree of overlap between the predicted segmented region and the real region.

[0079] Compute Unified Device Architecture (CUDA) is a parallel computing platform and programming model.

[0080] The Feature Pyramid Network (FSP) is a network structure used in object detection to extract multi-scale features. Feature maps of different scales are used to detect targets of different sizes.

[0081] Centered kernel alignment (CKA) is a metric used to measure the similarity between two neural network representations. In this application, it is used to evaluate the degree of feature alignment between the teacher model and the student model.

[0082] The Straight-Through Estimator (STE) is a technique used in quantization-aware training to approximate the gradient of non-differentiable quantization operations.

[0083] Automated Machine Learning (AutoML) is a technology that automates the entire process of machine learning, from data preparation to model deployment, including feature engineering, model selection, and hyperparameter optimization.

[0084] In the current field of autonomous driving, deep learning models undergo a series of optimization operations after training and before being deployed on the target hardware platform. The goal of these optimization operations is to reduce the model's computational complexity, number of parameters, and memory consumption while maintaining its accuracy, so that it can meet the real-time performance, power consumption, and resource constraints of the target deployment platform.

[0085] After training a model, algorithm engineers must first manually evaluate its performance metrics on both the training and validation datasets. For object detection models, they need to evaluate metrics such as mAP (e.g., mAP@0.5, mAP@0.5:0.95) at different IoU thresholds, average precision (AP), recall, and precision for each category. For semantic segmentation models, they need to evaluate metrics such as mIoU and pixel accuracy. For 3D object detection models, they also need to evaluate the detection accuracy in 3D space. Simultaneously, they need to measure efficiency metrics such as inference latency, throughput (measured in FPS), number of parameters, model size, and GPU memory usage during inference to establish baseline data for model optimization in terms of accuracy and efficiency.

[0086] Subsequently, algorithm engineers need to formulate model optimization strategies based on their personal experience, taking into account the hardware characteristics of the target deployment platform (such as GPU model and computing power version, memory size, power consumption limits, supported numerical precision types, and dedicated inference accelerator types) and business performance requirements (such as target inference latency upper limit, target precision lower limit, and maximum memory usage limit). This decision-making process involves the selection and balancing of multiple technical directions, including determining the pruning granularity and proportion, selecting quantization methods and precision levels, designing distillation strategies, and arranging the execution order of various optimization strategies.

[0087] In the structured pruning stage, algorithm engineers need to manually write pruning scripts, select the pruning granularity (channel-level pruning, hierarchical pruning, or attention head pruning), set the channel importance evaluation method (such as based on L1 norm, BN scaling factor, geometric median, etc.), set the pruning ratio, and execute the pruning operation. For YOLO series models, pruning usually focuses on the convolutional layers of the backbone network and feature pyramid; for Transformer-type models, pruning may involve pruning attention heads and slimming down feedforward network layers. After pruning, it is necessary to manually evaluate whether the accuracy loss of the model after pruning is within an acceptable range. If the accuracy loss is too large (e.g., mAP drops by more than 2%), it is necessary to manually reduce the pruning ratio or change the pruning strategy and re-execute the pruning operation. This trial-and-error process may need to be repeated multiple times.

[0088] During the quantization process, algorithm engineers must manually select the quantization method. While post-training quantization (PTQ) eliminates the need for retraining, it can lead to significant accuracy loss for certain model architectures (such as depthwise separable convolutions and softmax layers in attention mechanisms). Quantization-aware training (QAT) better preserves accuracy but requires additional training time and computational resources. Algorithm engineers also need to manually determine the quantization precision for each layer—maintaining FP16 precision for quantization-sensitive layers and using INT8 quantization for insensitive layers—a decision that requires extensive experience and numerous experimental validations.

[0089] In the knowledge distillation stage, algorithm engineers need to manually design teacher-student model pairing schemes. The teacher model is typically the original model before optimization or a larger-scale model, while the student model is a compressed model after pruning and quantization. Designing distillation strategies involves several technical choices: response distillation (using the teacher model's output logits as soft labels), feature distillation (aligning the intermediate layer feature representations of the teacher and student models), relation distillation (preserving the feature relationship structure between samples), and combinations of various distillation strategies and configuration of loss weights. Adjusting these parameters usually requires extensive experimentation and experience.

[0090] In the hardware adaptation and deployment phase, algorithm engineers face the challenge of adapting to various heterogeneous computing platforms in the autonomous driving field. Different target hardware platforms use different inference frameworks and model formats: Platform 1 uses the TensorRT engine format, Platform 2 uses the OpenVINO IR format, Platform 3 uses the OM format, and Platform 4 uses the BNN Runtime format. After each model update, model format conversion, operator compatibility checks, inference correctness verification, and performance benchmarking must be performed separately for each target hardware platform, resulting in a large amount of repetitive work and a high risk of errors.

[0091] The above solution has the following drawbacks:

[0092] (1) Long optimization cycle and extremely low iteration efficiency: The model optimization process involves multiple steps such as pruning, quantization, distillation, and hardware adaptation, each of which requires manual intervention for evaluation, debugging, and decision-making. A complete optimization iteration (from the original model to an optimized model that meets deployment requirements) usually takes several days to several weeks. For scenarios that need to be deployed to multiple heterogeneous platforms, the optimization cycle is further extended. This severely restricts the rapid iteration and deployment update requirements of models in autonomous driving systems, which is particularly prominent in an environment where business needs change rapidly.

[0093] (2) High dependence on human experience, resulting in poor consistency in optimization quality: Key aspects such as setting pruning ratios, selecting quantization precision, configuring distillation strategies, and adjusting hardware adaptation parameters all rely on the personal experience of algorithm engineers. Different engineers optimizing the same model may produce results with significantly different performance, and the consistency and reproducibility of optimization quality cannot be guaranteed. Especially for complex model architectures involved in the field of autonomous driving (such as multimodal fusion models, multi-task learning models, etc.), the optimization strategy space is huge, and manual search is extremely inefficient and prone to getting trapped in local optima.

[0094] (3) Lack of a global optimization perspective, with each optimization step executed in isolation: In the manual optimization process, each optimization step (pruning, quantization, distillation, operator fusion, etc.) is usually executed in isolation, lacking a global consideration of the model optimization effect. For example, a high pruning ratio may make subsequent quantization more prone to accuracy collapse (because the weight distribution changes after pruning), while an excessively low pruning ratio cannot fully release the model's compression potential (wasting the redundant space in the model). The mutual influence and cumulative effect between various optimization strategies are difficult to systematically evaluate and coordinate manually.

[0095] (4) Difficult hardware adaptation and large amount of repetitive work: The field of autonomous driving involves a variety of heterogeneous computing platforms, and the computing architecture, instruction set and operator support range of different platforms vary significantly. After each model update, model conversion, performance testing and parameter tuning must be re-executed for each target platform, which is repetitive and prone to errors. The lack of a unified cross-platform optimization management mechanism leads to extremely high maintenance costs for multi-platform deployment.

[0096] (5) Difficulty in achieving Pareto optimality in accuracy and performance: Model optimization is essentially a multi-objective optimization problem (maximizing detection accuracy, minimizing inference latency, and minimizing memory usage). It is difficult to find the Pareto optimal solution set manually in the huge and complex policy combination space. Existing trial-and-error methods can only find acceptable but not optimal compromises and cannot fully utilize the potential for model compression and acceleration.

[0097] (6) Lack of systematic recording and reuse capabilities in the optimization process: Information such as experience, parameter settings, and failed attempts in the manual optimization process is usually scattered in unstructured form in the engineer's personal notes or scripts, making it difficult to form a systematic knowledge accumulation and reuse mechanism. When it is necessary to optimize a new model or adjust the optimization objective, it is often necessary to start from scratch and explore again, wasting a lot of time and computing resources.

[0098] (7) Insufficient domain adaptation for autonomous driving scenarios: Autonomous driving data annotation models have characteristics such as multi-scale feature pyramid structure, multi-task detection heads, strict requirements for real-time performance, and complex post-processing logic, making it difficult to directly apply general optimization experience. During manual optimization, the accuracy of certain categories or scenarios is prone to serious degradation, while manual evaluation is difficult to cover all edge scenarios and key categories.

[0099] In some embodiments, the AutoML system can receive a user's task objective and a first dataset, and determine an initial AI model to help the user achieve their task objective. The system trains the initial AI model using the received first dataset to obtain a trained AI model. Further, the system analyzes the training of the initial AI model using the first dataset to obtain analysis results, including the impact of at least one type of data in the first dataset on the training of the initial AI model. Specifically, the system uses an incremental experiment method to analyze the impact of different types of data on AI model training: the training dataset is divided into a baseline set and an incremental set; a baseline AI model is trained using the baseline set; each type of data in the incremental set is divided into multiple parts, and the baseline AI model is trained by gradually adding data to obtain an intermediate AI model; the change in prediction accuracy of the intermediate AI model relative to the baseline AI model after each training iteration is calculated; and the benefit coefficient of each type of data on the prediction accuracy of the intermediate AI model is obtained based on the change. Based on the analysis results, the system provides optimization methods to the user, guiding the user to upload data of a type with a benefit coefficient greater than a preset threshold (i.e., a second dataset) to optimize the trained AI model.

[0100] The AutoML system architecture comprises seven functional modules: User Input / Output (I / O) module, Data Preprocessing module, Model Determination module, Model Training module, Model Optimization and Analysis module, Dataset Storage module, and AI Model Storage module. The User I / O module receives the user's task objectives and dataset, and provides optimization suggestions. The Data Preprocessing module performs preprocessing operations on the data, including size modification, validation, encoding conversion, attribute classification, and feature extraction. The Model Determination module automatically determines the initial AI model based on the task objectives. The Model Training module performs model training. The Model Optimization and Analysis module uses incremental experimentation to analyze the impact of different types of data on training. The Dataset Storage module and AI Model Storage module store the dataset and model, respectively.

[0101] However, the above solution has the following drawbacks:

[0102] (1) This approach only addresses data optimization during the training phase and does not involve post-training model deployment optimization: It focuses on analyzing the impact of different types of data in the training dataset on the training effect of the AI ​​model, and improves the model training accuracy by recommending users to upload more data types with high yield coefficients. Its optimization target is the composition of the training dataset, not the structure and parameters of the model itself. This approach does not involve any optimization operations after the model training is completed, including model compression and acceleration technologies such as structured pruning, model quantization (PTQ / QAT), knowledge distillation, and operator fusion, nor does it involve hardware adaptation and deployment optimization of the model. For autonomous driving data annotation scenarios, the core requirement is to automatically tune the model's hyperparameters through the intelligent agent platform, that is, the AI ​​automatically selects and adjusts the combination of optimization strategies such as pruning ratio, quantization accuracy, distillation strategy, and hardware compilation parameters to find the best parameter configuration that achieves the optimal balance between accuracy and performance. However, this approach can only indirectly improve the model accuracy by optimizing the composition of the training data and cannot achieve automatic search and tuning of model optimization parameters.

[0103] (2) Lack of closed-loop automatic iteration capability: The optimization process of this scheme is a linear process of "training → analysis → suggestion → user upload new data → retraining". The optimization decision depends on the user manually collecting and uploading new data according to the suggestion. The system itself does not have the closed-loop iteration capability to automatically adjust the strategy and re-execute based on the previous training results. Each optimization requires manual participation in the data collection and uploading process, and cannot achieve a fully automatic closed-loop model optimization.

[0104] (3) Lack of hardware awareness and cross-platform adaptability: The technical solution does not consider the characteristics of the target deployment hardware platform and does not involve hardware-aware compilation optimization for different computing architectures (such as GPU, BPU, NPU, etc.). The system outputs only the trained AI model and does not involve format conversion, operator compatibility verification and inference performance optimization of the model on different hardware platforms, and cannot solve the needs of multi-platform heterogeneous deployment in the field of autonomous driving.

[0105] (4) Lack of multi-objective optimization and Pareto optimal search capability: This scheme only uses the model prediction accuracy as the single optimization objective, without involving the joint optimization of multiple performance indicators such as inference latency and memory usage. It lacks the multi-objective Pareto optimal model search capability and cannot achieve the optimal balance between accuracy and performance.

[0106] (5) Lack of core technical means for automatic search of model optimization hyperparameters: This scheme does not involve model optimization techniques such as structured pruning, quantization, knowledge distillation, and operator fusion, nor does it involve automatic search and tuning of optimization hyperparameters such as pruning ratio, quantization accuracy, and distillation loss weight. It cannot automatically find the optimal parameter combination that achieves the best balance between accuracy and performance of the model through closed-loop iteration.

[0107] In some embodiments, analytical hints and machine learning algorithm technical documents can be input into a large language model (such as GPT-4) to extract key technical information and generate analytical results. Based on the analytical results, the large language model can be used to explore the potential innovations of existing algorithms. Based on the innovations, new machine learning algorithms can be generated through a meta-learning strategy, wherein the meta-learning model takes task features and the improvement points proposed by LLM as input and outputs the adjusted network structure. The feature contribution of the newly generated algorithms can be analyzed by combining the large language model with model interpretation tools (LIME, SHAP, etc.), and the analysis results of the interpretation tools can be encoded as text and input into the large language model to generate explanatory text. The model structure and prediction performance of multiple new algorithms can be evaluated using the large language model, optimization problems can be designed and solved through the large language model to obtain model fusion strategies, and the prediction results of multiple algorithms can be combined. Feature engineering strategies can be generated using the large language model, and data features can be extracted using automated feature tools. The final algorithm can be trained using the training dataset, and the parameters can be tuned by combining the large language model with AutoML tools to select the parameter configuration with the best performance.

[0108] The system architecture of the above solution includes seven functional modules: an algorithm analysis module (inputting analysis prompts and technical documents to LLM to generate analysis results), an algorithm mining module (using LLM to mine innovative points based on analysis results), a meta-learning module (generating new algorithms through meta-learning strategies), a model interpretation module (combining LLM and LIME / SHAP to analyze feature contribution), a model fusion module (using LLM to generate model fusion strategies), a feature extraction module (using LLM to generate feature engineering strategies and executing them through automated tools), and an algorithm training module (training the final algorithm and evaluating its performance).

[0109] However, the above solution has the following drawbacks:

[0110] (1) The solution addresses the problem of algorithm design improvement, not model deployment optimization: The above solution focuses on using large language models to analyze, improve, and generate new machine learning algorithms, with the core objective of improving the predictive performance and accuracy of the algorithms themselves. The optimization target of this solution is the design scheme of the algorithm (network structure, loss function, feature engineering strategy, etc.), rather than the pre-deployment optimization of the already trained model. This solution does not involve core technologies for model deployment optimization such as model compression (pruning, quantization), model acceleration (operator fusion, hardware compilation optimization), and cross-platform deployment adaptation. For autonomous driving data annotation scenarios, the core requirement is to automatically tune the model's hyperparameters through an intelligent agent platform, that is, for AI to automatically search and determine the optimal combination of optimization strategies such as pruning ratio, quantization accuracy, distillation loss weight, and hardware compilation parameters, so that the model achieves the best balance between accuracy and inference performance. However, this solution can only use large language models to propose improvement suggestions from the algorithm design level, and cannot perform automated hyperparameter tuning and closed-loop optimization of the trained model.

[0111] (2) Lack of determinism and reproducibility in the generation results of large language models: The core technical process of this scheme relies heavily on the text generation capabilities of large language models (such as GPT-4) to discover innovative points, generate new algorithms, and design fusion strategies. The generation results of large language models are random and uncertain; the same input may produce different outputs at different times or in different calls. This uncertainty leads to a lack of reproducibility in the algorithm improvement process—the same technical document and prompts may produce different "innovative points" and "new algorithms" in two runs, making it difficult to guarantee the consistency and reliability of the optimization effect. For safety-critical application areas such as autonomous driving, determinism and reproducibility of the model optimization process are basic requirements.

[0112] (3) It does not involve automatic search and tuning of model optimization hyperparameters: This scheme does not involve model optimization techniques such as structured pruning, post-training quantization, quantization-aware training, knowledge distillation, and operator fusion. It also does not involve automatic search and tuning of optimization hyperparameters such as pruning ratio, quantization accuracy, distillation loss weight, and hardware compilation parameters. It cannot automatically determine the optimal parameter combination that achieves the best balance between accuracy and inference performance through closed-loop iteration.

[0113] (4) Lack of closed-loop automatic iteration and convergence control mechanism: The process of this scheme is a linear execution process, which does not include automatic strategy adjustment and closed-loop iteration mechanism based on verification results. When the performance of the final model does not meet the requirements, the system cannot automatically analyze the cause of failure and adjust the strategy to re-execute, and lacks closed-loop scheduling capabilities such as convergence judgment and termination condition control.

[0114] (5) Lack of multi-objective optimization and Pareto optimality search: This scheme takes the algorithm's prediction performance as the single optimization objective and does not involve the joint optimization of multiple performance indicators such as inference latency and memory usage. In autonomous driving deployment scenarios, it is necessary to make multi-objective trade-offs between model accuracy, inference speed and resource usage, and this scheme does not have such multi-objective optimization capabilities.

[0115] (6) The meta-learning module relies on a pre-trained meta-model, which limits its generalization ability: The meta-learning module of this scheme requires a pre-trained meta-model to establish the relationship between task features and innovations. The generalization ability of this meta-model is limited by the types of tasks seen during training. For complex deep learning models in the field of autonomous driving (such as multimodal fusion models, 3D point cloud detection models, etc.), the meta-model may lack sufficient prior knowledge to generate effective improvement strategies.

[0116] In the course of their long-term practice of deploying deep learning models for autonomous driving, the inventors of this application have come to realize that traditional manual model optimization methods have become a core bottleneck restricting the rapid iteration and large-scale industrialization of autonomous driving technology. Not only are the optimization cycles lengthy and inefficient, but the optimization quality also depends heavily on the personal experience of engineers, making it difficult to guarantee consistency. Furthermore, it is impossible to simultaneously consider multiple conflicting optimization objectives such as accuracy, latency, and memory usage, making it difficult to find a globally optimal trade-off solution. At the same time, the repetitive adaptation work for different hardware platforms further exacerbates the development costs. Based on this, the inventors broke through the traditional single-path optimization approach by modularizing and intelligently encapsulating the expert capabilities of each step in the manual optimization process. This resulted in an automated optimization architecture that facilitates multi-agent collaboration. A dedicated agent responsible for model structure analysis replaces human intervention in comprehensive and accurate model analysis. Another agent, responsible for optimization strategy decision-making and execution, automatically selects and combines optimization strategies based on the analysis results and adapts them to the target hardware platform. A third agent, responsible for multi-objective decision-making, filters the globally optimal model from multiple candidate models. This completely eliminates reliance on human experience and automates the entire process from optimization to deployment on the target hardware platform. While significantly improving optimization efficiency and ensuring optimization quality stability, it automatically adapts to various heterogeneous hardware platforms, effectively meeting the urgent needs of rapid model iteration and large-scale industrial deployment in the autonomous driving field.

[0117] The execution entity of this application embodiment can be a model optimization system. The system consists of at least one server at the hardware level. The server can also be called a task node. This application does not limit the specific number of servers and can be flexibly configured according to actual business needs. The entire model optimization system can be deployed in an enterprise's internal computer room, private cloud platform, or hybrid cloud environment.

[0118] In some possible implementations, at the software level, the core functions of the model optimization system are jointly implemented by a first intelligent agent, a second intelligent agent, and a third intelligent agent.

[0119] The system comprises three agents: a first agent responsible for comprehensively analyzing and processing the received target deep learning model and generating a structured model analysis report; a second agent responsible for performing model optimization and hardware adaptation operations based on the model analysis report and preset optimization goals to generate multiple candidate optimization models adapted to the target hardware platform; and a third agent, serving as the central core of the system, responsible for determining the final deployment model from the candidate optimization models and coordinating the entire process. It should be noted that each agent can be deployed as an independent microservice on at least one server within the model optimization system.

[0120] It should be noted that the embodiments of this application do not limit the number of intelligent agents. In other words, the functions can be further modularized and encapsulated according to actual business needs to obtain multiple more granular intelligent agents, or multiple related functions can be merged into one intelligent agent to adapt to different deployment scenarios and task complexities. For example, the second intelligent agent can be further divided into a fourth intelligent agent and a fifth intelligent agent. The fourth intelligent agent is specifically responsible for automatically generating and executing model optimization operations such as structured pruning, quantization, knowledge distillation, and operator fusion based on the model analysis report and preset optimization goals. The fifth intelligent agent is specifically responsible for completing the compilation optimization and hardware adaptation processing of the model based on the characteristics of the target hardware platform. The third intelligent agent can be further divided into a sixth intelligent agent and a seventh intelligent agent. The sixth intelligent agent is specifically responsible for performing comprehensive accuracy evaluation and inference performance verification on all candidate optimization models. The seventh intelligent agent, as the global hub of the system, is responsible for optimization strategy review, iteration termination judgment, optimal model selection, and the generation of final deployment delivery documents. This modular architecture, which can be flexibly split, allows each agent to focus on a single responsibility, facilitating parallel development, independent upgrades, and fault isolation. It also supports dynamically adjusting the number of instances of each agent based on task load, enabling elastic resource scheduling.

[0121] Figure 1 This application provides a system architecture diagram for a model optimization system. (See diagram below.) Figure 1 The diagram illustrates the structure of the five types of intelligent agent capability modules integrated in the model optimization system, as well as the data flow and control flow interactions between the agents. The model optimization system comprises an input layer, an agent collaboration layer, and an output layer. The input layer includes a preset optimization objective and a pre-trained target deep learning model.

[0122] The first intelligent agent receives the target deep learning model and generates a model analysis report. The seventh intelligent agent handles process scheduling and data forwarding, driving the fourth intelligent agent to perform optimization processing on the target deep learning model. The optimized model is then input into the fifth intelligent agent to perform hardware adaptation processing for the target hardware platform, resulting in multiple candidate optimized models. These candidate optimized models are input into the sixth intelligent agent for evaluation processing, yielding evaluation results for each candidate model. The evaluation results are fed back to the seventh intelligent agent, which determines whether the preset iteration termination conditions are met. If met, the iteration terminates and the final model is determined from the candidate optimized models; otherwise, a new target optimization strategy (strategy adjustment instruction) is generated and sent to the fourth intelligent agent, driving it to perform iterative optimization processing, thus forming a fully automatic closed-loop iterative optimization mechanism. The model optimization system ultimately outputs the model information of the final model, including the final model file, deployment configuration file, and a full-process optimization report.

[0123] Alternatively, model analysis, optimization strategy selection, and execution can be integrated into a single agent, such as the optimization execution agent. Hardware adaptation and model evaluation can be integrated into another agent, such as the verification and evaluation agent. Closed-loop scheduling can be treated as a separate agent, forming a three-agent module architecture: "Optimization Execution Agent – ​​Verification and Evaluation Agent – ​​Closed-Loop Scheduling Agent." This approach reduces the number of modules and communication overhead, lowering platform complexity. However, it reduces the specialization of model analysis and hardware adaptation, potentially resulting in less depth in analysis reports and less granularity in hardware adaptation compared to the aforementioned five-agent module approach. It is suitable for optimization scenarios with relatively simple model structures (such as the standard YOLO single-stage detection model) or a single target platform (requiring adaptation to only one hardware platform).

[0124] The agents communicate using structured messages in a unified JSON format. Each message includes a sender identifier, a receiver identifier, a message type (command / status / report / configuration), a payload, and a timestamp. A third agent maintains a global message routing table, responsible for the correct distribution and flow of messages between all agents, ensuring the orderliness and traceability of data transmission. This loosely coupled, modular communication architecture allows each agent to be developed, upgraded, and extended independently without modifying the code logic of other modules, while also facilitating system operation and maintenance and troubleshooting.

[0125] It should be noted that the deployment method of each agent in this application embodiment is not limited. For example, all agents can be deployed in the cloud, or they can be deployed in a distributed manner, such as a distributed optimization deployment based on cloud-edge collaboration. That is, model analysis and policy selection are deployed on cloud servers (utilizing sufficient computing resources for deep model analysis and policy evaluation), while optimization execution and hardware adaptation are deployed on edge nodes (performing optimization and performance testing directly on the target hardware to obtain real inference performance data), and closed-loop scheduling is achieved through cloud-edge collaborative communication. The cloud is responsible for computationally intensive analysis tasks (such as gradient calculation and quantization sensitivity evaluation), while the edge is responsible for hardware-related adaptation and verification tasks (such as model conversion and inference performance testing). This approach can accelerate the policy selection process by utilizing cloud computing resources, while ensuring that inference performance is verified on real target hardware, making it suitable for scenarios such as large-scale fleet management or multi-platform parallel optimization.

[0126] In some embodiments, this application is applicable to automatic hyperparameter tuning and optimization scenarios for various deep learning models in autonomous driving data annotation systems, including but not limited to: automatic tuning of real-time object detection models on vehicle-mounted edge computing platforms, automatic search of model parameters for batch inference services in autonomous driving annotation platforms, automatic cross-hardware platform adaptation of multi-sensor fusion perception models, and automatic optimization of model inference performance for large-scale cloud-based annotation tasks. This application is particularly suitable for automatic hyperparameter tuning requirements that require the AI ​​agent to automatically search and determine the optimal combination of optimization strategies such as pruning ratio, quantization accuracy, distillation loss weight, and hardware compilation parameters to achieve the best balance between model accuracy and inference performance.

[0127] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0128] Figure 2 A flowchart illustrating the optimization method for the deep learning model provided in this application. Figure 1 ,like Figure 2 As shown, the method includes:

[0129] S101. Based on the first intelligent agent, analyze and process the received target deep learning model to generate a model analysis report of the target deep learning model.

[0130] For example, the first intelligent agent is a functional module specifically responsible for model structure parsing and performance quantification analysis, possessing multi-format model parsing capabilities and architecture-specific analysis logic.

[0131] The target deep learning model can be a deep learning model that has been trained and is ready for deployment for a specific task. For example, it can be an autonomous driving data annotation model in the field of autonomous driving, such as an object detection model, a semantic segmentation model, a 3D point cloud detection model, or a multimodal fusion model. It should be noted that the model types involved in this application can cover the mainstream deep learning model architectures in autonomous driving perception tasks, including but not limited to: single-stage object detection models represented by the YOLO series, two-stage object detection models represented by Faster R-CNN and Cascade R-CNN, end-to-end detection models based on Transformer represented by DETR and Deformable DETR, 3D point cloud object detection models represented by PointPillars and SECOND, multimodal bird's-eye view fusion models represented by BEVFormer and BEVDet, and semantic segmentation models represented by the DeepLab series and SegFormer.

[0132] The target deep learning model can originate from a training server, version management repository, model registry, or model synchronization node in a cloud-edge collaborative platform within the model training environment. The receiving methods can include pulling model files via a network interface, downloading model entities after receiving model release notifications via a message queue, loading model weights and network structure description files via a local file system, or directly receiving intermediate representation files exported by the training framework.

[0133] It should be noted that the embodiments of this application do not limit the format type of the target deep learning model, and support PyTorch, Open Neural Network Exchange (ONNX), TensorFlow, etc.

[0134] Model analysis reports can be generated using structured data formats, such as JSON (JavaScript Object Notation), tabular metadata files, or database records. The content can include basic model information, task type, input resolution, output definition, module topology, parameter statistics for each layer, computational statistics for each layer, sparsity indices for each module, sensitivity ranking results, a list of potential bottleneck operators, preliminary matching results for target hardware platform support, and recommended optimization directions.

[0135] In one possible implementation, when the first agent analyzes and processes the target deep learning model, it first performs computational graph parsing to identify the backbone feature extraction part, neck feature fusion part, detection head or segmentation head part, upsampling and downsampling paths, attention module, normalization layer, activation layer, and branch connection relationships in the network. For autonomous driving data annotation models, if the target deep learning model is an object detection model, the first agent can identify the convolutional blocks, feature pyramid structure, and classification and regression branches in the prediction head of the backbone network; if the target deep learning model is a semantic segmentation model, it can further identify the encoder, decoder, multi-scale context aggregation module, and pixel-level output layer; if the target deep learning model is a 3D object detection or bird's-eye view perception model, it can also separately label the view transformation module, voxel feature encoding module, cross-modal fusion module, and spatial transformation operator. Through this computational graph parsing process, the first agent can establish structural mapping relationships from the node level, layer level to the module level.

[0136] In further implementations, the first intelligent agent can also perform statistical analysis on the parameter and computational characteristics of the target deep learning model. Parameter characteristics can include the number of parameters in each layer, parameter distribution, proportion of non-zero parameters, weight sparsity, number of channels, convolutional kernel size, number of groups, and the proportion of parameters in different modules; computational characteristics can include the floating-point operation volume, memory access volume, intermediate feature map size, input and output tensor shape, inter-layer data transfer cost, and the theoretical parallelism matching on the target hardware platform.

[0137] The first intelligent agent can also perform sparsity analysis and sensitivity analysis on the weight parameters of the target deep learning model. For example, the potential pruning space can be characterized by the proportion of parameters with absolute values ​​below a set threshold in the statistical weight matrix or convolution kernel. The sensitivity can be obtained by observing changes in validation set accuracy after freezing the parameters of the remaining layers, applying a certain proportion of channel pruning to a single layer or module, using low-bit quantization, or substitutional approximation. This sensitivity can be measured by the relative decrease in accuracy, the change in key category recall, or the change in multi-task comprehensive score. In this way, the model analysis report includes not only static structural information but also dynamic assessment information reflecting optimization risks.

[0138] The first intelligent agent can also call the operator support library and compilation constraint configuration of the target hardware platform to perform compatibility pre-checks on the operator types, tensor layout, dynamic dimension usage, and special post-processing operators in the model, thereby identifying which nodes may not be directly deployed on the target hardware platform and which nodes need to be replaced, fused, or rewritten.

[0139] By generating the structured model analysis report mentioned above, the subsequent optimization process no longer relies on manual review of the network structure item by item. Instead, it is based on a unified, reproducible, and traceable understanding of the structured model, thereby reducing the fluctuations in results caused by differences in human experience.

[0140] Optionally, based on the first intelligent agent, after receiving the target deep learning model, a model validity check can be performed first to confirm that the model file is complete, the weight tensor is parseable, the input and output node definitions are complete, and the version identifier is consistent with the task identifier. After the check is passed, the analysis and processing stage begins.

[0141] This step, by analyzing the model structure, parameters, sparsity, sensitivity, and hardware compatibility in advance, can clarify the model structure, the optimizable space, and the regions that cannot be over-compressed, making subsequent optimization processes more targeted and constrained. This helps to alleviate the problems of unstable optimization paths, high trial-and-error costs, and repetitive cross-platform analysis in existing technologies.

[0142] S102. Based on the second intelligent agent, the target deep learning model is optimized according to the model analysis report and the preset optimization target to obtain multiple candidate optimized models adapted to the target hardware platform.

[0143] For example, the second agent is a functional module specifically responsible for model optimization strategy decision-making and execution, as well as hardware adaptation, integrating various mainstream model compression acceleration algorithms and hardware compilation tools. The second agent can be deployed on the same computing device as the model analysis node, or it can run on a separate optimization processing node.

[0144] Preset optimization objectives can be automatically generated from user input, platform default strategies, historical deployment records, or business rules. Specifically, these objectives can include the target hardware platform's computing power limit, GPU or RAM capacity, inference latency threshold (target inference latency cap), throughput requirements, power budget, model file size limits, and minimum acceptable accuracy constraints (target accuracy lower limit). In autonomous driving applications, optimization objectives can be further refined into constraints on key category recognition accuracy, long-distance target recall rate, nighttime scene stability, or robustness in specific weather conditions.

[0145] The target hardware platform refers to the computing device on which the model is ultimately deployed and runs, including but not limited to autonomous driving domain controllers, edge computing boxes, servers, embedded development boards, etc. For example, it could be an autonomous driving computing platform.

[0146] Candidate optimization models refer to multiple model versions that have been adapted to hardware after being processed by combining different optimization strategies, each corresponding to different accuracy and performance trade-offs.

[0147] In one possible implementation, the second agent can perform one or more combined processes of structured pruning, quantization, and knowledge distillation on the target deep learning model, based on the analysis results and optimization objectives indicated in the model analysis report.

[0148] For example, for structured pruning, the second agent can assign different pruning ratios to channels, convolutional kernels, attention heads, or feature layers based on the sensitivity scores of each layer or module. Layers with lower sensitivity and higher computational cost can be pruned with a higher proportion of channels, while the pruning intensity is limited for key feature extraction layers and output heads with higher sensitivity, thereby avoiding a significant decline in core recognition capabilities.

[0149] For quantization processing, the second agent can select fixed-point quantization, integer quantization, or mixed-precision quantization based on the numerical format supported by the target hardware platform, and determine the quantization bit width of activation values ​​and weights, the size of the calibration sample set, the quantization range estimation method, and the precision preservation strategy for special layers. If the target hardware platform only supports 8-bit integer inference for some operators, but supports 16-bit or floating-point retention for a few key layers, the second agent can generate a mixed bit width configuration to ensure that the model maintains as much precision as possible while meeting hardware support requirements.

[0150] For knowledge distillation, the second agent can use the original model or a high-precision teacher model as a supervision source to retrain the pruned or quantized student model. This allows for the recovery of performance loss caused by compression through feature distillation, response distillation, or multi-layer alignment distillation. These processes can be executed sequentially, such as analysis followed by pruning, distillation, and quantization, or they can be performed in parallel according to a preset strategy to obtain multiple candidate optimization models with different optimization parameter configurations.

[0151] In another possible implementation, the second agent does not only generate a single optimization result, but also forms a set of candidate models around pre-defined multi-objective constraints. For example, for the same objective deep learning model, the second agent can generate multiple candidate optimization models, such as low-latency versions, low-memory versions, balanced versions, and high-precision versions.

[0152] To ensure the candidate optimized model is practically adaptable to the target hardware platform, the second agent can also perform deployment-oriented hardware adaptation processing. For example, the optimized model can be exported to a model format supported by the target inference engine, and the computation graph can undergo operator fusion, constant folding, redundant node removal, tensor layout rearrangement, static graph solidification, and replacement or plugin registration of incompatible operators. If the target hardware platform is an in-vehicle computing platform, the second agent can generate a dedicated compilation configuration based on its inference engine constraints; if the target hardware platform is a roadside edge node, the runtime graph can be lightweighted and rearranged based on its memory limitations and batch size settings; if the target hardware platform is a cloud-based inference cluster, a deployment version suitable for batch processing can be generated based on throughput and parallel scheduling strategies.

[0153] Optionally, the second intelligent agent can select one or more of the above-mentioned structured pruning, quantization and knowledge distillation as the target optimization strategy based on preset decision logic.

[0154] Alternatively, it can be an end-to-end optimization policy search based on reinforcement learning. This approach does not use rule-driven policy selection logic. Instead, it trains a reinforcement learning agent that uses features from the model analysis report (such as the proportion of parameters in each layer, sparsity distribution, quantization sensitivity grading, etc.) as its state, optimization policy combinations (pruning ratio, quantization accuracy, distillation weights, etc.) as its action, and the multi-objective score of the optimized model (a weighted score combining accuracy, latency, and memory) as its reward. It learns the optimal policy selection strategy through interaction with the environment (the model optimization process). This approach offers greater flexibility in policy selection and can discover policy combinations that are difficult to cover with manual rules. However, it requires a large number of training samples (historical optimization records) for policy learning, and the interpretability of policy selection is relatively weak. It is suitable for scenarios with a large amount of historical optimization data and a high pursuit of optimization performance.

[0155] Alternatively, optimization strategy search can be based on evolutionary algorithms. Optimization strategy combinations are encoded as chromosome individuals (each gene corresponds to the parameter configuration of one optimization strategy). Using the multi-objective score of the optimized model as the fitness function, a genetic algorithm or evolutionary strategy is used to search for the optimal combination in the policy space. Evolutionary algorithms generate a new generation of policy combinations through selection, crossover, and mutation operations, gradually approaching the global optimum. This approach does not rely on prior knowledge or training data and can explore a broader policy space, but its search efficiency is relatively low (it requires evaluating a large number of candidate policy combinations, and each evaluation requires a complete optimization-validation process). It is suitable for scenarios with ample optimization time budgets and a high pursuit of optimization performance.

[0156] Optionally, for each candidate optimization model, the second agent can also record its optimization link information, parameter configuration, estimated performance, and compilation logs so that the third agent can make traceable comparisons later.

[0157] This step combines model analysis reports with preset optimization objectives, transforming the optimization process from experience-driven to data-driven and objective constraint-driven. It can not only automatically generate multiple candidate optimization models adapted to different deployment orientations, but also significantly reduce the workload of repetitive manual parameter tuning and repeated compilation and verification during cross-platform adaptation, thereby improving the optimization efficiency and stability under multi-objective constraints.

[0158] S103. Based on the third intelligent agent, determine the final model adapted to the target hardware platform deployment from the candidate optimization models.

[0159] For example, the third agent is a functional module specifically responsible for verification and evaluation, global scheduling, and multi-objective decision-making, serving as the central coordinator of the entire system to coordinate the execution of each agent. The third agent can receive the candidate optimization model set, optimization configuration records of each candidate model, compilation results, static analysis results, and runtime test results for the target hardware platform output by the second agent.

[0160] The final model refers to the deployment model that achieves the optimal balance in multiple dimensions such as accuracy, inference latency, and memory usage, while satisfying all preset optimization objectives.

[0161] In one possible implementation, based on a third-party intelligent agent, performance evaluation is first performed on each candidate optimization model to obtain evaluation results. These results can include accuracy metrics, inference latency, throughput, peak memory usage, model file size, compilation success rate, operator compatibility status, runtime stability, and performance under specific business scenarios. Accuracy metrics can vary depending on the task type. For example, object detection tasks can use average precision, key category recall, and false negative rate; semantic segmentation tasks can use intersection-union ratio (IU) and category pixel precision; and 3D detection tasks can use detection accuracy under different distance segments. Inference latency can be measured using single-frame average latency, tail latency, or latency statistics within a specified confidence interval. Peak memory usage can be collected in real-time by the runtime monitoring module on the target hardware platform. Through these evaluation results, the third-party intelligent agent can perform horizontal evaluation of multiple candidate optimization models within the same comparison dimension.

[0162] In a further implementation, the third agent can construct a multi-objective evaluation space and determine the final model within this space. If there are explicit deployment thresholds, the third agent can first perform constraint filtering, i.e., eliminate candidate optimization models that do not meet the minimum accuracy requirements, do not satisfy the maximum latency limit, exceed the memory budget, or cannot run stably on the target hardware platform; among the remaining models, the final model is then selected according to comprehensive evaluation rules. These comprehensive evaluation rules can employ weighted scoring, Pareto frontier screening, or hierarchical decision-making methods.

[0163] For example, when using weighted scoring, a comprehensive scoring function G= can be constructed. ×Standardization Precision+ 2×Standardized Stability- 3×Standardized Delay- 4×normalized memory footprint, of which to These correspond to the varying degrees of importance of different evaluation criteria. When using Pareto frontier screening, the third agent can retain a set of models that are not completely dominated by other candidate models in terms of accuracy, latency, and memory usage, and then select the final model from this set based on business preferences. When using a hierarchical decision-making approach, accuracy can be used as the first-level screening criterion, followed by latency and memory usage as the second-level screening criteria, to ensure that safety-related perception capabilities in autonomous driving scenarios are prioritized. Thus, the third agent can automatically and interpretably determine the final model from multiple candidate optimization models.

[0164] In another possible implementation, the third agent can also assess the deployment effectiveness of candidate optimization models based on the deployment requirements of the target hardware platform, and determine the corresponding candidate optimization model as the final model when the assessment result meets preset conditions. Specifically, the third agent can trigger a trial deployment process of the candidate optimization model on the target hardware platform or its high-fidelity simulation environment, verifying the model's stability and resource consumption fluctuations under continuous operation through real input samples, standard replay datasets, or typical scene fragments. For example, in an in-vehicle platform, the third agent can monitor whether the model experiences excessive latency jitter, memory fragmentation accumulation, or post-compilation operator rollback under scenarios of multi-sensor synchronous input, complex traffic flow, and long-term operation; in a roadside edge platform, it can monitor throughput degradation under concurrent input; and in a cloud inference cluster, it can monitor resource expansion efficiency under batch processing conditions.

[0165] Optionally, the third agent can also incorporate iteration termination conditions, such as terminating the selection process when a candidate optimization model simultaneously meets the lower limit of accuracy, the upper limit of latency, and the upper limit of memory, and the comprehensive score reaches a preset threshold; if no model meets the conditions in the current candidate set, constraint conflict information or performance bottleneck information is fed back to the second agent to trigger a new round of optimization. In this way, a closed-loop linkage is formed between the three stages of analysis, optimization, and determination, enabling the model optimization process to continuously converge to a deployable result.

[0166] The third-party intelligent agent establishes a comprehensive screening mechanism by uniformly evaluating, constraining, filtering, and ultimately determining candidate optimization models, considering accuracy, inference latency, memory usage, and hardware compatibility. Compared to the multi-round trial-and-error approach relying on manual methods, this step ensures higher consistency, reproducibility, and interpretability in the final model selection, and effectively avoids the problem of deployment failure or unsatisfactory business results caused by selecting a model based on only a single indicator.

[0167] The deep learning model optimization method provided in this application involves a first intelligent agent performing a comprehensive quantitative analysis of the target deep learning model and outputting a standardized model analysis report. A second intelligent agent, based on the model analysis report and optimization objectives, automatically executes various differentiated optimization strategy combinations and generates multiple hardware-adapted candidate optimization models. A third intelligent agent performs multi-dimensional objective evaluation of all candidate optimization models and selects the optimal model suitable for deployment on the target hardware platform. This approach completely eliminates the reliance on human experience in traditional model optimization processes. Engineers are no longer required to manually analyze model structure, identify performance bottlenecks, select optimization strategies, or repeatedly debug parameters, significantly shortening the model optimization cycle and improving optimization efficiency. Simultaneously, the standardized analysis process and data-driven decision-making logic ensure the stability and consistency of optimization quality, avoiding significant differences in optimization results among different engineers. Furthermore, it can simultaneously consider multiple conflicting optimization objectives such as accuracy, inference latency, and memory usage, finding the globally optimal trade-off. It also automatically adapts the model to the target hardware platform, eliminating the need for re-optimization for different hardware platforms, further reducing development costs and effectively meeting the urgent needs of rapid model iteration and large-scale industrial deployment in fields such as autonomous driving.

[0168] Figure 3 A flowchart illustrating the optimization method for the deep learning model provided in this application. Figure 2 ,like Figure 3 As shown, in this embodiment... Figure 2 Based on the examples, the optimization method for deep learning models is described in detail, and the method includes:

[0169] S201. Based on the first intelligent agent, according to the parsing method corresponding to the format of the target deep learning model, the structure of the target deep learning model is parsed to obtain the computation graph of the target deep learning model.

[0170] For example, the parsing method refers to the computational graph extraction algorithm corresponding to different model formats.

[0171] A computation graph is a directed acyclic graph that uses nodes and edges to represent the computational logic of a model. Nodes represent operators, and edges represent data flow. In other words, a computation graph can be stored using a directed acyclic graph data structure. Nodes are used to represent operators such as convolution, normalization, activation, attention, pooling, and fully connected operations, while edges represent tensor flow. Node attributes can record tensor shape, parameter dimensions, execution order, and operator type.

[0172] In one example, as mentioned above, the target deep learning model can be in the format of an intermediate representation format exported from ONNX, TensorFlow, PyTorch, or other model file formats that facilitate graph structure parsing. The first agent can call a parsing method that matches the format to read the model file, extract nodes, and restore the topology, thereby converting the original model representation into a computation graph containing input tensors, operator nodes, parameter tensors, and connection relationships.

[0173] For example, the first agent first identifies the format of the target deep learning model. For PyTorch format models, the complete computation graph is obtained through TorchScript dynamic tracing or torch.fx symbolic tracing. For ONNX format models, the computation graph is obtained by directly parsing the operator sequences and connection relationships in the ONNX file. For TensorFlow format models, the computation graph structure is extracted through the signature definition of SavedModel.

[0174] S202. Based on the first intelligent agent, determine the architecture type of the target deep learning model according to the combination pattern of operators in the computation graph.

[0175] For example, the combination pattern of operators refers to the characteristic structure formed by specific operators in a computation graph according to a fixed order and connection relationship. Architecture type refers to the overall network architecture category to which the target deep learning model belongs; different architecture types have different structural characteristics and optimization properties.

[0176] In one example, the first agent can pre-set a rule base, which can establish a correspondence between operator combinations and architecture categories, so as to map similar models to a unified analysis framework.

[0177] After the computation graph is constructed, the first agent traverses all operators in the computation graph to detect whether there are characteristic structures such as the YOLO series' unique Detect head operator combination, the Transformer series' unique Multi-Head Attention operator combination, and the PointPillars' unique PillarScatter operator combination. Based on the detected operator combination patterns and rule base, the specific architecture type of the target deep learning model is determined.

[0178] S203. Based on the first intelligent agent, perform at least one analysis process on the computation graph according to the architecture analysis template matched by the architecture type, and generate a model analysis report.

[0179] For example, an architecture analysis template refers to a pre-customized analysis process for different architecture types, which includes key component identification rules and performance analysis methods specific to that architecture.

[0180] Analysis and processing refers to various quantitative analysis operations performed on the computation graph. Analysis and processing can include layer-by-layer parameter statistical processing, sparsity detection processing, sensitivity assessment processing, bottleneck location processing, and baseline establishment processing.

[0181] A model analysis report is a structured document that contains all analysis results. A model analysis report may include at least one of the following information: architecture description information, layer-by-layer parameter statistics, channel importance score information, quantization sensitivity classification information, bottleneck location annotation information, layer-by-layer optimization suggestions, and baseline data for accuracy and efficiency.

[0182] In one example, after determining the architecture type, the first agent invokes an architecture analysis template compatible with that architecture type to perform at least one of the following processes on the computation graph: parameter statistics, sparsity detection, sensitivity assessment, bottleneck localization, and baseline establishment. Specifically, layer-by-layer parameter statistics can statistically analyze the number of weights, biases, input / output channels, and computational cost of each layer; sparsity detection can analyze weight distribution, activation distribution, or channel response to generate channel importance scores; sensitivity assessment can evaluate the performance changes of each layer under quantization perturbation, pruning perturbation, or input perturbation to form quantization sensitivity grading information; bottleneck localization can identify computationally intensive and parameter-intensive bottleneck layers; and baseline establishment can determine the accuracy and performance benchmark of the original model. The model analysis report can further summarize the architecture description information, layer-by-layer optimization suggestions, and the above statistical results, providing a unified basis for subsequent optimization.

[0183] Specifically, (1) Layer-by-layer parameter statistical processing can statistically determine the number of parameters, computational complexity, memory usage, inference delay and other layer-by-layer parameter statistical information.

[0184] Specifically, the number of parameters is calculated by determining the number of weight parameters and bias parameters in each layer of the target deep learning model, and then determining the proportion of each layer's parameters to the total number of model parameters. Optionally, specific statistics are performed for Transformer-type models, breaking down the number of parameters in the attention layer (including Q / K / V projection matrices and output projection matrices) and the feedforward network layer (including two fully connected layers) to analyze the parameter ratio between the attention layer and the feedforward network layer, providing a quantitative basis for subsequent pruning granularity selection and pruning ratio setting.

[0185] Computational complexity: Based on the input size specifications of the target deep learning model (such as image resolution, point cloud voxelization parameters, etc.), the number of floating-point operations (FLOPs) for each layer is calculated, and the proportion of each layer's computational cost to the total computational cost of the model is statistically analyzed. Optionally, specific statistics are performed on the multi-scale feature pyramid network in the detection model to calculate the FLOPs of feature layers at different scales, accurately identifying the scale levels with concentrated computational costs, and providing support for prioritizing optimization strategies.

[0186] Memory Usage: Based on the number of model parameters and the size of intermediate feature maps during inference, estimate the memory usage of each layer during inference, clearly distinguishing between static memory (weight parameter storage) and dynamic memory (activation value storage). Focus on shallow convolutional layers with large feature map sizes (high dynamic memory usage) and deep fully connected layers with large parameter numbers (high static memory usage), providing a basis for memory optimization strategies and hardware resource allocation.

[0187] Inference latency: Based on the computational cost and parameter count of each layer, combined with reference performance data of the target hardware platform, the inference latency of each layer is estimated to identify the latency bottleneck layer of the model. The estimation process fully considers the performance differences of operator types, distinguishing the different latency characteristics of computationally intensive operators such as convolution operations and memory-access-intensive operators such as element-wise operations, providing precise guidance for hardware-aware compilation optimization and optimization strategy adjustment.

[0188] (2) The sparsity detection process can be performed by calculating the channel importance score information of the output channel of the convolutional layer of the target deep learning model according to the following steps.

[0189] 2a) L1 Norm Calculation: Calculate the L1 norm of the convolution kernel corresponding to each output channel. The smaller the L1 norm, the lower the contribution of the channel to the model output. The calculation formula is as follows:

[0190] ;

[0191] in, This represents the set of convolutional kernels corresponding to the c-th output channel. Let be the weight tensor of the i-th convolutional kernel. This is the L1 norm. The L1 norm metric is used to reflect the absolute magnitude of the channel weight magnitude; the smaller the weight magnitude, the more likely the channel is to be a redundant channel.

[0192] 2b) BN scaling factor calculation: Channel importance is evaluated using the scaling parameter of the Batch Normalization (BN) layer. Channels with a BN scaling factor close to zero contribute negligibly to the model output. The calculation formula is as follows:

[0193] ;

[0194] in, Let c be the BN scaling parameter for the c-th channel. Let be the standard deviation parameter of the BN layer for the c-th channel. This metric is based on the training results of the BN layer and reflects the effective scaling learned by the channel during training. The smaller the scaling factor, the more the training process automatically suppresses the contribution of that channel.

[0195] 2c) Taylor expansion importance metric calculation: Based on the first-order Taylor expansion approximation, the impact of removing channels on the loss function is evaluated, and the calculation formula is as follows:

[0196] ;

[0197] in, The model loss function, Let be the weight of the c-th channel, and N be the number of calibration samples. This metric is the most direct indicator of channel importance, but it requires additional gradient computation overhead. In practice, gradient computation can be completed using 50-100 calibration images.

[0198] 2d) The L1 norm measurement results BN scaling factor measurement results The results of the importance measurement with Taylor were discussed. Normalize to the [0,1] interval to obtain the normalized index. , , The three normalized indicators are weighted and fused to obtain the channel importance score for each channel. The calculation formula is as follows:

[0199] ;

[0200] in, This provides the channel importance score information for the c-th channel. , , For the weighting coefficients, satisfying The default value is =0.3、 =0.3、 =0.4, the importance of Taylor expansion is given a high weight because it directly reflects the impact on the loss function.

[0201] Optionally, the first agent can dynamically adjust the weights based on the correlation between the metrics. If the three metrics are highly consistent (giving similar importance rankings for the same channel), the default weights are used. If a metric is less correlated with the other two, the weight of that metric is reduced to decrease the impact of noise.

[0202] Furthermore, based on channel-level analysis, an overall redundancy assessment is performed on each layer of the target deep learning model. For modules containing multiple layers of similar structures (such as multiple Bottleneck layers in the CSP module of the YOLO series, stacked encoder layers in the Transformer, etc.), the contribution differences of each layer are evaluated. Specifically, the average channel importance score of each layer is calculated. (in For the first If the average importance of a layer is significantly lower than that of other layers in the same module (e.g., lower than one standard deviation of the module average), then that layer is marked as a candidate for hierarchical pruning.

[0203] Optionally, for models containing multi-head attention mechanisms (such as BEVFormer, DETR, etc.), a redundancy assessment of attention heads is further performed based on channel-level analysis. Redundant attention heads are identified from two dimensions: output contribution and attention pattern diversity. The specific steps are as follows: Forward inference is performed on the model using 50-100 calibration dataset samples, and the output feature tensors and attention weight matrices of all attention heads in each multi-head attention layer are collected throughout the process. The L2 norm of the output features of each attention head is used as a contribution metric for that head. The smaller the L2 norm, the lower the output activation intensity of the attention head and the smaller its contribution to the final output of the model. The cosine similarity between the output features of any two attention heads in the same multi-head attention layer is calculated as a quantitative indicator of attention pattern diversity. The higher the cosine similarity, the closer the feature patterns learned by the two attention heads are, and the greater the possibility of functional overlap. Based on the evaluation results of both contribution and diversity, attention heads that simultaneously satisfy the conditions of having a small output L2 norm and a high cosine similarity with other attention heads in the same layer are marked as attention head pruning redundancy candidates, providing a precise basis for subsequent attention head pruning strategies.

[0204] (3) Sensitivity assessment processing can be performed on each layer of the model according to the following steps to obtain quantitative sensitivity classification information.

[0205] 3a) Weight Quantization Error Calculation: Perform INT8 quantization simulation on the weight tensor of each layer of the model. First, map the original weights with FP32 precision to the integer range [-128, 127]. Then, map the quantized integer weights back to FP32 precision. Calculate the relative error of the weight tensor before and after quantization. The calculation formula is as follows:

[0206] ;

[0207] in, For the first The original weight tensor of the layer, This refers to the weight tensor that has been simulated and quantized before being mapped back to FP32 precision. It is the Frobenius norm. Relative error. The larger the value, the more sensitive the weight of that layer is to quantization operations, and the greater the risk of accuracy loss after quantization.

[0208] 3b) Activation value distribution characteristic analysis: Forward inference was performed on the model using 50-100 calibration dataset samples, and statistical information of the output activation values ​​of each layer was collected throughout the process, including mean, variance, dynamic range, and kurtosis. Layers with extremely large dynamic ranges or extremely skewed distributions of activation values ​​are usually more sensitive to quantization because fixed-precision quantization methods are difficult to accurately represent such non-uniformly distributed data.

[0209] 3c) Quantization Sensitivity Level Classification: Based on the calculated relative error of the weight quantization and combined with the activation value distribution characteristics, the first agent divides each layer of the model into three quantization sensitivity levels and provides corresponding quantization strategy suggestions:

[0210] Low-sensitivity layer ( <0.01): INT8 quantization can be performed safely, and the precision loss caused by quantization is negligible;

[0211] Medium-sensitivity layer (0.01≤ <0.05): INT8 quantization can be performed, but it is recommended to use it in conjunction with quantization-aware training to restore accuracy;

[0212] Highly sensitive layer ( ≥0.05): It is recommended to maintain half-precision floating-point (FP16) precision, or to use special quantization strategies such as non-uniform quantization or group quantization to reduce precision loss.

[0213] Furthermore, the first intelligent agent can obtain the quantization sensitivity classification information of each layer of the model based on the above quantization sensitivity levels.

[0214] (4) The bottleneck location processing of the model can be carried out by following the steps below to obtain the bottleneck location labeling information.

[0215] 4a) Location of computational bottleneck layer: Identify the layer in the model with the highest proportion of floating-point operations (FLOPs) to total computation. These layers are usually deep convolutional kernel layers or the projection matrix of attention layers, and are the main source of model inference latency.

[0216] 4b) Parameter bottleneck layer location: Identify the layer with the highest proportion of parameters in the model. This type of layer is the main source of model file size and static memory usage during inference.

[0217] 4c) Low contribution layer location: Identify channels whose importance score information is below a preset pruning threshold. (The default value is) Channels or levels with a value of 0.1 have a very low contribution to the final output of the model and are the main candidates for subsequent structured pruning operations.

[0218] 4d) Quantization-sensitive layer location: Identify layers marked as "highly sensitive" in the quantization sensitivity assessment. These layers will suffer a large loss of accuracy when using conventional INT8 quantization, and need to be processed using special quantization strategies.

[0219] (5) Baseline establishment can be performed by following the steps below to obtain baseline data with high accuracy and efficiency.

[0220] 5a) Accuracy baseline establishment: Perform a full accuracy test on the original target deep learning model using the standard evaluation dataset, and record the full accuracy metrics by task type and evaluation dimension:

[0221] For object detection tasks, the mean average precision (mAP) (including industry-standard metrics such as mAP@0.5 and mAP@0.5:0.95) of the model under different Intersection over Union (IoU) thresholds, as well as the mean average precision (AP) and mean recall (AR) for each category, are calculated and recorded. At the same time, the AP and AR metrics for small, medium, and large objects are calculated separately according to the difficulty level of the object size.

[0222] For semantic segmentation tasks, the mean Intersection over Union (mIoU), the IoU for each class, the overall pixel accuracy, and the class pixel accuracy metrics are calculated and recorded.

[0223] For 3D target detection tasks, additional records are made of 3D mAP in three-dimensional space, bird's-eye view (BEV) mAP, and detection accuracy indicators at different distance ranges.

[0224] 5b) Efficiency Baseline Establishment: Based on a unified reference hardware platform and standard testing conditions, measure and record the full efficiency metrics of the original target deep learning model: total number of parameters, number of trainable parameters, and number of non-trainable parameters; model file size at FP32, FP16, and INT8 precision levels; FLOPs; P50, P95, and P99 quantiles of inference latency, average inference latency, FPS, and peak memory usage during inference. Simultaneously record environmental information such as the reference hardware platform model, driver version, and inference framework version to ensure the reproducibility and comparability of the baseline data.

[0225] After the automatic parsing described above, the first agent can generate a structured model analysis report. This report may include: a description of the overall model architecture and identification results of key components, a layer-by-layer parameter statistics table, a histogram of channel importance distribution, a quantization sensitivity grading table, bottleneck location annotations, optimization suggestions for each layer (suggested pruning ratios, suggested quantization accuracy, distillation applicability assessment, etc.), and baseline data for accuracy and efficiency. This model analysis report is transmitted to the seventh agent in structured JSON format. Upon receiving the report, the seventh agent forwards it to the fourth agent.

[0226] In this embodiment, the first agent transforms the model judgment process, which originally relied on human experience, into a structured analysis process through format adaptation parsing, computation graph recognition, architecture classification, and templated analysis. This enables the stable extraction of model features and the generation of reusable reports. After the model analysis report is output, subsequent agents can directly determine pruning, quantization, distillation, or compilation adaptation strategies based on the same analysis criteria, thereby reducing repeated trial and error and improving optimization consistency.

[0227] After adopting the aforementioned steps, the structural information of the target deep learning model can be accurately restored and its architectural attributes can be identified. This leads to the generation of a model analysis report containing multi-dimensional analysis results, providing a clear basis for subsequent optimization processes, reducing reliance on human experience, improving the stability, reproducibility, and processing efficiency of cross-platform model analysis, and helping to enhance the deployment and adaptation capabilities of autonomous driving perception models on edge devices and heterogeneous hardware.

[0228] S204. Based on the fourth intelligent agent, determine multiple target optimization strategies for the target deep learning model according to the model analysis report, optimization objectives, and preset decision logic.

[0229] For example, the fourth agent is a functional module specifically responsible for optimizing policy decision-making and execution, and integrates a variety of mainstream model compression and acceleration algorithms.

[0230] The preset decision logic refers to the predefined optimization strategy selection rules, which automatically match the optimal strategy based on the model analysis report and optimization objectives.

[0231] Objective optimization strategies refer to specific combinations of methods used to accelerate model compression. Objective optimization strategies include at least one of the following strategies: structured pruning, quantization, knowledge distillation, and operator fusion.

[0232] In one example, the fourth agent receives the model analysis report and the optimization target set by the user. According to the preset decision logic, when there are channels with importance scores below the threshold, structured pruning is enabled. A suitable quantization scheme is selected according to the quantization sensitivity distribution. When the compression ratio is large, knowledge distillation is automatically enabled. At the same time, all fusionable operator patterns are detected. Finally, multiple target optimization strategies with different pruning ratios, different quantization accuracies, and different strategy combinations are generated.

[0233] For structured pruning, the structure to be pruned can be determined based on channel importance scores, layer output contribution, or redundancy thresholds. Redundant parts can be removed by channel, convolution kernel, or module to reduce computational cost and parameter count. For quantization, weight precision and activation precision can be determined based on the quantization sensitivity of each layer, and floating-point parameters can be converted to fixed-point or low-bit parameters to reduce model size and improve inference speed. For knowledge distillation, distillation constraints can be constructed based on teacher model output or intermediate features to ensure that the optimized model maintains recognition accuracy after compression. For operator fusion, adjacent operators such as convolution, batch normalization, and activation can be merged into a single computation graph node to reduce runtime operator scheduling overhead.

[0234] For example, the fourth agent automatically selects an optimization strategy based on the following decision-making logic:

[0235] (1) Pruning strategy selection: When the model analysis report shows that the channel importance score information is lower than the pruning threshold. When using a channel, structured pruning is enabled. The pruning granularity is automatically determined based on the redundancy distribution characteristics of the model.

[0236] If redundant channels are concentrated in a specific layer (such as output channels with fewer categories in the detector head, or deep channels in the feature pyramid), channel-level pruning is adopted, and the differentiated pruning ratio is set according to the layer.

[0237] If the redundancy of the entire layer or module is high (e.g., the average channel importance of a deep encoder layer is significantly lower than that of a shallow layer), hierarchical pruning is used to remove the redundant layer as a whole.

[0238] If there are redundant heads in a multi-head attention mechanism (attention heads with low contribution and highly similar to other heads), attention head pruning is adopted.

[0239] (2) Quantization strategy selection: Select a quantization scheme based on the model's deployment objectives and quantization sensitivity grading information:

[0240] If the low-sensitivity and medium-sensitivity layers in the model account for more than 80% of the total computation, and the accuracy margin is sufficient (the model accuracy is significantly higher than the minimum accuracy required by the business), post-training quantization (PTQ) should be used first to achieve quantization deployment with the minimum computational cost.

[0241] If post-training quantization causes an accuracy loss that exceeds an acceptable range (determined through rapid calibration evaluation), enable Quantization-Aware Training (QAT) to simulate quantization operations during fine-tuning to restore accuracy.

[0242] If there are highly sensitive layers in the model, FP16 precision is maintained for the highly sensitive layers, while INT8 quantization is used for the insensitive layers, forming a mixed precision quantization scheme.

[0243] (3) Knowledge distillation strategy selection: When a single compression strategy (pruning only or quantization only) cannot simultaneously meet the accuracy and performance requirements, or when the compression ratio is large (pruning ratio exceeds 30% or quantization accuracy drops from FP32 to INT8), knowledge distillation is automatically activated to assist in accuracy recovery. The distillation strategy is automatically selected based on the structural relationship between the teacher model and the student model:

[0244] If the teacher model and the student model have the same architecture but different depths (e.g., pruning from YOLOv5l to YOLOv5s), feature distillation is used to align the feature representations of corresponding layers in the backbone network and the feature pyramid.

[0245] If the teacher model and the student model have different architectures (such as distillation from a large model to a lightweight model), relation distillation is used to preserve the relational structure of features between samples.

[0246] If versatility and robustness are desired, a comprehensive distillation strategy can be adopted, combining response distillation, characteristic distillation, and relational distillation.

[0247] (4) Operator fusion strategy selection: For detected fusionable operator patterns (such as Conv+BN+ReLU, Conv+BN+Sigmoid, MatMul+Softmax, Conv+DepthwiseConv, etc.), they are automatically marked as fusion candidates and fusion operations are performed during the hardware adaptation stage.

[0248] The fourth agent generates an optimization execution plan (described in YAML format) from the selected multiple target optimization strategies and their parameter configurations. This plan includes the execution order of optimization operations, the parameter configurations for each operation, the estimated expected optimization effect, and the dependencies between operations. The execution plan is submitted to the seventh agent for review and then executed sequentially.

[0249] S205. Based on the fourth intelligent agent, the target deep learning model is optimized according to the target optimization strategy to obtain the optimized model.

[0250] For example, optimization processing refers to compression and acceleration operations performed on the original model according to the target optimization strategy. The optimized model refers to the intermediate model obtained after processing through a combination of single optimization strategies.

[0251] In one example, the fourth agent optimizes the strategy for each objective independently, performing optimization operations on the original objective deep learning model. For the structured pruning strategy, it removes low-contribution channels and fine-tunes them according to channel importance; for the quantization strategy, it performs post-training quantization or quantization-aware training; for the knowledge distillation strategy, it uses the original model as the teacher model to guide the training of the optimized model; and for the operator fusion strategy, it labels all fusionable operator combinations, ultimately obtaining multiple different optimized models.

[0252] Optionally, based on the fourth intelligent agent, the target optimization strategy is sent to the third intelligent agent; based on the third intelligent agent, the target optimization strategy is reviewed and the review result is returned; based on the fourth intelligent agent, the target deep learning model is optimized according to the review result to obtain the optimized model.

[0253] The review process refers to the process of checking and verifying the legality, rationality, and feasibility of the target optimization strategy. The review result refers to the conclusion returned by the third-party intelligent agent after reviewing the target optimization strategy, including approval, rejection, and corresponding modification suggestions.

[0254] For example, after generating the target optimization strategy, the fourth agent encapsulates the corresponding strategy parameters, applicable scope, and expected optimization target into an audit request and sends it to the third agent. The third agent, combining preset thresholds, model structure constraints, and hardware platform constraints, performs consistency checks and feasibility assessments on the strategy and outputs the audit results. When the audit result is satisfactory, the fourth agent directly calls the model compression operator, reparameterization operator, or training constraint update operator based on the target optimization strategy to perform structural adjustments, weight reallocation, or parameter recalibration on the target deep learning model, thereby forming the optimized model. When the audit result is unsatisfactory, the fourth agent can modify the target optimization strategy based on the constraint information contained in the audit result and then perform optimization processing again to avoid using an incompatible strategy that would lead to a significant decrease in model accuracy or the inability to compile the computation graph correctly.

[0255] Optionally, the third intelligent agent may also include a seventh intelligent agent. After the fourth intelligent agent generates multiple target optimization strategies, it encapsulates each target optimization strategy into a JSON format review request message containing a strategy identifier, strategy content, parameter configuration, and expected optimization effect, and sends it to the seventh intelligent agent in the third intelligent agent. Upon receiving the review request, the seventh intelligent agent performs multi-dimensional reviews sequentially. First, it checks whether the target optimization strategy meets the user-defined optimization target constraints, such as whether the pruning ratio exceeds the maximum allowable value and whether the quantization accuracy meets the requirements of the target hardware platform. Second, it checks the rationality of the strategy combination, such as whether there are conflicting optimization strategies and whether the parameter configuration is within the preset range. Within reasonable limits; the feasibility of the optimization strategy is checked again, such as whether there are operators or precision types that are not supported by the target hardware platform; finally, the seventh agent generates the review result, returning "approved" for target optimization strategies that meet the requirements, and "rejected" for target optimization strategies that do not meet the requirements, clearly pointing out the problems and modification suggestions; after receiving the review result, the fourth agent immediately starts to perform the corresponding optimization operation on the original target deep learning model for target optimization strategies that have passed the review, and adjusts the strategy parameters or reselects the strategy combination according to the returned modification suggestions for target optimization strategies that have been rejected, and resubmits for review until it is approved, finally obtaining the optimized model.

[0256] By having a third intelligent agent pre-approving the target optimization strategy, the fourth intelligent agent can perform model optimization under the condition of meeting the approval constraints, thereby improving the applicability of the target optimization strategy and the reliability of the optimization process, reducing repeated adjustments caused by inconsistent strategies, out-of-bounds parameters or hardware incompatibility, and thus improving the optimization efficiency and result stability of the autonomous driving perception model in a heterogeneous deployment environment.

[0257] Specifically, when the target optimization strategy includes structured pruning, the specific steps are as follows:

[0258] (1) Scoring information based on channel importance All candidate pruning channels in the model are sorted in ascending order of importance from low to high;

[0259] (2) Determine the importance threshold based on the global pruning ratio p recommended in the model analysis report (usually ranging from 0.2 to 0.5). This threshold is calculated from the pruning ratio p and the corresponding quantile of the channel importance distribution. That is, channels with the bottom p proportion of importance ranking are selected to form the initial pruning channel subset. It satisfies the formula: ;

[0260] Based on this, the pruning ratio is dynamically adjusted according to the redundancy of each layer: for layers with low average channel importance and high redundancy, the pruning ratio is appropriately increased; for layers with high average channel importance and low redundancy, the pruning ratio is appropriately decreased, ultimately resulting in differentiated pruning channel subsets for each layer.

[0261] (3) Perform pruning operations: remove The convolutional layer determines the channels and their corresponding BN parameters, and simultaneously adjusts the input and output dimensions of adjacent layers to ensure the integrity and effectiveness of the model structure. Specifically, when the convolutional layer... When the c-th output channel is pruned, the layer is deleted along the output channel dimension. The c-th slice of the weight tensor is used to remove the corresponding batch normalization (BN) parameters, including the scaling factor, from that channel. offset factor running average and operating variance If layer If it is a convolutional layer, then delete the layer along the input channel dimension. The c-th slice of the weight tensor.

[0262] (4) Fine-tuning after pruning: Short-cycle fine-tuning training is performed on the pruned model. The number of fine-tuning training rounds is usually 10%~20% of the original training rounds. A subset of the original training dataset (usually 20%~30% of the complete dataset) is used for the training dataset to reduce computational overhead. The fine-tuning process adopts a knowledge distillation-assisted strategy, using the original model before pruning as the teacher model to guide the training of the pruned student model. The total loss function is:

[0263] ;

[0264] in, This refers to the original task loss of the model (such as cross-entropy classification loss and bounding box regression loss in object detection tasks). This is an auxiliary weight for distillation loss, with a default value of 0.5.

[0265] When the objective optimization strategy includes quantization, the fourth agent performs one or a combination of the following quantization operations:

[0266] (1) Post-Training Quantization (PTQ)

[0267] (a) Sample a batch of representative calibration data from the calibration dataset. The sample size is usually 200 to 500 images or equivalent point cloud data to ensure that the distribution of calibration data can fully cover the data distribution that may be encountered in the actual inference scenario.

[0268] (b) Perform forward inference on the model using the sampled calibration data and collect activation value statistics (minimum, maximum, and distribution histogram) for each layer. For weight quantization, directly calculate the range of values ​​for the weight tensor of each layer; for activation quantization, calculate the minimum, maximum, and distribution histogram of the output activation values ​​of each layer during the inference process.

[0269] (c) Calculate the quantization parameters for each layer based on the collected activation value statistics, including the scaling factor scale(s) and the zero point (z). The calculation formula is as follows:

[0270] ;

[0271] ;

[0272] in, , The upper and lower limits of the value range for the target quantization format (0 and 255 for INT8 asymmetric quantization, and -127 and 127 for INT8 symmetric quantization). , These represent the upper and lower limits of observation for the corresponding tensor. For weight tensors, a uniform symmetric quantization strategy (zero point z=0) is used; for activation tensors, a symmetric or asymmetric quantization strategy is automatically selected based on the symmetry of their distribution.

[0273] (d) Based on the calculated quantization parameters, convert the model's weights and activation values ​​from FP32 precision to INT8 precision. For the mixed-precision quantization scheme, strictly follow the quantization sensitivity evaluation results, maintain FP16 precision for layers marked as highly sensitive, and perform INT8 quantization only on insensitive layers.

[0274] (e) Calculate the relative error after quantization for each layer. Where W is the original FP32 weight tensor, This is the weight tensor that is quantized and then mapped back to FP32. If the quantization error of a certain layer exceeds a preset threshold... (If the default value is 0.05), the layer will be automatically rolled back to FP16 precision to avoid excessive precision loss.

[0275] (2) Quantization-Aware Training (QAT)

[0276] (a) Insert fake quantization nodes before and after all convolutional layers, fully connected layers, and activation layers of the original model to simulate quantization-dequantization operations during forward propagation. The calculation formula is as follows:

[0277] ;

[0278] in, The `clamp` function represents rounding, restricting the value to within the quantization range. Pseudo-quantized nodes simulate the information loss caused by quantization during training, allowing the model parameters to gradually adapt to the lower-precision representation during training.

[0279] (b) Use a straight-through estimator (STE) to approximate the gradient during backpropagation. That is, perform the full quantization operation in forward propagation, skip the gradient calculation of the quantization operation in backpropagation, and pass the gradient directly to the original floating-point parameters to ensure that the gradient can be backpropagated normally and update the model parameters.

[0280] (c) Execute the complete quantization-aware training process. The number of training rounds is usually 30% to 50% of the total number of training rounds of the original model. The initial learning rate is set to 1 / 10 of the maximum learning rate of the original training. A cosine annealing learning rate scheduling strategy is used to allow the model to gradually adapt to quantization noise. Knowledge distillation loss is also introduced during the training process, using the original floating-point model as the teacher model to further assist the quantization model in recovering accuracy.

[0281] When the objective optimization strategy includes knowledge distillation, the fourth agent performs the following steps:

[0282] (1) Automatic construction of teacher-student model pairing

[0283] Automatically establish the pairing relationship between the teacher model and the student model: The teacher model adopts the original model before optimization, maintaining the original structure and FP32 accuracy. All parameters are frozen during training, and gradient calculation is not performed. It is only used to provide supervision signals. The student model is a compressed model after structured pruning and / or quantization, which serves as the target model to be trained.

[0284] (2) Configuration of distillation loss function

[0285] Based on the preset distillation strategy, configure the comprehensive distillation loss function and the weight coefficients of each component. for:

[0286] ;

[0287] The specific definitions and configurations of each component are as follows:

[0288] Response to distillation loss Align the final output distributions of the teacher model and the student model, and use KL divergence to measure the difference between the two output distributions. The calculation formula is as follows:

[0289] ;

[0290] in, , These are the raw logits (without softmax activation) output from the classification heads of the teacher and student models, respectively. T is the temperature parameter (the default value is 4; a higher temperature can make the output distribution smoother and expose more inter-class relationships). This is the softmax activation function.

[0291] Characteristic distillation loss To align the feature representations of the intermediate layers of the teacher and student models, the mean squared error (MSE) is used to measure the difference in feature maps. The calculation formula is as follows:

[0292] ;

[0293] in, , The teacher model and the student model are respectively the first The output feature map of the layer, Let L be the feature adaptation function (achieving channel dimension alignment through 1×1 convolution and spatial dimension alignment through bilinear interpolation), and L be the total number of intermediate layers participating in feature distillation. Distillation layers are automatically selected for different model architectures: detection models typically select output features from each scale layer of the feature pyramid network; Transformer models typically select output features from each encoder layer.

[0294] Relationship to distillation loss To maintain consistency between the teacher model and the student model in terms of the feature relationship structure between samples, the calculation formula is as follows:

[0295] ;

[0296] Where P is the set of feature pairs (sampled from different layers or different spatial locations of the same batch of samples). The relation metric function is (using cosine similarity or L2 distance of the feature vectors).

[0297] Weighting coefficient , , The default values ​​are 0.3, 0.4, and 0.3. Feature distillation is given a higher weight because it provides the richest intermediate layer supervision signal. During training, the system dynamically adjusts the weights based on the teacher-student feature alignment: if the response distillation loss decreases slowly while the feature distillation loss decreases rapidly, the weights are increased. The weights are used to strengthen the alignment constraints on the output distribution.

[0298] (3) Distillation training execution

[0299] Distillation training is performed using the original training dataset or a specified subset thereof. The same input data is fed into both the teacher model and the student model simultaneously. The original task loss (such as the cross-entropy loss for classification tasks, the sum of the classification loss and regression loss for detection tasks) and the comprehensive distillation loss are calculated separately. The two are weighted and summed to obtain the total loss function. The parameters of the student model are updated only through backpropagation.

[0300] (4) Monitoring and early termination of training process

[0301] During training, the alignment of feature representations between the teacher and student models is dynamically monitored, and the feature similarity is quantified using the Centered Kernel Alignment (CKA) metric. When the feature alignment reaches a preset standard (e.g., feature cosine similarity greater than 0.95), the distillation training is automatically terminated early to save computational resources.

[0302] When the objective optimization strategy includes operator fusion, the fourth agent performs the following steps:

[0303] (1) Fusionable operator pattern scanning recognition

[0304] The system traverses the complete computational graph of the scanning model, automatically identifying all fusionable operator patterns that conform to preset rules. Common patterns include, but are not limited to, Conv2d+BatchNorm2d+ReLU, Conv2d+BatchNorm2d+Sigmoid, MatMul+Scale+Softmax, etc. It performs focused scanning on different model architectures: for detection models, it focuses on identifying numerous repetitive convolution-normalization-activation combinations in the backbone network and feature pyramid network; for Transformer models, it focuses on identifying combinations of linear projection and layer normalization.

[0305] (2) Execution of operator fusion operation

[0306] All identified fusionable operator patterns are replaced with their corresponding single fusion operators. Taking Conv2d+BatchNorm2d fusion as an example, all parameters of the BatchNorm layer are folded into the weights and biases of the preceding Conv2d layer. The calculation formula is as follows:

[0307] ;

[0308] ;

[0309] in, , , , Here, W represents the scaling factor, offset factor, running mean, and running standard deviation of the BatchNorm layer, respectively, while W and b are the weights and biases of the original Conv2d layer. The fused single convolutional layer does not require independent BatchNorm computation during inference, saving additional computational steps and memory access overhead.

[0310] (3) Verification of the correctness of the fused model

[0311] Select 5-10 calibration samples and input them into the models before and after fusion, respectively, and compare the numerical differences in their outputs. The maximum absolute error of the outputs for all samples should not exceed [a certain value]. If the fusion operation is confirmed to have not introduced any calculation errors, the operator fusion process is completed; if the error exceeds the threshold, the fusion operation is rolled back and the abnormal operator combination is recorded.

[0312] S206. Based on the fifth intelligent agent, the optimized model is subjected to hardware adaptation processing according to the target hardware platform to obtain candidate optimized models.

[0313] For example, the fifth intelligent agent is a functional module specifically responsible for hardware perception compilation and optimization, and has built-in feature description libraries for multiple mainstream hardware platforms.

[0314] Hardware adaptation refers to the process of converting a general-purpose optimized model into a deployment model that can be efficiently executed on the target hardware platform.

[0315] Candidate optimization models refer to model versions that, after hardware adaptation, can run directly on the target hardware platform.

[0316] In one example, the fifth agent, based on the target hardware platform specified by the user, extracts information such as the platform's computing power, memory level, supported operator set, and numerical precision from a hardware feature description library. It then performs compilation optimization operations such as model format conversion, operator compatibility verification, and inference parameter optimization on each optimized model, ultimately obtaining multiple candidate optimized models adapted to the target hardware platform. For example, Table 1 provides an example of a hardware feature description library provided in an embodiment of this application.

[0317] Table 1 Hardware Feature Description Library

[0318]

[0319] Specifically, based on the fifth intelligent agent, the platform information of the target hardware platform is determined according to the target hardware platform and the preset hardware feature description library; based on the fifth intelligent agent, the optimized model is compiled and optimized according to the platform information of the target hardware platform to obtain a candidate optimized model; wherein, the compilation and optimization process includes at least one of the following operations: model format conversion operation, operator compatibility verification operation, inference performance prediction operation, and inference parameter determination operation.

[0320] The preset hardware feature description library is a pre-built structured database that stores information such as the computing power, memory hierarchy, supported operator set, supported numerical precision, inference framework interface, and performance model parameters of each hardware platform.

[0321] Platform information comprises all relevant technical feature parameters obtained by matching the target hardware platform with the hardware feature description library, used to characterize the platform's deployment environment. Compilation optimization refers to the deployment-oriented transformation and verification of the optimized model based on the platform information, ensuring the model meets the execution requirements of the target hardware platform. The fifth agent can extract information such as the target hardware platform's computing power, memory hierarchy, supported operator set, supported numerical precision, and inference framework interface from the hardware feature description library based on the target hardware platform's identifier, generating a hardware adaptation configuration.

[0322] The model format conversion operation is used to convert the optimized model into a model file or intermediate representation that can be accepted by the target hardware platform. The operator compatibility verification operation is used to check whether the operators in the model can be mapped and executed normally on the target hardware platform. The inference performance prediction operation is used to predict inference latency, throughput and resource consumption by combining model structure and platform characteristics. The inference parameter determination operation is used to generate inference configuration parameters that are compatible with the platform to ensure that the candidate optimized model can be deployed stably.

[0323] In practical implementation, the fifth intelligent agent can first query and extract the complete platform information corresponding to the target hardware platform from the preset hardware feature description library based on the target hardware platform identifier, including the model format supported by the platform, operator whitelist, numerical precision range, memory bandwidth, number of computing cores, etc. Subsequently, the fifth agent performs a model format conversion operation, converting the optimized model from the common PyTorch or ONNX format into a dedicated format supported by the target hardware platform, such as TensorRTEngine, OpenVINO IR, or ONNX Runtime format. It then performs an operator compatibility verification operation, checking whether all operators in the model are in the target hardware platform's supported operator set. Unsupported operators are automatically replaced with equivalent operator combinations or decomposed into multiple basic operators; for operators that cannot be replaced, custom operator templates are generated. Next, it performs an inference performance prediction operation, estimating the model's inference latency, throughput, and memory usage on the target hardware platform based on the platform's performance model. Finally, it performs an inference parameter determination operation, automatically determining the optimal inference batch size, number of inference threads, number of streaming parallelisms, number of warm-up cycles, and other inference parameters based on the prediction results and hardware resources. After completing at least one of the above compilation optimization operations, a candidate optimized model is generated. In practical applications, this compilation optimization process can also execute only one of these operations or combine multiple operations; this embodiment does not limit this.

[0324] Specifically, the fifth intelligent agent performs a full-process hardware-aware compilation and optimization on the optimized model based on the platform information of the target hardware platform, generating a candidate optimized model that can run efficiently on the target hardware platform. The specific steps are as follows:

[0325] (1) Unified model format conversion across multiple platforms

[0326] A two-stage standardized conversion strategy of "PyTorch → Open Neural Network Exchange (ONNX) → Platform-specific format" is adopted to ensure cross-platform compatibility and the reproducibility of the conversion process.

[0327] Phase 1: Export the optimized PyTorch model to the standard ONNX format, specifying the opset version as 11 or higher to ensure operator compatibility. During the export process, basic optimizations of the computation graph are automatically performed, including constant folding, dead code elimination, and redundant node removal.

[0328] The second stage involves converting the standardized ONNX model into a format specific to the target hardware platform and performing platform-specific deep optimizations. The specific conversion process for each platform is as follows:

[0329] For Platform A: ONNX → Inference Optimization Engine (TensorRT) Engine, it utilizes TensorRT's core functions such as layer fusion, automatic kernel tuning, and dynamic shape support to generate a high-efficiency inference engine, automatically selects the optimal implementation of each operator on the target GPU (including Tensor Core acceleration, Winograd convolution, etc.), and determines the optimal kernel parameters through multiple batch tests.

[0330] For D: ONNX → OpenVINO (Open Visual Inference and Optimization Toolkit) IR, leveraging OpenVINO's operator optimization and CPU / GPU / iGPU heterogeneous inference acceleration capabilities, it automatically performs optimizations such as operator fusion, layout optimization (adjusting data memory layout to adapt to hardware cache structure), and constant propagation.

[0331] For C:ONNX→OM (offline model), it utilizes operator optimization from the Ascend Computing Language (ACL) library and hardware acceleration capabilities of the processor's Cube / Vector computing unit;

[0332] For B:ONNX→BNN Runtime models, the BPU's dedicated quantization inference acceleration capabilities are utilized to automatically perform calibration quantization adapted to the BPU hardware characteristics during the conversion process.

[0333] (2) Automatic verification and adaptation of operator compatibility

[0334] During the model format conversion process, each operator in the model is checked to see if it is in the official operator support list of the target platform. For unsupported operators, hierarchical adaptation is automatically performed:

[0335] If there is an equivalent combination of standard operators that can achieve the same computational logic, the operator decomposition and replacement will be performed automatically. For example, SiLU will be decomposed into Sigmoid+Multiply, and Mish will be decomposed into Softplus+Tanh+Multiply.

[0336] If no equivalent standard operator combination can be used as a replacement, the operator will be marked as a custom operator, and a complete framework code template for the custom operator plugin will be automatically generated, including input and output specification definitions, core computational logic framework, compilation and build instructions, and integration interface descriptions, for subsequent manual integration.

[0337] (3) Hardware-based perception inference performance prediction

[0338] Based on the performance model of the target hardware platform, and combining the model's layer-by-layer computational complexity and memory access characteristics, the inference latency of the optimized model on the target platform is estimated. The calculation formula is:

[0339] ;

[0340] in, For the first The computational cost of a layer (float-point operations). This is the reference throughput of the corresponding operator on the target hardware (pre-stored from the hardware feature description library). For the first Total memory access volume of the layer (weight read volume + activation value read / write volume). The corresponding memory bandwidth of the target hardware. The framework scheduling overhead and data copy latency are estimated based on empirical values ​​of historical statistics.

[0341] (4) Automatic configuration of dynamic inference parameters

[0342] Automatically configure optimal inference runtime parameters based on the hardware characteristics of the target platform and the requirements of the business scenario:

[0343] Optimal batch size calculation: Based on the target hardware's memory limitations and business throughput requirements, the optimal batch size is calculated using an incremental trial-and-error method: starting from batch size 1 and gradually increasing, each time estimating the memory usage of the model inference under the corresponding batch, and selecting the maximum batch size within the memory limitation range;

[0344] Streaming inference configuration: For video stream or continuous data input scenarios, the pipeline parallel inference mode is automatically configured to enable model inference to be executed in parallel with data preprocessing and postprocessing to maximize throughput. Specific configuration parameters include pipeline depth (number of frames processed at the same time), input queue length, output queue length, and timeout threshold.

[0345] Multi-stream parallel configuration: For hardware platforms that support multi-stream parallelism (such as GPUs on platform A that support multiple Compute Unified Device Architecture (CUDA) streams for parallel execution), the optimal number of parallel streams is automatically configured based on the number of hardware parallel computing units and the computational density of model inference, making full use of hardware computing resources.

[0346] Through the above processing, the fifth agent can establish a correspondence between the optimized model and the execution characteristics of the target hardware platform, and complete the comprehensive adaptation of format, compatibility, performance and parameters during the compilation stage, thereby reducing the risk of subsequent deployment, reducing the number of repeated trials and errors, and improving the deployability and stability of the candidate optimized model in heterogeneous hardware environments.

[0347] Through the aforementioned steps, the fourth agent first generates an executable target optimization strategy based on the model structure and optimization objectives to compress and accelerate the target deep learning model. Subsequently, the fifth agent, in conjunction with the target hardware platform, adapts the format, operators, and runtime parameters, enabling the optimization results to be directly used for subsequent deployment and verification. This approach divides the tasks of strategy selection, model optimization, and hardware adaptation in a collaborative manner, reducing reliance on human experience and improving the consistency and stability of cross-platform deployment.

[0348] After adopting the aforementioned steps, a suitable optimization path can be automatically determined based on the model analysis report, reducing the cost of repeated trial and error, and making the optimized model more closely match the resource constraints and operating characteristics of the target hardware platform. Since the optimization strategy and hardware adaptation are completed collaboratively by different intelligent agents, the model compression efficiency, deployment and adaptation efficiency, and overall stability under multi-objective constraints can be improved, which in turn facilitates the rapid reuse and consistent deployment of autonomous driving perception models in vehicle platforms, edge nodes, and cloud environments.

[0349] S207. Based on the sixth intelligent agent, evaluate the candidate optimization models to obtain the evaluation results of each candidate optimization model.

[0350] For example, the sixth agent is a functional module specifically responsible for model validation and evaluation, possessing standardized accuracy and performance testing capabilities. Evaluation processing refers to the comprehensive testing of the accuracy and inference performance of candidate optimized models. The evaluation results include quantitative data on various performance indicators of the candidate models.

[0351] In one example, the sixth agent performs accuracy evaluation on each candidate optimization model on a standard evaluation dataset, calculating accuracy metrics; performs inference performance tests on the target hardware platform, measuring metrics such as inference latency, throughput, and memory usage; and finally integrates all test results into the evaluation results for each candidate optimization model.

[0352] Specifically, the sixth agent can perform accuracy evaluation on candidate optimization models on a standard evaluation dataset to obtain accuracy metrics, including:

[0353] (1) Comprehensive evaluation of global accuracy

[0354] The overall accuracy index of the optimized model on the complete standard evaluation dataset is calculated and compared with the accuracy baseline in the model analysis report to quantify the overall optimization effect.

[0355] Core accuracy metrics calculation: For object detection tasks, calculate and record the mean average precision (mAP), including mAP@0.5 and mAP@0.5:0.95; for semantic segmentation tasks, calculate and record the mean intersection over union (mIoU).

[0356] Precision decrease Calculation: Compare the accuracy of the baseline data with the baseline data, and calculate the decrease in accuracy of the optimized model. Taking object detection as an example, the calculation formula is:

[0357] ;

[0358] in, The mean of the baseline accuracy of the original model on the standard evaluation dataset. This represents the mean accuracy of the optimized model on the same dataset.

[0359] Optimization effect quantification: Calculate the compression ratio of model parameters and the percentage reduction in FLOPs The compression and acceleration effects of the quantization model are calculated using the following formulas:

[0360] ;

[0361] ;

[0362] in, The total number of parameters in the original model recorded in the model analysis report. The total number of parameters in the optimized model. The number of floating-point operations recorded in the model analysis report for the original model. This represents the total number of floating-point operations in the optimized model.

[0363] (2) Fine-grained accuracy dimensional assessment

[0364] We conduct a detailed analysis of the model's accuracy from multiple dimensions, identify concentrated areas of accuracy degradation, and ensure that the performance of key categories and edge scenarios meets safety requirements.

[0365] Divide by target category: Calculate the accuracy index (AP or IoU) for each category and compare it with the baseline to identify categories with a significant decrease in accuracy;

[0366] Target scale division: Strictly follow the COCO dataset standard to divide target scales. Targets smaller than 32×32 pixels are small targets, targets between 32×32 and 96×96 pixels are medium targets, and targets larger than 96×96 pixels are large targets. Detection or segmentation accuracy of targets at different scales is statistically analyzed.

[0367] Classification by scenario conditions: Based on the annotation information or metadata in the dataset, the dataset is divided into typical scenarios such as day / night, sunny / rainy, and urban roads / highways, and the model accuracy under each scenario is statistically analyzed.

[0368] Key Category Accuracy Verification: This involves closely monitoring accuracy changes in key safety-related categories such as pedestrians, cyclists, and motor vehicles. If the accuracy decrease in any key category exceeds a preset threshold... (The default value is 2%), which will automatically trigger the optimization strategy adjustment process.

[0369] (3) Numerical consistency verification

[0370] Randomly select 100-200 evaluation samples and compare the numerical differences in the model output before and after optimization to verify the numerical stability of the model optimization process:

[0371] For object detection models: compare the deviation of the detection box coordinates and the deviation of the classification confidence score before and after optimization, under the same input;

[0372] For semantic segmentation models: compare the pixel-level differences in the segmentation masks output by the model before and after optimization, under the same input;

[0373] Deviation threshold requirements: Ensure that the maximum bounding box coordinate deviation of all sampled samples does not exceed 1 pixel, the classification confidence deviation does not exceed 0.05, and confirm that the optimization operation does not introduce significant numerical errors.

[0374] Specifically, the sixth agent can perform full-process actual inference performance testing on the target hardware platform. If the target hardware platform is temporarily unavailable, performance prediction will be completed in a simulation environment based on the hardware performance model, including:

[0375] (1) Inference Delay Test

[0376] The measurement model's complete inference latency on the target hardware platform (or simulation environment) covers the entire process from data preprocessing, model inference, to post-processing, leaving no step out. During the test, the P50, P95, and P99 quantiles of the single inference latency are recorded. The fluctuation of inference latency is analyzed by using the quantile values ​​to evaluate the stability of the model's inference latency, ensuring that latency fluctuations are within an acceptable range and meet the real-time requirements of autonomous driving scenarios.

[0377] (2) Throughput test

[0378] For different batch sizes (covering common batch configurations such as 1, 2, 4, 8, and 16), the inference throughput of the model was measured (in terms of frames per second). Based on the test results, a batch size-throughput relationship curve was plotted to visually present the trend of throughput change with batch size, determine the optimal batch size when the throughput reaches its maximum value, and provide an accurate reference for batch parameter configuration in actual deployment.

[0379] (3) Resource usage test

[0380] Real-time monitoring and recording of core resource usage during model inference:

[0381] Memory / GPU memory usage: Measures the peak memory / GPU memory usage during the inference process, covering all usage aspects such as model weight storage, intermediate activation value reading and writing, and inference framework overhead.

[0382] Power consumption test: Measure the average power consumption during the model inference process. For in-vehicle autonomous driving hardware platforms, the focus is on verifying whether the power consumption meets the platform's heat dissipation capacity and energy consumption constraints, so as to avoid affecting the normal operation of the platform due to excessive power consumption.

[0383] (4) Long-term stability test

[0384] Execute continuous inference tests on the model. The default test duration is 1 hour, during which inference performance changes are monitored in real time.

[0385] Memory stability: Monitor changes in memory / video memory usage to confirm there is no memory leak (i.e., video memory / memory usage does not continuously increase or fail to release).

[0386] Performance stability: Monitor changes in inference latency and throughput to ensure no performance degradation (i.e., inference latency does not gradually increase and throughput does not gradually decrease), and guarantee the reliability of the model in long-term continuous operation scenarios.

[0387] S208. Based on the seventh agent, determine whether the preset iteration termination condition is met.

[0388] For example, the seventh intelligent agent is a functional module specifically responsible for global scheduling and decision-making, serving as the central control core of the entire system. The preset iteration termination condition refers to a pre-defined rule used to determine whether to stop iterative optimization. For example, the iteration termination condition can be any of the following: reaching the preset maximum number of iterations, the existence of at least one candidate model satisfying all preset optimization objectives, no candidate model with better performance being generated in multiple consecutive iterations, or the optimization effect having converged, etc. The specific condition can be set according to actual needs, and this embodiment of the application does not limit it.

[0389] In one example, the seventh agent receives the evaluation results of all candidate optimization models returned by the sixth agent and determines whether any of the preset iteration termination conditions are met. If met, step S209 is executed; otherwise, step S210 is executed.

[0390] Specifically, after each round of verification and evaluation is completed, the seventh agent determines the termination condition based on the evaluation results output by the sixth agent. If any of the following conditions are met, the iterative optimization process is automatically terminated, and step S210 is executed to output the final model:

[0391] Both accuracy and performance meet the requirements: the model's accuracy loss is within an acceptable range, and its inference performance meets deployment requirements. The criteria for this determination are as follows: At the same time, it requires that the inference latency be lower than the preset target upper limit, or the throughput be higher than the preset target lower limit.

[0392] Maximum number of iterations reached: The preset maximum number of iterations has been reached. The default value is 5 iterations to prevent meaningless iterations from causing continuous consumption of computing resources.

[0393] Convergence criterion: The improvement in the overall optimization effect of the two most recent consecutive iterations is less than the minimum improvement threshold. .in: This represents the overall performance improvement between two adjacent iterations. The minimum effective improvement threshold is preset, with a default value of 0.1%, meaning that further iterations will not bring significant performance improvements.

[0394] S209. Based on the seventh agent, determine the final model from the candidate optimization models according to the evaluation results of the candidate optimization models.

[0395] For example, the final model refers to the deployment model that achieves the optimal trade-off in multiple dimensions such as accuracy, inference latency, and memory usage, while satisfying all preset optimization objectives.

[0396] In one example, a three-dimensional multi-objective optimization space is constructed based on the seventh agent with accuracy, inference latency, and memory usage as three optimization objectives. All candidate optimization models are mapped to coordinate points in the space. The set of models that meet the Pareto optimality conditions is identified to form the Pareto front. The comprehensive score of each model on the Pareto front is calculated according to the user-defined preference weights. Finally, the model with the highest comprehensive score is selected as the final deployment model.

[0397] Specifically, based on the seventh agent, a three-dimensional coordinate space is constructed using the accuracy, inference latency, and memory usage in the evaluation results. Each candidate optimization model is then mapped to a coordinate point in the three-dimensional coordinate space based on its evaluation results. Based on the seventh agent, all coordinate points satisfying the Pareto optimality condition are designated as the Pareto front. Based on the seventh agent, for each candidate optimization model on the Pareto front, its score is determined based on its evaluation results. The candidate optimization model with the highest score is then selected as the final model.

[0398] Accuracy is used to characterize the candidate optimization model's ability to identify the target task; inference latency is used to characterize the single inference time of the candidate optimization model on the target hardware platform; and memory usage is used to characterize the storage resources required for the candidate optimization model to run. The three-dimensional coordinate space can be composed of three independent evaluation dimensions, each corresponding to accuracy, inference latency, and memory usage, respectively, thus enabling a unified expression and comparison of the multi-objective performance of different candidate optimization models. The Pareto optimality condition is used to determine whether a candidate optimization model is superior to other models in at least one evaluation dimension without significantly deteriorating in the remaining dimensions. The resulting Pareto front can retain a set of candidate models with a good balance across multiple objectives. The score can be calculated by the seventh agent based on the accuracy weight, latency weight, and memory weight, and the importance of each evaluation indicator can be normalized according to different deployment scenarios to ensure that the selected final model meets the actual deployment requirements.

[0399] For example, after receiving the evaluation results output by the sixth agent, the seventh agent uses the accuracy value as the first coordinate of the coordinate point, the inference latency value as the second coordinate, and the memory usage value as the third coordinate, and uses these to locate the candidate optimization model in three-dimensional space. Subsequently, the seventh agent performs a non-dominated sorting of all coordinate points. When a model corresponding to a certain coordinate point is not simultaneously completely outperformed by other models in terms of higher accuracy, lower latency, and smaller memory usage, the coordinate point is determined to satisfy the Pareto optimality condition. For models located on the Pareto front, the seventh agent further calculates a score according to a preset scoring function. The scoring function can set high accuracy as a positive contribution and high latency and high memory usage as negative contributions. After processing with unified dimensions, a comparable comprehensive score is obtained, and the model with the highest score is selected as the final model. In practical applications, this scoring function can also use different normalization coefficients and weight configurations; this embodiment does not limit this.

[0400] For example, the seventh agent can select the Pareto optimal model based on multi-objective optimization theory. The specific steps are as follows:

[0401] (1) Constructing a multi-objective optimization space

[0402] A three-dimensional optimization space is constructed with accuracy (mAP), inference latency (Latency), and memory usage (Memory) as the three optimization objectives. For each candidate optimization model i, the coordinates of the model in the three-dimensional optimization space are calculated. ),in, Let be the mean accuracy of the i-th candidate model; Let be the inference delay value of the i-th candidate model; Let be the memory usage value of the i-th candidate model.

[0403] (2) Identifying the Pareto Front

[0404] A multi-objective trade-off analysis is performed on all candidate optimization models to select the Pareto optimal model set, which constitutes the Pareto front. Models on the Pareto front must meet the following core conditions: no other candidate model exists; the model is not inferior to the Pareto model in any of the three optimization objectives; and the model is strictly superior to the Pareto model in at least one objective.

[0405] The Pareto front is essentially the boundary of the optimal trade-off between three objectives: accuracy, latency, and memory. Each model on the front represents an optimal state where "it is impossible to improve all objectives at the same time" (e.g., improving accuracy may lead to increased latency, while reducing memory usage may lead to decreased accuracy).

[0406] (3) Calculate the comprehensive score of candidate models and select the optimal model.

[0407] Based on the user-defined preference weights, a score is calculated for each candidate model on the Pareto front, and the final recommended model is selected, as follows:

[0408] The formula for the scoring function:

[0409] ;

[0410] in, The score of the i-th candidate model (the higher the score, the better the overall performance of the model). , , The user sets three optimization objectives with corresponding preference weights (default values ​​of 0.4, 0.4, and 0.2 respectively), with higher weights for accuracy and latency, which aligns with the core needs of autonomous driving scenarios. Let be the mean accuracy of the i-th candidate model; The highest average accuracy among all candidate models (used for normalization to eliminate the influence of dimensions). The minimum inference delay among all candidate models (used for normalization); The inference delay for the i-th candidate model; The minimum memory footprint among all candidate models (used for normalization); Let be the memory usage of the i-th candidate model.

[0411] Furthermore, based on the overall score The model with the highest score is selected as the final model, which is the optimal solution after weighing accuracy, latency, and memory.

[0412] (4) Output Pareto Front Visualization Report

[0413] The Pareto front and the coordinates of all candidate models are generated into a visualization report in the form of a 3D chart or a 2D projection chart. The report clearly shows the position of each candidate model in the three-dimensional space of "accuracy-latency-memory" and the Pareto front curve. At the same time, the key performance parameters of each model are marked to help users intuitively understand the trade-off relationship between different optimization schemes and assist users in further confirming or adjusting their preference weights.

[0414] Through the above methods, candidate optimization models can be screened within a unified multi-dimensional evaluation space, avoiding the imbalances in accuracy, latency, and resource consumption caused by selecting models based on a single metric. The introduction of the Pareto front first eliminates obviously degraded candidate models, then performs fine-tuning on models with high balance, thereby improving the stability and deployment adaptability of the final model selection. Thus, the determined final model can balance recognition accuracy, real-time performance, and resource constraints in autonomous driving perception deployment scenarios, which is beneficial for improving the overall inference performance on the target hardware platform.

[0415] Through the aforementioned steps, the sixth agent first conducts a unified evaluation of multiple candidate optimization models, ensuring that different models are comparable under the same evaluation criteria. The seventh agent then makes a decision when the iteration converges or the stopping condition is met, thereby avoiding premature selection or repeated trial and error. By assigning evaluation and decision-making to different agents, the model selection process can be made more consistent and traceable, and the fluctuations caused by human intervention can be reduced.

[0416] By adopting this implementation method, candidate optimization models can be screened automatically and uniformly, thereby improving the efficiency of determining the deployment of the target hardware platform, reducing the number of repeated verifications, and improving the stability of the final model in terms of the balance between accuracy and performance, thus better meeting the deployment requirements of autonomous driving perception models.

[0417] S210. Based on the seventh agent, a new target optimization strategy is generated according to the evaluation results of the candidate optimization model, and sent to the second agent so that the second agent can perform iterative optimization processing on the target deep learning model according to the new target optimization strategy.

[0418] For example, the new target optimization strategy describes the direction of the next round of adjustments to the target deep learning model. The adjustment direction may correspond to structural compression, quantization bit width adjustment, distillation weight update, operator fusion relationship adjustment, or hardware adaptation parameter correction, etc. After receiving the new target optimization strategy, the second agent can call the fourth and fifth agents again to re-optimize the target deep learning model and generate candidate optimized models again so that the subsequent evaluation results can be compared with the results of the previous round.

[0419] In one example, based on the evaluation results of all candidate models in the previous round analyzed by the seventh agent, the reasons for excessive loss of accuracy or failure to meet performance standards are identified. The pruning ratio, quantization scheme or distillation parameters are adjusted accordingly to generate a new target optimization strategy and send it to the fourth agent. The fourth agent re-executes the optimization operation based on the original target deep learning model and starts a new round of iteration.

[0420] For example, if the model accuracy loss exceeds the acceptable range but the maximum number of iterations has not been reached, the closed-loop scheduling agent automatically traces the cause of accuracy degradation based on the fine-grained accuracy evaluation results and adjusts the optimization strategy accordingly: if the accuracy loss is mainly caused by structured pruning operations (determined by comparing the model's accuracy changes before and after pruning), the overall pruning ratio is reduced by 20%~30%, or pruning is rolled back to the pruning-sensitive layer where the accuracy drop is most concentrated; if the accuracy loss is mainly caused by quantization operations (determined by comparing the model's accuracy changes before and after quantization), the quantization-sensitive layer is rolled back from INT8 accuracy to FP16 accuracy, or Quantization-Aware Training (QAT), which has a stronger accuracy recovery capability, is switched on; if a single optimization strategy adjustment still cannot restore the model accuracy, a knowledge distillation optimization step is added, or the feature distillation loss weight is increased. This strengthens the transmission of intermediate layer feature supervision signals from the teacher model to the student model, thereby improving the accuracy of the compressed model.

[0421] Through this method, the seventh agent can continuously drive the second agent to perform closed-loop iterative optimization of the target deep learning model until the termination condition is met. This allows the optimization direction to be corrected in real time based on the evaluation results, thus avoiding the local optimum problem caused by a one-time optimization. This approach enables the candidate optimization model to gradually approach the deployment requirements in multiple iterations and improves the balance between accuracy, latency, and resource consumption, thereby enhancing the deployment stability and optimization convergence efficiency of the autonomous driving perception model on heterogeneous hardware platforms.

[0422] Optionally, the seventh agent performs structured recording and persistent storage of the entire optimization process for each iteration, generating an optimization report. This report may include: the iteration number, the combination of optimization strategies used in this iteration and all parameter configurations, the core performance indicators of the optimized model (accuracy, inference latency, memory usage, parameter compression ratio, FLOPs reduction ratio, etc.), comparison data with the original baseline and the previous iteration, and the decision basis for strategy adjustments in this iteration. This optimization report can be stored in a JSON structured format for subsequent optimization effect review, fault tracing, and reuse and transfer of historical optimization experience.

[0423] S211. Based on the third intelligent agent, generate the model information of the final model.

[0424] The model information includes the final model file, deployment configuration file, and optimization report.

[0425] The third agent can be an intelligent analysis unit used for final determination and result encapsulation. After obtaining the final model, it can uniformly organize the model ontology and the information required for its deployment. The model file is used to characterize the parameters, computation graph, and network structure of the final model. It can usually be saved in a file format adapted to the target inference engine (such as TensorRT Engine, OpenVINO IR, etc.) for easy loading, transmission, and retrieval later. It can be directly used for inference deployment without additional conversion steps. The deployment configuration file (JSON format) is used to characterize the running configuration of the final model on the target hardware platform. It includes the model file path, inference engine type and version, model input and output specifications (data type, shape, preprocessing parameters), recommended batch size, inference thread number configuration, warm-up strategy (number of warm-ups for the first inference), etc., so that the model can complete initialization and inference execution according to the predetermined deployment environment. The optimization report (Markdown format) is used to characterize the final model's processing results during the optimization process. The report may include a complete comparative analysis of the original and optimized models (multi-dimensional comparison of accuracy, latency, memory usage, number of parameters, FLOPs, etc.), quantitative analysis of the effects of each optimization stage, detailed records of optimization strategy combinations, performance trends during iterations, deployment considerations, and known limitations, to facilitate version data retention and subsequent iteration analysis. Output the above model information to the specified directory to complete the automatic model optimization process.

[0426] When generating model information, the third-party agent can, after confirming that the final model meets the deployment conditions, call the storage module to export the model file of the final model and simultaneously generate the deployment configuration file and optimization report corresponding to the model. Then, the three are combined into unified model information. This model information can be written to the version management system according to a preset directory structure, or it can be transmitted to the deployment server or edge inference node via the network to support model release, rollback, and reuse.

[0427] Through the above methods, once the final model is determined, complete model information can be generated synchronously, ensuring a consistent encapsulation relationship between the model itself, deployment parameters, and optimization results. This improves deployment and delivery efficiency, reduces the cost of manual organization and repetitive configuration, and enhances the convenience of model version management, cross-platform migration, and subsequent issue tracking. This, in turn, improves the stability and engineering maintainability of autonomous driving perception models in heterogeneous hardware environments.

[0428] The deep learning model optimization method provided in this application involves a first agent performing multi-format computation graph parsing, automatic architecture type identification, and multi-dimensional quantitative analysis of the target deep learning model, generating a standardized structured model analysis report. A fourth agent, based on the model analysis report and user-preset optimization objectives, automatically generates multiple differentiated target optimization strategies according to preset decision logic and independently executes optimization operations. A fifth agent performs hardware-aware compilation optimization on each optimized model to generate candidate optimization models adapted to the target hardware platform. A sixth agent performs comprehensive accuracy evaluation and inference performance testing on all candidate models. A seventh agent determines whether the iteration termination condition is met. If the condition is met, the globally optimal final deployment model is selected from all candidate models based on multi-objective optimization theory. If the condition is not met, a new optimization strategy is generated based on the evaluation results to drive the system to perform closed-loop iterative optimization. Finally, a third agent generates a complete deployment delivery file. This approach automates the entire process of deep learning model deployment from raw input to target hardware, completely eliminating reliance on human experience, significantly shortening the model optimization cycle, and improving optimization efficiency. Simultaneously, standardized analysis processes and data-driven decision-making logic ensure the stability and consistency of optimization quality, avoiding significant discrepancies in optimization results among different engineers. Through a closed-loop iterative mechanism and multi-objective Pareto optimal selection, it can simultaneously address multiple conflicting optimization objectives such as accuracy, inference latency, and memory usage, finding the globally optimal trade-off. Furthermore, it automatically adapts the model to different hardware platforms, eliminating the need for re-optimization for different hardware, further reducing development costs, and effectively meeting the urgent needs of rapid model iteration and large-scale industrial deployment in fields such as autonomous driving.

[0429] In some embodiments, the above-described model optimization system initializes the following agents upon startup and establishes communication protocols, message formats, and scheduling mechanisms among the agents.

[0430] (1) Model Analysis Agent (First Agent): Responsible for comprehensive automatic analysis of the trained autonomous driving data annotation model (target deep learning model). The model analysis agent receives the trained model (supports multiple model formats, including PyTorch model files, ONNX model files, TensorFlow SavedModel, etc.), automatically performs analysis tasks such as model structure parsing, layer-by-layer parameter statistics, computational complexity analysis, sparsity detection, bottleneck location and accuracy-efficiency baseline establishment, and outputs a structured model analysis report.

[0431] The model analysis agent incorporates parsing rules for various autonomous driving perception models, covering the identification logic of mainstream model architectures and their key components. For single-stage detection models, it identifies components such as the backbone network, feature pyramid network (FPN / PANet / BiFPN), and detection head; for two-stage detection models, it identifies the Region Candidate Network (RPN) and RoI Pooling / Align modules; for Transformer-based models, it identifies components such as the encoder-decoder structure, multi-head attention layer, feedforward network layer, and position encoding; for 3D point cloud detection models, it identifies components such as the voxelization module, sparse convolutional layer, and bird's-eye view feature extraction network; and for multimodal fusion models, it identifies feature extraction branches for each modality and cross-modal fusion modules.

[0432] For each model architecture, the model analysis agent maintains a characteristic architecture analysis template, which can identify key components in the model and their connections, and select appropriate importance assessment methods and optimization strategies according to the component type.

[0433] (2) Optimization Strategy Agent (Fourth Agent): Responsible for automatically selecting and executing optimization strategy combinations based on model analysis reports. The optimization strategy agent incorporates various optimization techniques such as structured pruning, post-training quantization (PTQ), quantization-aware training (QAT), knowledge distillation, and operator fusion. It can automatically select the optimal optimization strategy combination and generate an optimization execution plan based on model structure characteristics, sparsity distribution, bottleneck location, target hardware characteristics, and performance requirements. The optimization strategy agent maintains a knowledge base on the applicability of each optimization technique to different model architectures and hardware platforms, which can avoid using inappropriate optimization methods for specific structures (e.g., avoiding over-pruning for depthwise separable convolutions, and avoiding INT8 quantization for Softmax layers in attention mechanisms).

[0434] (3) Hardware Adaptation Agent (Fifth Agent): Responsible for automatic detection of target hardware platforms, hardware perception compilation optimization, and cross-platform deployment adaptation. The hardware adaptation agent has built-in hardware feature libraries of various autonomous driving computing platforms, maintains hardware description information such as computing power, storage level, supported operator set, supported numerical precision, and inference framework interface of each platform, and can automatically complete model format conversion, operator mapping, compilation optimization, and inference performance prediction for the target platform.

[0435] (4) Verification and Evaluation Agent (Sixth Agent): Responsible for the accuracy evaluation of the optimized model, inference performance verification, Pareto optimal model selection, and regression testing. The verification and evaluation agent has a built-in evaluation index system dedicated to autonomous driving scenarios, covering various task types such as object detection (mAP@different IoU thresholds, AP of various categories, AR, etc.), semantic segmentation (mIoU, pixel accuracy, etc.), and 3D object detection (3D mAP, BEV mAP, etc.). It supports multi-dimensional fine-grained analysis and automatically selects the Pareto optimal model based on multi-objective optimization theory.

[0436] (5) Closed-loop scheduling agent (seventh agent): responsible for coordinating the workflow and execution order of the above four agents, iterative optimization decisions, termination condition judgment, model version management and optimization trajectory recording. As the central scheduler of the entire platform, the closed-loop scheduling agent controls the iterative rhythm of "model analysis → strategy optimization → hardware adaptation → verification and evaluation", maintains the global optimization state (current iteration round, historical best index, optimization convergence trend, etc.), and automatically outputs the optimal deployment model and deployment configuration when the termination condition is met.

[0437] During the initialization phase, each agent performs the following preparatory work: the model analysis agent loads the model parsing rule base and various architecture analysis templates; the optimization strategy agent loads the optimization technical parameter library and historical optimization experience data; the hardware adaptation agent loads the hardware feature description library and inference framework configuration templates; the verification and evaluation agent loads the evaluation metric definition and evaluation dataset configuration; and the closed-loop scheduling agent initializes the global state manager and message routing table. The model optimization system supports specifying optimization objectives (accuracy lower limit, latency upper limit, target platform, etc.) and constraints (maximum number of iterations, maximum optimization time, etc.) through configuration files or command-line parameters.

[0438] In some embodiments, Figure 4 A flowchart illustrating an optimization method for a deep learning model provided in this application is shown below. Figure 4 As shown, the trained target deep learning model is input into the model optimization system. First, the agent initialization and configuration steps are executed. Then, the first agent is started to complete the automatic model analysis and generate a model analysis report. Based on the analysis report, the optimization strategy selection and execution stage is entered. The corresponding optimization operations are executed sequentially through three decision branches: if a pruning strategy is included, structured pruning and fine-tuning are performed; if a quantization strategy is included, post-training quantization (PTQ) or quantization-aware training (QAT) is performed; if a distillation strategy is included, knowledge distillation training is performed. After all optimization operations are completed, operator fusion is performed uniformly. The optimized model is input into the fifth agent to complete hardware adaptation and compilation optimization, generating multiple candidate optimization models. The candidate optimization models are handed over to the sixth agent to perform verification evaluation and Pareto optimization to obtain the evaluation results of each candidate model. Finally, the seventh agent performs the iteration termination condition judgment: if the termination condition is met, the final model and deployment configuration are output; if the termination condition is not met, for scenarios where the accuracy loss exceeds the standard, the pruning ratio is reduced or the quantization is rolled back; for scenarios where the performance does not meet the standard, the optimization parameters are adjusted to generate a new target optimization strategy and return to the optimization strategy selection and execution stage, thus forming a fully automatic closed-loop iterative optimization mechanism.

[0439] In some embodiments, a manual confirmation step can be introduced into the termination decision-making process of the seventh agent. After completing one round of optimization iterations, the model optimization system displays the optimization results (accuracy comparison, performance comparison, strategy effect analysis, etc.) and suggested next steps (continue iteration and adjust the strategy, accept the current results, or change the optimization target) to the user for confirmation before continuing execution. This approach retains the flexibility of manual supervision, allowing users to adjust the optimization direction according to actual business needs. It is suitable for scenarios with extremely high requirements for model accuracy, where manual verification of the optimization process is required when using the model optimization system for the first time, or where the optimization target needs to be dynamically adjusted during the process.

[0440] In some embodiments, Figure 5A schematic diagram of an optimization strategy selection decision tree provided in this application is shown below. Figure 5 As shown, the fourth agent can automatically generate the entire decision-making logic for optimizing the execution plan based on core indicators such as channel importance distribution, quantization sensitivity grading, and precision margin in the model analysis report. The fourth agent receives the model analysis report, first determining if the model has low-importance channels; if so, it activates a structured pruning strategy. Next, it determines if the model's precision margin is sufficient; if so, it directly activates post-training quantization (PTQ); if insufficient, it further determines if the PTQ precision is acceptable; if acceptable, it activates PTQ; otherwise, it activates quantization-aware training (QAT). Then, it determines if a single compression strategy is sufficient to meet the optimization objective; if insufficient, it additionally activates a knowledge distillation strategy. Afterward, it determines if there are fusionable operator patterns in the model's computational graph; if so, it performs operator fusion operations. Finally, it outputs a complete optimized execution plan.

[0441] In summary, the embodiments of this application have the following technical advantages:

[0442] 1. Automated Hyperparameter Tuning Significantly Shortens Optimization Cycle: This application constructs an automated optimization platform integrating multiple intelligent agent capability modules, achieving automated search and tuning of model optimization hyperparameters. Compared to the days to weeks required for manual trial-and-error adjustments of parameters such as pruning ratios, quantization accuracy, and distillation weights, this application can automatically complete multiple rounds of optimization iterations (including model analysis, automatic selection of policy parameters, optimization execution, hardware adaptation, and verification evaluation) within hours, automatically searching for the optimal parameter combination that achieves the best balance between accuracy and performance, improving optimization efficiency by more than 10 times. Unlike methods that can only improve model accuracy through data optimization during the training phase, this application uses intelligent agents to collaboratively and automatically tune model optimization hyperparameters, covering the automatic search of the entire optimization parameter space, including pruning, quantization, distillation, and hardware adaptation.

[0443] 2. Intelligent strategy selection eliminates reliance on human experience, possessing determinism and reproducibility: The optimization strategy agent automatically selects the optimal combination of optimization strategies based on quantitative data (channel comprehensive importance score, quantitative sensitivity classification, bottleneck distribution, accuracy margin, etc.) from the model analysis report, eliminating the reliance on the personal experience of algorithm engineers in traditional methods. Unlike methods that rely on LLM-generated results, which suffer from randomness and non-reproducibility, the strategy selection in this application is based on deterministic numerical analysis and rule-driven decision-making. Under the same model input and optimization objective, it produces consistent optimization results, ensuring the reproducibility and reliability of the optimization process, which is crucial for safety-critical applications such as autonomous driving.

[0444] 3. Multi-strategy collaborative automatic hyperparameter search, overcoming the performance bottleneck of single strategies: This application organically integrates multiple optimization techniques such as structured pruning, quantization (PTQ / QAT), knowledge distillation, and operator fusion into a closed-loop automatic tuning framework. The hyperparameters of each strategy (pruning ratio, quantization accuracy, distillation loss weight, etc.) are automatically determined and dynamically adjusted by the agent based on model analysis results. For example, knowledge distillation is automatically introduced to assist in accuracy recovery during the fine-tuning stage after pruning, and distillation loss weights are automatically configured simultaneously during quantization-aware training. The hyperparameters of each optimization strategy are continuously optimized through closed-loop feedback. Compared to manually setting optimization parameters one by one, the multi-strategy collaborative automatic tuning of this application can search for better hyperparameter combinations under the same accuracy constraints, achieving greater model optimization effects. Unlike automatic search and tuning that does not involve model optimization hyperparameters, this application provides a complete solution for the automatic search of the model optimization parameter space.

[0445] 4. Hardware-aware optimization enables automatic cross-platform adaptation: The hardware-adaptive intelligent agent incorporates feature descriptions of various autonomous driving computing platforms, enabling it to automatically adjust optimization strategies and model format conversion processes based on the target hardware's computing architecture, operator support, and numerical accuracy. A single model optimization can automatically adapt to multiple target platforms, eliminating the need to relearn toolchains and manually adjust optimization parameters for each hardware, significantly reducing the workload and error probability of cross-platform deployment.

[0446] 5. Closed-loop iteration ensures an optimal balance between accuracy and performance: The closed-loop scheduling agent, through an "analysis-optimization-verification-decision" closed-loop iterative mechanism, automatically adjusts the optimization strategy and re-executes when accuracy loss exceeds expectations, until the optimal model that satisfies both accuracy constraints and performance goals is found. The Pareto optimal model selection mechanism ensures that a globally optimal balance point is found in the multi-objective space (accuracy, latency, memory), rather than a locally acceptable solution obtained through manual trial and error. Compared to the linear "training → analysis → suggestion" process and the linear execution process, the closed-loop iterative mechanism of this application has stronger adaptability and convergence guarantee.

[0447] 6. Systematized accumulation and reuse of optimization experience: The closed-loop scheduling agent maintains a complete record of optimization trajectories, including structured data such as strategy combinations, parameter configurations, and effect quantifications for each iteration. When optimizing models with the same or similar architectures, the system can refer to historical optimization experience to accelerate strategy selection (such as directly reusing the historically optimal initial pruning ratio and quantization configuration), avoiding repeated trial and error and continuously improving optimization efficiency.

[0448] 7. Ensure key performance in autonomous driving scenarios: The verification and evaluation agent supports fine-grained accuracy analysis across multiple dimensions, including target category, target scale, and scenario conditions, ensuring that model optimization does not result in unacceptable accuracy degradation in key categories (such as safety-related categories like pedestrians and vehicles) and key scenarios (such as edge scenarios like nighttime and severe weather). Stricter accuracy degradation thresholds are set for key categories to ensure the safety of the autonomous driving system in edge scenarios.

[0449] 8. Adaptive quantization sensitivity to reduce the risk of quantization accuracy loss: This application automatically identifies highly sensitive layers through quantization sensitivity evaluation and adopts a hybrid precision quantization strategy, avoiding the accuracy collapse caused by one-size-fits-all quantization. For specific quantization-sensitive structures (such as depthwise separable convolutions, attention temperature parameter layers, feature pyramid upsampling layers, etc.), FP16 precision is automatically maintained, minimizing the risk of accuracy loss while ensuring the overall quantization effect.

[0450] Figure 6 A schematic diagram of the structure of the optimization device for the deep learning model provided in this application is shown below. Figure 6 As shown, the deep learning model optimization device 300 provided in this embodiment includes:

[0451] Analysis module 301 is used to analyze and process the received target deep learning model based on the first intelligent agent and generate a model analysis report of the target deep learning model.

[0452] The optimization module 302 is used to optimize the target deep learning model based on the second intelligent agent, according to the model analysis report and the preset optimization target, to obtain multiple candidate optimized models adapted to the target hardware platform.

[0453] The determination module 303 is used to determine the final model adapted to the deployment of the target hardware platform from the candidate optimization models based on a third intelligent agent.

[0454] In one possible implementation, the second intelligent agent includes a fourth intelligent agent and a fifth intelligent agent, and the optimization module 302 is used for:

[0455] Based on the fourth intelligent agent, multiple target optimization strategies for the target deep learning model are determined according to the model analysis report, optimization objectives, and preset decision logic; among them, the target optimization strategies include at least one of the following strategies: structured pruning, quantization, knowledge distillation, and operator fusion.

[0456] Based on the fourth intelligent agent, the target deep learning model is optimized according to the target optimization strategy to obtain the optimized model;

[0457] Based on the fifth intelligent agent, the optimized model is hardware adapted according to the target hardware platform to obtain candidate optimized models.

[0458] In one possible implementation, the optimization module 302 is configured to:

[0459] Based on the fifth intelligent agent, the platform information of the target hardware platform is determined according to the target hardware platform and the preset hardware feature description library;

[0460] Based on the fifth intelligent agent, the optimized model is compiled and optimized according to the platform information of the target hardware platform to obtain a candidate optimized model; wherein, the compilation and optimization process includes at least one of the following operations: model format conversion operation, operator compatibility verification operation, inference performance prediction operation, and inference parameter determination operation.

[0461] In one possible implementation, the optimization module 302 is configured to:

[0462] Based on the fourth intelligent agent, the target optimization strategy is sent to the third intelligent agent;

[0463] Based on a third-party intelligent agent, the target optimization strategy is reviewed and the review result is returned;

[0464] Based on the fourth intelligent agent, the target deep learning model is optimized according to the review results to obtain the optimized model.

[0465] In one possible implementation, the analysis module 301 is used for:

[0466] Based on the first intelligent agent, the structure of the target deep learning model is parsed according to the parsing method corresponding to the format of the target deep learning model, and the computation graph of the target deep learning model is obtained.

[0467] Based on the first intelligent agent, the architecture type of the target deep learning model is determined according to the combination pattern of operators in the computation graph;

[0468] Based on the first intelligent agent, according to the architecture analysis template matched by the architecture type, at least one analysis process is performed on the computation graph to generate a model analysis report. The analysis process includes layer-by-layer parameter statistical processing, sparsity detection processing, sensitivity assessment processing, bottleneck location processing, and baseline establishment processing. The model analysis report includes at least one of the following information: architecture description information, layer-by-layer parameter statistical information, channel importance score information, quantization sensitivity classification information, bottleneck location annotation information, layer-by-layer optimization suggestions, and baseline data.

[0469] In one possible implementation, the third intelligent agent includes a sixth intelligent agent and a seventh intelligent agent; the determining module 303 is configured to:

[0470] Based on the sixth intelligent agent, the candidate optimization models are evaluated to obtain the evaluation results of each candidate optimization model;

[0471] Based on the seventh agent, when the preset iteration termination condition is met, the final model is determined from the candidate optimization models according to the evaluation results of the candidate optimization models.

[0472] In one possible implementation, the determining module 303 is configured to:

[0473] Based on the seventh agent, a three-dimensional coordinate space is constructed using the accuracy, inference latency, and memory usage in the evaluation results. Based on the evaluation results of each candidate optimization model, the candidate optimization model is mapped to coordinate points in the three-dimensional coordinate space.

[0474] Based on the seventh agent, all coordinate points that satisfy the Pareto optimality condition are taken as the Pareto front.

[0475] Based on the seventh agent, for candidate optimization models on the Pareto front, the score of the candidate optimization model is determined according to the evaluation results of the candidate optimization model;

[0476] The candidate optimization model with the highest score is selected as the final model.

[0477] In one possible implementation, the determining module 303 is configured to:

[0478] Based on the seventh agent, when the iteration termination condition is not met, a new target optimization strategy is generated according to the evaluation results of the candidate optimization model and sent to the second agent, so that the second agent can perform iterative optimization processing on the target deep learning model according to the new target optimization strategy.

[0479] In one possible implementation, the determining module 303 is configured to:

[0480] Based on the third intelligent agent, the model information of the final model is generated; the model information includes the model file, deployment configuration file, and optimization report of the final model.

[0481] The deep learning model optimization device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0482] Figure 7 A schematic diagram of the structure of the electronic device provided in this application. Figure 7As shown, the electronic device 400 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the electronic device 400 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus. This electronic device can be the aforementioned server.

[0483] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0484] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0485] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0486] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0487] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0488] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0489] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0490] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0491] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0492] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0493] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0494] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0495] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0496] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0497] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for optimizing a deep learning model, characterized in that, include: Based on the first intelligent agent, the received target deep learning model is analyzed and processed to generate a model analysis report of the target deep learning model; Based on the second intelligent agent, the target deep learning model is optimized according to the model analysis report and the preset optimization target to obtain multiple candidate optimized models adapted to the target hardware platform. Based on a third intelligent agent, a final model adapted to the deployment of the target hardware platform is determined from the candidate optimization models.

2. The method according to claim 1, characterized in that, The second intelligent agent includes a fourth intelligent agent and a fifth intelligent agent. Based on the second intelligent agent, and according to the model analysis report and preset optimization objectives, the target deep learning model is optimized to obtain multiple candidate optimized models adapted to the target hardware platform, including: Based on the fourth intelligent agent, and according to the model analysis report, the optimization objective, and the preset decision logic, multiple target optimization strategies for the target deep learning model are determined; wherein, the target optimization strategies include at least one of the following strategies: structured pruning, quantization, knowledge distillation, and operator fusion. Based on the fourth intelligent agent, the target deep learning model is optimized according to the target optimization strategy to obtain the optimized model; Based on the fifth intelligent agent, and according to the target hardware platform, the optimized model is subjected to hardware adaptation processing to obtain the candidate optimized model.

3. The method according to claim 2, characterized in that, The step of performing hardware adaptation processing on the optimized model based on the fifth intelligent agent and the target hardware platform to obtain the candidate optimized model includes: Based on the fifth intelligent agent, the platform information of the target hardware platform is determined according to the target hardware platform and the preset hardware feature description library; Based on the fifth intelligent agent, according to the platform information of the target hardware platform, the optimized model is compiled and optimized to obtain the candidate optimized model; wherein, the compilation and optimization process includes at least one of the following operations: model format conversion operation, operator compatibility verification operation, inference performance prediction operation, and inference parameter determination operation.

4. The method according to claim 2, characterized in that, The optimization process, based on the fourth intelligent agent and according to the target optimization strategy, optimizes the target deep learning model to obtain the optimized model, including: Based on the fourth agent, the target optimization strategy is sent to the third agent; Based on the third intelligent agent, the target optimization strategy is reviewed, and the review result is returned; Based on the fourth intelligent agent, the target deep learning model is optimized according to the review results to obtain the optimized model.

5. The method according to claim 1, characterized in that, The step of analyzing and processing the received target deep learning model based on the first intelligent agent to generate a model analysis report of the target deep learning model includes: Based on the first intelligent agent, the structure of the target deep learning model is parsed according to the parsing method corresponding to the format of the target deep learning model, so as to obtain the computation graph of the target deep learning model. Based on the first intelligent agent, the architecture type of the target deep learning model is determined according to the combination pattern of operators in the computation graph; Based on the first intelligent agent, according to the architecture analysis template matched by the architecture type, at least one analysis process is performed on the computation graph to generate the model analysis report; wherein, the analysis process includes layer-by-layer parameter statistical processing, sparsity detection processing, sensitivity assessment processing, bottleneck location processing, and baseline establishment processing; the model analysis report includes at least one of the following information: architecture description information, layer-by-layer parameter statistical information, channel importance score information, quantization sensitivity classification information, bottleneck location annotation information, layer-by-layer optimization suggestions, and baseline data.

6. The method according to claim 1, characterized in that, The third intelligent agent includes a sixth intelligent agent and a seventh intelligent agent; the step of determining the final model adapted to the deployment of the target hardware platform from the candidate optimization models based on the third intelligent agent includes: Based on the sixth intelligent agent, the candidate optimization models are evaluated to obtain the evaluation result of each candidate optimization model; Based on the seventh agent, when the preset iteration termination condition is met, the final model is determined from the candidate optimization models according to the evaluation results of the candidate optimization models.

7. The method according to claim 6, characterized in that, Based on the seventh agent, and according to the evaluation results of the candidate optimization models, the final model is determined from the candidate optimization models, including: Based on the seventh agent, a three-dimensional coordinate space is constructed using the accuracy, inference latency, and memory usage in the evaluation results. Then, according to the evaluation results of each candidate optimization model, the candidate optimization model is mapped to coordinate points in the three-dimensional coordinate space. Based on the seventh agent, all coordinate points that satisfy the Pareto optimality condition among the coordinate points are taken as the Pareto front. Based on the seventh agent, for the candidate optimization model on the Pareto front, the score of the candidate optimization model is determined according to the evaluation result of the candidate optimization model; The candidate optimization model with the highest score is selected as the final model. Accordingly, the method further includes: Based on the seventh agent, when the iteration termination condition is not met, a new target optimization strategy is generated according to the evaluation results of the candidate optimization model and sent to the second agent, so that the second agent can perform iterative optimization processing on the target deep learning model according to the new target optimization strategy.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Based on the third intelligent agent, model information of the final model is generated; wherein, the model information includes the model file, deployment configuration file, and optimization report of the final model.

9. An optimization device for a deep learning model, characterized in that, include: The analysis module is used to analyze and process the received target deep learning model based on the first intelligent agent, and generate a model analysis report of the target deep learning model. An optimization module is used to optimize the target deep learning model based on the second intelligent agent, according to the model analysis report and preset optimization objectives, to obtain multiple candidate optimized models adapted to the target hardware platform. The determination module is used to determine, based on a third intelligent agent, the final model adapted to the deployment of the target hardware platform from the candidate optimization models.

10. An electronic device / computer-readable storage medium / computer program product, characterized in that, The electronic device includes: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-8; The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8; The computer program product includes a computer program that, when executed by a processor, is used to implement the method as described in any one of claims 1-8.

Images