A method and system for post-deployment training of convolutional neural networks
By standardizing and scripting the CNN training and deployment process, the problems of lengthy deployment process, poor reproducibility, insufficient quantization accuracy control and limited throughput in the existing technology are solved, realizing efficient and reproducible FPGA convolutional neural network deployment and optimizing end-to-end performance.
Patent Information
- Application Number
- CN202610598024.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-30
- Publication Date
- 2026-07-21
AI Technical Summary
Existing FPGA-based convolutional neural network deployment methods suffer from problems such as lengthy and unstandardized deployment processes, poor reproducibility, insufficient control over accuracy loss during quantization, limited single-board throughput, lack of standardized organizational mechanisms for multi-board clusters, and end-to-end inference performance constrained by pre- and post-processing links.
The CNN training and deployment process is broken down into steps such as environment initialization, model training, computation graph freezing, frozen graph accuracy evaluation, quantization calibration, quantization graph accuracy evaluation, DPU model compilation for the target FPGA platform, deployment package generation, and inference verification. These steps are executed in sequence through scripting to establish a complete quantization calibration and accuracy evaluation mechanism, supporting parallel execution of inference tasks under single-board or multi-board clusters.
It significantly improves deployment efficiency and engineering level, ensures the consistency and reproducibility of deployment results, unleashes potential through multi-board parallelism, provides objective basis for comparison of acceleration effects, and optimizes the pre- and post-processing links to improve end-to-end performance.
Smart Images
Figure CN122433808A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of hardware acceleration and artificial intelligence inference deployment technology, and in particular to a method and system for deploying convolutional neural networks after training. Background Technology
[0002] Convolutional neural networks (CNNs) have achieved remarkable results in computer vision tasks such as image classification, object detection, and semantic segmentation, and have been widely applied in practical scenarios such as edge computing, intelligent security, and industrial inspection. With the increasing complexity of models and the growing demands for real-time performance, how to achieve high-throughput, low-latency CNN inference deployments on edge or server sides with limited power consumption and cost has become a crucial issue for the engineering implementation of artificial intelligence.
[0003] Field-programmable gate arrays (FPGAs) are increasingly being used to accelerate CNN inference due to their reconfigurability, high parallel computing capabilities, and excellent energy efficiency. Current mainstream solutions (such as Xilinx / AMD's Vitis AI toolchain) typically adopt a basic process of "training-quantization-compilation-deployment" to convert floating-point models into fixed-point models that are adapted to deep learning processing units on FPGAs, thereby achieving hardware acceleration.
[0004] However, existing FPGA-based CNN deployment methods still have the following technical shortcomings in practical applications:
[0005] (1) The deployment process is lengthy and lacks standardization, resulting in poor reproducibility. From model training, freezing the computation graph, quantization calibration, compiling and generating the DPU executable file to board verification, multiple toolchain links and environment configurations (such as Docker containers, Conda environment, dependency library versions, etc.) are involved. Mismatches in path settings, environment variables, or versions at any link may lead to process interruption or deployment failure. Existing solutions mostly rely on manual step-by-step operation and debugging, lacking a standardized and scriptable deployment link, resulting in low deployment efficiency, high error rate, and difficulty in ensuring the consistency of deployment results across different batches.
[0006] (2) Insufficient control over accuracy loss during quantization. Quantizing a 32-bit floating-point (FP32) model into an 8-bit integer (INT8) model is a critical step in deploying to a DPU, but parameter truncation and activation value mapping during quantization may introduce accuracy loss. In existing methods, the selection of quantization calibration datasets, the rationality of calibration strategies, and the reliability of post-quantization accuracy evaluation lack systematic guarantees, making it difficult to achieve the optimal balance between accuracy and acceleration.
[0007] (3) Single-board throughput is limited, and multi-board clusters lack standardized organizational mechanisms. The inference throughput of a single FPGA accelerator card has a physical upper limit. In application scenarios that require higher throughput, although performance can be improved by increasing the number of boards to build a cluster, the existing technology lacks a standardized process for unified scheduling, task allocation and performance statistics of multi-board clusters, which results in the multi-board parallel potential not being fully realized, and it is difficult to quantitatively compare and verify the acceleration effect.
[0008] (4) End-to-end inference performance is constrained by the pre- and post-processing links. In practical applications, inference not only includes neural network computation on the DPU, but also involves video decoding, image preprocessing, result post-processing, data transmission, and other links. Existing solutions mostly focus on optimizing the DPU core itself, while ignoring the fact that the pre- and post-processing links may become end-to-end performance bottlenecks. For example, in video streaming mode, there may be frame loss and increased latency. There is a lack of analysis and optimization guidance for the entire link. Summary of the Invention
[0009] In view of the problems of the existing deployment process being lengthy and lacking standardization, poor reproducibility, insufficient control of accuracy loss in the quantization process, limited single-board throughput, lack of standardized organization mechanism for multi-board clusters, and end-to-end inference performance being constrained by the pre- and post-processing links, this invention is proposed.
[0010] Therefore, the purpose of this invention is to provide a method for deploying a convolutional neural network after training. The purpose is to decompose the CNN training and deployment process into a series of orderly steps, such as environment initialization, model training, computation graph freezing, frozen graph accuracy evaluation, quantization calibration, quantization graph accuracy evaluation, DPU model compilation for the target FPGA platform, deployment package generation, DPU activation and inference verification, and to execute each step in a series through scripting.
[0011] To address the aforementioned technical problems, this invention provides the following technical solution: a method for deploying a convolutional neural network after training, comprising the following steps:
[0012] S1: Environment Initialization
[0013] Start the deployment toolchain runtime environment, activate the specified deep learning framework environment, and load the unified configuration file; the unified configuration file shall at least contain model size parameters, training hyperparameters, and path settings;
[0014] S2: Model Training and Freezing
[0015] The training script is executed within the deep learning framework environment to train the convolutional neural network model and obtain the training state model; the freeze script is executed to freeze the computation graph of the training state model and generate a static computation graph file.
[0016] S3: Accuracy Assessment of Frozen Image
[0017] Execute the frozen graph evaluation script, load the static computation graph file, perform inference evaluation based on the validation dataset, and generate frozen graph accuracy evaluation results;
[0018] S4: Quantification and Quantification Accuracy Assessment
[0019] Execute the quantization script to perform quantization calibration on the static computation graph file using the calibration dataset, and generate a quantization model file; execute the quantization evaluation script to perform accuracy evaluation on the quantization model file, and generate quantization accuracy evaluation results;
[0020] S5: Compile to generate deployment model
[0021] The compilation script is executed to compile the quantization model file for the deep learning processing unit (DPU) of the target FPGA platform, generating an executable model file adapted to the DPU.
[0022] S6: Deployment Package Generation and DPU Activation
[0023] Execute the deployment preparation script to package the executable model file and related dependency files into a deployment package; execute the DPU activation script to mount and activate the DPU core on the target FPGA board.
[0024] S7: Inference Verification and Performance Statistics
[0025] Run the inference verification program, load the deployment package, perform batch inference based on the test dataset, and statistically analyze and output the inference throughput and inference accuracy.
[0026] As a preferred embodiment of the method for deploying a convolutional neural network after training according to the present invention, the deployment toolchain runtime environment in step S1 is the Vitis AI Docker container environment, and the deep learning framework environment is the TensorFlow environment.
[0027] As a preferred embodiment of the method for deploying a convolutional neural network after training according to the present invention, wherein: the training script in step S2 performs model training for a preset number of training rounds; the freezing script converts the training-state model into a static computation graph file using the freeze_graph tool, and the static computation graph file is in .pb format.
[0028] As a preferred embodiment of the method for deploying a convolutional neural network after training according to the present invention, the quantization calibration in step S4 is to quantize a 32-bit floating-point model into an 8-bit integer model, and the quantized model file includes a quantized computation graph file.
[0029] As a preferred embodiment of the method for deploying a convolutional neural network after training according to the present invention, in step S5 the target FPGA platform is an FPGA acceleration board equipped with a DPU, and the executable model file is a .xmodel format file.
[0030] As a preferred embodiment of the method for deploying a convolutional neural network after training according to the present invention, the inference verification program in step S7 supports configuring a single-board FPGA inference mode or a multi-board FPGA cluster inference mode, and outputs the total inference time, total inference data volume, throughput and accuracy corresponding to the current configuration after the inference is completed.
[0031] As a preferred embodiment of the system for deploying a convolutional neural network after training, as described in this invention, it includes:
[0032] A training and compilation host is used to execute the following steps in the method of any one of claims 1 to 6: environment initialization, model training and freezing, frozen graph accuracy evaluation, quantization and quantization accuracy evaluation, compilation to generate deployment model and deployment package generation.
[0033] At least one FPGA acceleration board is communicatively connected to the training and compilation host, and a deep learning processing unit (DPU) is deployed on the FPGA acceleration board for loading and executing the executable model file;
[0034] The inference verification module runs on the training and compilation host or FPGA acceleration board and is used to perform the inference verification and performance statistics steps, and output inference throughput and inference accuracy.
[0035] As a preferred embodiment of the convolutional neural network training and deployment system described in this invention, the FPGA acceleration board consists of multiple boards forming an FPGA board cluster; the inference verification module supports parallel inference and unified performance statistics in a multi-board cluster mode.
[0036] As a preferred embodiment of the deployment system for training convolutional neural networks described in this invention, the inference verification module further supports image inference mode and video inference mode; in image inference mode, the system performs inference on single or batch image files; in video inference mode, the system performs frame-level inference on real-time video streams or video files.
[0037] As a preferred embodiment of the deployment system for training convolutional neural networks according to the present invention, the training and compilation host stores intermediate product files generated by the method, including static computation graph files, quantization model files, executable model files and corresponding evaluation log files, to achieve traceability and reproducibility of the deployment process.
[0038] Compared with the prior art, the present invention has at least the following beneficial effects:
[0039] 1. This invention constructs a standardized, scriptable, and reproducible end-to-end deployment chain, which decomposes the CNN post-training deployment process into a series of orderly steps, including environment initialization, model training, computation graph freezing, frozen graph accuracy evaluation, quantization calibration, quantization graph accuracy evaluation, DPU model compilation for the target FPGA platform, deployment package generation, DPU activation, and inference verification. Each step is executed in series through scripting, and this chain effectively reduces the probability of human error, ensures the consistency and reproducibility of deployment results, and significantly improves deployment efficiency and engineering level.
[0040] 2. This invention establishes a comprehensive quantization calibration and accuracy evaluation mechanism, which sets up freeze graph accuracy evaluation and quantization graph accuracy evaluation links before and after the quantization step, respectively. The quantization parameters are calibrated by calibrating the calibration dataset and quantifiable evaluation indicators are output. At the same time, this mechanism makes the accuracy loss in the quantization process controllable and traceable, providing a reliable guarantee for achieving efficient fixed-point inference while ensuring model accuracy.
[0041] 3. This invention supports batch execution of inference tasks in single-board or multi-board cluster configurations through a unified inference verification program, and automatically calculates key indicators such as throughput (FPS) and accuracy. Furthermore, this solution effectively unleashes the parallel potential of multiple boards and provides objective and quantitative comparisons of acceleration effects under different configurations. It also provides two inference demonstration methods: image mode and video mode. Actual testing shows that image mode runs stably and smoothly, while video mode may experience frame loss and latency due to bottlenecks in the pre- and post-processing links. This conclusion provides a clear direction for mode selection in actual deployment and subsequent end-to-end optimization, enhancing the practicality and implementation guidance value of the solution. Attached Figure Description
[0042] Figure 1 This is a schematic diagram of the system architecture (training / compilation host + FPGA board + DPU + inference program) for the method of deploying a convolutional neural network after training according to the present invention;
[0043] Figure 2 This is a board-side demo flowchart of the method for deploying a convolutional neural network after training according to the present invention (KL106: path, command, exit);
[0044] Figure 3 This is a flowchart outlining the entire deployment process of the method for deploying a convolutional neural network after training, as described in this invention (docker-environment-script 0-7 setup-app).
[0045] Figure 4This is a flowchart illustrating the steps of the method for deploying a convolutional neural network after training, as described in this invention. Detailed Implementation
[0046] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0047] Reference Figures 1-4 As an embodiment of the present invention, a system for deploying a trained convolutional neural network is provided, comprising the following components:
[0048] (1) Training and compilation host: Used to run the Vitis AI Docker container environment and perform a series of steps such as model training, computation graph freezing, quantization calibration, accuracy evaluation, DPU model compilation and deployment package generation. This host can be a workstation or server configured with a GPU accelerator card, or a general-purpose computer used only for compilation.
[0049] (2) FPGA acceleration board: Connected to the training and compilation host via PCIe interface or network communication. The FPGA acceleration board is equipped with a Deep Learning Processing Unit (DPU) for loading and executing the compiled executable model file. The FPGA acceleration board can be a single board or a cluster of multiple boards (such as two Xilinx Alveo U50s).
[0050] (3) Inference verification module: It runs as a verification program (such as app_mt.py) on the host connected to the training and compilation host or FPGA acceleration board. It is used to load the deployment package and perform batch inference based on the test dataset, and to count and output the inference throughput (FPS) and inference accuracy.
[0051] A method for deploying a convolutional neural network after training includes the following steps:
[0052] S1: Environment Initialization
[0053] Start the deployment toolchain runtime environment, activate the specified deep learning framework environment, and load the unified configuration file; the unified configuration file shall at least contain model size parameters, training hyperparameters, and path settings;
[0054] The deployment toolchain runs in a Vitis AI Docker container environment, and the deep learning framework environment is a TensorFlow environment.
[0055] S2: Model Training and Freezing
[0056] The training script is executed within the deep learning framework environment to train the convolutional neural network model and obtain the training state model; the freeze script is executed to freeze the computation graph of the training state model and generate a static computation graph file.
[0057] The training script performs model training for a preset number of training rounds; the freeze script uses the freeze_graph tool to convert the trained model into a static computation graph file, which is in .pb format.
[0058] S3: Accuracy Assessment of Frozen Image
[0059] Execute the frozen graph evaluation script, load the static computation graph file, perform inference evaluation based on the validation dataset, and generate frozen graph accuracy evaluation results;
[0060] S4: Quantification and Quantification Accuracy Assessment
[0061] Execute the quantization script to perform quantization calibration on the static computation graph file using the calibration dataset, and generate a quantization model file; execute the quantization evaluation script to perform accuracy evaluation on the quantization model file, and generate quantization accuracy evaluation results;
[0062] The quantization calibration is to quantize the 32-bit floating-point model into an 8-bit integer model, and the quantization model file includes the quantized computation graph file;
[0063] S5: Compile to generate deployment model
[0064] The compilation script is executed to compile the quantization model file for the deep learning processing unit (DPU) of the target FPGA platform, generating an executable model file adapted to the DPU.
[0065] The target FPGA platform is an FPGA acceleration board equipped with a DPU, and the executable model file is a .xmodel format file;
[0066] S6: Deployment Package Generation and DPU Activation
[0067] Execute the deployment preparation script to package the executable model file and related dependency files into a deployment package; execute the DPU activation script to mount and activate the DPU core on the target FPGA board.
[0068] The inference verification program supports configuring a single-board FPGA inference mode or a multi-board FPGA cluster inference mode, and outputs the total inference time, total inference data volume, throughput and accuracy corresponding to the current configuration after the inference is completed.
[0069] S7: Inference Verification and Performance Statistics
[0070] Run the inference verification program, load the deployment package, perform batch inference based on the test dataset, and statistically analyze and output the inference throughput and inference accuracy.
[0071] Example 1: Accelerated Inference Method for CNN (MNIST) Based on Dual U50 FPGA Cluster
[0072] Hardware and software environment:
[0073] 1. FPGA Hardware: Two Xilinx Alveo U50 boards, used to build an FPGA inference cluster;
[0074] 2. Software toolchain: Vitis AI (entering the container environment via docker_run.sh);
[0075] 3. Deep learning framework environment: Activate vitis-ai-tensorflow within the container;
[0076] 4. DPU core: Mounted / activated using commands like `setup.sh DPucAHx8H`;
[0077] 5. Verification program: app_mt.py, used for batch inference and statistics of throughput (FPS) and accuracy.
[0078] Training and model generation steps (corresponding) Figure 2 , Figure 4 ):
[0079] Step S1: Start the Vitis AI Docker environment
[0080] Enter the VitisAI folder;
[0081] Execute: sudo . / docker_run.sh to enter the container environment.
[0082] Step S2: Activate the TensorFlow environment
[0083] Execute: conda activate vitis-ai-tensorflow.
[0084] Step S3: Enter the project directory and load the unified configuration.
[0085] Navigate to the specified project directory (cd to the target path);
[0086] Execute: source . / 0_setenv.sh
[0087] This step is used to load size parameters, number of training epochs, hyperparameter settings, path settings, etc., to ensure that subsequent scripts can be executed reproducibly.
[0088] Step S4: Train the CNN model
[0089] Execute: source. / 1_train.sh
[0090] The preset training rounds were 10; the final training accuracy reached approximately 98.10%.
[0091] Step S5: Freeze the model calculation graph
[0092] Execute: source . / 2_freeze.sh
[0093] The trained model is converted into a static computation graph using the freeze_graph tool; generating:
[0094] frozen_graph.pb
[0095] freeze.log
[0096] Step S6: Evaluate the accuracy of the frozen image model
[0097] Execute: source. / 3_eval_frozen_graph.sh
[0098] Output evaluation metrics and generate eval_frozen_graph.log; Material example: Evaluation accuracy is approximately 98.10%.
[0099] Step S7: Quantize the model (FP32-INT8)
[0100] Execute: source. / 4_quant.sh
[0101] The quantization parameters are calibrated using the calibration dataset, and the output is a quantization model suitable for DPU operation.
[0102] Step S8: Evaluate the accuracy of the quantification model
[0103] Execute: source . / 5_eval_quant_graph.sh
[0104] Generate the quantized version of quantize_eval_model.pb; Material example: quantization evaluation accuracy is approximately 98.12%.
[0105] Step S9: Compile and generate the DPU deployment model (xmodel)
[0106] Execute: source . / 6_compile_u50.sh
[0107] Compile the quantization model into a *.xmodel file adapted for the U50DPU.
[0108] Step S10: Preparations before deployment
[0109] Execute: source . / 7_make_target_u50.sh
[0110] Complete tasks such as loading data and organizing the target structure before deployment.
[0111] DPU activation and inference verification steps (corresponding) Figure 2 , Figure 3 ):
[0112] Step S11: Mount / Activate DPU
[0113] Executing `source / worksapce / setup / alveo / setup.sh DPUCAHX8H` puts the FPGADPU into a runnable state.
[0114] Step S12: Run the inference verification program and perform performance analysis.
[0115] Execute: / usr / bin / python3 app_mt.py
[0116] The material provides the test output for the dual-board cluster:
[0117] Total data volume: 10000
[0118] Total time: 0.5121s
[0119] FPS: 19527.13 fps
[0120] Correct: 9871; Incorrect: 129
[0121] Accuracy: 98.71%
[0122] Comparison of single-ply and double-ply effects
[0123] To verify the effect of multi-board clusters on improving inference throughput, this embodiment also conducted a comparative test between single-board U50 and dual-board U50 clusters. The results are shown in the table below:
[0124] Single-board U50 Approximately 10908.92 98.70% Dual-board U50 cluster Approximately 19527.13 98.71%
[0125] Therefore, it can be seen that the solution of the present invention can significantly improve the inference throughput of CNN by using multiple FPGAs in parallel / cluster while basically maintaining accuracy.
[0126] Example 2 (Optional): Inference Execution Method for On-Board DPU Demonstration (KL106)
[0127] Hardware preparation (materials preparation)
[0128] KL106 circuit board (custom-made)
[0129] Monitor: 2K DisplayPort (4K may flicker due to driver issues)
[0130] DP cable, USB extender (board with single USB port)
[0131] Mouse and keyboard
[0132] SD card (≥32GB)
[0133] Basic Operations and Exit Mechanism
[0134] After booting up and entering the desktop, double-click the Terminal icon on the desktop to open the terminal;
[0135] Unified Exit: Press Ese or q on the keyboard during the demonstration to exit the program.
[0136] Demo execution path and commands (material compilation)
[0137] Face detection:
[0138] cd ~ / face_demo_dpu
[0139] Image detection Python3 facedetdpujpeg_demo.py
[0140] Video detection: python3 facedetdpu_video_demo.py
[0141] Smoking behavior detection:
[0142] cd ~ / smoke_demo_dpu
[0143] Image detection: python3 smokedetdpu_jpeg_demo.py
[0144] Video detection: python3 smokedetdpu_video_demo.py
[0145] Stability and performance phenomenon analysis
[0146] Based on the measured phenomena recorded in the disclosure materials:
[0147] Image detection mode: It has good stability, runs continuously for 30 minutes without abnormal exit, and the screen is smooth;
[0148] Video detection mode: The inference frame rate (FPS) is only 1.6 to 3.2, with a 1 to 2 second frame lag; two frame loss phenomena occurred in the smoking detection demonstration.
[0149] The reason for this is that in video mode, the pre- and post-processing stages, such as real-time video stream decoding, frame buffer accumulation, and data transmission, become the bottleneck for end-to-end performance. This conclusion suggests that end-to-end inference performance in practical applications depends not only on the computing power of the DPU cores but also on the overall efficiency of the pre- and post-processing links, including image decoding, buffer management, and data transmission.
[0150] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for deploying a convolutional neural network after training, characterized in that, Includes the following steps: S1: Environment Initialization Start the deployment toolchain runtime environment, activate the specified deep learning framework environment, and load the unified configuration file; the unified configuration file shall at least contain model size parameters, training hyperparameters, and path settings; S2: Model Training and Freezing The training script is executed within the deep learning framework environment to train the convolutional neural network model and obtain the training state model; the freeze script is executed to freeze the computation graph of the training state model and generate a static computation graph file. S3: Accuracy Assessment of Frozen Image Execute the frozen graph evaluation script, load the static computation graph file, perform inference evaluation based on the validation dataset, and generate frozen graph accuracy evaluation results; S4: Quantification and Quantification Accuracy Assessment Execute the quantization script to perform quantization calibration on the static computation graph file using the calibration dataset, and generate a quantization model file; execute the quantization evaluation script to perform accuracy evaluation on the quantization model file, and generate quantization accuracy evaluation results; S5: Compile to generate deployment model The compilation script is executed to compile the quantization model file for the deep learning processing unit (DPU) of the target FPGA platform, generating an executable model file adapted to the DPU. S6: Deployment Package Generation and DPU Activation Execute the deployment preparation script to package the executable model file and related dependency files into a deployment package; execute the DPU activation script to mount and activate the DPU core on the target FPGA board. S7: Inference Verification and Performance Statistics Run the inference verification program, load the deployment package, perform batch inference based on the test dataset, and statistically analyze and output the inference throughput and inference accuracy.
2. The method for deploying a convolutional neural network after training according to claim 1, characterized in that: The deployment toolchain runtime environment mentioned in step S1 is the Vitis AI Docker container environment, and the deep learning framework environment is the TensorFlow environment.
3. The method for deploying a convolutional neural network after training according to claim 1, characterized in that: In step S2, the training script performs model training for a preset number of training rounds; the freeze script uses the freeze_graph tool to convert the trained model into a static computation graph file, which is in .pb format.
4. The method for deploying a convolutional neural network after training according to claim 1, characterized in that: The quantization calibration in step S4 involves quantizing the 32-bit floating-point model into an 8-bit integer model, and the quantization model file includes the quantized computation graph file.
5. The method for deploying a convolutional neural network after training according to claim 1, characterized in that: The target FPGA platform mentioned in step S5 is an FPGA acceleration board equipped with a DPU, and the executable model file is a .xmodel format file.
6. The method for deploying a convolutional neural network after training according to claim 1, characterized in that: The inference verification program described in step S7 supports configuring a single-board FPGA inference mode or a multi-board FPGA cluster inference mode, and outputs the total inference time, total inference data volume, throughput and accuracy corresponding to the current configuration after the inference is completed.
7. A system for deploying a convolutional neural network after training, characterized in that, include: A training and compilation host is used to execute the following steps in the method of any one of claims 1 to 6: environment initialization, model training and freezing, frozen graph accuracy evaluation, quantization and quantization accuracy evaluation, compilation to generate deployment model and deployment package generation. At least one FPGA acceleration board is communicatively connected to the training and compilation host, and a deep learning processing unit (DPU) is deployed on the FPGA acceleration board for loading and executing the executable model file; The inference verification module runs on the training and compilation host or FPGA acceleration board and is used to perform the inference verification and performance statistics steps, and output inference throughput and inference accuracy.
8. The system for deploying a convolutional neural network after training, as described in claim 7, is characterized in that: The FPGA acceleration board consists of multiple boards, forming an FPGA board cluster; the inference verification module supports parallel inference and unified performance statistics in a multi-board cluster mode.
9. The system for deploying a convolutional neural network after training, as described in claim 8, is characterized in that: The inference verification module also supports image inference mode and video inference mode; in image inference mode, the system performs inference on single or batch image files; in video inference mode, the system performs frame-level inference on real-time video streams or video files.
10. The system for deploying a convolutional neural network after training, as described in claim 8, is characterized in that: The training and compilation host stores intermediate output files generated by the method, including static computation graph files, quantization model files, executable model files, and corresponding evaluation log files, which are used to achieve traceability and reproducibility of the deployment process.