Deep learning model compiling optimization system for reconnaissance equipment

Abstract

The invention discloses a deep learning model compiling optimization system for reconnaissance equipment, and belongs to the field of model compiling. According to the optimization system, an ONNX model is converted into a MegEngine (MGE) model or a TraceedModule model by using a MMeConvert tool, an mgb-to-tinynn tool is executed, the converted MGE model or the TraceedModule model is compiled into a Kernel of a pure C code, and a model file corresponding to the Kernel is generated; a model path is specified through a JSON file, a data shape and optimization parameters are input, an mgb-to-tinynn command is executed, and model operation files suitable for different hardware platforms are compiled and generated by using a specified target architecture. The lightweight model compiler has the beneficial effects that the lightweight model compiler is utilized to better realize lightweight high-performance calculation in mobile investigation equipment for public security work and realize functions of target identification and the like through the mobile investigation equipment, so that mobile public security can identify and track personnel and articles more efficiently, the efficiency and the accuracy of the public security work are improved, and the working efficiency and the working efficiency of the public security are improved. And public security and safety are maintained.

Description

Technical Field

[0001] The present invention relates to a deep learning model compilation optimization system for reconnaissance equipment, and belongs to the field of model compilation. Background Art

[0002] Early neural network deployment focused on frameworks and operator libraries, and had relatively weak links with compilers. Therefore, in early optimization, the focus was on modifying the framework and improving the performance of the operator library. However, this intuitive deployment method also exposed some problems: new operators continue to emerge, making it increasingly difficult to develop and maintain operator libraries; newly added operators not only need to be implemented, but also need to be optimized and tested in combination with hardware characteristics to fully utilize hardware performance. However, dedicated hardware needs to provide similar optimization libraries, resulting in over-reliance on these libraries and inability to effectively utilize deep learning chips.

[0003] Due to the diversity of hardware, the important task is how to effectively map calculations, so portability becomes a necessity; most NPUs are ASICs that are specially optimized for computing, storage, and data transmission in neural network scenarios, so they have better energy efficiency than CPUs and GPUs. At the same time, the instruction set architectures of NPUs vary, and they usually lack tool chains such as GCC and LLVM, which makes it difficult to port operator libraries that have been optimized for CPUs and GPUs to NPUs in the short term and fully utilize hardware capabilities to achieve good performance. Summary of the invention

[0004] The purpose of the present invention is to provide a deep learning model compilation optimization system for reconnaissance equipment, which can effectively solve the above-mentioned problems.

[0005] In order to solve the above technical problems, the present invention is achieved through the following technical solutions: S1: Use the MgeConvert tool to convert the ONNX model to a MegEngine (MGE) model or a TracedModule model. The opset range supported by MgeConvert is 7 to 12. S2: By executing the mgb-to-tinynn tool, the converted MGE or TracedModule model is compiled into a pure C code Kernel and the corresponding model file is generated; S3: During the compilation process, the model path, input data shape, and optimization parameters are specified through a JSON file, supporting the description of multiple models and multiple input shapes; S4: By executing the mgb-to-tinynn command, the specified target architecture is used to compile and generate model run files suitable for different hardware platforms; S5: Compile the compiled Kernel file and the corresponding model file together with the Runtime pre-written by MegCC to generate a library file that can eventually run on the target device.

[0006] Further: In S3, define a directory for storing or outputting certain results and store the JSON file in that directory; list the model to be compiled, including the model's name, path, shape of the input data, and whether to enable dedicated optimization methods for optimization.

[0007] Further: For the S4 step, the target hardware platform can be one of baremetal, arm64, armv7, armv7_with_dot, arm64v7, arm64v7_with_dot.

[0008] Further: For the S4 step, build the scripts for the compilation and runtime environment, compile the compiled model and kernel together with the runtime source code provided by MegCC, and generate the final executable file and library; the generated files will be used for inference testing on the target device.

[0009] Further: For step S4, the compilation process supports cross-compilation on the Android operating system and the Linux operating system based on the aarch64 platform.

[0010] The deep learning model compilation and optimization system for reconnaissance equipment according to claim 5 is characterized in that: The equipment used in this optimization system includes: A processor executes each step of the above compilation method; Storage device, which stores ONNX models, MGE models, JSON configuration files, compilation tools, and generated C code and model files; Compilation environment, including MgeConvert tool, mgb-to-tinynn tool, MegCC tool and compilation scripts; Input interface, used to receive configuration and compilation parameters provided by the user; Output interface, used to store compiled files and support model reasoning.

[0011] The beneficial effects are: This study can use this lightweight model compiler to better realize lightweight high-performance computing in mobile reconnaissance equipment for public security work. By realizing functions such as target recognition through mobile reconnaissance equipment, mobile public security can more efficiently identify and track people and objects, improve the efficiency and accuracy of public security work, and maintain social order and safety.

[0012] At the same time, the comprehensive application of MegCC and YOLOX enables efficient target detection algorithm models to be efficiently used on mobile reconnaissance equipment, with only a low runtime, achieving low-energy consumption, high-quality and efficient operations, providing an effective solution for target identification on mobile reconnaissance equipment, and also providing technical support for the informatization and intelligent development of public security work. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] For ease of explanation, the present invention is described in detail with reference to the following specific implementations and the accompanying drawings.

[0014] Figure 1 It is a flowchart of the present invention; DETAILED DESCRIPTION

[0015] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0016] It should be noted that, in the description of the present invention, unless otherwise specified, “plurality” means two or more than two; the orientations or positional relationships indicated by terms such as “upper”, “lower”, “left”, “right”, “inside”, “outside”, “front end”, “rear end”, “head” and “tail” are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0017] Furthermore, the terms “first,” “second,” “third,” etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

[0018] Meanwhile, in the description of the present invention, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0019] See also Figure 1This is an embodiment of a deep learning model compilation and optimization system for reconnaissance equipment of the present invention. S1: Use the MgeConvert tool to convert the ONNX model to a MegEngine (MGE) model or a TracedModule model. The opset range supported by MgeConvert is 7 to 12. S2: By executing the mgb-to-tinynn tool, the converted MGE or TracedModule model is compiled into a pure C code Kernel and the corresponding model file is generated; S3: During the compilation process, the model path, input data shape, and optimization parameters are specified through a JSON file, supporting the description of multiple models and multiple input shapes; S4: By executing the mgb-to-tinynn command, the specified target architecture is used to compile and generate model run files suitable for different hardware platforms; S5: Compile the compiled Kernel file and the corresponding model file together with the Runtime pre-written by MegCC to generate a library file that can eventually run on the target device.

[0020] In S3, define a directory for storing or outputting certain results, and store the JSON file in the directory; list the models to be compiled, including the model's name, path, shape of the input data, and whether to enable special optimization methods for optimization. The corresponding instructions are: dump_dir: Specify the location where the compiled model and C code are stored. models: List the models to be compiled, including the model's name, path, shape of the input data, and whether to enable certain optimizations. At the same time, for a model, if it has multiple input shapes, they can be specified in the file and separated by colons. The enable_nchw44 option is used to indicate whether to use an optimization method to help speed up the model.

[0021] For the S4 step, the target hardware platform can be one of baremetal, arm64, armv7, armv7_with_dot, arm64v7, arm64v7_with_dot. The corresponding instructions are: --arm64v7, which generates a model that can run on Arm64 and ArmV7 devices. --baremetal, generates pure C code that is only suitable for MCUs. --armv7_with_dot, for ArmV7 architecture and enables dotprod instructions, which can improve performance.

[0022] For the S4 step, build the scripts for the compilation and runtime environment, compile the compiled model and kernel, and the runtime source code provided by MegCC together to generate the final executable file and library; the generated files will be used for inference testing on the target device. The corresponding instructions are: execute the . / runtime / scripts / runtime_build.py script to compile the compiled code and model together with the runtime program. Use the tinynn_test_lite tool to test the compiled model on the target device to ensure that it can run normally.

[0023] For step S4, the compilation process supports cross-compilation on the Android operating system and the Linux operating system based on the aarch64 platform.

[0024] The equipment used in this optimization system includes: A processor executes each step of the above compilation method; Storage device for storing ONNX models, MGE models, JSON configuration files, compilation tools, and generated C code and model files; compilation environment, including MgeConvert tool, mgb-to-tinynn tool, MegCC tool, and compilation scripts; input interface for receiving configuration and compilation parameters provided by the user; output interface for storing compiled files and supporting model reasoning.

[0025] When implementing this deep learning model compilation and optimization system, the first thing to do is: Step 1: Create a working environment: Hardware facilities: Processor is Intel(R) Core(TM) i7-10875HCPU@2.30GHz, main frequency is 2.30GHz; graphics card is NVIDIAGeForceRTX20608G; machine has 16.0G RAM, frequency is 3200MHz; hard disk is 512G; operating system is Windows11 Professional Edition.

[0026] Software facilities: The programming language is Python; the compilation environment is Python3.8; the development environment is numpy1.19.5, pytorch2.1.0+cu121, PIL10.2.0, tqdm4.66.2, matplotlib3.3.3, onnx1.11.0.

[0027] The compiled model can currently be compiled using the executable file mgb-to-tinynn, or using the ready-made script . / script / ppl_gen.sh in the Release package.

[0028] Step 2: Convert the ONNX model to the MakeEngine model: MgeConvert is a converter for MegEngine models. Taking onnx as an example, the following command will install the onnx converter and handle related dependencies: pip3installmgeconvert--user--install-option=--targets=onnx; The purpose of this instruction is to: "pip3installmgeconvert": Use pip3 to install a Python library called mgeconvert. mgeconvert is a tool for converting MegEngine models to other formats such as ONNX.

[0029] "--user": specifies to install to the user's local directory instead of installing globally. This method does not require administrator privileges.

[0030] "--install-option=--targets=onnx": pass custom installation options when installing via pip. Here --targets=onnx is a custom parameter that may tell mgeconvert that the installation process needs to support conversion to the ONNX format.

[0031] Then convert the format: The conversion code used by this system is: "convertmge_to_onnx-imodel.mge-out.onnx converttracedmodule_to_onnx-itracedmodule.tm-oout.onnx convertonnx_to_mge-imodel.onnx-out.mge convertonnx_to_tracedmodule-itracedmodule.onnx-out.tm".

[0032] The purpose of this code is to realize the conversion of model formats between different deep learning frameworks or tools, including: conversion between MegEngine (.mge) and ONNX (.onnx); conversion between TracedModule (.tm) and ONNX (.onnx).

[0033] Among them, the conversion between MegEngine (.mge) and ONNX (.onnx) is mainly to improve cross-platform and framework interoperability; MegEngine to ONNX: ONNX is a universal model exchange format that supports multiple deep learning frameworks (such as PyTorch, TensorFlow, MXNet, etc.). Convert MegEngine's ".mge" model to ".onnx" format, which can be loaded, debugged, or continued training in other frameworks.

[0034] ONNX to MegEngine: If a model was originally trained in another framework (and saved in ONNX format), after converting it to MegEngine’s “.mge” format, it can be optimized and deployed using MegEngine’s high-performance inference engine and toolchain.

[0035] At the same time, ONNX has become the industry standard for model formats. The interoperability of the ONNX format can simplify the process and reduce development costs during collaborative development, model sharing, and deployment. Therefore, supporting the conversion between ".mge" and ".onnx" will help better integrate into the existing deep learning tool chain and ecosystem.

[0036] Moreover, during the training and inference stages of the model, the interoperability between MegEngine and other frameworks makes it easy to compare the performance and accuracy of the model on different frameworks.

[0037] It also has debugging and repair functions: If the model encounters problems when running on MegEngine, it can be debugged by converting it to ONNX format and loading it into other frameworks that support ONNX.

[0038] The purpose of converting between TracedModule ".tm" and ONNX ".onnx" is to optimize model performance, enhance cross-platform interoperability, and meet the needs of specific application scenarios.

[0039] ONNX is a universal model format that supports multiple deep learning frameworks (such as PyTorch, TensorFlow, etc.). By converting ".tm" to ".onnx", the model can be loaded in other frameworks for further development, training, or debugging.

[0040] Convert ONNX to TracedModule: If the original model is trained with other frameworks (and saved in .onnx format), convert it to ".tm" and it can be adapted to the framework or inference engine that supports TracedModule for efficient deployment.

[0041] ONNX's wide compatibility: ONNX supports multiple hardware backends (such as NVIDIA TensorRT, ONNX Runtime, Intel OpenVINO), and can be directly deployed to these hardware platforms by converting ".tm" to ".onnx".

[0042] TracedModule-specific optimizations: Some inference frameworks may be optimized specifically for the TracedModule format (such as efficient operator implementation or hardware acceleration support). These optimizations can be utilized after converting ".onnx" to ".tm" format. It can also serve as a debugging and repair function.

[0043] For the first conversion code "convertmge_to_onnx-imodel.mge-oout.onnx", its function is: convert a MegEngine format model (extension .mge) to an ONNX format model (extension .onnx); "-imodel.mge": specifies the input file, indicating that the MegEngine format model file is "model.mge"; "-oout.onnx": specifies the output file, indicating that the converted ONNX format model is saved as "out.onnx".

[0044] For the second conversion code "converttracedmodule_to_onnx-itracedmodule.tm-oout.onnx", its function is: convert the TracedModule format model (extension .tm) to the ONNX format model; among them, "-itracedmodule.tm": specifies the input file, indicating that the model file in the TracedModule format is "tracedmodule.tm"; "-oout.onnx": specifies the output file, indicating that the converted ONNX format model is saved as "out.onnx".

[0045] For the third conversion code "convertonnx_to_mge-imodel.onnx-oout.mge", its function is to convert an ONNX format model (with the extension .onnx) to a MegEngine format model (with the extension .mge). Among them, "-imodel.onnx" specifies the input file, indicating that the model file in the ONNX format is "model.onnx"; "-oout.mge" specifies the output file, indicating that the converted MegEngine format model is saved as "out.mge".

[0046] For the fourth conversion code "convertonnx_to_tracedmodule-itracedmodule.onnx-oout.tm", its function is: convert the ONNX format model (extension name .onnx) to the TracedModule format model (extension name .tm); "-itracedmodule.onnx": specifies the input file, indicating that the ONNX format model file is "tracedmodule.onnx"; "-oout.tm": specifies the output file, indicating that the converted TracedModule format model is saved as "out.tm".

[0047] Step 3: Write the JSON file The mgb-to-tinynn tool is mainly used in the model compilation stage. After the compilation is completed, the corresponding pure C code Kernel and the corresponding model will be generated in the directory given by the user. In order to compile the model, the mgb-to-tinynn tool requires the user to provide a Json file to describe the specific details of the compilation.

[0048] The code is: "dump_dir":". / kernel_yolox_s_arm / ", "models":[ { "model_name":"yolox_s", "model_path":". / yolox_s.mge", "input_shape_str":"data=(1,3,640,640)", "enable_nchw44":true This command is a configuration file or parameter setting snippet that describes the running configuration and path information of a YOLOX-S model, which may be used as input and output for loading models and setting inference in deep learning inference frameworks (such as MegEngine).

[0049] The following are the specific meanings and functions of each part of the code: dump_dir: defines the directory used to store or output certain results. ". / kernel_yolox_s_arm / " means that some related files will be stored in the kernel_yolox_s_arm directory.

[0050] models: is a model configuration list that supports configuring multiple models; here only one model is included, indicating that a YOLOX-S model will be loaded.

[0051] model_name: specifies the name of the model (logical name); the value here is "yolox_s", indicating that the model type is YOLOX-S, a lightweight target detection model.

[0052] model_path: specifies the physical path of the model file; here the path is ". / yolox_s.mge", indicating that the model file is a yolox_s.mge file in MegEngine format.

[0053] input_shape_str: defines the input shape of the model; data=(1,3,640,640) means that the shape of the input tensor is (batch_size=1,channels=3,height=640,width=640), which is usually used in image processing, where 3 is the number of channels of RGB images and 640x640 is the width and height of the input image.

[0054] enable_nchw44: indicates whether to enable the NCHW44 format; NCHW is a storage format for tensor data, and NCHW44 is an optimized version, which is usually used to accelerate inference calculations, especially on ARM architecture devices.

[0055] The end of the last command is set to true, which enables optimization.

[0056] The main purpose of writing JSON files is to specify the file path, input shape, and optimization method of the YOLOX-S model; to load and infer the model with the deep learning inference engine; it is often used in deployment scenarios (such as embedded devices or mobile devices) and is optimized for the ARM architecture.

[0057] The Json file needs to specify the model name compiled using mgb-to-tinynn, the model path, the model input data, and some optimization parameters. If you need to form a pipeline for multiple models in the actual deployment, you need to specify multiple models. If a model may require multiple input shapes during the actual reasoning process, you need to specify them separately in input_shape_str and separate them with:.

[0058] Step 4: Model compilation (1) Use ready-made scripts to compile models There is a ppl_gen.sh file under the script directory in the Release package. Execute it directly: .. / bin / mgb-to-tinynn --json=". / yolox_arm.json" --arm64v7 The . / script / ppl_gen.sh script will compile the model and package the resources required by the Runtime in a compressed package to facilitate the subsequent Runtime compilation. Decompressing the compressed package will yield: ├──buildruntimebuild path ├──immigrationgeneralIntrinsic header file │└──include ├──kern model kernel file including cv operator ├──yolox_arm.json configuration file required for model dump ├──model │└──yolox_s.tiny ├──model_info model input information │└──yolox_s.tiny.txt ├──ppl_build.sh ├──runtimeruntime source code │├──CMakeLists.txt │├──example │├──flatcc │├──include │├──schema │├──script │└──src └──test_model.py model test script In the compressed package after decompression, the function of buildruntimebuild is to store the path of the files generated by the runtime build, which may be binary files and library files generated after compilation.

[0059] immigrationgeneralintrinsicinclude: contains header files, which may be declarations of Intrinsic implementations such as general tool functions or mathematical operators to support core functions.

[0060] Function of kern: Stores kernel-related implementation files, including the definition and implementation of computer vision (CV) operators, which are the key calculation parts in model operation.

[0061] Function of yolox_arm.json: The configuration file required for model dump (export), which defines the relevant information on how to serialize or debug the model.

[0062] Modelyolox_s.tiny Function: Stores the actual YOLOX model file, which may be the trained weights or the deployed lightweight model file.

[0063] Model_infoyolox_s.tiny.txt is used to store the model input information, which may include input dimensions, preprocessing parameters, etc., to guide how to load and use the model.

[0064] Function of ppl_build.sh: Script file used to build the entire project. It may call CMake or other build tools to complete source code compilation and generate runtime files.

[0065] Runtime: Subdirectory Contents: CMakeLists.txt is used to define the build rules of the runtime part and supports cross-platform compilation.

[0066] Example Sample code that shows how to use the runtime library or related functions.

[0067] Flatcc and FlatBuffers related implementations or generated intermediate files for efficient model serialization and deserialization.

[0068] Include: Contains the runtime header file, which defines the API interface and data structure.

[0069] Schema: defines the structure and data format of the model file, which is used for loading and parsing the model.

[0070] Script: Stores script files, which may be automation tools to assist runtime.

[0071] Src: Contains the source code of the runtime, which implements the specific functions of the runtime.

[0072] Function of test_model.py: Model test script, used to load, run and verify the model functions, which may contain performance testing and result verification logic.

[0073] (2) Compile using executable file Use mgb-to-tinynn and the Json file written above to execute: mgb-to-tinynn--json= / path / to / json--[target] After the model is compiled, the generated Kernel that runs the model, the model files bound to these Kernel, and the cv operators are placed in the directory specified in the Json file. Target: can be baremetal, arm64, armv7, armv7_with_dot, arm64v7, arm64v7_with_dot; baremetal: The generated Kernel is in pure C format that can be run by the microcontroller; arm64v7: Generates two sets of kernels and their corresponding models that can run on both Arm64 and ArmV7. At this time, the model file may be larger than when the target is arm64.

[0074] armv7_with_dot: When running in AArch32 mode of Arm64 and the device supports dotprod, armv7_with_dot can be used to generate an armv7 kernel using dot instructions, which usually has better performance than kernels that do not use dot instructions.

[0075] For example, to compile the yolox model in the Release package, the target machine is an arm64 machine, and run the following command: mkdirkernel_yolox_s_arm . / bin / mgb-to-tinynn --json=. / example / yolox_arm.json --arm64 --dumpkernel_yolox_s_arm The purpose of this command is to convert the MegEngine format model of YOLOX to the lightweight format of TinyNN, which is adapted to the ARM64 architecture. The applicable scenarios include deploying the YOLOX model on embedded or mobile terminals; optimizing model reasoning for the ARM64 platform to improve operating efficiency; and preparing code and model files for subsequent integration or development.

[0076] This command will generate all kernels for running this model in kernel_yolox_s_arm, as well as the model input information: yolox_s.tiny.txt file, and the newly generated model file: yolox_s.tiny.

[0077] (3) Runtime compilation 1) Use script compilation MegCC provides a script to facilitate the above compilation operation: unzip the megcc_ppl_gen.tar.gz generated after compiling with the ready-made script above, execute tar-xvfmegcc_ppl_gen.tar.gz and enter the compressed directory.

[0078] If the compilation platform is arm64, execute: . / ppl_build.sh If the compilation platform is armv7, execute . / ppl_build.sh --cross_build_target_arch=armv7-a If the compilation platform is armv7_with_dot, execute . / ppl_build.sh --cross_build_target_arch=armv7-a --enable_aarch32_dot.

[0079] The model file required for the final runtime is yolox_s.tiny in the model directory after decompression.

[0080] 2) Compile separately The kernel and model compiled using the executable file above are saved in the specified generated file. In order to generate the final library file, it is also necessary to compile it together with the runtime pre-written by MegCC.

[0081] The kernel and model compiled using the executable file above are saved in the specified generated file. In order to generate the final library file, it is also necessary to compile it together with the runtime pre-written by MegCC. Execute: . / runtime / scripts / runtime_build.py --kernel_dir . / kernel_yolox_s_arm --cross_build This command is mainly used to build a YOLOX model runtime environment adapted to the ARM architecture. It will: use the specified kernel file (. / kernel_yolox_s_arm) as part of the build process, which are usually optimized computing kernels (such as ARM-specific operator codes); enable cross-compilation so that the compiled runtime can run on an ARM device instead of the current development machine.

[0082] The runtime_build.py script compiles by default to support aarch64 compilation on Android. Users can use: --cross_build_target_arch: specifies the target compilation platform, currently supported: ['x86_64', 'i386', 'aarch64', 'armv7-a', 'cortex-m', 'armv7-a-qemu'].

[0083] --enable_aarch32_dot: This option must be enabled when the arch specified when compiling the model is armv7_with_dot.

[0084] --cross_build_target_os: Specify the target operating system for compilation. Currently supported: ['ANDROID', 'LINUX', 'IOS', 'NOT_STANDARD_OS'] --kernel_dir: specifies the compiled Kernel path.

[0085] On the target machine, you can use tinynn_test_lite and the compiled model file kernel_yolox_s_arm / yolox_s.tiny and certain input data to perform inference on the yolox model.

[0086] After the above command is executed, the Kernel in the previously generated kernel_yolox_s_arm and the Runtime in the Release package are compiled and installed in the runtime / install directory of kernel_yolox_s_arm. The files in the directory are as follows: ├──bin │└──tinynn_test_lite: Integrates the following libTinyNN.a file and compiles the executable file. Users can run it directly. ├──include: Integrate the header files required by libTinyNN.a below │├──lite-c ││├──common_enum_c.h ││├──global_c.h ││├──macro.h ││├──network_c.h ││└──tensor_c.h │├──tinycv_c.h: Header file required to use the compiled cv operator │└──tinynn_callback.h └──lib └──libTinyNN.a: static library generated by compiling runtime and compiled Kernel; In terms of model optimization, the present invention uses the MegCC compiler to implement a series of optimization techniques, including weight quantization, network structure pruning, channel rearrangement, and computational graph optimization. The MegCC compiler runtime library is very lightweight, with a size of only 81KB, and performs well on the Am platform. Each Kernel has been carefully tuned, with better performance than MegEngine, and can be easily ported to different platforms. It has low memory usage and fast startup speed. Memory planning is automatically performed during model compilation, and static binding is used to reduce operating overhead. The advantages of MegCC and a variety of optimization techniques can reduce the size and computational complexity of the model, and improve the compilation efficiency and performance of the deep learning model. In addition, MegCC also supports automatic selection of the optimal computational graph partitioning strategy and operator fusion strategy to further improve the performance of the model.

[0087] At the same time, this project uses the YOLOX target detection algorithm model, combined with the MegCC lightweight deep learning model compiler, to enable high-precision, high-speed, and high-efficiency target recognition and tracking on mobile reconnaissance equipment while maintaining low energy consumption.

[0088] In terms of code generation, this study uses the MegCC compiler to efficiently convert the optimized model into efficient runtime code in real time. It supports a variety of hardware platforms, including CPU, GPU, and AI accelerators. By generating code for different hardware platforms, MegCC can make full use of the parallel computing capabilities of the hardware and achieve high-performance operation of the model on the mobile terminal.

[0089] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A deep learning model compilation and optimization system for reconnaissance equipment, characterized in that : The steps for using the system are: S1: Use the MgeConvert tool to convert the ONNX model into a MegEngine model or a TracedModule model. The opset range supported by MgeConvert is 7~12; S2: By executing the mgb-to-tinynn tool, the converted MGE or TracedModule model is compiled into a pure C code Kernel, and the corresponding model file is generated; S3: During the compilation process, the model path, input data shape, and optimization parameters are specified through a JSON file, and the description of multiple models and multiple input shapes is supported; S4: By executing the mgb-to-tinynn command, the specified target architecture is used to compile and generate model run files suitable for different hardware platforms; S5: Compile the compiled Kernel file and the corresponding model file together with the Runtime pre-written by MegCC to generate a library file that can eventually run on the target device.

2. The deep learning model compilation and optimization system for reconnaissance equipment according to claim 1, characterized in that : In S3, define a directory for storing or outputting certain results and store the JSON file in the directory; list the models to be compiled, including the model name, path, shape of the input data, and whether to enable dedicated optimization methods for optimization.

3. The deep learning model compilation and optimization system for reconnaissance equipment according to claim 2 is characterized in that :For the S4 step, the target hardware platform can be one of baremetal, arm64, armv7, armv7_with_dot, arm64v7, arm64v7_with_dot.

4. The deep learning model compilation and optimization system for reconnaissance equipment according to claim 3 is characterized in that : For the S4 step, build the scripts for the compilation and runtime environment, compile the compiled model and kernel, and the runtime source code provided by MegCC together to generate the final executable file and library; the generated files will be used for inference testing on the target device.

5. The deep learning model compilation and optimization system for reconnaissance equipment according to claim 4 is characterized in that :For step S4, the compilation process supports cross-compilation on the Android operating system and the Linux operating system based on the aarch64 platform.

6. The deep learning model compilation and optimization system for reconnaissance equipment according to claim 5, characterized in that The devices used in the optimization system include: a processor that executes the various steps of the above compilation method; a storage device that stores ONNX models, MGE models, JSON configuration files, compilation tools, and generated C code and model files; a compilation environment, including the MgeConvert tool, the mgb-to-tinynn tool, the MegCC tool, and a compilation script; an input interface for receiving configuration and compilation parameters provided by the user; and an output interface for storing compiled files and supporting model reasoning.

Images