Precision performance adjusting and optimizing method, system and equipment based on network block topological sequence and storage medium
By using a precision performance tuning method based on network block topology sequences, the topological relationships of network blocks are dynamically adjusted and iterative hybrid precision configuration is adopted. This solves the problem of complex and time-consuming precision settings in large-scale deep learning model deployment, achieves the best balance between precision and performance, and improves model inference speed and deployment efficiency.
Patent Information
- Application Number
- CN202511022068.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-24
- Publication Date
- 2025-11-28
AI Technical Summary
In large-scale deep learning model deployments, the existing technology involves complex and time-consuming manual layer precision settings, resulting in low efficiency in model conversion and precision evaluation, making it difficult to achieve a balance between accuracy and performance.
A precision and performance tuning method based on network block topology sequences is adopted. Through Python scripts and TensorRT tools, the topological relationship of network blocks is dynamically adjusted to perform iterative hybrid precision configuration. Combining topology sequence partitioning and dynamic adjustment mechanisms, the accuracy and performance of the model are optimized.
It achieves the best balance between accuracy and performance on the NVIDIA chip platform, improving model inference speed by 2-5 times, significantly optimizing model deployment efficiency, and meeting real-time requirements.
Smart Images
Figure CN121031280A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent driving technology and relates to deep learning GPU inference optimization. Specifically, it relates to a method, system, device and storage medium for precision performance tuning based on network block topology sequence. Background Technology
[0002] As deep learning models continue to grow in scale, memory usage and computational power consumption have become core bottlenecks in engineering deployment. In deep learning model deployment, using FP32 (single-precision floating-point) and FP16 (half-precision floating-point) for computation and storage can reduce memory usage, improve inference performance, and lower hardware power consumption. For layers that are not precision-sensitive but computationally intensive, switching to FP16 can be safely implemented to improve performance and save memory. Traditional manual layer precision setting methods require analyzing and adjusting the precision of each model layer layer by layer when specifying whether to use FP32 or FP16. For modern models with hundreds of cascaded layers, this is complex and labor-intensive. Furthermore, each model conversion and precision evaluation requires repeated execution, significantly extending the optimization cycle for model precision tuning and overall time consumption. Summary of the Invention
[0003] To address the aforementioned problems, the main objective of this invention is to design a method, system, device, and storage medium for precision performance tuning based on network block topology sequences. By leveraging the topological relationships within dynamic network blocks, this invention solves the problems of high workload and complex operation in manually setting layered precision, as well as the time-consuming model conversion and precision evaluation.
[0004] To achieve the above objectives, the present invention adopts the following technical solution: A method for precision performance tuning based on network block topology sequences is proposed. This method runs on an NVIDIA chip platform and is implemented using Python scripts and the TensorRT tool. The specific steps include: S1, Full-precision module inference truth value: The ONNX source model is loaded in Python using the ONNXRuntime library, unlabeled image samples are input into the model, FP32 inference is performed and the output results are saved. The inference results are used as the full-precision model truth value GT. S2. Initialize FP32 accuracy and performance: Use TensorRT's trtexec tool to convert the ONNX source model to FP32 full-precision model, perform inference to obtain the model inference accuracy Prec_gt and time consumption Time_gt, which serve as the benchmark for FP32 full-precision model optimization. S3. Obtain the topology sequence: Extract the topology sequences Lys_0, Lys_1, ..., Lys_i of multiple input ONNX source models and downstream nodes; S4. Topology Sequence Blocking: Based on the principle of top-down network layer division, each group of topology sequences is divided into blocks to obtain multiple sequence blocks Block_0, ..., Block_i for each group of topology sequences; S5. Perform iterative blending precision configuration optimization, including initializing intra-block blending precision, model precision and time consumption supervision, dynamically optimizing intra-block blending precision, inter-block blending precision, and dynamically adjusting the block size (block_size). S6. Output model: After the optimization process in step S5, the target platform model that meets the requirements in terms of model accuracy and inference performance is obtained, which is the final model Model_end.
[0005] As a further description of the present invention, in step S3, the topology sequence is a sorting of network layers with connection dependencies, and multiple topology sorting sequences are obtained, denoted as Lys_0, Lys_1, ..., Lys_i.
[0006] As a further description of the present invention, in step S4, the sequence block size block_size and the number of sequence blocks block_num are set, where block_size represents the number of FP16 network layers in each sequence block, and block_num = N / block_size, where N is the total number of layers in the sequence.
[0007] As a further description of the present invention, step S5, iterative mixed precision configuration optimization, includes the following steps: S501. Initialize mixed precision within sequence blocks: For each set of topological sequences, process each sequence block Block_i (i=0,1,..., M-1, M is the total number of blocks) in sequence, set the precision of all layers in Block_i to FP16, and the precision of all layers in the remaining blocks to FP32, and use TensorRT's trtexec tool to perform the transformation to generate a mixed precision model Model_i. S502, Model Accuracy and Time Consumption Supervision: Supervising the accuracy Prec_i and time consumption Time_i of the mixed-precision model Model_i: If the precision Prec_i and the time consumption Time_i meet the threshold conditions, the network layer precision of the sequence block is set to a fixed FP16, and the process proceeds to the S504 sequence block mixing precision step to iterate the next sequence block. The precision threshold condition is: Prec_i - Prec_gt| ≤ δ, and the time consumption optimization condition is: Time_i ≤ Time_gt, where δ is a preset tolerance coefficient. If the accuracy Prec_i does not meet the threshold condition, then proceed to the S503 dynamic optimization of the mixing accuracy within the sequence block to optimize the mixing accuracy within the sequence block until the accuracy meets the standard. If the precision Prec_i meets the threshold condition and the time consumption Time_i does not decrease compared to Time_gt, then proceed to the S504 inter-block precision mixing step to iterate the next block. The process continues until all blocks are optimized, at which point a mixed-precision model that meets both accuracy and performance targets is output.
[0008] As a further description of the present invention, in step S503 above, the mixing precision within the sequence block is dynamically optimized: for the current sequence block of the current topology sequence, starting from the block_size layer, it is adjusted to FP32 one layer in sequence. For each layer adjusted, a mixing precision model conversion is performed, and the model precision and time consumption supervision in S502 is executed until the precision meets the requirements. Then, the mixing precision layer of the current sequence block is determined, and the process proceeds to step S504, the mixing precision between sequence blocks, to iterate the next sequence block.
[0009] As a further description of the present invention, in step S504 above, the mixing accuracy between sequence blocks is: for the next sequence block of the current topological sequence, S501 is executed to initialize the mixing accuracy within the sequence block, and after model transformation, S502 is executed to supervise model accuracy and time consumption. If the accuracy meets the requirements, repeat step S504 for mixing accuracy between sequence blocks; if the accuracy does not meet the requirements, proceed to step S503 for dynamic optimization of mixing accuracy within the sequence block, and perform model accuracy and time monitoring again. If the accuracy meets the requirements, continue until all sequence blocks of the topology sequence are optimized, then proceed to step S4 for topology sequence partitioning, and perform sequence block partitioning and mixing accuracy optimization for the next topology sequence. Once the mixed precision optimization of all topological sequences is completed, proceed to S6 to output the model result; If the time taken does not meet the requirements, the process switches to S505 to dynamically adjust the sequence block size (block_size).
[0010] As a further description of the present invention, in step S505 above, the sequence block size block_size is dynamically adjusted: the initial value of block_size is 16, and it is decreased by powers of 2, and then the topological sequence is re-divided in step S4 until the optimal block_size is found.
[0011] A precision performance tuning system based on network block topology sequences, running on an NVIDIA chip platform, includes: The benchmark building module is used to generate the truth value GT by performing FP32 full-precision inference through the ONNXRuntime library, and to obtain the FP32 benchmark precision Prec_gt and time consumption Time_gt through the trtexec tool of TensorRT. The topology analysis module is used to parse the inter-layer dependencies of the ONNX source model and output multiple sets of topology sequences Lys_0, Lys_1, ..., Lys_i; The block partitioning module is used to divide each group of topological sequences into sequence blocks of size block_size, Block_0, ..., Block_i, according to the top-down principle, where block_size is a power of 2; The hybrid precision optimization module iteratively performs the following operations until the precision and time consumption thresholds are met: Initialize the current sequence block to FP16 precision, and the remaining sequence blocks to FP32 precision; Dynamically adjust the number of FP16 layers within the sequence block, i.e., backtrack from the block_size layer to FP32 layer by layer; Inter-block precision synchronization optimization dynamically reduces block size; The output module is used to generate the optimized final model, Model_end.
[0012] An electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus, and the memory is used to store computer programs; The processor is configured to perform the above-described method by running the computer program stored in the memory.
[0013] A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method.
[0014] Compared with the prior art, the technical advantages of the present invention are as follows: This invention provides a method, system, device, and storage medium for precision performance tuning based on network block topology sequences. By partitioning network block topology sequences and iteratively optimizing mixed precision, it achieves the best balance between precision and performance on the NVIDIA chip platform. It adopts block-level granular mixed precision configuration and combines it with a dynamic adjustment mechanism. Compared with traditional global mixed precision methods, it can improve inference speed by 2-5 times while maintaining the original precision. Through parallel processing of multiple topology sequences and the trtexec tool, it significantly optimizes the deployment efficiency of complex models and meets the real-time requirements of various application scenarios. Attached Figure Description
[0015] Figure 1 This is a schematic diagram of the performance tuning method of the present invention; Figure 2 This is a diagram showing the comparison before and after optimization of this performance tuning method. Detailed Implementation
[0016] The present invention will now be described in detail with reference to the accompanying drawings: In one embodiment of the present invention, a method for precision performance optimization based on network block topology sequences is disclosed, with reference to... Figure 1-2 As shown, by using the topological relationships of the layers in the dynamic network block, the model is set to mixed precision in a search-like manner, resulting in a hybrid FP32 and FP16 model that is basically aligned with the full-precision model. This reduces the overall model time while ensuring optimal inference performance.
[0017] Specifically, in this embodiment, the process of the above performance tuning method includes the following: Operating scenario description: In this embodiment, the application scenarios for performance tuning include, but are not limited to, image classification, object detection, semantic segmentation, and other models, nor are they limited to the application scenarios of algorithms in the autonomous driving industry. Any other field or application involving mixed precision models in GPU inference can use the method of this embodiment to achieve performance tuning. The method of this embodiment is a general mixed precision model inference performance tuning strategy.
[0018] Operating platform description: The performance tuning method in this embodiment runs on NVIDIA chip platforms such as Orin and Xavier, and the entire optimization process can be achieved using Python scripts and the TensorRT tool.
[0019] Specifically, the steps include the following: S1. Full-precision module inference truth value: The ONNX source model is loaded in Python using the ONNXRuntime library. A certain amount of unlabeled image samples are input into the model, FP32 inference is performed and the output results are saved. The inference results are used as the full-precision model truth value, denoted as GT.
[0020] S2. Initialize FP32 accuracy and performance: Use TensorRT's trtexec tool to convert the ONNX source model to FP32 full-precision model. Perform inference on the above samples to obtain the model inference accuracy and time consumption, which serve as the benchmark for FP32 full-precision model optimization, denoted as accuracy Prec_gt and time consumption Time_gt.
[0021] S3. Obtain the topology sequence: For a multi-input ONNX source model, obtain the topology sequence of each input and downstream node. The topology sequence is the sorting of network layers with connection dependencies. Obtain multiple topology sorting sequences layersequence, denoted as Lys_0, Lys_1, ..., Lys_i.
[0022] S4. Topology Sequence Block Division: For the topology sorted sequence, based on the principle of network layers from top to bottom, each topology sequence Lys_i is divided into blocks. The block size block_size and the number of blocks block_num are set. The multiple blocks of each topology sequence are denoted as Block_0, ..., Block_i. Here, block_size represents the number of FP16 network layers in each block, and block_num = N / block_size, where N is the total number of layers in the sequence, and block_num is rounded up.
[0023] S5. Perform iterative blending precision configuration optimization, including initializing intra-block blending precision, model precision and time consumption supervision, dynamically optimizing intra-block blending precision, inter-block blending precision, and dynamically adjusting the block size (block_size). S501. Initialize the mixed precision within the sequence block: For each group of topological sequences in step S4, process each sequence block Block in sequence, starting from Block_i (i=0,1,...,M-1, M is the total number of blocks), set the precision of all layers in Block_i to FP16, and the precision of all layers in the remaining blocks to FP32. Use TensorRT's trtexec tool to convert the mixed precision ONNX model to generate the mixed precision model Model_i. S502, Model Accuracy and Time Consumption Supervision: Supervising the accuracy Prec_i and time consumption Time_i of the mixed-precision model Model_i: If the precision Prec_i and the time consumption Time_i meet the threshold conditions, the network layer precision of the sequence block is set to a fixed FP16, and the process proceeds to the S504 sequence block mixing precision step to iterate the next sequence block. The precision threshold condition is: Prec_i - Prec_gt| ≤ δ, and the time consumption optimization condition is: Time_i ≤ Time_gt, where δ is a preset tolerance coefficient. If the accuracy Prec_i does not meet the threshold condition, then proceed to the S503 dynamic optimization of the mixing accuracy within the sequence block to optimize the mixing accuracy within the sequence block until the accuracy meets the standard. If the precision Prec_i meets the threshold condition and the time consumption Time_i does not decrease compared to Time_gt, then proceed to the S504 inter-block precision mixing step to iterate the next block. S503, Dynamically optimize the mixing precision within the sequence block: For the current sequence block of the current topology sequence, starting from the block_size layer, adjust it to FP32 one layer ahead. For each layer adjustment, perform a mixing precision model conversion and execute the model precision and time consumption supervision in S502 until the precision meets the requirements. Then, determine the mixing precision layer of the current sequence block and proceed to the mixing precision step between sequence blocks in S504 to iterate the next sequence block. S504, Inter-block Mixed Precision: For the next block of the current topology sequence, i.e., block_i+1, execute S501 to initialize the intra-block mixed precision. After model transformation, execute S502 to supervise model precision and time consumption. If the accuracy requirement is met, repeat the S504 sequence block mixing accuracy; If the accuracy does not meet the requirements, proceed to S503 for dynamic optimization of the mixing accuracy within the sequence block, and perform model accuracy and time monitoring again. If the accuracy meets the requirements, continue until all sequence blocks of the topology sequence are optimized, then proceed to S4 for topology sequence partitioning, and perform sequence block partitioning and mixing accuracy optimization for the next topology sequence. Once the mixed precision optimization of all topological sequences is completed, proceed to S6 to output the model result; If the precision does not meet the requirements, then switch to S505 to dynamically adjust the sequence block size (block_size). S505, Dynamically adjust the sequence block size block_size: with the initial value of block_size being 16, decrease it in powers of 2, go back to S4 topological sequence partitioning and re-partition, and finally find the optimal block_size based on the principle of obtaining the best accuracy and time consumption.
[0024] S6. Output model: After the optimization process in step S5, the target platform model that meets the requirements in terms of model accuracy and inference performance is obtained, which is the final model Model_end.
[0025] The above embodiments disclose the performance tuning method of the present invention, which has the following advantages compared with the prior art: 1. The performance tuning method of this invention is based on the standardized implementation of TensorRT, which is easy to deploy, is a general tuning strategy, automatically adjusts the block_size, and is suitable for various application scenarios and model types; 2. The performance tuning method of the present invention adopts hybrid precision configuration iteration of network topology search, and achieves dynamic decision-making through dual indicators of precision and time consumption, thereby reducing the overall model time and improving model performance; 3. This invention achieves hierarchical granularity optimization through topological sequence block partitioning, which is more refined than traditional global mixed precision methods; 4. This invention uses a topology sequence acquisition method, which can process multiple network paths in parallel and improve computational efficiency.
[0026] In another embodiment of the present invention, a precision performance tuning system based on network block topology sequences is disclosed. This system runs on an NVIDIA chip platform and includes: The benchmark building module is used to generate the truth value GT by performing FP32 full-precision inference through the ONNXRuntime library, and to obtain the FP32 benchmark precision Prec_gt and time consumption Time_gt through the trtexec tool of TensorRT. The topology analysis module is used to parse the inter-layer dependencies of the ONNX source model and output multiple sets of topology sequences Lys_0, Lys_1, ..., Lys_i; The block partitioning module is used to divide each group of topological sequences into sequence blocks of size block_size, Block_0, ..., Block_i, according to the top-down principle, where block_size is a power of 2; The hybrid precision optimization module iteratively performs the following operations until the precision and time consumption thresholds are met: Initialize the current sequence block to FP16 precision, and the remaining sequence blocks to FP32 precision; Dynamically adjust the number of FP16 layers within the sequence block, i.e., backtrack from the block_size layer to FP32 layer by layer; Inter-block precision synchronization optimization dynamically reduces block size; The output module is used to generate the optimized final model, Model_end.
[0027] In another embodiment of the invention, an electronic device is also included, which may include a processor and a memory storing instructions from a computer program.
[0028] Specifically, in this embodiment, the processor may include a central processing unit (CPU), a specific integrated circuit, or one or more integrated circuits that can be configured in this embodiment; the memory may include a mass storage device for data or instructions, including but not limited to a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these; where appropriate, the memory may include removable or non-removable (or fixed) media; in a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.
[0029] The processor described above implements the performance tuning method disclosed in this invention by reading and executing computer program instructions stored in the memory.
[0030] It should also be noted that the electronic device in this embodiment may further include a communication interface and a communication bus. The processor, memory, and communication interface are connected via the communication bus to complete communication between them. The communication interface is mainly used to realize communication between the various units, modules, devices, or equipment in this embodiment of the invention.
[0031] The aforementioned communication bus comprises hardware, software, or a combination of both, coupling components of online data traffic devices together. Where appropriate, the communication bus may include one or more buses.
[0032] In addition, in conjunction with the performance tuning method in the above embodiments, the embodiments of the present invention can be implemented by providing a computer storage medium, on which computer program instructions are stored; the computer program instructions are executed by a processor using the above performance tuning method.
[0033] It should be clarified that the present invention is not limited to the methods, systems, and devices disclosed above, but also includes various changes, modifications, and additions made by those skilled in the art based on the ideas of the present invention, or changes in the order of steps.
[0034] When implemented in hardware, this invention can be electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. These programs or code segments can be stored on a machine-readable medium or transmitted via a data signal carried on a carrier wave through a transmission medium or communication link. "Machine-readable medium" can include any medium capable of storing or transmitting information, such as electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, optical disks, hard disks, fiber optic media, radio frequency links, etc. The code segments can be downloaded via computer networks such as the Internet or intranets.
[0035] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention, as long as they do not depart from the spirit and scope of the technical solutions of the present invention, should be covered within the scope of the claims of the present invention.
Claims
1. A method for precision performance tuning based on network block topology sequences, characterized in that, This method runs on the NVIDIA chip platform and is implemented using Python scripts and the TensorRT tool. The specific steps include: S1, Full-precision module inference truth value: The ONNX source model is loaded in Python using the ONNXRuntime library, unlabeled image samples are input into the model, FP32 inference is performed and the output results are saved. The inference results are used as the full-precision model truth value GT. S2. Initialize FP32 accuracy and performance: Use TensorRT's trtexec tool to convert the ONNX source model to FP32 full-precision model, perform inference to obtain the model inference accuracy Prec_gt and time consumption Time_gt, which serve as the benchmark for FP32 full-precision model optimization. S3. Obtain the topology sequence: Extract the topology sequences Lys_0, Lys_1,..., Lys_i of multiple input ONNX source models and downstream nodes; S4. Topology Sequence Blocking: Based on the principle of top-down network layer division, each group of topology sequences is divided into blocks to obtain multiple sequence blocks Block_0, ..., Block_i for each group of topology sequences; S5. Perform iterative blending precision configuration optimization, including initializing intra-block blending precision, model precision and time consumption supervision, dynamically optimizing intra-block blending precision, inter-block blending precision, and dynamically adjusting the block size (block_size). S6. Output model: After the optimization process in step S5, the target platform model that meets the requirements in terms of model accuracy and inference performance is obtained, which is the final model Model_end.
2. The accuracy performance tuning method based on network block topology sequence according to claim 1, characterized in that: In step S3, the topology sequence is a sorting of network layers with connection dependencies, and multiple topology sorting sequences layersequence are obtained, denoted as Lys_0, Lys_1, ..., Lys_i.
3. The accuracy performance tuning method based on network block topology sequence according to claim 1, characterized in that: In step S4, the sequence block size block_size and the number of sequence blocks block_num are set, where block_size represents the number of FP16 network layers in each sequence block, and block_num = N / block_size, where N is the total number of layers in the sequence.
4. The accuracy performance tuning method based on network block topology sequence according to claim 3, characterized in that: In step S5, the iterative mixed precision configuration optimization includes the following steps: S501. Initialize mixed precision within sequence blocks: For each set of topological sequences, process each sequence block Block_i (i=0,1,..., M-1, M is the total number of blocks) in sequence, set the precision of all layers in Block_i to FP16, and the precision of all layers in the remaining blocks to FP32, and use TensorRT's trtexec tool to perform the transformation to generate a mixed precision model Model_i. S502, Model Accuracy and Time Consumption Supervision: Supervising the accuracy Prec_i and time consumption Time_i of the mixed-precision model Model_i: If the precision Prec_i and the time consumption Time_i meet the threshold conditions, the network layer precision of the sequence block is set to a fixed FP16, and the process proceeds to the S504 sequence block mixing precision step to iterate the next sequence block. The precision threshold condition is: Prec_i - Prec_gt| ≤ δ, and the time consumption optimization condition is: Time_i ≤ Time_gt, where δ is a preset tolerance coefficient. If the accuracy Prec_i does not meet the threshold condition, then proceed to the S503 dynamic optimization of the mixing accuracy within the sequence block to optimize the mixing accuracy within the sequence block until the accuracy meets the standard. If the precision Prec_i meets the threshold condition and the time consumption Time_i does not decrease compared to Time_gt, then proceed to the S504 inter-block precision mixing step to iterate the next block. The process continues until all blocks are optimized, at which point a mixed-precision model that meets both accuracy and performance targets is output.
5. The accuracy performance tuning method based on network block topology sequence according to claim 4, characterized in that: In step S503 above, the mixing precision within the sequence block is dynamically optimized: for the current sequence block of the current topology sequence, starting from the block_size layer, it is adjusted to FP32 one layer at a time. For each layer adjusted, a mixing precision model conversion is performed, and the model precision and time consumption supervision in S502 is executed until the precision meets the requirements. Then, the mixing precision layer of the current sequence block is determined, and the process proceeds to the mixing precision step between sequence blocks in S504 to iterate the next sequence block.
6. A method for precision performance optimization based on network block topology sequences according to claim 4 or 5, characterized in that: In step S504 above, the mixing accuracy between sequence blocks is as follows: For the next sequence block of the current topology sequence, S501 is executed to initialize the mixing accuracy within the sequence block, and after model transformation, S502 is executed to supervise model accuracy and time consumption. If the accuracy meets the requirements, repeat step S504 for mixing accuracy between sequence blocks; if the accuracy does not meet the requirements, proceed to step S503 for dynamic optimization of mixing accuracy within the sequence block, and perform model accuracy and time monitoring again. If the accuracy meets the requirements, continue until all sequence blocks of the topology sequence are optimized, then proceed to step S4 for topology sequence partitioning, and perform sequence block partitioning and mixing accuracy optimization for the next topology sequence. Once the mixed precision optimization of all topological sequences is completed, proceed to S6 to output the model result; If the time taken does not meet the requirements, the process switches to S505 to dynamically adjust the sequence block size (block_size).
7. The accuracy performance optimization method based on network block topology sequence according to claim 6, characterized in that: In step S505 above, the sequence block size block_size is dynamically adjusted: the initial value of block_size is 16, and it decreases by powers of 2. Then, the topological sequence is re-divided in step S4 until the optimal block_size is found.
8. A precision performance tuning system based on network block topology sequences according to any one of claims 1-7, characterized in that: Running on NVIDIA chip platforms, including: The benchmark building module is used to generate the truth value GT by performing FP32 full-precision inference through the ONNXRuntime library, and to obtain the FP32 benchmark precision Prec_gt and time consumption Time_gt through the trtexec tool of TensorRT. The topology analysis module is used to parse the inter-layer dependencies of the ONNX source model and output multiple sets of topology sequences Lys_0, Lys_1, ..., Lys_i; The block partitioning module is used to divide each group of topological sequences into sequence blocks of size block_size, Block_0, ..., Block_i, according to the top-down principle, where block_size is a power of 2; The hybrid precision optimization module iteratively performs the following operations until the precision and time consumption thresholds are met: Initialize the current sequence block to FP16 precision, and the remaining sequence blocks to FP32 precision; Dynamically adjust the number of FP16 layers within the sequence block, i.e., backtrack from the block_size layer to FP32 layer by layer; Inter-block precision synchronization optimization dynamically reduces block size; The output module is used to generate the optimized final model, Model_end.
9. An electronic device comprising a processor, a communication interface, a memory, and a communication bus, wherein, The processor, the communication interface, and the memory communicate with each other via the communication bus, wherein the memory is used to store computer programs; The processor is configured to perform the method of any one of claims 1-7 by running the computer program stored in the memory.
10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the method of any one of claims 1-7.