Static image mode pattern generation system and method supporting dynamic resolution

Abstract

The present disclosure relates to a static graph mode graph generation system supporting dynamic resolution. The system comprises: an original logic graph construction component, for a first specific graph generation task, tracking original computing logic by executing a construction function of a graph to construct an original logic graph; a logic graph optimization component, forming a first logic graph for the constructed original logic graph; a node insertion component, inserting a communication node and a runtime control node required by distributed computing for the first logic graph to generate a first runtime logic graph for multi-device collaboration; a runtime plan generation component, generating a first runtime execution plan based on a data dependency relationship of the first runtime logic graph; and a runtime initialization component, initializing a first specific graph runtime instance according to the first runtime execution plan to complete dynamic allocation of computing resources and configuration of an execution environment.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer information processing, and in particular, to a technical solution for supporting dynamic resolution in a static graph mode graph generation system and method. BACKGROUND

[0002] Diffusion models are a cutting-edge generation model that restores noisy images or videos to clear content step by step, achieving high-quality image and video generation. This technology is based on a reverse diffusion process, simulating the diffusion of noise from data and inversely inferring the original data to generate realistic visual content. By applying diffusion models, stylized images, highly realistic video clips, and even virtual environments with complex scenes and details can be created. Diffusion models not only have great potential in artistic creation, but also play an important role in medical image generation, autonomous driving simulation, augmented reality, and other fields. Its unique generation mechanism enables it to capture and reproduce delicate details and complex textures, providing a powerful tool for artificial intelligence-driven visual creation.

[0003] Static graph acceleration technology is an important means to improve model training and inference efficiency in deep learning. Unlike dynamic graphs, static graphs have fixed structures when the computation graph is defined, allowing for global optimization and acceleration during the execution phase. Through compiler technology, static graph acceleration can perform optimizations such as operation fusion, constant folding, and subgraph partitioning, reducing redundant calculations and memory access, and significantly improving execution speed. TensorFlow, PyTorch's TorchScript, and NVIDIA's TensorRT are typical static graph acceleration tools. Static graph acceleration is particularly important in large-scale deep learning tasks, effectively reducing training time and improving inference efficiency.

[0004] When using static graph acceleration technology to accelerate diffusion models, dynamic shape support is required. Diffusion models that generate images and videos generally have a variety of resolutions for the generated content (such as images). For example, "Vertical (9:16)" corresponds to a resolution of "768x1344", "Square (1:1)" corresponds to a resolution of "1024x1024", and "Widescreen (16:9)" corresponds to a resolution of "1365x768".

[0005] Stable diffusion is a diffusion model-based generative artificial intelligence technology that can generate high-quality images, videos, or animations according to text descriptions or image prompts. Its core is to transform random noise into visual content that meets user needs through a "step-by-step denoising" process. As an open-source model, it allows developers to freely modify, optimize, or integrate into other applications. Stable diffusion needs to support multi-shape inference in actual deployment, and the current method is to use one graph for each resolution (shape). Using multiple graphs has two problems. One is that constant folding will generate new tensors that are not shared between multiple graphs, resulting in redundant overhead. Each graph that has the same input but is different will do constant folding at compile time. Constant folding calculates constants that are unrelated to the input at compile time, so there is no need to calculate at runtime. The problem caused by this repeated compilation is that new tensors are generated, and if each graph generates its own new tensors, there will be a lot of redundant computation and storage between multiple graphs. The other is that multiple graphs take a long time to start and compile when they are online, causing customers to complain that the online machine has weak CPU and slow compilation. Therefore, people hope to compile offline and directly load it online. Because the prediction interface triggers compilation, it takes about 150 seconds to compile 9 graphs (6.5 seconds for each, a total of 58.5 seconds) and execute 9 input inferences in stable diffusion u-net, which results in a long wait.

[0006] Therefore, it is desirable to eliminate the redundancy of new tensors in the existing static graph mode image generation system for a large number of similar graph generation processes and to provide an offline compilation process.

[0007] The above information disclosed in the BACKGROUND section is only intended to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0008] In view of the above, the present disclosure provides a static graph mode image generation system that supports dynamic resolution and can at least eliminate the redundancy of new tensors.

[0009] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0010] According to an aspect of the present disclosure, a static graph mode graph generation system supporting dynamic resolution is provided, comprising: an original logical graph construction component, for a first specific graph generation task containing a first input resolution attribute, tracking original computing logic by executing a construction function of a graph to construct an original logical graph; a logical graph optimization component, for the constructed original logical graph, performing at least constant folding and automatic mixed precision processing to form a first logical graph; a node insertion component, for the first logical graph, inserting communication nodes and runtime control nodes required by distributed computing to generate a first runtime logical graph for multi-device collaboration; a runtime plan generation component, based on data dependency relationships of the first runtime logical graph, generating a first runtime execution plan containing at least computing resource configuration proportion and execution order of each execution body, task scheduling strategy, memory reuse strategy, and pipeline parallel strategy; and a runtime initialization component, according to the first runtime execution plan, initializing a first specific graph runtime instance to complete dynamic allocation of computing resources and configuration of an execution environment.

[0011] According to the static graph mode graph generation system supporting dynamic resolution of the present disclosure, when a second specific graph generation task containing a second input resolution attribute is received, the runtime plan generation component modifies the first runtime execution plan into a second runtime execution plan corresponding to the second specific graph generation task based on comparison differences between the second input resolution attribute and the first input resolution attribute, based on data dependency relationships of the first runtime logical graph; and the runtime initialization component initializes a second specific graph runtime instance according to the second runtime execution plan to complete dynamic allocation of computing resources and configuration of an execution environment.

[0012] According to the static graph mode graph generation system supporting dynamic resolution of the present disclosure, the runtime plan generation component modifies the first runtime execution plan into a second runtime execution plan corresponding to the second specific graph generation task by comparing the second input resolution attribute with the first input resolution attribute, only updating a differential logical graph part in the first runtime logical graph, enabling parameter mirror copying for the second specific graph generation task with the same hardware configuration, and performing parameter self-adaptive conversion for a heterogeneous device environment.

[0013] According to the static graph mode graph generation system supporting dynamic resolution of the present disclosure, the runtime initialization component modifies the first runtime execution plan into a second runtime execution plan corresponding to the second specific graph generation task, and creates the second runtime execution plan with a shared memory area of the first runtime execution plan.

[0014] According to the static graph mode graph generation system supporting dynamic resolution of the present disclosure, when the runtime initialization component compares the difference between the second input resolution attribute and the first input resolution attribute, it detects the same subgraph structure features between two or more specific graphs, verifies the compatibility of the device environment parameters with the first specific graph, and receives the user-specified parameter inheritance instruction.

[0015] According to another aspect of the present disclosure, a static graph mode graph generation method supporting dynamic resolution is provided, comprising: by an original logical graph construction component, generating a task for a first specific graph containing a first input resolution attribute, tracking the original computing logic by executing the construction function of the graph to construct an original logical graph; by a logical graph optimization component, at least performing constant folding and automatic mixed precision processing on the constructed original logical graph to form a first logical graph; by a node insertion component, inserting communication nodes and runtime control nodes required for distributed computing into the first logical graph to generate a first runtime logical graph for multi-device collaboration; by a runtime plan generation component, generating a first runtime execution plan based on the data dependency relationship of the first runtime logical graph, the first runtime execution plan at least containing the computing resource configuration proportion and execution order of each execution body, task scheduling strategy, memory reuse strategy, and pipeline parallel strategy; and by a runtime initialization component, initializing the first specific graph runtime instance according to the first runtime execution plan, completing dynamic allocation of computing resources and execution environment configuration.

[0016] According to the static graph mode graph generation method supporting dynamic resolution of the present disclosure, when the runtime initialization component compares the difference between the second input resolution attribute and the first input resolution attribute, it detects the same subgraph structure features between two or more specific graphs, verifies the compatibility of the device environment parameters with the first specific graph, and receives the user-specified parameter inheritance instruction.

[0017] According to the static graph mode graph generation method supporting dynamic resolution of the present disclosure, the runtime plan generation component modifies the first runtime execution plan into a second runtime execution plan corresponding to the second specific graph generation task based on the comparison difference between the second input resolution attribute and the first input resolution attribute, and based on the data dependency relationship of the first runtime logical graph; and the runtime initialization component initializes the second specific graph runtime instance according to the second runtime execution plan, completes dynamic allocation of computing resources and execution environment configuration.

[0018] According to the static graph mode graph generation method supporting dynamic resolution of the present disclosure, the runtime initialization component modifies the first runtime execution plan into a second runtime corresponding to a second specific graph generation task, and creates a second runtime execution plan having a shared memory area with the first runtime execution plan.

[0019] According to the static graph mode graph generation method supporting dynamic resolution of the present disclosure, when the runtime initialization component compares the difference between the second input resolution attribute and the first input resolution attribute, it detects the same subgraph structure features between two or more specific graphs, verifies the compatibility of the device environment parameters with the first specific graph, and receives the user-specified parameter inheritance instruction.

[0020] According to the static graph mode graph generation system and method supporting dynamic resolution of the present disclosure, by changing the inference logic based on the graph resolution (shape) without changing the entire optimized graph structure, the resolution and attributes of the changed graph can be adjusted, a large amount of repeated compilation time can be saved, and the result of memory reuse can be achieved. Therefore, the traditional mixed static graph mode graph generation method is divided into multiple stages that can be independently called, so that the generation processing of graphs with different resolutions shares the achievements of the early stages, and the later stages of the static graph mode graph generation can be compiled based on the shared early stages for specific graphs, eliminating the redundancy of new tensors. More importantly, since the compilation for the first specific graph can be compiled offline, the waiting time for online compilation is eliminated, providing a good user experience.

[0021] It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. The drawings described below are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.

[0023] Figure 1 is a block diagram of an embodiment of a static graph mode graph generation system supporting dynamic resolution according to the present disclosure according to an exemplary embodiment.

[0024] Figure 2 is a flowchart of a static graph mode graph generation method supporting dynamic resolution according to the present disclosure according to an exemplary embodiment. DETAILED DESCRIPTION

[0025] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0026] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0027] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0028] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0029] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first computing device discussed below may be referred to as the second computing device without departing from the teachings of this disclosure. As used herein, the term "and / or" includes all combinations of any and more of the associated listed items.

[0030] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of exemplary embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing this disclosure, and therefore cannot be used to limit the scope of protection of this disclosure.

[0031] Figure 1 yes Figure 1 This is a block diagram illustrating an embodiment of a static graph pattern generation system 100 supporting dynamic resolution according to an exemplary embodiment of the present disclosure. Figure 1As shown, the static graph mode graph generation system 100 that supports dynamic resolution includes an original logical graph construction component 110, a logical graph optimization component 120, a node insertion component 130, a run plan generation component 140, and a runtime initialization component 150. This is to split the traditional hybrid integrated static graph mode graph generation method into multiple stages that can be independently called, so that the generation process of graphs of different resolutions does not need to regenerate new tensors due to constant folding, and makes the later stages of static graph mode graph generation based on the shared early stages be able to compile for specific graphs, eliminating the redundancy of new tensors.

[0032] Specifically, for the first first specific graph generation task, the original logical graph construction component 110 constructs an original logical graph by executing a graph construction function for the first specific graph generation task containing the first input resolution attribute. Specifically, when the static graph mode graph generation system 100 runs to the build function, the system scans the deep learning model code (such as PyTorch or TensorFlow code) written by you line by line. At this time, the calculation is not really performed, but the connection relationship and parameter setting of each operation step are recorded to form the original logical graph of the first specific graph generation task. This process mainly includes establishing a flowchart of the calculation steps, labeling dynamic changing parameters (such as image resolution) for subsequent processing, and locking the operation type and connection mode of each step to avoid errors caused by temporary modification of the structure at runtime. Tracing the original calculation logic is to simulate running through the calculation steps, which involves parameter dimension determination, such as whether the convolution kernel size is 3x3 or 5x5; data flow: label the input and output ports of each operation node; dynamic labeling: draw a question mark for the variable parameters (such as image size), indicating that it will be filled in later (for example, if it is found that the output size of a layer will change according to the input image size, a question mark will be marked in the graph [?, 256] (the question mark represents a dynamic dimension), which provides an entry for subsequent sharing). Therefore, the core function of this step is to generate a complete calculation step specification, which labels all key information, reserves modification entries for variable parameters, and can be flexibly adjusted later (such as changing from 128x128 graph to 512x512 graph), and discovers structural contradictions in advance (such as the output size of a layer cannot be used as the input of the next layer), etc. For example, if the deep learning model code contains conv and relu operations, the construction function will mark the conv and relu operations and record their front and back connection relationship. This construction process can be performed in a conventional manner, and will not be repeated here.

[0033] Subsequently, the logical graph optimization component 120 performs at least constant folding and automatic mixed precision processing on the constructed original logical graph to form a first logical graph. Specifically, the original logical graph is rewritten to generate an optimized logical graph. As mentioned above, constant folding calculates constants independent of input at compile time, so that runtime does not need to calculate, which is a means of reducing runtime overhead. Automatic mixed precision is also a means of reducing computation. Automatic mixed precision is a deep learning training optimization technique that intelligently mixes different precision floating point numbers (such as FP16 and FP32), automatically selects FP16 / FP32 calculation mode according to hardware computing power, and improves training efficiency and reduces memory occupation under the premise of maintaining model accuracy. Using constant folding optimization and automatic mixed precision optimization strategies, the operator topology of the original logical graph is rewritten to generate a memory occupation reduction of 20%-40%. The optimization process can also merge convolutional layers and activation function nodes based on operator fusion rules, use tensor slicing technology to optimize memory access paths, perform operator fusion optimization on static graph topology, and optimize memory access mode according to hardware characteristics.

[0034] The node insertion component 130 inserts communication nodes and runtime control nodes required for distributed computing into the first logical graph to generate a first runtime logical graph for multi-device collaboration. In simple terms, the optimized logical graph is completed to generate a runtime logical graph, such as inserting special ops required for oneflow runtime execution, AllReduce nodes related to NCCL communication protocol, adding IO control nodes for data loading and result output, etc., such as Tick control nodes for coordinating execution timing. This insertion process is based on the need. Through this process, a logical graph that can run is formed. Completing the optimized logical graph can also be used for NCCL communication node groups for multi-card data synchronization, Tick / Tock node pairs for controlling the execution timing of the computing flow, and DMA control nodes for CPU-GPU data transfer.

[0035] Upon receiving a second specific graph generation task containing a second input resolution attribute, the runtime plan generation component 140 modifies the first runtime execution plan to a second runtime execution plan corresponding to the second specific graph generation task based on a comparison difference between the second input resolution attribute and the first input resolution attribute based on the data dependency relationship of the first runtime logical graph; and the runtime initialization component initializes the second specific graph runtime instance according to the second runtime execution plan, completes dynamic allocation of computing resources and configuration of execution environment.

[0036] The runtime plan generation component 140 modifies the first runtime execution plan into a second runtime execution plan corresponding to the second specific graph generation task, updates only the differential logical graph part in the first runtime logical graph through comparison of the second input resolution attribute and the first input resolution attribute, enables parameter mirror copying for the second specific graph generation task with the same hardware configuration, and performs parameter adaptive conversion for a heterogeneous device environment.

[0037] Subsequently, the runtime plan generation component 140 generates a first runtime execution plan based on the data dependency relationship of the first runtime logical graph, wherein the first runtime execution plan at least includes the computing resource configuration proportion and execution order of each execution body, the task scheduling strategy, the memory reuse strategy, and the pipeline parallel strategy. For example, it can also include the determination of execution priority based on DAG topological sorting, load balancing distribution according to device computing capacity, and generation of a resource allocation table including a memory reuse strategy. When generating the first runtime execution plan, the priority queue of task preemption scheduling can also be supported based on the elastic memory allocation table of the dynamic resolution parameter, and the topology configuration file of the inter-device communication bandwidth is recorded.

[0038] Finally, the runtime initialization component 150 initializes the first specific graph runtime instance according to the first runtime execution plan, completes the dynamic allocation of computing resources and the configuration of the execution environment. In simple terms, the device allocation instructions in the execution plan are parsed, the distributed communication group is initialized, the GPU memory resources are dynamically allocated according to the execution plan, the cross-device communication link group is established, the memory buffer is pre-allocated and the pipeline execution channel is established, the deployment of the computing graph in the heterogeneous hardware environment is completed, and the like.

[0039] The runtime initialization component 150 modifies the first runtime execution plan into a second runtime execution plan corresponding to the second specific graph generation task, and creates a second runtime execution plan with a shared memory area with the first runtime execution plan. When comparing the difference between the second input resolution attribute and the first input resolution attribute, the runtime initialization component 150 detects the same subgraph structure features between two or more specific graphs, verifies the compatibility of the device environment parameters with the first specific graph, and receives the user-specified parameter inheritance instruction.

[0040] In summary, since the graph generation process can be directly performed by the runtime initialization component 150 and the runtime initialization component 150 based on the input parameters in the subsequent graph generation process, the static graph mode graph generation system 100 can perform the re-reasoning of the blob desc according to the new input resolution (input shape), that is, when the input data size changes (such as the picture changes from 256x256 to 512x512), the system will automatically recalculate the memory specification (blob description) of the related data container to ensure that the new size of the data can be accommodated, that is, to build a dimension parameterized symbolic tensor representation system, to establish a dynamic shape derivation rule library at the compilation period, and to dynamically adjust the memory allocation strategy at the runtime according to the input data. In short, the constant folding parameters in the shared optimization logic graph of the first specific graph generation task are shared; the AMP mixed precision configuration strategy of the first specific graph generation task is inherited; and the inserted runtime node topology structure is reused. In the sharing process, only the differential logic graph part is updated through version comparison; the parameter mirror copy is enabled for the graph with the same hardware configuration; and the parameter adaptive conversion is performed for the heterogeneous device environment.

[0041] Figure 2 is a flow chart of a static graph mode graph generation method supporting dynamic resolution according to the present disclosure according to an exemplary embodiment. As shown in Figure 2As shown, at step S210, the first concrete graph generation task is generated for the first specific graph containing the first input resolution attribute by the original logic graph building component 110, and the original computation logic is tracked to build the original logic graph by executing the build function of the graph. Specifically, when the static graph mode graph generation system 100 runs to the build function, the system scans the deep learning model code (such as PyTorch or TensorFlow code) written by you line by line. At this time, the calculation is not really performed, but the connection relationship and parameter setting of each operation step are recorded to form the original logic graph of the first concrete graph generation task. This process mainly includes establishing a flowchart of the calculation steps, labeling dynamic parameters (such as image resolution) for subsequent processing, and locking the operation type and connection mode of each step to avoid errors caused by temporary modification of the structure at runtime. Tracking the original computation logic is to simulate running through the calculation steps, which involves parameter dimension determination, such as whether the convolution kernel size is 3x3 or 5x5; data flow: marking the input and output ports of each operation node; dynamic labeling: marking the parameters that will change (such as image size) with a question mark to indicate that it will be filled in later (for example, if it is found that the output size of a certain layer will change according to the input image size, [?, 256] will be marked in the graph (? represents dynamic dimension), which provides an entry for subsequent sharing). Therefore, the core function of this step is to generate a complete calculation step specification, which labels all key information, reserves modification entries for variable parameters, and can be flexibly adjusted later (such as changing from 128x128 images to 512x512 images), and can also find structural contradictions in advance (such as the output size of a certain layer cannot be used as the input of the next layer). For example, if the deep learning model code contains conv and relu operations, the build function will mark the conv and relu operations and record their front and back connection relationship. This construction process can be performed in a conventional manner and will not be described here.

[0042] Subsequently, at step S220, the logical graph optimization component 120 performs at least constant folding and automatic mixed precision processing on the constructed original logical graph to form a first logical graph. Specifically, the original logical graph is rewritten to generate an optimized logical graph. As mentioned previously, constant folding calculates constants that are independent of input at compile time, so that the runtime does not need to calculate, which is a means of reducing runtime overhead. Automatic mixed precision is also a means of reducing computation. Automatic mixed precision is a deep learning training optimization technique that intelligently mixes different precision floating-point numbers (such as FP16 and FP32), automatically selects FP16 / FP32 calculation modes according to hardware computing power, improves training efficiency and reduces memory occupancy under the premise of maintaining model accuracy. Using constant folding optimization and automatic mixed precision optimization strategies, the operator topology structure of the original logical graph is rewritten to generate an optimized logical graph with a 20%-40% reduction in memory occupancy. The optimization process can also merge convolutional layers and activation function nodes based on operator fusion rules, optimize memory access paths using tensor slicing techniques, optimize operator fusion for static graph topology structures, and optimize memory access modes based on hardware characteristics.

[0043] Next, at step S230, the node insertion component 13030 inserts communication nodes and runtime control nodes required for distributed computing into the first logical graph to generate a first runtime logical graph for multi-device collaboration. In other words, the optimized logical graph is completed to generate a runtime logical graph, such as inserting special ops required for oneflow runtime execution, AllReduce nodes related to NCCL communication protocols, adding IO control nodes for data loading and result output, and Tick control nodes for coordinating execution timing. This insertion process is based on the need. Through this process, a logical graph that can be run is formed. Completing the optimized logical graph can also be used for NCCL communication node groups for multi-card data synchronization, Tick / Tock node pairs for controlling the execution timing of the computing flow, and DMA control nodes for CPU-GPU data transfer.

[0044] Subsequently, at step S240, the runtime plan generation component 140 generates a first runtime execution plan based on the data dependency relationship of the first runtime logical graph, which at least includes the computing resource configuration proportion and execution order of each execution body, task scheduling strategy, memory reuse strategy, and pipeline parallel strategy. For example, it can also include determining execution priority based on DAG topological sorting, load balancing allocation according to device computing capacity, generating a resource allocation table containing a memory reuse strategy. When generating the first runtime execution plan, the priority queue of task preemption scheduling can also be supported based on the elastic memory allocation table of the dynamic resolution parameter, and the topology configuration file of the communication bandwidth between devices is recorded.

[0045] Finally, at step S250, the runtime initialization component 150 initializes the first concrete graph runtime instance according to the first runtime execution plan, completes the dynamic allocation of computing resources and the configuration of the execution environment. In simple terms, the device allocation instructions in the execution plan are parsed, the distributed communication group is initialized, the GPU memory resources are dynamically allocated according to the execution plan, the communication link group across devices is established, the memory buffer is pre-allocated and the pipeline execution channel is established, and the deployment of the computation graph in the heterogeneous hardware environment is completed.

[0046] Finally, in the case of a new graph generation task, return to step S240 for execution until all graph generation tasks are completed.

[0047] In summary, according to the static graph mode graph generation system and method supporting dynamic resolution of the present disclosure, by changing the inference logic based on the graph resolution (shape) without changing the entire optimized graph structure, the resolution and attributes of the changed graph are adjusted, a large amount of repeated compilation time is saved, and the memory is reused, so that the traditional hybrid static graph mode graph generation method is divided into multiple stages that can be independently called, the generation processing of graphs of different resolutions shares the achievements of the early stages, the later stages of the static graph mode graph generation are based on the shared early stages and can be compiled for specific graphs, and the redundancy of new tensors is eliminated. More importantly, since the compilation for the first specific graph can be compiled offline, the waiting time for online compilation is eliminated, and a good user experience is provided.

[0048] Those skilled in the art can understand that the above-mentioned modules can be distributed in the device according to the description of the embodiments, and can also be changed in one or more devices different from the embodiments. The modules of the above-mentioned embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0049] Through the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a plurality of instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

[0050] The exemplary embodiments of this disclosure are specifically illustrated and described herein. But, it is to be understood that the disclosure is not limited to the detailed construction, arrangements, or implementation methods described herein; rather, the disclosure is intended to cover any and all modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A static graph pattern generation system supporting dynamic resolution, comprising: The original logic graph construction component, for a first specific graph generation task containing a first input resolution attribute, traces the original computational logic to construct the original logic graph by executing the graph construction function; The logic graph optimization component performs at least constant folding and automatic mixed precision processing on the original logic graph to form a first logic graph. The node insertion component inserts communication nodes and runtime control nodes required for distributed computing into the first logical graph, generating a first runtime logical graph for multi-device collaboration. The runtime plan generation component generates a first runtime execution plan based on the data dependencies of the first runtime logic graph. The first runtime execution plan includes at least the computing resource configuration ratio and execution order of each executor, task scheduling strategy, memory reuse strategy, and pipeline parallel strategy. as well as The runtime initialization component initializes the first specific graph runtime instance according to the first runtime execution plan, and completes the dynamic allocation of computing resources and the configuration of the execution environment.

2. The static graph pattern generation system supporting dynamic resolution according to claim 1, wherein, Upon receiving a second specific graph generation task containing a second input resolution attribute, the runtime plan generation component modifies the first runtime execution plan to a second runtime execution plan corresponding to the second specific graph generation task based on the comparison difference between the second input resolution attribute and the first input resolution attribute, and based on the data dependency relationship of the first runtime logic graph; and the runtime initialization component initializes the runtime instance of the second specific graph according to the second runtime execution plan, thereby completing the dynamic allocation of computing resources and the configuration of the execution environment.

3. The static graph pattern generation system supporting dynamic resolution according to claim 2, wherein, The runtime plan generation component modifies the first runtime execution plan into a second runtime corresponding to the second specific graph generation task. The execution plan updates only the differentiated logic graph part of the first runtime logic graph by comparing the second input resolution attribute with the first input resolution attribute. It also enables parameter mirroring for the second specific graph generation task with the same hardware configuration and performs parameter adaptive conversion for heterogeneous device environments.

4. The static graph pattern generation system supporting dynamic resolution according to claim 2, wherein, The runtime initialization component modifies the first runtime execution plan into a second runtime corresponding to the second specific graph generation task, and creates a second runtime execution plan that shares a memory area with the first runtime execution plan.

5. The static graph pattern generation system supporting dynamic resolution according to claim 2, wherein, When comparing the difference between the second input resolution attribute and the first input resolution attribute, the runtime initialization component detects the same subgraph structure features between two or more specific graphs, verifies the compatibility between the device environment parameters and the first specific graph, and receives the parameter inheritance instruction specified by the user.

6. A method for generating static graph pattern diagrams that supports dynamic resolution, comprising: The original logic graph is constructed by building components from the original logic graph. For the first specific graph generation task containing the first input resolution attribute, the original computational logic is traced to construct the original logic graph by executing the graph construction function. By using the logic graph optimization component, at least constant folding and automatic mixed precision processing are performed on the original logic graph to form the first logic graph; By inserting a node component, communication nodes and runtime control nodes required for distributed computing are inserted into the first logical graph to generate a first runtime logical graph for multi-device collaboration. The runtime execution plan is generated by the runtime plan generation component based on the data dependencies of the first runtime logic graph. The first runtime execution plan includes at least the computing resource configuration ratio and execution order of each executor, task scheduling strategy, memory reuse strategy and pipeline parallel strategy. as well as By initializing the component at runtime, the first specific graph runtime instance is initialized according to the first runtime execution plan, thereby completing the dynamic allocation of computing resources and the configuration of the execution environment.

7. The method for generating static graph pattern diagrams supporting dynamic resolution according to claim 6, wherein, Upon receiving a second specific graph generation task containing a second input resolution attribute, the runtime plan generation component modifies the first runtime execution plan to a second runtime execution plan corresponding to the second specific graph generation task based on the comparison difference between the second input resolution attribute and the first input resolution attribute, and based on the data dependency relationship of the first runtime logic graph; and the runtime initialization component initializes the runtime instance of the second specific graph according to the second runtime execution plan, thereby completing the dynamic allocation of computing resources and the configuration of the execution environment.

8. The method for generating static graph pattern diagrams supporting dynamic resolution according to claim 7, wherein, The runtime plan generation component modifies the first runtime execution plan into a second runtime corresponding to the second specific graph generation task. The execution plan updates only the differentiated logic graph part of the first runtime logic graph by comparing the second input resolution attribute with the first input resolution attribute. It also enables parameter mirroring for the second specific graph generation task with the same hardware configuration and performs parameter adaptive conversion for heterogeneous device environments.

9. The method for generating static graph pattern diagrams supporting dynamic resolution according to claim 7, wherein, The runtime initialization component modifies the first runtime execution plan into a second runtime corresponding to the second specific graph generation task, and creates a second runtime execution plan that shares a memory area with the first runtime execution plan.

10. The method for generating static graph pattern diagrams supporting dynamic resolution according to claim 7, wherein, When comparing the difference between the second input resolution attribute and the first input resolution attribute, the runtime initialization component detects the same subgraph structure features between two or more specific graphs, verifies the compatibility between the device environment parameters and the first specific graph, and receives the parameter inheritance instruction specified by the user.

Images