Region-aware neural network hybrid graph execution optimization method

Abstract

The invention discloses a region-aware neural network hybrid graph execution optimization method. The method comprises the following steps: step S1, neural network model analysis and region division; s2, constructing and optimizing a static region calculation graph; s3, generating a dynamic region execution path; and S4, executing the mixed graph and integrating the result. By automatically dividing the dynamic and static areas and supporting efficient deployment of the AI in the dynamic scene, the defects of insufficient global dynamic graph optimization and poor global static graph adaptability in the prior art are overcome, the problem that manual explicit division depends on artificial experience and high-complexity operation is avoided, balance between performance and flexibility is achieved, and the method is suitable for large-scale popularization and application. Meanwhile, hardware resource utilization is optimized, the calculation cost is reduced, and green calculation is assisted.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence model optimization, and relates to a method for optimizing the execution of a computational graph, specifically to a method for optimizing the execution of a computational graph in a deep learning framework for dynamic input shape scenarios. Background Art

[0002] With the rapid development of deep learning technology, neural network models have achieved remarkable results in the fields of computer vision, natural language processing, autonomous driving, etc. As the core tool for supporting the development and deployment of neural network models, deep learning frameworks usually use a computational graph to describe the computational process of the model. According to the construction and execution methods of the computational graph, the existing technologies are mainly divided into two modes: static graph and dynamic graph.

[0003] In the static graph mode, a complete computational graph is predefined and compiled before training or inference. Users need to explicitly specify the network structure, connection method, and input and output shapes. The framework then improves the running efficiency through techniques such as operation fusion, memory optimization, and parallel computing. This mode performs well in scenarios with fixed input shapes, such as convolutional neural networks (CNNs) processing images with fixed resolutions, or recurrent neural networks (RNNs) processing sequence data with fixed lengths. Typical static graph frameworks such as early TensorFlow (version 1.x) and Caffe are widely used in large-scale distributed training and industrial deployment due to their high efficiency and stability. However, the static graph lacks flexibility. Once compiled, it is difficult to adapt to dynamic changes in the input shape or network structure during runtime, resulting in low efficiency or even inapplicability when dealing with dynamic shape scenarios such as variable-length sequences or heterogeneous data.

[0004] In the dynamic graph mode, the computational graph is constructed instantaneously according to the input data during runtime, which has high flexibility and is suitable for rapid iterative development and dynamic shape scenarios, such as processing variable-length sentences in natural language processing or multi-resolution inputs in image processing. Representative frameworks include PyTorch and the Eager Execution mode of TensorFlow 2.0, which improve the debugging efficiency by executing operations instantaneously. However, due to the lack of pre-compilation and global optimization, the performance of the dynamic graph is usually inferior to that of the static graph in large-scale inference tasks with fixed shapes. In practical applications, many scenarios involve both fixed shape and dynamic shape requirements, such as the mixed processing of fixed-resolution images and variable-length sensor data in autonomous driving, or the combination of fixed embedding layers and variable-length sentences in natural language processing. A single mode is difficult to balance efficiency and flexibility.

[0005] To make up for the deficiencies of a single mode, existing technologies have attempted to use a hybrid between the two modes. However, the hybrid execution mechanism of existing frameworks relies on manual prediction of dynamic boundaries. Developers need to manually mark static code blocks and handle conversion and optimization issues during switching. The marking itself poses a certain threshold for developers, which may lead to insufficiently fine or overly conservative or even incorrect granularity division of the computational graph, causing some areas that could have been statically optimized to still be regarded as dynamic, and areas that should have been dynamically optimized to be regarded as static, thus reproducing the defects of a single execution mode and reducing the overall execution efficiency. This undoubtedly increases the complexity of development. These problems make it difficult for existing technologies to achieve the best balance between running efficiency and flexibility in dynamic shape scenarios, restricting the performance of neural network models in diverse applications. Therefore, there is an urgent need for a new method that can automatically sense and optimize the execution of computational graphs in dynamic shape scenarios to improve the execution efficiency and adaptive capabilities of neural network models in complex applications and provide strong support for the further development of deep learning frameworks. Summary of the Invention

[0006] To solve the problems of low running efficiency and insufficient flexibility of existing technologies when dealing with dynamic shape scenarios, the present invention provides a region-aware neural network hybrid graph execution optimization method. This method automatically divides the computational graph of a neural network model into static regions with fixed shapes and dynamic regions with non-fixed shapes, and adopts a hybrid graph execution strategy to achieve the best balance between running efficiency and flexibility in dynamic shape scenarios and avoid potential risks brought by manual division. The present invention improves efficiency through compilation optimization of static regions, and flexible execution of dynamic regions avoids frequent compilation, overcomes the defects of existing technologies, supports the efficient deployment of dynamic shape scenarios such as autonomous driving and medical imaging, and reduces computational costs and development cycles.

[0007] The object of the present invention is achieved through the following technical solutions:

[0008] A region-aware neural network hybrid graph execution optimization method, comprising the following steps:

[0009] Step S1, neural network model parsing and region division:

[0010] Obtain the complete definition of the neural network model to be optimized, parse its computational logic into an initial computational graph, and divide the computational graph into static regions with fixed shapes and dynamic regions with non-fixed shapes through graph structure and operation semantics analysis and symbolic inference technology to generate a region marking table; the specific steps are as follows:

[0011] Step S11, parse the computational logic of the model into an initial computational graph composed of operation nodes and data dependency relationships between nodes;

[0012] Step S12: Based on the type and topological structure of the operation nodes, perform a preliminary partition, and combine the shape dependencies propagated along the computational graph from the input nodes to generate a temporary marking table;

[0013] Step S13: Use symbolic inference technology to assign symbolic variables to the tensors in the computational graph, propagate symbolic expressions, verify and adjust the preliminary partition results, generate a region marking table, mark the region belonging of each node as "static" or "dynamic", and record the operation type and dependencies;

[0014] Step S2: Static region computational graph construction and optimization:

[0015] Extract the operation nodes in the static region from the region marking table, construct an independent static computational graph, and perform compilation optimization on it to generate an executable static computational graph file; the specific steps are as follows:

[0016] Step S21: Filter static nodes from the region marking table, retain their original operation definitions and data flow dependencies, and construct a static computational graph;

[0017] Step S22: Perform compilation optimization on the static computational graph, including operator fusion, memory reuse, and parallel computing scheduling;

[0018] Step S23: Save the optimized static computational graph as an executable file and define its input and output interfaces;

[0019] Step S3: Dynamic region execution path generation:

[0020] Extract the operation nodes in the dynamic region from the region marking table, generate a dynamic execution path, and organize it in the form of a callable function; the specific steps are as follows:

[0021] Step S31: Extract dynamic nodes from the region marking table, retain their original operation logic and dependencies, and generate a dynamic execution sequence;

[0022] Step S32: Introduce control flow processing logic in the dynamic region to adapt to shape changes;

[0023] Step S33: Store the dynamic execution path in the form of a callable function to ensure that its output tensor can be docked with the input interface of the static region;

[0024] Step S4: Hybrid graph execution and result integration:

[0025] According to the dependencies in the region marking table and the topological sorting of the computational graph, integrate the static computational graph and the dynamic execution path, sequentially execute the static region and the dynamic region, collect and integrate the output tensors to form the final result, and the specific steps are as follows:

[0026] Step S41: Determine the execution order of the static region and the dynamic region according to the dependency relationship in the region marking table and the topological sorting of the computational graph.

[0027] Step S42: Load the static computational graph file, run the static region, and obtain the intermediate output tensors.

[0028] Step S43: Pass the intermediate output tensors into the dynamic execution path, run the dynamic region, and generate subsequent intermediate results or the final output.

[0029] Step S44: Collect all output tensors and integrate them into the final prediction result.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] By automatically dividing the dynamic and static regions, the present invention supports the efficient deployment of AI in dynamic scenarios, overcomes the deficiencies of insufficient global dynamic graph optimization and poor adaptability of global static graphs in the prior art, avoids the problems of manual explicit division relying on manual experience and high-complexity operations, achieves a balance between performance and flexibility, optimizes the utilization of hardware resources at the same time, reduces the computing cost, and contributes to green computing. Description of the Drawings

[0032] Figure 1 It is a flowchart of the method for executing a hybrid graph of a region-aware neural network;

[0033] Figure 2 It is a schematic diagram of neural network model parsing and region division;

[0034] Figure 3 It is a schematic diagram of constructing and optimizing a computational graph for the static region;

[0035] Figure 4 It is a schematic diagram of generating an execution path for the dynamic region. Detailed Embodiments

[0036] The technical solutions of the present invention will be further described below in conjunction with the drawings, but are not limited thereto. Any modification or equivalent replacement of the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention shall be covered by the protection scope of the present invention.

[0037] The present invention provides an optimization method for executing a hybrid graph of a region-aware neural network, as Figure 1 shown. The method includes the following steps:

[0038] Step S1: Neural network model parsing and region division:

[0039] Obtain the complete definition of the neural network model to be optimized, parse its computational logic into an initial computational graph, and divide the computational graph into static regions with fixed shapes and dynamic regions with non-fixed shapes through graph structure and operation semantics analysis as well as symbolic inference techniques to generate a region marking table.

[0040] In this step, obtaining the complete definition of the neural network model to be optimized is usually provided in the form of Python code, such as model descriptions based on frameworks like PyTorch, TensorFlow, or MindSpore. Parse the computational logic of the model into an initial computational graph, which consists of a series of operation nodes (such as convolution, pooling, fully connected, etc.) and the data dependencies between the nodes. The parsing process can be achieved through the computational graph generation tools provided by the framework. Next, through the combination of graph structure and operation semantics analysis as well as symbolic inference techniques, perform region division on the computational graph to distinguish static regions with fixed shapes and dynamic regions with non-fixed shapes, such as Figure 2As shown. The specific operation starts from the initial computational graph and makes a preliminary division based on the type and topological structure of the operation nodes: For each node, check its operation type. For example, convolutional layers and pooling layers tend to be classified into the static region because the convolution kernel or pooling window size is fixed, while adaptive pooling layers tend to be classified into the dynamic region because the output shape may depend on the dynamic dimensions of the input data. At the same time, propagate the shape dependencies along the topological order of the computational graph from the input node (which can be achieved through the depth-first search algorithm). If the input of a node comes from a node with a fixed shape, then this node is more likely to belong to the static region. If the input directly depends on external dynamic inputs, it tends to be in the dynamic region. This process can be achieved by using torch.fx in PyTorch to extract the operation type and parameter information of the nodes, or by using tf.Graph in TensorFlow to read the node attributes, generating a temporary label table to record the region tendency of each node and its dependencies. To ensure the accuracy of the division, further use symbolic inference to verify the preliminary results: Assign symbolic variables to the tensors in the computational graph (such as batch_size = N, channels = C, height = H, width = W), and propagate the symbolic expressions along the computational graph starting from the input layer. For example, if the input tensor is (N, C, H, W), after passing through a convolutional layer (convolution kernel 3×3, stride 1, no padding), the output shape is (N, C_out, H - 2, W - 2). Then check the shape expressions of each node to determine whether its dimensions are fixed values or variables. If all dimensions are constants (such as C_out is determined by the convolution kernel, and H - 2 is derived from the fixed input), then confirm that this node belongs to the static region. If it contains variables (such as N or H depends on external inputs), then confirm that it belongs to the dynamic region. If the preliminary division is inconsistent with the symbolic inference result (for example, a fully connected layer is misjudged as dynamic due to fixed weights), then adjust the node attribution according to the symbolic inference. Finally, combining the preliminary division and the symbolic inference results, generate a region label table, clearly mark the region attribution of each node as "static" or "dynamic", and record its operation type (such as "Conv2D", "FC") and dependencies (such as the input node number). The label table is stored in tabular form for subsequent steps to call.

[0041] Step S2, Construction and Optimization of the Static Region Computational Graph:

[0042] Extract the operation nodes in the static region from the region label table, construct an independent static computational graph, and perform compilation optimization on it to generate an executable static computational graph file.

[0043] In this step, for the operation nodes marked as the static region in step S1, extract their sub-computational graphs and construct an independent static computational graph, such as Figure 3As shown in the figure. Filter out all static nodes from the region marking table, retain their original operation definitions (such as convolution parameters, activation functions, etc.) and their data flow dependencies, and use a static graph construction tool (such as tf.function in TensorFlow or the static graph compiler in MindSpore) to convert them into the static computation graph format. After construction, perform compilation optimization on the static computation graph. The optimization content includes but is not limited to: operator fusion (combining consecutive operations into a single efficient operation, such as fusing convolution and activation), memory reuse (analyzing the data life cycle and reusing the memory space of intermediate results), and parallel computing scheduling (adjusting the operation execution order according to hardware characteristics, such as thread allocation on GPU). The specific implementation of the optimization can rely on the built-in compiler of the framework, such as XLA (Accelerated Linear Algebra) in TensorFlow or the JIT compiler in PyTorch. During the optimization process, it is necessary to ensure that the input interface of the static region is compatible with the output interface of the dynamic region, that is, by defining a clear tensor transfer protocol (such as constraints on tensor shape and data type), to ensure seamless connection of subsequent hybrid execution. After optimization, save the static computation graph as an executable file (such as the SavedModel format in TensorFlow or the TorchScript model in PyTorch), and record the definition of its input and output interfaces.

[0044] Step S3, Generation of dynamic region execution path:

[0045] Extract the operation nodes of the dynamic region from the region marking table, generate the dynamic execution path, and organize it in the form of a callable function.

[0046] In this step, for the operation nodes marked as the dynamic region in step S1, generate their dynamic execution paths, as Figure 4 shown in the figure. Extract the dynamic nodes from the region marking table, retain their original operation logic and dependencies, but do not pre-compile them into a static graph. Instead, organize them into a dynamic execution sequence. The process of generating the dynamic execution path is as follows: According to the dependency order of the nodes, record the call logic of each operation one by one (such as Python functions or native operations of the framework). To improve flexibility, control flow processing logic can be introduced in the dynamic region, such as using Python's if-else statements or conditional operations supported by the framework (such as tf.cond), to adapt to different shape change scenarios. After generation, the dynamic execution path is stored in the form of a callable function (such as a Python function or a subclass of nn.Module in PyTorch), and ensure that its output tensor can be docked with the input interface of the static region. The dynamic region does not need to be pre-compiled and generates the computation graph and executes it immediately according to the actual input during runtime.

[0047] Step S4, Hybrid graph execution and result integration:

[0048] Integrate the static computational graph and the dynamic execution path according to the dependencies in the region marking table and the topological sorting of the computational graph, execute the static region and the dynamic region in sequence, collect and integrate the output tensors to form the final result.

[0049] In this step, integrate the static computational graph generated in step S2 and the dynamic execution path generated in step S3 into a hybrid execution process to complete the full operation of the neural network model. Determine the execution order of the static region and the dynamic region according to the dependencies in the region marking table and the topological sorting of the computational graph. During execution, first load the static computational graph file saved in step S2, import it into the runtime environment through the model loading interface of the framework, and then run the static region in sequence to obtain the intermediate output tensors. At the same time, pass the intermediate output tensors as inputs to the dynamic execution path in step S3, call its function interface to execute the dynamic region operations, and generate subsequent intermediate results or the final output. After the execution is completed, collect all the output tensors and integrate them to form the final prediction result of the model.

[0050] Embodiment:

[0051] This embodiment is a specific application of the above-mentioned neural network hybrid graph execution optimization method for region perception in the autonomous driving scenario, processing inputs that simultaneously include fixed-resolution images (from cameras) and variable-length point cloud data (from lidar).

[0052] Step S1, Autonomous driving model parsing and region division:

[0053] Obtain the complete definition of the autonomous driving multi-modal model, including the CNN module (processing RGB images), the PointNet module (processing point cloud data), the fusion layer (fully connected layer), and the decision layer (outputting control instructions). Parse the computational logic of the model through the torch.fx tool of PyTorch to generate an initial computational graph. The nodes in the graph include convolutional layers, pooling layers, the MLP layer of PointNet, pooling layers (adaptive max pooling), fully connected layers, etc., and the nodes are connected through data dependencies.

[0054] Perform a preliminary division based on the operation node type and topological structure. The convolutional layer and pooling layer of the CNN module are initially marked as static regions because the input image shape is fixed ([batch_size, 3, H, W]) and the convolutional kernel size is fixed. The PointNet module processes variable-length point cloud inputs ([batch_size, M, 4]), and its MLP layer and adaptive max pooling need to adapt to the dynamic number of points M, so it is initially marked as a dynamic region. The fusion layer receives the CNN output ([batch_size, C_out, H_out, W_out], fixed) and the PointNet output ([batch_size, C_point], fixed), and the decision layer receives the output of the fusion layer ([batch_size, C_fusion], fixed). Since the input shapes of the fusion layer and the decision layer are fixed (the PointNet output dimension C_point is a constant), they are initially marked as static regions. Through the depth-first search algorithm, the shape dependence is propagated along the computational graph from the input nodes (images and point clouds) to generate a temporary marking table that records the region tendency and dependence relationship of each node.

[0055] Use symbolic inference technology to verify the division result. Assign symbolic variables to the input tensors. For example, the image tensor is (N, 3, H, W), and the point cloud tensor is (N, M, 4), where H is the image height, W is the image width, N is the batch size, and M is the dynamic number of points. Propagate the symbolic expressions from the input layer. For example, the output shape of the convolutional layer of the CNN is (N, C_out, H - 2, W - 2), where C_out is determined by the convolutional kernel, and H and W are both constants, confirming that it belongs to the static region; the output shape of the adaptive max pooling of PointNet is (N, C_point), C_point is fixed, but its calculation process depends on the dynamic number of points M, confirming that PointNet belongs to the dynamic region; the input shapes (N, C_point) and (N, C_out, H_out, W_out) of the fusion layer and the decision layer are both fixed dimensions, and symbolic inference confirms that they are static regions. If the preliminary division is inconsistent with the symbolic inference (for example, the fusion layer is misjudged as dynamic because it depends on PointNet), it is adjusted to the static region according to the fixed shape. Finally, generate a region marking table, mark the CNN module, the fusion layer, and the decision layer as static regions, the PointNet module as a dynamic region, and record the operation types (such as "Conv2D", "MaxPool3D") and dependence relationships.

[0056] Step S2: Construction and optimization of the static region computational graph:

[0057] Extract static region nodes (CNN module, fusion layer, decision layer) from the region marking table to construct an independent static computational graph. Retain the convolutional layer, pooling layer, and activation function of the CNN, the feature concatenation and fully connected operations of the fusion layer, and the fully connected operation and its data flow dependencies in the decision layer. Use the torch.jit.trace tool in PyTorch to convert it into the static computational graph format.

[0058] Perform compilation optimization on the static computational graph. Apply operator fusion to merge the convolutional layer with ReLU activation and the fusion layer with fully connected operations into a single efficient operation to reduce computational overhead. Through memory reuse analysis, reuse the memory space of intermediate feature maps and fully connected layers to reduce video memory occupancy. The optimization process uses the TorchScript compiler in PyTorch to ensure that the input interfaces (image [N, 3, 720, 1280] and PointNet output [N, C_point]) and output interface (control instruction [N, C_decision]) of the static region are clearly defined. After optimization, save the static computational graph as a TorchScript model file and record its input and output tensor formats.

[0059] Step S3, Generation of dynamic region execution path:

[0060] Extract dynamic region nodes (PointNet module) from the region marking table to generate a dynamic execution path. Retain the MLP layer, adaptive max pooling, and their dependencies in PointNet to generate a dynamic execution sequence.

[0061] Introduce control flow logic in the dynamic region to adapt to the shape changes of the point cloud. For example, M in the input point cloud tensor (N, M, 4) of PointNet is a dynamic value, and different processing logics are dynamically adjusted through Python conditional statements; adaptive max pooling generates a fixed-dimension output [N, C_point] based on the number of input points M. The dynamic execution path is stored as a callable function in the form of a subclass of nn.Module in PyTorch to ensure that the output tensor [N, C_point] is docked with the input interface of the fusion layer in the static region, and a standardized tensor format is adopted to ensure compatibility.

[0062] Step S4, Hybrid graph execution and result integration:

[0063] Determine the execution order according to the dependencies in the region marking table and the topological sorting of the computational graph: first execute the dynamic region (PointNet module), and then execute the static region (CNN module, fusion layer, decision layer). Load the TorchScript model file saved in step S2 and import it into the runtime environment through the torch.jit.load interface of PyTorch. Run the PointNet module, input the point cloud tensor (N, M, 4), and generate the output tensor (N, C_point).

[0064] Feed the output tensor (N, C_point) of PointNet and the image input (N, 3, H, W) into the static computational graph together. Run the CNN module to generate the feature tensor (N, C_out, H_out, W_out), integrate the CNN and PointNet features through the fusion layer to generate (N, C_fusion), and finally the decision layer outputs the control instructions (such as steering angle, acceleration, (N, C_decision)). Collect all the output tensors and integrate them into the final prediction result (control instructions).

Claims

1. A method for optimizing the execution of a neural network hybrid graph with regional perception, characterized in that The method includes the following steps: Step S1, neural network model parsing and region partitioning: Obtain the complete definition of the neural network model to be optimized, parse its computational logic into an initial computational graph, and through graph structure and operation semantics analysis as well as symbolic inference techniques, partition the computational graph into static regions with fixed shapes and dynamic regions with non-fixed shapes, generating a region label table; Step S2, static region computational graph construction and optimization: Extract the operation nodes of the static regions from the region label table, construct an independent static computational graph, and perform compilation optimization on it to generate an executable static computational graph file; Step S3, dynamic region execution path generation: Extract the operation nodes of the dynamic regions from the region label table, generate a dynamic execution path, and organize it into a callable function form; Step S4, hybrid graph execution and result integration: According to the dependency relationships in the region label table and the topological sorting of the computational graph, integrate the static computational graph and the dynamic execution path, sequentially execute the static regions and the dynamic regions, collect and integrate the output tensors to form the final result.

2. The optimization method for executing a region-aware neural network hybrid graph according to claim 1, wherein The specific steps of step S1 are as follows: Step S11, parse the computational logic of the model into an initial computational graph composed of operation nodes and data dependency relationships between nodes; Step S12, perform a preliminary partition based on the type and topological structure of the operation nodes, and combine the shape dependencies propagated from the input nodes along the computational graph to generate a temporary label table; Step S13, use symbolic inference techniques to assign symbolic variables to the tensors in the computational graph, propagate symbolic expressions, verify and adjust the preliminary partition results, generate a region label table, label the region belonging of each node as "static" or "dynamic", and record the operation type and dependency relationships.

3. The region-aware neural network hybrid graph execution optimization method according to claim 2, characterized in that The specific steps of step S12 are as follows: For each node, check its operation type; at the same time, propagate the shape dependencies from the input nodes along the topological order of the computational graph. If the input of a certain node comes from a node with a fixed shape, then this node is more likely to belong to the static region. If the input directly depends on external dynamic inputs, it tends to be in the dynamic region. This process is achieved by extracting the operation type and parameter information of the nodes through torch.fx of PyTorch or by reading the node attributes through tf.Graph of TensorFlow, generating a temporary label table to record the region tendency of each node and its dependency relationships.

4. The method for optimizing the execution of the region-aware neural network hybrid graph according to claim 2, wherein The specific steps of step S13 are as follows: Assign symbolic variables to the tensors in the computational graph, propagate the symbolic expressions from the input layer along the computational graph, and then check the shape expressions of each node to determine whether its dimensions are fixed values or variables. If all dimensions are constants, then confirm that this node belongs to the static region. If it contains variables, then confirm that it belongs to the dynamic region. If the preliminary partition is inconsistent with the symbolic inference result, then adjust the node attribution according to the symbolic inference. Finally, integrate the preliminary partition and the symbolic inference results to generate a region label table, clearly label the region belonging of each node as "static" or "dynamic", and record its operation type and dependency relationships.

5. The optimization method for performing region-aware neural network hybrid graph according to claim 1, characterized in that The specific steps of step S2 are as follows: Step S21: Screen static nodes from the region marking table, retain their original operation definitions and data flow dependencies, and construct a static computational graph; Step S22: Perform compilation optimization on the static computational graph, including operator fusion, memory reuse, and parallel computing scheduling; Step S23: Save the optimized static computational graph as an executable file and define its input and output interfaces.

6. The region-aware neural network hybrid graph execution optimization method according to claim 1, characterized in that The specific steps of step S3 are as follows: Step S31: Extract dynamic nodes from the region marking table, retain their original operation logic and dependencies, and generate a dynamic execution sequence; Step S32: Introduce control flow processing logic in the dynamic region to adapt to shape changes; Step S33: Store the dynamic execution path in the form of a callable function to ensure that its output tensor can be docked with the input interface of the static region.

7. The method for optimizing the execution of the region-aware neural network hybrid graph according to claim 1, characterized in that In step S3, the generation process of the dynamic execution path is: according to the dependency order of the nodes, record the call logic of each operation one by one.

8. The region-aware neural network hybrid graph execution optimization method according to claim 1, characterized in that The specific steps of step S4 are as follows: Step S41: Determine the execution order of the static region and the dynamic region according to the dependency relationship in the region marking table and the topological sorting of the computational graph; Step S42: Load the static computational graph file, run the static region, and obtain the intermediate output tensor; Step S43: Pass the intermediate output tensor into the dynamic execution path, run the dynamic region, and generate subsequent intermediate results or the final output; Step S44: Collect all output tensors and integrate them into the final prediction result.

Images