Dynamic neural network compiling execution method and device, equipment and medium

Abstract

The invention discloses a dynamic neural network compiling execution method and device, equipment and a medium. Comprising the following steps: calculating an optimal fusion slice corresponding to each dynamic fusion fragment according to chip parameters in a compiling stage; static fusion fragments matched with the corresponding dynamic fusion fragments are obtained according to the optimal fusion slices, and all the dynamic fusion fragments and the matched static fusion fragments form a fusion unit; and in the execution stage, the cyclic scalar control logic of the dynamic neural network is obtained, and the fusion unit is dynamically called based on the cyclic scalar control logic. The optimal fusion slice, suitable for the hardware cache size, of each dynamic fusion fragment is obtained under the condition of not depending on shape information, the dynamic fusion fragments are converted into static fusion fragments based on the optimal fusion slices so as to eliminate related computing nodes of dynamic shapes, dynamic scheduling planning can be conducted on the static fusion fragments in the execution stage, and therefore the dynamic fusion fragments can be dynamically scheduled and planned. And the execution efficiency of the dynamic neural network is greatly improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer technology, and in particular to a method, device, equipment and medium for compiling and executing a dynamic neural network. Background Art

[0002] Compared with static neural networks, the input shape of dynamic neural networks is uncertain. With the application of neural networks in scenarios such as large language models and image processing, dynamic neural networks with variable shapes are increasingly used in various fields.

[0003] However, since dynamic neural networks cannot obtain exact shape information during the compilation phase, they cannot perform compile-time cache optimization. All data is obtained from the chip's external memory space, which significantly reduces the performance and execution efficiency of dynamic neural networks. Summary of the invention

[0004] The embodiment of the present invention provides a compilation and execution method of a dynamic neural network, so as to achieve compilation optimization of the dynamic neural network in the compilation stage.

[0005] In a first aspect, an embodiment of the present invention provides a compilation and execution method of a dynamic neural network, comprising: obtaining a dynamic fusion fragment according to each operator node of the dynamic neural network in a compilation stage;

[0006] Obtaining chip parameters of a multi-level storage chip carrying the dynamic neural network, and calculating the best fusion slice corresponding to each of the dynamic fusion segments according to the chip parameters;

[0007] According to the optimal fusion slice, a static fusion segment matching the corresponding dynamic fusion segment is obtained, and each of the dynamic fusion segments and the matching static fusion segment constitutes a fusion unit, wherein the fusion unit performs tensor transport in the shared storage layer of the multi-level storage chip;

[0008] The loop scalar control logic of the dynamic neural network is obtained during the execution phase, and the fusion unit is dynamically called based on the loop scalar control logic.

[0009] In a second aspect, an embodiment of the present invention provides a compilation and execution device for a dynamic neural network, including: a dynamic fusion fragment acquisition module, which is used to acquire dynamic fusion fragments according to each operator node of the dynamic neural network during the compilation stage;

[0010] An optimal fusion slice acquisition unit, used to acquire chip parameters of a multi-level storage chip carrying the dynamic neural network, and calculate the optimal fusion slice corresponding to each of the dynamic fusion segments according to the chip parameters;

[0011] A fusion unit acquisition module, used for acquiring a static fusion segment matching the corresponding dynamic fusion segment according to the optimal fusion slice, and forming a fusion unit with each of the dynamic fusion segments and the matching static fusion segment, wherein the fusion unit performs tensor transport in the shared storage layer of the multi-level storage chip;

[0012] The fusion unit calling module is used to obtain the loop scalar control logic of the dynamic neural network during the execution phase, and dynamically call the fusion unit based on the loop scalar control logic.

[0013] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described above when executing the program.

[0014] In a fourth aspect, an embodiment of the present invention provides a storage medium of computer executable instructions, on which a computer program is stored, characterized in that the program implements the method described above when executed by a processor.

[0015] The present invention obtains dynamic fusion fragments in the compilation stage, obtains the best fusion slices of each dynamic fusion fragment that are suitable for the hardware cache size without relying on shape information, and converts the dynamic fusion fragments into static fusion fragments based on the best fusion slices to eliminate the relevant computing nodes of the dynamic shape, thereby realizing the optimization of the dynamic neural network in the compilation stage, and dynamically scheduling the static fusion fragments in the execution stage, thereby dynamically converting the internal processing of the original operator level into the dynamic processing of the scheduling logic, which greatly improves the execution efficiency of the dynamic neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a method for compiling and executing a dynamic neural network provided by the first embodiment of the present invention;

[0017] Figure 2 is a schematic diagram of the structure of a dynamic neural network provided in Embodiment 1 of the present invention;

[0018] Figure 3 is a schematic diagram of the structure of a compiled dynamic neural network provided in the first embodiment of the present invention;

[0019] Figure 4 is a flow chart of a method for compiling and executing a dynamic neural network provided in Embodiment 2 of the present invention;

[0020] Figure 5 It is a structural diagram of a dynamic neural network compilation and execution device provided by Embodiment 3 of the present invention;

[0021] Figure 6 It is a structural diagram of a computer device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0023] Embodiment 1

[0024] Figure 1 This is a flowchart of a method for compiling and executing a dynamic neural network provided in the first embodiment of the present invention. This embodiment is applicable to the case of compiling and executing a dynamic neural network. The method can be executed by a compiling and executing device for a dynamic neural network, which can be implemented in the form of hardware and / or software. Figure 1 As shown, the method includes:

[0025] Step S101, obtaining dynamic fusion fragments according to each operator node of the dynamic neural network in the compilation stage.

[0026] Optionally, in the compilation stage, a dynamic fusion fragment is obtained according to each operator node of the dynamic neural network, including: in the compilation stage, a fusion operator node is screened out from each operator node of the dynamic neural network; and a dynamic fusion fragment is obtained according to the fusion operator node, wherein the dynamic fusion fragment includes at least two fusion operator nodes having a dependent relationship.

[0027] Specifically, a dynamic neural network is a directed acyclic graph composed of several operator nodes and data dependencies, and an operator node includes input, calculation process and output, where the calculation process is usually the type of the operator node itself, for example, convolution, pooling, addition and subtraction, and the attributes of the operator stage itself are also determined, such as the step size of the convolution, the size of the convolution kernel, etc. This information is also certain for the dynamic neural network, and the only uncertainty is some specific dimensions of some inputs, such as the width and height of the convolution, or the specific batch size. And for a dynamic neural network in a specific field, which dimensions are dynamic dimensions have been determined in the compilation stage of the network, and all operator nodes in the network will basically follow it. Therefore, the dynamic dimensions of all operator node inputs in the entire dynamic neural network can also be determined at compile time. For example Figure 2The structure diagram of the dynamic neural network in this embodiment is shown, which includes seven dynamic operator nodes, namely dynamic OP1, dynamic OP2…dynamic OP7, and the input shape of each operator node is uncertain, so the data is obtained from the external memory space of the chip. Of course, this embodiment is only an example, and does not limit the number of operator nodes included in the dynamic neural network. In addition, in this embodiment, in order to optimize the dynamic neural network in the compilation stage, the fusion operator nodes will be screened out from the operator nodes of the dynamic neural network, and the dynamic fusion fragments will be obtained based on the fusion operator nodes.

[0028] Optionally, fusion operator nodes are screened out from the operator nodes of the dynamic neural network during the compilation phase, including: obtaining the weight transfer type of each operator node in the dynamic neural network, wherein the weight transfer type includes weight transfer sparse type and weight transfer intensive type; obtaining the dynamic dimension and reduced calculation dimension of each operator node in the dynamic neural network; and taking the operator nodes in the dynamic neural network whose weight transfer type is weight transfer sparse type and whose dynamic dimension does not overlap with the reduced calculation dimension as fusion operator nodes.

[0029] Optionally, the weight transfer type of each operator node in the dynamic neural network is obtained, including: obtaining the weight size and chip bandwidth of each operator node in the dynamic neural network, and obtaining the weight transfer overhead according to the ratio of the weight size to the chip bandwidth; obtaining the data transfer overhead and the data calculation overhead of each operator node in the dynamic neural network, and obtaining the overhead threshold according to the sum of the data transfer overhead and the data calculation overhead; judging whether the weight transfer overhead of the operator node is less than the overhead threshold, if so, determining the weight transfer type of the operator node to be weight transfer sparse type, otherwise, determining the weight transfer type of the operator node to be weight transfer intensive type.

[0030] Specifically, in this embodiment, when analyzing each operator node in the dynamic neural network one by one, it is specifically screened from the two aspects of functional evaluation and performance evaluation, and the operator nodes that meet the requirements are used as fusion operator nodes. On the one hand, when performing performance evaluation, the weight transfer type of the operator node is specifically determined, and the operator node with a weight transfer type of weight transfer sparse type is used as a candidate operator node, wherein the weight transfer type includes weight transfer sparse type and weight transfer intensive type, and when determining the weight transfer type of the operator node, the weight size, chip bandwidth, data transfer overhead and data calculation overhead of the operator node are specifically obtained, wherein the weight size and chip bandwidth are known quantities that can be directly obtained, and the data transfer overhead and data calculation overhead need to be calculated based on the operation characteristics of the operator node using a certain calculation method to obtain, and the specific calculation method is not the focus of this application, so it will not be repeated in this embodiment. In this embodiment, the weight transfer overhead is obtained according to the ratio of the weight size and the chip bandwidth, and the overhead threshold is obtained according to the data transfer overhead and the data calculation overhead, so as to determine the weight transfer type of the operator node based on the relationship between the weight transfer overhead of the operator node and the overhead threshold. For example, when the weight transfer overhead is less than the overhead threshold, the operator node is determined to be a weight transfer sparse type, and when the weight transfer overhead is greater than the overhead threshold, the operator node is determined to be a weight transfer intensive type. Of course, this embodiment is only an example, and does not limit the specific determination method of the weight transfer type of the operator node. For example, through performance evaluation, six operator nodes, namely dynamic OP1, dynamic OP2, dynamic OP3, dynamic OP4, dynamic OP5 and dynamic OP6, are obtained as six candidate operator nodes that meet the performance requirements. On the other hand, after the performance evaluation, the present embodiment will also perform a functional evaluation on the candidate operator nodes, specifically to determine whether the dynamic dimension and the reduced calculation dimension of the operator node overlap, and use the non-overlapping candidate operator nodes as fusion operator nodes. For example, after the functional evaluation, five fusion operator nodes that meet the functional requirements, such as dynamic OP2, dynamic OP3, dynamic OP4, dynamic OP5 and dynamic OP6, are obtained. Among them, the dynamic dimension of the operator node has been determined in the compilation stage of the network and is therefore a known quantity, while the reduced calculation dimension of the operator node can be obtained by looking up the reduced dimension table, wherein the reduced dimension table records the dimensions of the reduced calculation corresponding to different types of operator nodes, as shown in Table 1 below as an example of a reduced dimension table:

[0031] Table 1

[0032]

[0033] In particular, due to space limitations, Table 1 only uses three types of operator nodes as examples for illustration, and in actual applications, this implementation does not limit the specific number of operator types recorded in the specification dimension table.

[0034] Optionally, obtaining a dynamic fusion fragment according to the fusion operator nodes includes: obtaining the input-output ratio of each fusion operator node; searching and fusing the fusion operator nodes whose input-output ratio is within a first numerical range and has a dependency relationship using a greedy algorithm to obtain at least one dynamic fusion fragment.

[0035] Specifically, in this embodiment, after obtaining fusion operator nodes, such as dynamic OP2, dynamic OP3, dynamic OP4, dynamic OP5 and dynamic OP6, the input-output ratio of each fusion operator node is obtained, and the fusion operator nodes with input-output ratios within the first numerical range and with dependencies are fused to obtain dynamic fusion segments. In the fusion process, a greedy algorithm is used to search for dynamic fusion segments of the dynamic neural network, and the number of fusion operator nodes included in a dynamic fusion segment is at least 2. Although the principle of "all fusions should be fused" is followed in the search process, in actual applications, fusion is usually interrupted by functional limitations or performance limitations, so the dynamic fusion segment is not infinite.

[0036] Among them, from the performance limitation, although the specific shape information of each fusion operator node, that is, the absolute size, cannot be obtained in the compiler, when multiple fusion operator nodes are connected in series, their multiple relationship, that is, the relative size, is determined. For example, fusion operator node a is an element-level operation operator, and the size of its input dynamic dimension is x. The output of a is given to fusion operator node b, and b is a convolution operator with a step size of 2. The input of b is equal to the output x of a, and the output of b becomes 1 / 2x. That is to say, although the exact value of the dynamic dimension of the output of operator b is unknown, it must be half of the input x. This type of multiple relationship can be derived and determined at compile time. That is, although the actual memory size of the dynamic shape network cannot be obtained at compile time, its memory size trend curve can be obtained. Therefore, in this embodiment, the fusion operator nodes with consistent memory trends, that is, input-output ratios within the first numerical range are specifically fused, wherein the first numerical range can be 1-1.3, that is, the fusion operator nodes with input-output ratios close to or even the same are fused. In addition, from the functional limitation, in the process of fusion using the greedy algorithm, if the dynamic dimension of the newly fused dynamic fusion segment does not overlap with the reduced calculation dimension, it can be fused together, but when the dynamic dimension overlaps with the reduced calculation dimension, it will be interrupted. For example, the dynamic dimension of the dynamic fusion segment obtained after the fusion of dynamic OP4, dynamic OP5 and dynamic OP6 does not overlap with the reduced calculation dimension, but when the dynamic dimension of the dynamic fusion segment re-obtained by amplification after the fusion of dynamic OP3 overlaps with the reduced calculation dimension, then an interruption will occur between dynamic OP4 and dynamic OP3, so that it can be determined that dynamic OP4, dynamic OP5 and dynamic OP6 constitute a dynamic fusion segment 1, and the dynamic OP3 is used as the new anchor point to re-search upward greedily, and a dynamic fusion segment 2 containing dynamic OP2 and dynamic OP3 is obtained. Of course, in this embodiment, only two dynamic fusion segments are obtained as an example for explanation, and the specific number of dynamic fusion segments obtained is not limited.

[0037] Step S102, obtaining chip parameters of a multi-level storage chip that carries a dynamic neural network, and calculating the best fusion slice corresponding to each dynamic fusion segment according to the chip parameters.

[0038] Optionally, the chip parameters include the chip cache size, and the optimal fusion slice corresponding to each dynamic fusion fragment is calculated according to the chip parameters, including: setting an initial slice value for the dynamic dimension of each dynamic fusion fragment, and calculating the memory cache size occupied by the dynamic fusion fragment under the initial slice value; judging whether the memory cache size is larger than the chip cache size, if so, reducing the initial slice value to obtain the updated slice value, otherwise, increasing the initial slice value to obtain the updated slice value; calculating the difference between the updated slice value and the chip cache size, and when the difference is within a second numerical range, taking the updated slice value as the optimal fusion slice.

[0039] Optionally, after calculating the best fusion slice corresponding to each dynamic fusion fragment according to the chip parameters, it also includes: aligning the dimensional size of the best fusion slice to an even number; calculating the data volume size corresponding to the best fusion slice after the even number alignment, and aligning the data volume size to an integer multiple with reference to the chip handling bandwidth.

[0040] Specifically, in this embodiment, a multi-level storage chip may be used to carry out the operation of the dynamic neural network, and the chip parameters of the chip are known, such as the chip cache size and chip bandwidth of each level. Since the dynamic dimension of each fusion operator node is known, the dynamic dimension of the dynamic fusion fragment obtained after the corresponding fusion is also known. Therefore, in this embodiment, an initial slice value may be set for each dynamic fusion fragment to perform an attempted calculation to obtain the memory cache size of the dynamic fusion fragment under the given initial slice value, so as to determine whether the current slice can be completely cached on the chip. If the memory cache size is smaller than the chip cache size, it means that it can be accommodated. In this case, an attempt is made to increase the initial slice value to obtain an updated slice value. If the memory cache size is larger than the chip cache size, it means that it cannot be accommodated. In this case, an attempt is made to reduce the initial slice value to obtain an updated slice value. The above operation is repeated until the maximum value of the dynamic dimension of the dynamic fusion fragment that the current chip can cache is found. This value is the limit that the chip can just cache. In the specific calculation, the difference between the updated slice value and the chip cache size is obtained. When the difference is within the second numerical range, the updated slice value is used as the best fused slice. For example, the second numerical range can be 0-0.5. Of course, this embodiment is only an example, and the specific content of the second numerical range is not limited. For example, when the dynamic fusion segment 1 has only one dynamic dimension, the best fused slice obtained is 320. Correspondingly, when there are two dynamic dimensions, the best fused slices 320 and 340 corresponding to each dynamic dimension will be obtained, that is, two best fused slices. Of course, this embodiment is only an example, and the specific values ​​of the best fused slices corresponding to each dynamic fusion segment are not limited.

[0041] It should be noted that, in this embodiment, after obtaining the best fusion slice corresponding to each dynamic fusion fragment, the best fusion slice will also be aligned to ensure that the hardware resources such as chip computing power and bandwidth are fully utilized. Among them, when performing the alignment process, the alignment is specifically performed from two angles. At the first angle, the dimension size of the best fusion slice is evenly aligned, that is, aligned to a power of 2 and other chip computing unit-friendly values. For example, when the best fusion slice is 323, it will be aligned to 320; the second angle is to align the data volume size corresponding to the best fusion slice with reference to the chip handling bandwidth. For example, the original size of the dynamic fusion fragment 1 is 367*? *360, where "?" represents the position of the dynamic dimension. When the best fusion slice is determined, it is substituted into the position of the dynamic dimension to obtain the data volume "367×320×360". It is necessary to ensure that the product result is an integer multiple of 512 bits. If not, the best fusion slice needs to be aligned and adjusted. Of course, this embodiment is only an example, and does not limit the specific alignment method of the best fusion slice.

[0042] Step S103, obtaining the static fusion segment that matches the corresponding dynamic fusion segment according to the best fusion slice, and forming a fusion unit with each dynamic fusion segment and the matching static fusion segment.

[0043] Optionally, a static fusion fragment that matches the corresponding dynamic fusion fragment is obtained according to the best fusion slice, including: filling each aligned best fusion slice into the dynamic dimension of the corresponding dynamic fusion fragment to obtain a static fusion fragment that matches the dynamic fusion fragment; performing constant folding and algebraic simplification on each static fusion fragment to obtain a static fusion fragment after ordinary layer optimization; performing static operator fusion and redundant operator node elimination on the static fusion fragment after ordinary layer optimization to obtain a static fusion fragment after cache layer optimization.

[0044] Specifically, in this embodiment, after finding the aligned best fusion slice corresponding to each dynamic fusion fragment, the subsequent best fusion slice can be filled into the dynamic dimension of the corresponding dynamic fusion fragment, and a static fusion fragment matching the dynamic fusion fragment can be obtained. And after obtaining the static fusion fragment, a series of internal optimizations can be performed, such as common layer optimizations such as constant folding and algebraic simplification, and cache layer optimizations such as static operator fusion and redundant operator node elimination. Since the specific optimization process of the static neural network is not the focus of this application, it will not be repeated in this embodiment. At the same time, for the operator node, since the determined size information has been obtained, the operator node can compile and select the data flow and calculation flow that is most suitable for the current size, instead of the dynamic operator grounding that needs to apply various input shapes and use more general data flows and calculation flows, thereby improving the computing performance in a variety of specific scenarios.

[0045] It should be noted that a prerequisite for converting dynamic fusion segments into static fusion segments in the present embodiment is that all dynamic dimensions in the dynamic fusion segments need to be segmented, and the dynamic dimensions are replaced with the obtained segmented sizes and static values, so as to achieve the transformation from static to dynamic. If any dimension is still dynamic, it cannot be converted into a static fusion segment. In addition, the original dynamic fusion segment in the present embodiment will also be retained, and the converted static fusion segment and the corresponding dynamic fusion segment will be used to form a fusion unit. The fusion unit performs tensor transfer in the shared storage layer of the multi-level storage chip, that is, the data in the fusion operator nodes in the fusion unit are all obtained from the internal space of the chip, without the need to frequently transfer from the outside of the chip, thereby improving the overall performance of the dynamic neural network. Among them, such as Figure 3 The structure diagram of the compiled dynamic neural network is shown, and the corresponding loop scalar control logic is pre-compiled for each dynamic neural network. During the subsequent operation, when each fusion unit is executed, the loop scalar control logic will be called once to determine the calling method for each fusion unit. Of course, this embodiment is only described by taking the dynamic neural network containing two fusion units as an example, and the specific number of fusion units in the dynamic neural network is not limited in this embodiment.

[0046] Step S104, obtaining the loop scalar control logic of the dynamic neural network during the execution phase, and dynamically calling the fusion unit based on the loop scalar control logic.

[0047] Optionally, the fusion unit is dynamically called based on the loop scalar control logic, including: obtaining the best fusion slice corresponding to the fusion unit to be executed currently and the actual input size of the dynamic neural network; inputting the actual input size and the best fusion slice into the loop scalar control logic to obtain a calling method for the fusion unit, and dynamically calling the fusion unit to be executed currently based on the calling method.

[0048] Optionally, the actual input size and the best fusion slice are input into the loop scalar control logic to obtain a calling method for the fusion unit, including: when the actual input size is smaller than the best fusion slice, determining that the calling method for the fusion unit by the loop scalar control logic is to call the dynamic fusion fragment in the fusion unit only once; when the actual input size is not smaller than the best fusion slice and is divisible, obtaining the integer value of the actual input size divided by the best fusion slice, and determining that the calling method for the fusion unit by the loop scalar control logic is to cyclically call the static fusion fragment in the fusion unit of the integer value; when the actual input size is larger than the best fusion slice and is not divisible, obtaining the integer value and remainder value of the actual input size divided by the best fusion slice, and determining that the calling method for the fusion unit by the loop scalar control logic is to cyclically call the static fusion fragment in the fusion unit of the integer value, and call the dynamic fusion fragment in the fusion unit of the remainder value.

[0049] Specifically, in the present embodiment, the actual input size of the dynamic neural network can be determined during the execution phase. At this time, the specific calling method of each fusion unit for the current input size will be calculated based on the loop scalar control logic, that is, how many dynamic fusion fragments and how many static fusion fragments need to be called. Among them, the loop scalar control logic in the present embodiment is mainly applied to the following two application scenarios: Scenario 1 is when a dynamic dimension needs to be looped and expanded N times, it can be guaranteed that the first (N - 1) loops enter the static graph calculation. If the Nth loop cannot divide the slice size evenly, it needs to be calculated by the dynamic graph; Scenario 2 is when multiple dynamic dimensions need to be sliced ​​and expanded, for example, the convolution width / height is a dynamic dimension, and the width / height is expanded M / N times respectively, then it can be guaranteed that (M-1)*(N-1) loops enter the static graph. If the remaining part cannot be divided evenly, it needs to be calculated by the dynamic graph. Assuming the optimal static slice size is H_step=64 / W_step=64, when the dynamic shape is H=128 / W=128, the static graph can be hit for 2 cycles ((128 / 64)*(128 / 64)); when the dynamic shape is H=32 / W=128, the static graph cannot be hit. In summary, the larger the dynamic shape, the more cycles hit the static graph.

[0050] In a specific implementation, when the fusion unit 1 is run, the loop scalar control logic will be called, and the actual input size of the current dynamic neural network and the best fusion slice A will be input into the loop scalar control logic, and the loop scalar control logic will determine the number of calls to the dynamic fusion segment 1 and the static fusion segment 1. For example, when the actual input size is smaller than the best fusion slice, it means that the current data processing volume is relatively small, and the fusion unit 1 does not need any segmentation or the loop only needs to call the dynamic fusion segment once; when the actual input size is not smaller than the best fusion slice and is divisible, the integer value m of the actual input size divided by the best fusion slice is calculated, and the static fusion segment 1 is called m times; when the actual input size is larger than the best fusion slice and is not divisible, the integer value p and the remainder value q of the actual input size divided by the best fusion slice are calculated, and the static fusion segment 1 is called p times and the dynamic fusion segment 1 is called q times, and usually the number of calls to the dynamic fusion segment 1 is very small, and most of the calls are to the static fusion segment. And when the dynamic fusion fragment 1 and the static fusion fragment 1 are called at the same time, according to the actual calculated runtime calculation logic, after the static fusion fragment 1 and the dynamic fusion fragment 1 are called, the results of each fusion fragment calculation will be spliced ​​and written back to the original output memory space, thereby completing the calculation and output of the entire fusion result. Through the above steps, the dynamic nature of a dynamic neural network is transformed from the internal processing of the complete operator to the dynamic scheduling of the layer plus the static operator, and the dynamic scheduling on the host side usually has a scalar calculation overhead of almost 0, thereby realizing the conversion of a dynamic network into a static slice network, so that the performance of the layer operators is fully optimized, and the performance of the entire dynamic neural network is guaranteed. Of course, in this embodiment, only the call of fusion unit 1 is used as an example for explanation, and the method of calling other fusion units is roughly the same, and will not be repeated in this embodiment.

[0051] It is worth mentioning that the present embodiment does not rely on shape information, and can automatically find the best fusion slice suitable for the hardware cache size, providing the possibility of fusion for the dynamic network; according to the searched best fusion slice of a determined size, the dynamic shape of the neural network is converted into a static shape, and the related computing nodes of the dynamic shape can be eliminated; the fusion slice converted into a static shape is further optimized, and various optimization methods in the static network can be fully utilized, including algebraic optimization, static operator fusion and constant folding, which greatly improves the performance of the fusion slice; the original dynamic operator in the network is converted into a dynamic scheduling plus a static operator, and the ultimate performance of the static operator is obtained with extremely small scheduling overhead.

[0052] This implementation method obtains dynamic fusion fragments in the compilation stage, obtains the best fusion slices of each dynamic fusion fragment that are suitable for the hardware cache size without relying on shape information, and converts the dynamic fusion fragments into static fusion fragments based on the best fusion slices to eliminate the relevant computing nodes of the dynamic shape, thereby optimizing the dynamic neural network in the compilation stage. In the execution stage, the static fusion fragments can be dynamically scheduled and planned, thereby dynamically converting the internal processing of the original operator level into the dynamic processing of the scheduling logic, which greatly improves the execution efficiency of the dynamic neural network.

[0053] Embodiment 2

[0054] Figure 4 This is a flowchart of a dynamic neural network compilation and execution method provided by the second embodiment of the present invention. This embodiment is based on the above embodiment and further includes: detecting the calling results of each fusion unit of the dynamic neural network after dynamically calling the fusion unit based on the loop scalar control logic. Figure 4 As shown, the method includes:

[0055] Step S201, obtaining dynamic fusion fragments according to each operator node of the dynamic neural network in the compilation stage.

[0056] Optionally, in the compilation stage, a dynamic fusion fragment is obtained according to each operator node of the dynamic neural network, including: in the compilation stage, a fusion operator node is screened out from each operator node of the dynamic neural network; and a dynamic fusion fragment is obtained according to the fusion operator node, wherein the dynamic fusion fragment includes at least two fusion operator nodes having a dependent relationship.

[0057] Optionally, fusion operator nodes are screened out from the operator nodes of the dynamic neural network during the compilation phase, including: obtaining the weight transfer type of each operator node in the dynamic neural network, wherein the weight transfer type includes weight transfer sparse type and weight transfer intensive type; obtaining the dynamic dimension and reduced calculation dimension of each operator node in the dynamic neural network; and taking the operator nodes in the dynamic neural network whose weight transfer type is weight transfer sparse type and whose dynamic dimension does not overlap with the reduced calculation dimension as fusion operator nodes.

[0058] Optionally, the weight transfer type of each operator node in the dynamic neural network is obtained, including: obtaining the weight size and chip bandwidth of each operator node in the dynamic neural network, and obtaining the weight transfer overhead according to the ratio of the weight size to the chip bandwidth; obtaining the data transfer overhead and the data calculation overhead of each operator node in the dynamic neural network, and obtaining the overhead threshold according to the sum of the data transfer overhead and the data calculation overhead; judging whether the weight transfer overhead of the operator node is less than the overhead threshold, if so, determining the weight transfer type of the operator node to be weight transfer sparse type, otherwise, determining the weight transfer type of the operator node to be weight transfer intensive type.

[0059] Optionally, obtaining a dynamic fusion fragment according to the fusion operator nodes includes: obtaining the input-output ratio of each fusion operator node; searching and fusing the fusion operator nodes whose input-output ratio is within a first numerical range and has a dependency relationship using a greedy algorithm to obtain at least one dynamic fusion fragment.

[0060] Step S202, obtaining chip parameters of a multi-level storage chip that carries a dynamic neural network, and calculating the best fusion slice corresponding to each dynamic fusion segment according to the chip parameters.

[0061] Optionally, the chip parameters include the chip cache size, and the optimal fusion slice corresponding to each dynamic fusion fragment is calculated according to the chip parameters, including: setting an initial slice value for the dynamic dimension of each dynamic fusion fragment, and calculating the memory cache size occupied by the dynamic fusion fragment under the initial slice value; judging whether the memory cache size is larger than the chip cache size, if so, reducing the initial slice value to obtain the updated slice value, otherwise, increasing the initial slice value to obtain the updated slice value; calculating the difference between the updated slice value and the chip cache size, and when the difference is within a second numerical range, taking the updated slice value as the optimal fusion slice.

[0062] Optionally, after calculating the best fusion slice corresponding to each dynamic fusion fragment according to the chip parameters, it also includes: aligning the dimensional size of the best fusion slice to an even number; calculating the data volume size corresponding to the best fusion slice after the even number alignment, and aligning the data volume size to an integer multiple with reference to the chip handling bandwidth.

[0063] Step S203: acquiring the static fusion segment that matches the corresponding dynamic fusion segment according to the best fusion slice, and forming a fusion unit with each dynamic fusion segment and the matching static fusion segment.

[0064] Optionally, a static fusion fragment that matches the corresponding dynamic fusion fragment is obtained according to the best fusion slice, including: filling each aligned best fusion slice into the dynamic dimension of the corresponding dynamic fusion fragment to obtain a static fusion fragment that matches the dynamic fusion fragment; performing constant folding and algebraic simplification on each static fusion fragment to obtain a static fusion fragment after ordinary layer optimization; performing static operator fusion and redundant operator node elimination on the static fusion fragment after ordinary layer optimization to obtain a static fusion fragment after cache layer optimization.

[0065] Step S204, obtaining the loop scalar control logic of the dynamic neural network during the execution phase, and dynamically calling the fusion unit based on the loop scalar control logic.

[0066] Optionally, the fusion unit is dynamically called based on the loop scalar control logic, including: obtaining the best fusion slice corresponding to the fusion unit to be executed currently and the actual input size of the dynamic neural network; inputting the actual input size and the best fusion slice into the loop scalar control logic to obtain a calling method for the fusion unit, and dynamically calling the fusion unit to be executed currently based on the calling method.

[0067] Optionally, the actual input size and the best fusion slice are input into the loop scalar control logic to obtain a calling method for the fusion unit, including: when the actual input size is smaller than the best fusion slice, determining that the calling method for the fusion unit by the loop scalar control logic is to call the dynamic fusion fragment in the fusion unit only once; when the actual input size is not smaller than the best fusion slice and is divisible, obtaining the integer value of the actual input size divided by the best fusion slice, and determining that the calling method for the fusion unit by the loop scalar control logic is to cyclically call the static fusion fragment in the fusion unit of the integer value; when the actual input size is larger than the best fusion slice and is not divisible, obtaining the integer value and remainder value of the actual input size divided by the best fusion slice, and determining that the calling method for the fusion unit by the loop scalar control logic is to cyclically call the static fusion fragment in the fusion unit of the integer value, and call the dynamic fusion fragment in the fusion unit of the remainder value.

[0068] Step S205, detecting the calling results of each fusion unit of the dynamic neural network.

[0069] Specifically, in this embodiment, after the dynamic neural network is executed, the call results of each fusion unit will be detected, for example, whether the fusion unit calls the dynamic fusion segment and the static fusion segment according to the calculated number of calls, or whether the calculation output of the fusion unit is garbled or other obvious errors. When it is determined that the fusion unit is not called according to the calculated number of calls, it means that the call logic part of the cyclic scalar control logic may have a fault. At this time, an alarm prompt information will be generated to prompt the supervisor to maintain the cyclic scalar control logic in time; when the calculation output of the fusion unit is garbled, it means that the calculation part of the fusion unit may have a fault. At this time, an alarm prompt will be generated to prompt the supervisor to maintain the abnormal fusion unit in time. Of course, this embodiment is only an example, and does not limit the specific content of the detection.

[0070] It should be noted that in this embodiment, a test report will be generated after the test is completed. The test report includes test items, test results and maintenance suggestions. The test report and a visual form will be displayed to supervisors to facilitate rapid discovery of abnormal points based on the test report and rapid maintenance of abnormal points based on maintenance suggestions, thereby improving the repair efficiency of the cyclic scalar control logic or fusion unit.

[0071] This implementation method obtains dynamic fusion fragments in the compilation stage, obtains the best fusion slices of each dynamic fusion fragment that are suitable for the hardware cache size without relying on shape information, and converts the dynamic fusion fragments into static fusion fragments based on the best fusion slices to eliminate the relevant computing nodes of the dynamic shape, thereby optimizing the dynamic neural network in the compilation stage. In the execution stage, the static fusion fragments can be dynamically scheduled and planned, thereby dynamically converting the internal processing of the original operator level into the dynamic processing of the scheduling logic, which greatly improves the execution efficiency of the dynamic neural network.

[0072] Embodiment 3

[0073] Figure 5 This is a schematic diagram of the structure of a dynamic neural network compilation and execution device provided by the third embodiment of the present invention. Figure 5 As shown, the device includes: a dynamic fusion segment acquisition module 310, an optimal fusion slice acquisition unit 320, a fusion unit acquisition module 330 and a fusion unit calling module 340.

[0074] The dynamic fusion segment acquisition module 310 is used to acquire the dynamic fusion segment according to each operator node of the dynamic neural network during the compilation stage;

[0075] The best fusion slice acquisition unit 320 is used to acquire chip parameters of the multi-level storage chip carrying the dynamic neural network, and calculate the best fusion slice corresponding to each dynamic fusion segment according to the chip parameters;

[0076] A fusion unit acquisition module 330 is used to acquire a static fusion segment matching the corresponding dynamic fusion segment according to the best fusion slice, and form a fusion unit with each dynamic fusion segment and the matching static fusion segment, wherein the fusion unit performs tensor transport in a shared storage layer of a multi-level storage chip;

[0077] The fusion unit calling module 340 is used to obtain the loop scalar control logic of the dynamic neural network during the execution phase, and dynamically call the fusion unit based on the loop scalar control logic.

[0078] Optionally, the dynamic fusion fragment acquisition module includes: a fusion operator node screening unit, which is used to screen out fusion operator nodes from each operator node of the dynamic neural network during the compilation stage;

[0079] The dynamic fusion segment acquisition unit is used to acquire the dynamic fusion segment according to the fusion operator node, wherein the dynamic fusion segment includes at least two fusion operator nodes having a dependent relationship.

[0080] Optionally, a fusion operator node screening unit is used to obtain a weight transfer type of each operator node in the dynamic neural network, wherein the weight transfer type includes a sparse weight transfer type and an intensive weight transfer type;

[0081] Obtain the dynamic dimension and reduced calculation dimension of each operator node in the dynamic neural network;

[0082] The operator nodes whose weight transfer type in the dynamic neural network is sparse weight transfer type and whose dynamic dimensions do not overlap with the reduced calculation dimensions are used as fusion operator nodes.

[0083] Optionally, a fusion operator node screening unit is used to obtain the weight size and chip bandwidth of each operator node in the dynamic neural network, and obtain the weight transfer overhead according to the ratio of the weight size to the chip bandwidth;

[0084] Obtaining the data handling cost and data calculation cost of each operator node in the dynamic neural network, and obtaining a cost threshold according to the sum of the data handling cost and the data calculation cost;

[0085] It is determined whether the weight transfer cost of the operator node is less than the cost threshold. If so, the weight transfer type of the operator node is determined to be weight transfer sparse type; otherwise, the weight transfer type of the operator node is determined to be weight transfer intensive type.

[0086] Optionally, a dynamic fusion fragment acquisition unit is used to obtain the input-output ratio of each fusion operator node;

[0087] The fusion operator nodes whose input-output ratios are within the first numerical range and have a dependency relationship are searched and fused using a greedy algorithm to obtain at least one dynamic fusion segment.

[0088] Optionally, the chip parameters include a chip cache size, an optimal fusion slice acquisition unit, and a memory cache size occupied by the dynamic fusion fragments under the initial slice value.

[0089] Determine whether the memory cache size is larger than the chip cache size. If so, reduce the initial slice value to obtain the updated slice value. Otherwise, increase the initial slice value to obtain the updated slice value.

[0090] The difference between the updated slice value and the chip cache size is calculated, and when the difference is within the second value range, the updated slice value is used as the best fused slice.

[0091] Optionally, the device further comprises an optimal fusion slice alignment module, which is used to align the dimensions of the optimal fusion slices into even numbers;

[0092] Calculate the data size corresponding to the best fused slice after even alignment, and align the data size as integer multiples with reference to the chip handling bandwidth.

[0093] Optionally, the fusion unit acquisition module includes a static fusion segment acquisition unit, which is used to fill each aligned best fusion slice into the dynamic dimension of the corresponding dynamic fusion segment to obtain a static fusion segment matching the dynamic fusion segment;

[0094] Perform constant folding and algebraic simplification on each static fusion fragment to obtain a static fusion fragment optimized for a common layer;

[0095] Static operator fusion and redundant operator node elimination are performed on the static fusion fragment after optimization of the common layer to obtain the static fusion fragment after optimization of the cache layer.

[0096] Optionally, a fusion unit calling module is used to obtain the best fusion slice corresponding to the fusion unit to be executed and the actual input size of the dynamic neural network;

[0097] The actual input size and the best fused slice are input into the loop scalar control logic to obtain a calling mode for the fused unit, and the fused unit to be executed is dynamically called based on the calling mode.

[0098] Optionally, the fusion unit calling module is further used to determine that the calling mode of the loop scalar control logic for the fusion unit is to call the dynamic fusion segment in the fusion unit only once when the actual input size is smaller than the optimal fusion slice;

[0099] When the actual input size is not less than the best fused slice and is divisible, obtain the integer value of the actual input size divided by the best fused slice, and determine that the calling mode of the loop scalar control logic for the fused unit is to loop call the static fused fragment in the fused unit of the integer value;

[0100] When the actual input size is larger than the best fused slice and is not divisible, the integer value and remainder value of the actual input size divided by the best fused slice are obtained, and the calling mode of the loop scalar control logic for the fusion unit is determined to loop call the static fusion fragment in the fusion unit of the integer value and call the dynamic fusion fragment in the fusion unit of the remainder value.

[0101] The dynamic neural network compilation and execution device provided in the embodiment of the present invention can execute the dynamic neural network compilation and execution method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0102] Embodiment 4

[0103] Figure 6 A schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention is shown in FIG. Figure 6 As shown, the computer device includes a processor 610, a memory 620, an input device 630, and an output device 640; the number of the processor 610 in the computer device can be one or more. Figure 6 A processor 610 is taken as an example; the processor 610, memory 620, input device 630 and output device 640 in the computer device can be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0104] The memory 620, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the method for compiling and optimizing the computational graph in the embodiment of the present invention. The processor 610 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 620, that is, implements the above-mentioned method for compiling and executing the dynamic neural network.

[0105] The memory 620 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 620 may further include a memory remotely arranged relative to the processor 610, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0106] The input device 630 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the computer device. The output device 640 may include a display device such as a display screen.

[0107] Embodiment 5

[0108] Embodiment 5 of the present invention further provides a storage medium including computer executable instructions, which, when executed by a computer processor, are used to execute a compilation and execution method of a dynamic neural network, including: obtaining a dynamic fusion fragment according to each operator node of the dynamic neural network in a compilation stage;

[0109] Obtain chip parameters of a multi-level storage chip carrying a dynamic neural network, and calculate the best fusion slice corresponding to each dynamic fusion segment according to the chip parameters;

[0110] According to the best fusion slice, a static fusion segment matching the corresponding dynamic fusion segment is obtained, and each dynamic fusion segment and the matching static fusion segment form a fusion unit, wherein the fusion unit performs tensor transport in a shared storage layer of a multi-level storage chip;

[0111] In the execution phase, the loop scalar control logic of the dynamic neural network is obtained, and the fusion unit is dynamically called based on the loop scalar control logic.

[0112] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention is not limited to the above method operations, and the computer executable instructions can also execute related operations in the compilation and execution method of the dynamic neural network provided by any embodiment of the present invention.

[0113] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0114] It is worth noting that in the above-mentioned embodiment of the compilation optimization device according to the computational graph, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0115] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for compiling and executing a dynamic neural network, characterized in that: include: In the compilation phase, dynamic fusion fragments are obtained according to each operator node of the dynamic neural network; Obtaining chip parameters of a multi-level storage chip carrying the dynamic neural network, and calculating the best fusion slice corresponding to each of the dynamic fusion segments according to the chip parameters; According to the optimal fusion slice, a static fusion segment matching the corresponding dynamic fusion segment is obtained, and each of the dynamic fusion segments and the matching static fusion segment constitutes a fusion unit, wherein the fusion unit performs tensor transport in the shared storage layer of the multi-level storage chip; The loop scalar control logic of the dynamic neural network is obtained during the execution phase, and the fusion unit is dynamically called based on the loop scalar control logic.

2. The method according to claim 1, characterized in that The step of obtaining the dynamic fusion fragment according to each operator node of the dynamic neural network in the compilation stage includes: In the compilation phase, a fusion operator node is selected from each operator node of the dynamic neural network; The dynamic fusion segment is obtained according to the fusion operator node, wherein the dynamic fusion segment includes at least two of the fusion operator nodes having a dependent relationship.

3. The method according to claim 2, characterized in that The step of selecting a fusion operator node from each operator node of the dynamic neural network during the compilation phase includes: Obtaining a weight transfer type of each operator node in the dynamic neural network, wherein the weight transfer type includes a sparse weight transfer type and an intensive weight transfer type; Obtaining the dynamic dimension and the reduced calculation dimension of each operator node in the dynamic neural network; The weight transfer type in the dynamic neural network is a weight transfer sparse type, and the operator node whose dynamic dimension does not overlap with the reduced calculation dimension is used as the fusion operator node.

4. The method according to claim 3, characterized in that The obtaining of the weight transfer type of each operator node in the dynamic neural network includes: Obtaining the weight size and chip bandwidth of each operator node in the dynamic neural network, and obtaining the weight transport overhead according to the ratio of the weight size to the chip bandwidth; Obtaining data handling overhead and data calculation overhead of each operator node in the dynamic neural network, and obtaining an overhead threshold according to the sum of the data handling overhead and the data calculation overhead; Determine whether the weight transfer overhead of the operator node is less than the overhead threshold; if so, determine that the weight transfer type of the operator node is the weight transfer sparse type; otherwise, determine that the weight transfer type of the operator node is the weight transfer intensive type.

5. The method according to claim 2, characterized in that: The acquiring the dynamic fusion segment according to the fusion operator node includes: Obtaining the input-output ratio of each fusion operator node; The fusion operator nodes whose input-output ratios are within a first numerical range and have a dependency relationship are searched and fused using a greedy algorithm to obtain at least one of the dynamic fusion fragments.

6. The method according to claim 1, characterized in that The chip parameters include a chip cache size, and calculating the best fusion slice corresponding to each of the dynamic fusion segments according to the chip parameters includes: Setting an initial slice value for the dynamic dimension of each of the dynamic fusion segments, and calculating the memory cache size occupied by the dynamic fusion segment under the initial slice value; Determine whether the memory cache size is greater than the chip cache size, if so, reduce the initial slice value to obtain an updated slice value, otherwise, increase the initial slice value to obtain an updated slice value; The difference between the updated slice value and the chip cache size is calculated, and when the difference is within a second numerical range, the updated slice value is used as the optimal fused slice.

7. The method according to claim 1, characterized in that After calculating the best fusion slice corresponding to each of the dynamic fusion segments according to the chip parameters, the method further includes: Aligning the dimensions of the best fused slices to an even number; The data size corresponding to the optimal fused slice after even-number alignment is calculated, and the data size is aligned as an integer multiple with reference to the chip transport bandwidth.

8. The method according to claim 7, characterized in that The step of obtaining a static fusion segment that matches a corresponding dynamic fusion segment according to the optimal fusion slice includes: Filling the aligned best fused slices into the dynamic dimension of the corresponding dynamic fused segment to obtain the static fused segment matching the dynamic fused segment; Performing constant folding and algebraic simplification on each of the static fusion fragments to obtain the static fusion fragment after ordinary layer optimization; Static operator fusion and redundant operator node elimination are performed on the static fusion fragment after the ordinary layer is optimized to obtain the static fusion fragment after the cache layer is optimized.

9. The method according to claim 1, characterized in that: The dynamically calling the fusion unit based on the loop scalar control logic includes: Obtaining the best fusion slice corresponding to the fusion unit to be executed currently and the actual input size of the dynamic neural network; The actual input size and the optimal fused slice are input into the loop scalar control logic to obtain a calling mode for the fusion unit, and the fusion unit to be executed currently is dynamically called based on the calling mode.

10. The method according to claim 9, characterized in that The step of inputting the actual input size and the optimal fused slice into the loop scalar control logic to obtain a calling mode for the fusion unit includes: When the actual input size is smaller than the optimal fused slice, determining that the calling mode of the loop scalar control logic for the fusion unit is to call the dynamic fusion segment in the fusion unit only once; When the actual input size is not less than the best fused slice and is divisible by an integer, obtaining an integer value of the actual input size divided by the best fused slice, and determining that the calling mode of the loop scalar control logic for the fusion unit is to loop call the static fusion segment in the fusion unit of the integer value; When the actual input size is larger than the optimal fusion slice and is not divisible, obtain the integer value and remainder value of the actual input size divided by the optimal fusion slice, and determine that the calling mode of the loop scalar control logic for the fusion unit is to cyclically call the static fusion fragment in the fusion unit of the integer value and call the dynamic fusion fragment in the fusion unit of the remainder value.

11. A compiling and executing device for a dynamic neural network, characterized in that: include: A dynamic fusion fragment acquisition module is used to acquire dynamic fusion fragments according to each operator node of the dynamic neural network during the compilation stage; An optimal fusion slice acquisition unit, used to acquire chip parameters of a multi-level storage chip carrying the dynamic neural network, and calculate the optimal fusion slice corresponding to each of the dynamic fusion segments according to the chip parameters; A fusion unit acquisition module, used for acquiring a static fusion segment matching the corresponding dynamic fusion segment according to the optimal fusion slice, and forming a fusion unit with each of the dynamic fusion segments and the matching static fusion segment, wherein the fusion unit performs tensor transport in the shared storage layer of the multi-level storage chip; The fusion unit calling module is used to obtain the loop scalar control logic of the dynamic neural network during the execution phase, and dynamically call the fusion unit based on the loop scalar control logic.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 10 is implemented.

13. A computer executable instruction storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images